Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/skills/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,6 @@ This directory contains specialized skills (recipes) to guide contributors and A
- [Add Task Pipeline](./add-task-pipeline/SKILL.md) — TypeScript task pipelines and React hooks.
- [Model Schema Validation](./model-schema-validation/SKILL.md) — Model specs, dynamic shapes, and schema validation.
- [Error Handling](./error-handling/SKILL.md) — Error codes, throwing across worklet and JSI boundaries, and catching.
- [Add API Tests](./add-api-tests/SKILL.md) — TypeScript API test suites and the fake native runtime.
- [Verify and Build](./verify-and-build/SKILL.md) — TypeScript typechecking, native rebuilding, and troubleshooting.
- [Skills Maintenance](./skills-maintenance/SKILL.md) — Keeping skills synchronized with core primitives.
148 changes: 148 additions & 0 deletions .agents/skills/add-api-tests/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
---
name: add-api-tests
description: Use when adding or changing anything under src/ — a task pipeline, a hook, a native op wrapper, a registry entry — and you need to cover it with the TypeScript API test suites.
metadata:
id: add_api_tests
scope: packages/react-native-executorch/__tests__/*
---

# Skill: Add TypeScript API Tests

Every change under `src/` belongs in the Jest suites at
[`packages/react-native-executorch/__tests__/`](../../../packages/react-native-executorch/__tests__/README.md).
They run on a laptop or a CI runner — no simulator, no device, no `.pte` — and
finish in a few seconds.

```bash
yarn workspace react-native-executorch test
yarn workspace react-native-executorch test __tests__/tasks # one directory
yarn workspace react-native-executorch test -u # update snapshots
```

Types come along for free: `yarn typecheck` already covers `__tests__/`.

---

## 🧩 The Fake Native Runtime

There is no stubbing of individual JSI calls. `__tests__/support/fakeJsi.ts`
implements the whole `__rnexecutorch_jsi__` contract in JavaScript — tensors
hold real data, `math`/`cv`/`speech` operators compute real values — so a task
pipeline runs end to end and its own logic is what the assertions measure.

A test describes the model it wants, then drives the real pipeline:

```typescript
import { f32, method } from '../../src/core/schema';
import { fakeJsi } from '../support/fakeJsi';
import { tracked } from '../support/lifetime';
import { STRETCH_PREPROCESSING, exported, imageBuffer, writesOutputs } from '../support/fixtures';

fakeJsi.registerModel('/models/task.pte', {
schema: exported(method('forward', [f32(1, 3, 4, 4)], [f32(1, 3)])),
execute: writesOutputs([1, 0, 2]),
});

const runner = tracked(await createMyTask({ modelPath: '/models/task.pte', modelOpts }));
expect(await runner.runTask(imageBuffer(8, 8))).toEqual(/* ... */);
```

Key helpers:

| Helper | Use |
| :--- | :--- |
| `fakeJsi.registerModel(path, program)` | Make `loadModel(path)` succeed with a given schema and `execute` |
| `fakeJsi.registerTokenizer(path, vocabulary)` | Same for `loadTokenizer` |
| `fakeJsi.registerLLMRunner(path, program)` | Same for `createLLMRunner`: a context window and the responses to generate, one per `generate` call |
| `fakePhonemizer.serve(text, phonemes)` | Script the native grapheme-to-phoneme converter |
| `fakeFs.write(path, contents)` | Put a file where a pipeline will read it (a charset, a `tokenizer_config.json`, a voice matrix) |
| `exported(spec)` | Reinterpret a spec built with `method`/`f32`/`i64` as an *exported* one (it verifies no symbolic dims are left) |
| `writesOutputs(...)`, `copiesInputToOutput()` | Ready-made `execute` implementations |
| `tracked(pipeline)` | Auto-dispose at the end of the test |
| `imageBuffer(w, h, format)` | A deterministic input image |
| `cachePathFor(url)` | Where the fetcher will download a URL, so a hook test can register its model up front |
| `fakeNet.serve(url, route)` | Script the server: status, body, `Range` support, and a `gate` to hold a download open |

---

## 🧠 What to Cover for a New Task Pipeline

1. **Schema acceptance** — one test per variant the pipeline declares
(`batched`, `unbatched`, ...), asserting the factory resolves.
2. **Schema rejection** — a model that matches no variant, asserting the error
names the mismatch (`Rank mismatch`, `inconsistent bindings`, ...). A caller
should learn what is wrong from the message.
3. **Configuration mismatch** — e.g. a `labels` array that disagrees with the
model's output dimension.
4. **Postprocessing** — the part that is yours: sorting, thresholding,
suppression, colormaps, coordinate scaling. Choose fixture values that make
the expected output obvious in the test.
5. **Options** — every default in `modelOpts`, and every per-call override.
6. **Disposal** — `dispose()` leaves `fakeJsi.liveTensors()` at 0 and
`fakeJsi.liveModels()` empty, and repeated calls do not accumulate scratch
tensors.
7. **Sync/async parity** — `runTaskWorklet(x)` equals `await runTask(x)`.

Only the *weights* are out of scope, not the pipeline that runs on them. Before
settling for schema-acceptance-and-disposal, check what is actually TypeScript:
a decode loop, a sliding window, a chunker, an argument check, a streaming
generator and a disposal path all run fine over a scripted `execute`. Reach for
the minimal treatment only when the assertion would be measuring the fixture.

For a new hook, add a case to `__tests__/hooks/`: not-ready before the download
lands, methods exposed after, errors surfaced through the shared `error` field,
and every native handle released on unmount.

---

## 🔒 Leak Checking

Native memory is not garbage collected, so the setup file asserts after **every
test** that nothing allocated through the fake was left undisposed. That gives
each pipeline suite disposal coverage for free.

- Wrap construction in `tracked()` — it disposes at the end of the test and
stops a failing assertion from cascading into a second, misleading error.
- A test that deliberately leaks calls `allowNativeLeaks()` with a comment
saying why.

---

## 📐 Source-Level Conventions

`__tests__/api/workletDirective.test.ts` parses `src/` with the TypeScript
compiler and enforces the conventions no type can express:

- every exported function that calls into `rnexecutorchJsi` starts with
`'worklet';`
- no `async` function is marked as a worklet
- only `src/native/bridge.ts` names the `__rnexecutorch_jsi__` global
- `core/` never imports from `extensions/`, and `hooks/` never imports from
`native/`

If you add a new native wrapper without the directive, that suite fails — add
the directive rather than the exception.

---

## 📋 Verification Checklist

When adding or changing code under `src/`, verify that:

- [ ] `yarn workspace react-native-executorch test` passes.
- [ ] A new task pipeline has a suite covering acceptance, rejection,
postprocessing, options and disposal.
- [ ] A new hook has a lifecycle case in `__tests__/hooks/`.
- [ ] A new registry entry passes `__tests__/api/modelRegistry.test.ts` without
the rules being loosened (https URL, pinned revision,
`modelname_backend_precision.pte`, a folder naming that backend, and a
`DEFAULT` that is one of the configs the group offers).
- [ ] A new error code is in `VALID_ERROR_CODES`, so `isRnExecuTorchError` does
not reject the library's own error — `__tests__/core/error.test.ts` reads
every code `src/` raises out of the source and checks it is listed.
- [ ] A new export is reflected in the `api/apiSurface` snapshot, and the change
is intentional (a removal or rename is a breaking change).
- [ ] Any new fake behaviour in `__tests__/support/` is faithful where fidelity
changes an assertion, and its simplifications are commented.
- [ ] No test was made to pass by calling `allowNativeLeaks()` without an
explanation.
1 change: 1 addition & 0 deletions .agents/skills/core-guidelines/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ Use the following index to locate the specific procedural guides for your task:
| **Add a new native operator or C++ binding** | [SKILL.md](../add-native-extension/SKILL.md) | Procedural guide to implementing C++ functions, exposing them via JSI, and writing TypeScript bridge wrappers. |
| **Create a task pipeline or hook** | [SKILL.md](../add-task-pipeline/SKILL.md) | Guide to building end-to-end TS pipelines (e.g. object detection) and exposing them via React hooks. |
| **Verify, rebuild, or troubleshoot changes** | [SKILL.md](../verify-and-build/SKILL.md) | Workflows for rebuilding TS/C++ and resolving common JSI runtime errors. |
| **Test TypeScript changes** | [SKILL.md](../add-api-tests/SKILL.md) | Covering `src/` with the Jest API suites and their fake native runtime. |
| **Validate model constraints & schemas** | [SKILL.md](../model-schema-validation/SKILL.md) | Guide on specifying model specs, dynamic shapes, and runtime constraints for model validation. |
| **Throw, catch, or classify an error** | [SKILL.md](../error-handling/SKILL.md) | The error code set, `RnExecuTorchError`, C++ `RnExecuTorchException`/`guarded`, and adding a code. |
| **Maintain or refactor codebase patterns** | [SKILL.md](../skills-maintenance/SKILL.md) | Guide to keeping workspace skills in sync with codebase state to prevent documentation decay. |
Expand Down
1 change: 1 addition & 0 deletions .agents/skills/skills-maintenance/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Use this guide when you introduce, modify, or deprecate core codebase patterns,
- [add-task-pipeline](../add-task-pipeline/SKILL.md) for TypeScript pipeline orchestration, pre-allocation, and lifecycle hooks.
- [model-schema-validation](../model-schema-validation/SKILL.md) for schema verification constraints.
- [verify-and-build](../verify-and-build/SKILL.md) for compilation and troubleshooting steps.
- [add-api-tests](../add-api-tests/SKILL.md) for the TypeScript test suites and the fake native runtime.

3. **Verify Example Correctness**:
- Ensure all code blocks and examples in updated skills compile/work and match actual usage in the repository.
Expand Down
11 changes: 10 additions & 1 deletion .agents/skills/verify-and-build/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,14 @@ To check types and compile the TypeScript source code:
```bash
yarn typecheck
```
- **Run the TypeScript API Tests**:
```bash
yarn workspace react-native-executorch test
```
_Jest suites over the public `src/` surface — hooks, task pipelines, core
primitives, the fetcher and the model registry — running against a fake
native runtime, so they need no simulator, device or `.pte` file. See the
[Add API Tests skill](../add-api-tests/SKILL.md)._
- **Build Bundles**:
```bash
yarn prepare
Expand Down Expand Up @@ -174,7 +182,7 @@ This project does **not** bundle local `.pte` model files inside the React Nativ

## 🚫 Avoid / Anti-Patterns

- **Do NOT run code without verification:** Do not test TypeScript changes in the app without first running `yarn typecheck` (verify types) and `yarn prepare` (build target bundles).
- **Do NOT run code without verification:** Do not test TypeScript changes in the app without first running `yarn typecheck` (verify types), `yarn workspace react-native-executorch test` (API suites) and `yarn prepare` (build target bundles).
- **Do NOT skip native rebuilds after C++ edits:** If any C++ files or config bindings are added/modified, do not attempt to run the app without executing `pod install` (for iOS) or letting Gradle sync (for Android).
- **Do NOT run `lint:cpp` with the system `clang-tidy`**: Use the Homebrew LLVM binary: `CLANG_TIDY=$(brew --prefix llvm)/bin/clang-tidy yarn workspace react-native-executorch lint:cpp`.
- **Do NOT trust a green macOS-only syntax check for platform-conditional C++:** see the clang-tidy notes above.
Expand All @@ -188,6 +196,7 @@ This project does **not** bundle local `.pte` model files inside the React Nativ
When verifying or compiling your modifications, check that:

- [ ] TypeScript typechecking passes without errors (`yarn typecheck`).
- [ ] The TypeScript API tests pass (`yarn workspace react-native-executorch test`), and any new `src/` behavior is covered by them.
- [ ] Bundles compile successfully (`yarn prepare`).
- [ ] `pod install` has been run inside `apps/<domain-app>/ios/` after any native C++ edits.
- [ ] `lint:cpp` passes cleanly: `CLANG_TIDY=$(brew --prefix llvm)/bin/clang-tidy yarn workspace react-native-executorch lint:cpp`.
Expand Down
13 changes: 13 additions & 0 deletions .cspell-wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -334,3 +334,16 @@ binarizes
unshrunk
DEEPSEEK
LLMKV
macrotask
microtask
microtasks
unbatched
sdcard
dontMock
BIES
binarize
binarized
unclip
phonemizations
phonemizes
həlˈoʊ
13 changes: 13 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,19 @@ module.exports = {
'no-console': 'warn',
},
},
{
// The JSDoc rules exist to keep the generated API reference complete.
// Test helpers are not part of that surface, and requiring a tag per
// parameter on a three-line fixture crowds out the prose that explains
// why the fixture exists.
files: ['packages/react-native-executorch/__tests__/**/*.{ts,tsx}'],
rules: {
'jsdoc/require-param': 'off',
'jsdoc/require-param-description': 'off',
'jsdoc/require-returns': 'off',
'jsdoc/require-returns-description': 'off',
},
},
{
files: ['**/*.md'],
processor: 'markdown/markdown',
Expand Down
18 changes: 18 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,3 +122,21 @@ jobs:

- name: Run C++ unit tests
run: scripts/run-native-tests.sh

api-tests:
name: TypeScript API tests
runs-on: ubuntu-latest
# A full run takes a few seconds; this only has to catch a hang.
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v6

- name: Setup
uses: ./.github/actions/setup

# No native libraries, no simulator, no `.pte` download - the suites run
# against the fake JSI runtime in `__tests__/support/`, so the setup
# action's TypeScript-only install is all this job needs.
- name: Run TypeScript API tests
run: yarn workspace react-native-executorch test --ci
107 changes: 107 additions & 0 deletions packages/react-native-executorch/__tests__/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# TypeScript API tests

Jest suites covering the public TypeScript surface under `src/` — the hooks,
the task pipelines, the core primitives, the resource fetcher and the model
registry. They run on a developer machine or a CI runner: no simulator, no
emulator, no device, no `.pte` file, and the whole run finishes in a few
seconds.

```bash
yarn workspace react-native-executorch test
yarn workspace react-native-executorch test --watch
yarn workspace react-native-executorch test __tests__/tasks # one directory
```

Types are checked by the existing `yarn typecheck`, which already covers this
directory.

## Why a fake native runtime, not stubs

Every path through `src/` bottoms out in `__rnexecutorch_jsi__`: a pipeline
allocates tensors, hands them to `model.execute`, and pushes them through
`softmax`, `resize` and `nms` on the way in and out. Stubbing those calls per
test would mean each assertion checks the stub rather than the pipeline — the
sorting in `classify`, the suppression in `detectObjects` and the colormap in
`segment` would all go untested.

So `support/fakeJsi.ts` implements the native contract in JavaScript instead:

| Piece | What it does |
| --- | --- |
| `support/fakeTensor.ts` | Typed-array-backed tensors with the real `setData`/`getData` byte semantics, `copyTo` windows, and use-after-dispose errors |
| `support/fakeOps.ts` | JS implementations of the `math`, `cv` and `speech` operators, plus the phonemizer host object |
| `support/fakeJsi.ts` | `createTensor`, `loadModel`, `loadTokenizer`, `createLLMRunner`, and the resource trackers |
| `support/blobUtilMock.ts` | In-memory filesystem plus a programmable server (status, body, `Range` support, `stateChange`, and a gate to hold a download open) |
| `support/workletsMock.ts` | Runs worklets inline — a worklet is an ordinary function marked for a second runtime |

A test describes the model it wants and drives the real pipeline over it:

```ts
fakeJsi.registerModel('/models/classifier.pte', {
schema: exported(method('forward', [f32(1, 3, 4, 4)], [f32(1, 3)])),
execute: writesOutputs([1, 0, 2]),
});

const classifier = tracked(await createClassifier(config));
expect((await classifier.classify(imageBuffer(8, 8))).map((r) => r.label))
.toEqual(['bird', 'cat', 'dog']);
```

The fake is faithful where fidelity changes an assertion — writing a float into
`uint8` storage rounds and clamps the way OpenCV's `saturate_cast` does, and the
tokenizer's methods are closures rather than prototype methods because a real
JSI host object's are — and deliberately simple elsewhere: `resize` is
nearest-neighbor whatever interpolation is asked for, and models the geometry
only. The numerical behavior of the real operators belongs to the C++ suites in
[`cpp/tests/`](../cpp/tests/README.md); what these suites own is the TypeScript
above them.

## Leak checking

Native memory is not garbage collected, so anything a test allocates through
the fake and does not dispose is a leak in the code under test. The setup file
asserts that after every test, so each pipeline suite gets disposal coverage
for free.

Wrap construction in `tracked()` and the harness disposes it at the end of the
test — which also keeps a failing assertion from cascading into a second,
misleading leak error. A test that means to leak calls `allowNativeLeaks()`.

## Layout

| Path | Contents |
| --- | --- |
| `core/` | `tensor`, `model`, `runtime`, the coded `error` type, and the `schema` spec matcher |
| `fetcher/` | `download` (caching, resume, cancellation, shared requests), telemetry, the Android backend |
| `tasks/` | One suite per task pipeline, plus the shared construction-failure behavior. `remainingTasks.ts` holds the pipelines that only get schema acceptance and disposal |
| `hooks/` | `useModel`, `useResourceDownload`, and the task hooks end to end |
| `extensions/` | The pure-TypeScript helpers: box/point scaling, seeded generators |
| `api/` | Export snapshot, model registry rules, label constants, source-level conventions |
| `support/` | The fake runtime, the mocks, and the fixtures |

## What is deliberately not covered

**Numerical behavior of the native operators.** `resize` interpolation,
`cvtColor` conversions and the exact `nms` arithmetic are the C++ suites' job;
duplicating them here would only test the fake.

**The weights.** Whisper's decode loop, the VAD rolling window and the SDXS
diffusion step depend on real model weights, so faking them would mostly assert
against the fixture. What they do get is schema acceptance, rejection of a
mismatched model, and full disposal — including Whisper's nested tokenizer and
VAD pipeline.

The line is drawn per pipeline rather than per suite, because it falls in a
different place for each. Kokoro's waveform is weights, but its chunking,
argument validation and streaming are not; the privacy filter's logits are
weights, but the BIOES decode and the sliding window over them are pure
TypeScript and are driven end to end; the LLM's generation belongs to the
native runner, but the chat session's history, KV cache bookkeeping and
tool-calling loop are covered against a scripted one; PaddleOCR's probability
map is weights, but the quad decode, CTC collapse and reading order run over a
map the test paints.

**The thread hop.** Worklets run inline here, so serialization onto a real
worklet runtime is not exercised. The `'worklet'` directive convention that
makes that hop possible *is* checked, by parsing `src/` in
`api/workletDirective.test.ts`.
Loading