Unofficial Flakiness.io Node.js Reporter

Warning

This is an independent, experimental package and is not an official Flakiness.io project. It is published under Max Schmitt's personal npm scope while upstream adoption is being discussed.

Package: @mxschmitt/flakiness-node

A custom reporter for the built-in node:test runner. It writes a Flakiness Report and can upload it to Flakiness.io.

Install

npm install --save-dev @mxschmitt/flakiness-node

Use the reporter alongside Node's spec reporter to retain normal terminal output:

node --test \
  --test-reporter=spec \
  --test-reporter=@mxschmitt/flakiness-node \
  --test-reporter-destination=stdout \
  --test-reporter-destination=stdout

The report is written to flakiness-report/report.json. Open it with:

npx flakiness show

Upload

Set the Flakiness.io project slug:

FLAKINESS_PROJECT=my-org/my-project node --test --test-reporter=@mxschmitt/flakiness-node

In GitHub Actions, grant OIDC permission so no secret is needed:

permissions:
  contents: read
  id-token: write

steps:
  - uses: actions/checkout@v6
  - uses: actions/setup-node@v6
    with:
      node-version: 24
  - run: npm ci
  - run: npm test
    env:
      FLAKINESS_PROJECT: my-org/my-project

Outside GitHub Actions, set FLAKINESS_ACCESS_TOKEN. Set FLAKINESS_DISABLE_UPLOAD=true to write the report without uploading it.

Configuration

Node does not pass reporter options through --test-reporter, so CLI usage is configured with environment variables.

Variable	Purpose
FLAKINESS_PROJECT	Flakiness.io project slug and GitHub OIDC audience
FLAKINESS_ACCESS_TOKEN	Upload token outside GitHub OIDC
FLAKINESS_ENDPOINT	Alternate service endpoint
FLAKINESS_OUTPUT_DIR	Output directory; defaults to flakiness-report
FLAKINESS_TITLE	Human-readable report title
FLAKINESS_NAME	Environment name; defaults to node:test
FLAKINESS_CONFIG_PATH	Optional config path relative to the git root
FLAKINESS_DUPLICATES	fail (default) or rename
FLAKINESS_DISABLE_UPLOAD	true or 1 disables auto-upload
FK_ENV_*	Custom environment metadata

For programmatic node:test usage, options can be passed directly:

import { createFlakinessReporter } from '@mxschmitt/flakiness-node';
import { run } from 'node:test';

run({ files: ['./test/example.test.js'] })
  .compose(createFlakinessReporter({
    flakinessProject: 'my-org/my-project',
    title: 'integration tests',
  }))
  .pipe(process.stdout);

When using run() with a custom cwd or isolation, pass the same values to createFlakinessReporter(). Node does not include these programmatic settings in reporter events.

Behavior

• Node suites become Flakiness suites.
• A test that owns subtests is reported as a test plus an anonymous suite with the same title, preserving both the parent result and its child hierarchy.
• skip, todo, and expectFailure become skip, fixme, and fail annotations.
• Persisted --test-rerun-failures attempts are represented individually. Node does not retain prior durations or errors, so earlier attempts are synthesized with zero duration.
• Native tags are consumed when provided by Node. On older Node versions, trailing @tag tokens in suite and test titles are supported as a fallback.
• Test file process slots are exposed as parallelIndex.
• File-loading failures, root-level after-hook failures, post-test asynchronous errors, and interrupted test files are reported as unattributed errors.
• Each completed watch cycle is written and uploaded independently. The local output folder contains the latest cycle.

Node's reporter API exposes stdout and stderr only at file scope, without test identity or timestamps. The reporter does not guess attribution. Newer Node versions that emit test-scoped test:log events are recorded as timed stdio.

Node 20 and Node 22.9 do not emit per-file summary events. If multiple entry files import the same test definition, those tests remain grouped under the definition file and may be reported as duplicates. Node 22.10 and newer provide the boundary needed for exact entry-file attribution. A skipped suite does not expose its child declarations, so its hidden children cannot be reported.

On Node versions that emit test:interrupted, the reporter writes the local report synchronously before Node forces the process to exit. There is not enough time to upload that interrupted report.

See features.md for the complete feature matrix and docs/node-test-reporter-api.md for the API mapping.

Development

npm ci
npm run check
npm test
npm run pack:check

Biome provides formatting, linting, and import organization. Run npm run check:fix to apply its safe fixes.

GitHub Releases publish to npm through trusted publishing; no npm token is stored in GitHub.