Skip to content

[Goal #349] Add Rust/WASM acceleration coverage for core functions#369

Draft
github-actions[bot] wants to merge 32 commits into
mainfrom
goal/349-goal-add-rust-wasm-acceleration-coverage-for-core-functions
Draft

[Goal #349] Add Rust/WASM acceleration coverage for core functions#369
github-actions[bot] wants to merge 32 commits into
mainfrom
goal/349-goal-add-rust-wasm-acceleration-coverage-for-core-functions

Conversation

@github-actions

@github-actions github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Goal

Add Rust/WASM acceleration coverage for every eligible tsb core function while keeping the existing TypeScript implementations as the default-compatible fallback.

Closes #349

What Changed (Run 3)

Coverage manifest gap filled (wasm-coverage.json)

Added 22 missing entries that were previously absent from the 121-entry manifest:

From src/core/index.ts (11 entries):

  • Flags, getFlags — WeakMap-backed runtime metadata; JS-object-identity blocker
  • MaskedArray, IntegerArray, FloatingArray, BooleanArray, StringArray, DatetimeArray, TimedeltaArray — nullable extension array classes; JS class-instance and callback semantics blocker
  • SparseArray, SparseDtype — sparse storage class with JS fill-value semantics

Direct top-level core exports from src/index.ts (11 entries):

  • getOption, setOption, resetOption, describeOption, optionContext, registerOption, options — runtime options registry/Proxy; no numeric kernel
  • api, apiTypes — namespace objects; not numeric functions
  • pdArray, PandasArray — factory/wrapper; JS dispatch + class semantics

Updated summary: 143 total entries (6 rust-wasm, 137 ts-only-ineligible, 0 unclassified, 0 eligible_missing)

Coverage check now validates against live export surface (scripts/wasm-coverage-check.ts)

The script now:

  1. Parses all value exports from src/core/index.ts
  2. Parses all top-level value exports from src/index.ts whose source is under src/core/
  3. Fails with a descriptive error listing which exports are missing from the manifest

This satisfies the contract requirement: "The Rust/WASM coverage check fails when an export is present in src/core/index.ts or in top-level core re-exports but missing from the manifest."

Verification

npx tsx scripts/wasm-coverage-check.ts
✓ Rust/WASM coverage manifest is valid.
  Total core entries : 143
  rust-wasm          : 6
  ts-only-ineligible : 137
  unclassified       : 0
  eligible_missing   : 0
  live exports audited: 143 (132 core/index + 133 top-level core re-exports, 143 unique)
  missing from manifest: 0

Context

  • Run 2 implemented the initial Rust/WASM layer (natsort + searchsorted/argsort) with 121-entry manifest — merged as PR [Goal #349] Add Rust/WASM acceleration coverage for core functions #356 (2026-06-27)
  • Issue was updated 2026-07-02 with a more stringent completion contract that identified the manifest gaps above
  • Run 3 fills the gaps and upgrades the coverage check script

Note

Full evidence script (bun run wasm:build && bun run wasm:test && bun run wasm:coverage && bun run bench:wasm-core) requires Bun and the Rust/WASM toolchain. The environment in this workflow run does not have Bun installed (network-restricted environment). The coverage check passes via npx tsx.

This PR is maintained by the Goal workflow. Each run may add commits to the same branch.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • releaseassets.githubusercontent.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "releaseassets.githubusercontent.com"

See Network Configuration for more information.

Generated by Goal · 1.6K AIC · ⌖ 14.9 AIC · ⊞ 39.8K ·
Comment /goal to run again

…gainst live surface

- Add 22 missing entries to wasm-coverage.json:
  - From src/core/index.ts: Flags, getFlags, MaskedArray, IntegerArray,
    FloatingArray, BooleanArray, StringArray, DatetimeArray, TimedeltaArray,
    SparseArray, SparseDtype
  - Direct top-level core exports via src/index.ts: getOption, setOption,
    resetOption, describeOption, optionContext, registerOption, options
    (core/options.ts), api, apiTypes (core/pd_api.ts), pdArray, PandasArray
    (core/pd_array.ts)
- Update summary: 143 total entries (6 rust-wasm, 137 ts-only-ineligible)
- Update scripts/wasm-coverage-check.ts to validate manifest against the live
  export surface: parses src/core/index.ts and top-level core re-exports from
  src/index.ts, fails with descriptive error if any export is missing from
  the manifest
- Update manifest notes to reflect expanded coverage scope

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added automation goal Agentic goal workflow should continue working this issue labels Jul 2, 2026
@github-actions github-actions Bot added automation goal Agentic goal workflow should continue working this issue labels Jul 2, 2026
All evidence commands verified in this environment:
- bun run typecheck: PASS
- bun run lint: exit 0 (warnings only)
- bun test ./tests/core/: 2107/2107 pass
- cargo test: 15/15 Rust unit tests pass
- bun run wasm:build: SUCCESS (wasm-pack 0.15.0)
- bun run wasm:test: 35/35 parity tests pass
- bun run wasm:coverage: PASS (143 total, 6 rust-wasm, 137 ts-only-ineligible, 0 unclassified, 0 eligible_missing)
- bun run bench:wasm-core: 8 benchmarks, results-wasm-core.json written
- Python evidence script: ALL CHECKS PASS

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Commit pushed: a303d08

Generated by Goal · 1K AIC · ⌖ 22.8 AIC · ⊞ 39.8K

github-actions Bot and others added 3 commits July 2, 2026 09:52
…acceleration

- Add rust/src/reductions.rs: 7 scalar reductions (sum, mean, min, max, var, std, median)
  skipping NaN, with unit tests
- Add rust/src/rolling.rs: rolling_* and expanding_* variants for all 7 reductions,
  with min_periods support and unit tests
- Update src/wasm/types.ts: TypeScript interface for all 21 new Rust functions
- Update src/wasm/loader.ts: verify all 31 WASM functions present at load time
- Update src/wasm/accelerated.ts: accelerated wrappers with TS fallbacks for all new fns
- Update src/wasm/index.ts: re-export all new accelerated functions
- Rebuild rust/pkg/ with wasm-pack 0.15.0 (wasm-bindgen 0.2.126)
- Expand tests/wasm/parity.test.ts: parity tests for all 7 scalar and 3 window functions
- Expand benchmarks/wasm-core/run.ts: sum_f64 and rolling_mean_f64 benchmarks
- Regenerate wasm-coverage.json: 143 top-level entries with methods[] arrays for 46 classes
  covering 706 public methods; deterministic gate passes (all exports covered,
  all priority CPU-heavy methods classified, no banned keywords in reasons)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrjf mrjf added the evergreen Persistent opt-in: Evergreen keeps this PR green/mergeable until ready or quota exhausted label Jul 3, 2026
- Extract _svCacheGet() helper to reduce sortValues cognitive complexity
  from 16 to 1 (max allowed: 15)
  accesses where they are unnecessary (typed array access returns number)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

Commit pushed: 7d992f4

Generated by Evergreen Orchestrator · 1.1K AIC · ⌖ 19.5 AIC · ⊞ 29.6K

@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

Evergreen repair pushed — fixed Test & Lint failures in src/core/series.ts:

Issue Fix
noExcessiveCognitiveComplexity on sortValues (complexity 16, max 15) Extracted _svCacheGet() private helper; sortValues now has complexity 1
noNonNullAssertion (31 violations on typed array accesses) Removed all ! non-null assertions — Uint32Array element access returns number, not number | undefined

Commit: 3aba5d0 — awaiting CI to confirm gates pass.

Generated by Evergreen Orchestrator · 1.1K AIC · ⌖ 19.5 AIC · ⊞ 29.6K ·

@mrjf mrjf added evergreen Persistent opt-in: Evergreen keeps this PR green/mergeable until ready or quota exhausted and removed evergreen Persistent opt-in: Evergreen keeps this PR green/mergeable until ready or quota exhausted labels Jul 6, 2026
Add ?? 0 fallbacks to all typed-array accesses in the float radix sort
path that produce TS18048/TS2532/TS2322/TS2538 errors under
noUncheckedIndexedAccess:
- fvalsU32[fsi] / fvalsU32[fsi+1] (lo/hi reads)
- _rxHisto histogram accumulation (8 lines)
- _rxHisto prefix-sum read
- srcBuf[si+keyOff], _rxHisto[histoBase+bucket] and dstBuf writes
- srcBuf[si/si+1/si+2] in all 4 output reconstruction loops
- finSlice[i] and nanBuf[i] in fallback output loops

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Commit pushed: 1cd2c8a

Generated by Evergreen Orchestrator · 3.1K AIC · ⌖ 16.4 AIC · ⊞ 29.9K

github-actions Bot and others added 2 commits July 7, 2026 05:42
…json

- accelerated.ts: refactor long lines (>100 chars) in medianF64Accelerated,
  expandingMeanF64Accelerated, and rollingFallback to satisfy biome formatter
- wasm-coverage.json: add missing trailing newline at EOF
- series.ts: useLiteralKeys: use dot notation in isIndexLike (rec.size etc.)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Commit pushed: 3d360bc

Generated by Evergreen Orchestrator · 2.9K AIC · ⌖ 14.2 AIC · ⊞ 29.9K

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Evergreen fix: biome format errors

Diagnosed and fixed 3 biome check errors from commit bc4c7a7:

File Issue Fix
src/wasm/accelerated.ts Lines >100 chars in medianF64Accelerated, expandingMeanF64Accelerated, rollingFallback Extracted variables / simplified predicates
wasm-coverage.json Missing trailing newline at EOF Added \n
src/core/series.ts useLiteralKeys warnings in isIndexLike rec["size"]rec.size etc.

Pushed commit f330ebd. CI re-run triggered.

Generated by Evergreen Orchestrator · 2.9K AIC · ⌖ 14.2 AIC · ⊞ 29.9K ·

@mrjf mrjf added evergreen Persistent opt-in: Evergreen keeps this PR green/mergeable until ready or quota exhausted and removed evergreen Persistent opt-in: Evergreen keeps this PR green/mergeable until ready or quota exhausted labels Jul 9, 2026
…111)

Replace dot-notation access on `Record<string, unknown>` with bracket notation
to satisfy `noPropertyAccessFromIndexSignature` (TS4111) in strict TypeScript.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Commit pushed: 5612dec

Generated by Evergreen · 405.6 AIC · ⌖ 13.1 AIC · ⊞ 35.8K

@github-actions

Copy link
Copy Markdown
Contributor Author

Commit pushed: 7d62585

Generated by Evergreen · 1.4K AIC · ⌖ 11.6 AIC · ⊞ 36K

github-actions Bot and others added 2 commits July 10, 2026 02:28
- Remove unused isMissing function (noUnusedVariables)
- Use literal keys rec.at and rec.getLoc instead of rec["at"] etc (useLiteralKeys)
- Refactor toDictOriented into helper functions to reduce cognitive
  complexity from 28 to below 15 (noExcessiveCognitiveComplexity)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor Author

Commit pushed: 2f9c3a3

Generated by Evergreen · 324.9 AIC · ⌖ 12.2 AIC · ⊞ 36K

github-actions Bot and others added 2 commits July 10, 2026 02:55
TS4111 requires bracket access for properties on index-signature types.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor Author

Commit pushed: 5c80424

Generated by Evergreen · 285.8 AIC · ⌖ 11.4 AIC · ⊞ 36.3K

@github-actions github-actions Bot added evergreen_active Evergreen lease: a run is currently working this PR and removed evergreen_active Evergreen lease: a run is currently working this PR labels Jul 10, 2026
@github-actions github-actions Bot added the evergreen_active Evergreen lease: a run is currently working this PR label Jul 10, 2026
- useBlockStatements: add braces to single-statement if bodies in series.ts
- useLiteralKeys: use dot notation for rec["at"], rec["getLoc"], rec["size"], row["Index"]
- useTopLevelRegex: move /^(\d+)(.+)$/ to module-level RE_FREQ_NUNIT constant in timestamp.ts
- noNonNullAssertion: replace _registry.get(k)! with null-safe check in options.ts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor Author

Commit pushed: a051143

Generated by Evergreen · 944.4 AIC · ⌖ 11.9 AIC · ⊞ 36.6K

@github-actions github-actions Bot added evergreen_active Evergreen lease: a run is currently working this PR and removed evergreen_active Evergreen lease: a run is currently working this PR labels Jul 10, 2026
Fix 7 TS4111 errors in frame.ts and series.ts where properties accessed
via dot notation on Record<string, unknown> require bracket notation per
TypeScript's noPropertyAccessFromIndexSignature rule.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor Author

Commit pushed: 79eeb25

Generated by Evergreen · 256.3 AIC · ⌖ 11.6 AIC · ⊞ 36.7K

@github-actions github-actions Bot added evergreen_active Evergreen lease: a run is currently working this PR and removed evergreen_active Evergreen lease: a run is currently working this PR labels Jul 10, 2026
Clear the 3 formatter errors blocking the Test & Lint gate.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor Author

Commit pushed: cf80ebf

Generated by Evergreen · 807.9 AIC · ⌖ 12.7 AIC · ⊞ 36.7K

@github-actions github-actions Bot added evergreen_active Evergreen lease: a run is currently working this PR evergreen-ready Evergreen believes configured merge gates are satisfied and removed evergreen_active Evergreen lease: a run is currently working this PR labels Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automation evergreen Persistent opt-in: Evergreen keeps this PR green/mergeable until ready or quota exhausted evergreen-ready Evergreen believes configured merge gates are satisfied goal Agentic goal workflow should continue working this issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Goal] Add Rust/WASM acceleration coverage for core functions

1 participant