feat(core-backend): spike OpenAPI→superstruct codegen for the Price API (ADR spike plan)#9446
feat(core-backend): spike OpenAPI→superstruct codegen for the Price API (ADR spike plan)#9446Prithpal-Sooriya wants to merge 5 commits into
Conversation
…lugin Adds a Kubb-based code generation pipeline to @metamask/core-backend, spiked against the Price API: - codegen/kubb-plugin-superstruct: a Kubb plugin that generates @metamask/superstruct structs from an OpenAPI document - codegen/kubb-plugin-query-core: a Kubb plugin that generates TanStack query-core bindings (query keys, query options, fetchers) that validate responses with the generated structs - codegen/openapi/price-api.json: curated Price API OpenAPI 3.1 document enriched with response schemas - src/generated/price-api: generated types (plugin-ts), structs, faker mocks (plugin-faker), MSW handlers (plugin-msw) and query bindings - new subpath exports: @metamask/core-backend/price-api and @metamask/core-backend/mocks - yarn codegen script wired into the package build step Co-authored-by: Prithpal Sooriya <prithpal.sooriya@gmail.com>
…stic formatted codegen - Pin msw to ~2.10.5 (2.11+ depends on ESM-only rettime, breaking Jest CJS) and @faker-js/faker to ^9 (v10 is ESM-only) - Add HttpHandler annotations to generated MSW handlers (fixes TS2742 declaration emit) - Format generated output with oxfmt as part of yarn codegen - Register the core-backend build script variant in yarn constraints - Fix ESLint violations in the codegen plugins and tests Co-authored-by: Prithpal Sooriya <prithpal.sooriya@gmail.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Caution MetaMask internal reviewing guidelines:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
- Export ApiRequestClient / ApiRequestArgs and PricesApiRequestClient from the package root - Add changelog entries for the generated Price API bindings, mocks entry point and request client Co-authored-by: Prithpal Sooriya <prithpal.sooriya@gmail.com>
… tsx Replaces the @kubb/cli invocation with a small tsx runner (codegen/run.ts) that imports the Kubb config directly and calls safeBuild from @kubb/core. This removes @kubb/cli and its extra dependency surface (config discovery, jiti config loading, subprocess runner) from the lockfile, addressing the Socket Security alerts that targeted @kubb/cli itself. Generated output is unchanged. Co-authored-by: Prithpal Sooriya <prithpal.sooriya@gmail.com>
Aligns the spike with the 'Generate Runtime Schema Validation and Mocks from OpenAPI Specs' ADR (MetaMask/decisions#234) while keeping the DX extras (MSW handlers, query-core bindings) generated from the same pipeline: - Wire generated structs into the existing PricesApiClient response path behind assert() (ADR spike step 3): supported networks, exchange rates, v1/v2/v3 spot prices, v1 single-token and v3 historical prices now reject malformed responses with StructError; unknown additional fields are still tolerated. One drifted test fixture surfaced by validation was corrected. - Move the vendored spec to specs/price-api.json (ADR layout) and add the /v1/exchange-rates/crypto endpoint so all exchange-rate methods are validated. - Rename the struct output directory from schemas/ to structs/ (ADR naming). - Add a codegen:check script (regeneration-is-clean check per the ADR's committed-output constraint). - Add response-validation tests, including one that serves generated mock data through the client to prove mocks and structs stay in sync. Co-authored-by: Prithpal Sooriya <prithpal.sooriya@gmail.com>
Explanation
Executes the spike plan from the "Generate Runtime Schema Validation and Mocks from OpenAPI Specs" ADR (MetaMask/decisions#234), prototyping Option A: Kubb + custom
@metamask/kubb-plugin-superstructagainst the Price API — plus two DX extras generated from the same pipeline (MSW handlers and query-core bindings, see below).Spike plan → status
kubb.config.tswithplugin-oas+plugin-ts+plugin-fakerspecs/price-api.json(see spec-quality finding below), config incodegen/kubb.config.tsplugin-zod's parser/generator shape)codegen/kubb-plugin-superstruct/PricesApiClient's response path behindassert()@metamask/core-backend/mocks; swapping an extension e2e fixture is the follow-up (different repo)1. The custom superstruct plugin (Option A)
packages/core-backend/codegen/kubb-plugin-superstruct/is a full Kubb v4 plugin (built withdefinePluginfrom@kubb/core, modeled on@kubb/plugin-zod):parser.tsconverts Kubb's keyword-based schema tree into superstruct expressions (type(),object()foradditionalProperties: false,record(),enums(),tuple(),union(),intersection(),optional()/nullable()/defaulted(),min/max/patternrefinements, direct struct references for$refs).generators/superstructGenerator.tsemits one file per component schema and one per operation, named<PascalCase>Structper repo convention.type()structs are the default; strictobject()only when the spec saysadditionalProperties: false.2. Structs wired into
PricesApiClient(spike step 3 — the crash-prevention payoff)The existing hand-written client now validates responses behind
assert()for every endpoint covered by the vendored spec (supported networks, exchange rates ×3, v1/v2/v3 spot prices, v1 single-token + v3 historical prices). Malformed responses reject with aStructErrorat the fetch boundary — callers degrade gracefully instead of the wallet crashing onnull.toLowerCase()later; benign additive backend changes still pass.Notably, validation immediately caught a drifted test fixture: a
fetchV1SpotPriceByCoinIdtest mocked{ ethereum: { usd: 2500 } }, a shape the endpoint never returns — exactly the drift class the ADR describes.3. Pinned spec snapshot + regeneration check (ADR constraints)
specs/price-api.jsonis the vendored snapshot; generation always runs offline (yarn codegen, also run by the packagebuildscripts). Runs are deterministic (byte-identical output).yarn codegen:checkverifies regeneration is clean against the committed output — ready to wire into CI; the spec-vs-livedocs-jsondiffing job is follow-up infra.src/generated/price-api/(ESLint-ignored, excluded from coverage).4. Mocks + MSW, importable as
@metamask/core-backend/mocksmocks/— seeded faker builders (createCoinGeckoSpotPrice(), ...) — the ADR's e2e-fixture feed (mockttp-compatible:mockServer.forGet(...).thenJson(200, createV3SpotPrices())).msw/— DX extra beyond the ADR's minimum ("not required", but free from the same pipeline): MSW request handlers serving the seeded mocks (getV3SpotPricesHandler(), aggregatehandlers) for unit/integration tests in repos that use MSW.mswand@faker-js/fakerare optional peer dependencies, so runtime entry points stay free of test-only dependencies.5. Query-core bindings — a working preview of the ADR's "Later" item
codegen/kubb-plugin-query-core/generates, per operation, a query key factory, a query options factory, and afetch*function whosequeryFnvalidates with the generated struct — mirroring the hand-writtengetV3SpotPricesQueryOptions/fetchV3SpotPricesnaming and consuming an injectableApiRequestClientcontract implemented byPricesApiRequestClient extends BaseApiClient. The ADR marks typed request functions/queryOptions as "Later"; this demonstrates the follow-up is mechanical once validation lands. (No@tanstack/react-queryhooks are generated, per the ADR's hard "No".)Everything is importable from
@metamask/core-backend/price-api(types + structs + query bindings).Findings (spike step 5)
docs-jsondeclares response DTOs for only 4 of ~27 endpoints.specs/price-api.jsonis therefore a curated snapshot enriched with response schemas matching the shapes the hand-written client already documents. Generation is only as good as the spec — upstream response-decorator coverage in the APIs is a prerequisite for rolling this out.additionalProperties), nullable record values, tuples (prefixItems), enums,$refgraphs, defaults. Not yet exercised: discriminators,allOfmerging, recursive$refs (kubb'splugin-oasnormalizes most of this before the parser sees it).@metamask/superstruct, already a dependency;@tanstack/query-corewas already a dependency. All codegen tooling is dev-only.plugin-oasdid the heavy OpenAPI lifting. Nothing surfaced that would force adopting Zod (Option B).@faker-js/fakerpinned to v9 (v10 is ESM-only, incompatible with the CJS build);mswpinned to~2.10.5(≥2.11 depends on ESM-onlyrettime, which Jest's CJS runtime can't load; peer range stays^2.0.0);@kubb/plugin-fakeremits{}foradditionalProperties-only record schemas (can be papered over withtransformers.schemalater); MSW handlers need a small post-processing step for portableHttpHandlertype annotations (codegen/annotate-msw-handlers.mjs).codegen/kubb-plugin-superstructinto its own workspace package (the ADR's@metamask/kubb-plugin-superstruct) or upstream to kubb-labs/plugins; kept in-package here for reviewability.Tests
src/api/prices/client.test.ts— response-validation cases through the realPricesApiClient: malformed responses reject with preciseStructErrors, additive fields pass, and generated mock data served as a response round-trips (mocks and structs share a source, so fixtures can't drift).src/api/prices/query-client.test.ts— the generated pipeline end-to-end: faker mocks validate against structs; generated query bindings fetch through the generated MSW handlers with validation on the way out (failure, path-param interpolation, handler override, and QueryClient dedup cases).Supply chain notes (Socket Security triage)
The Socket check needs a human sign-off; the alerts all come from transitive packages of two intentional dev-only additions and cannot be reduced further in code (dropping
@kubb/clialready removed its alerts — codegen callssafeBuildfrom@kubb/coredirectly viacodegen/run.ts).Mitigations that apply to everything below:
dist/.es5-ext,mswpostinstall) never execute in this repo:.yarnrc.ymlsetsenableScripts: falseandyarn allow-scripts autoreports no allowlist changes needed.specs/price-api.json, so the OpenAPI toolchain's remote-$ref/URL-fetching code paths are never exercised.Via
@kubb/oas(kubb's OpenAPI parser — intrinsic to kubb, effectively part of the Option A/B decision):es5-ext(protestware console notice; postinstall inert here),es6-promise(minified bundle flagged as "obfuscated"),@apidevtools/json-schema-ref-parser/@redocly/openapi-core/oas-normalize/oas-resolver/swagger2openapi/http2-client/node-fetch-h2(network-access alerts from remote-spec-fetching support we don't use),@redocly/ajv/mustache(AI anomaly heuristics),@readme/openapi-schemas/compute-gcd(publisher changed).Via
msw(test-only):mswpostinstall (inert),@mswjs/interceptors(AI anomaly — request interception is msw's documented purpose),@bundled-es-modules/cookie/mute-stream(publisher changed).Ready-to-paste triage comment for a reviewer who has verified the notes above (per internal guidelines this sign-off is a human decision, so it is intentionally not posted by the agent):
References
Checklist