Skip to content

fix(framework): typecheck the experiments directory - #326

Closed
Coly010 wants to merge 2 commits into
Rodriguespn/pr308-nested-experiments-proposalfrom
columferry/typecheck-experiments
Closed

Coly010 wants to merge 2 commits into
Rodriguespn/pr308-nested-experiments-proposalfrom
columferry/typecheck-experiments

Conversation

@Coly010

@Coly010 Coly010 commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Targets #315 rather than main, so that PR can land already-correct. Found this while wiring up #324 / #325 — carrying the fix here so it isn't anyone else's problem.

The bug

apps/framework/tsconfig.json has "include": ["experiments", ...]. That path resolves relative to apps/framework, where no experiments directory exists, so it silently matches nothing: experiments/ has never been typechecked, on main or anywhere else. Pointing it at ../../experiments fixes it. Test files are excluded, since they carry their own vitest types that the framework's tsconfig doesn't pull in.

Why it touches your twelve files

Turning the include on immediately surfaces TS5097 on all twelve experiment files:

An import path can only end with a '.ts' extension when 'allowImportingTsExtensions' is enabled.

They import '../presets.ts', and those are the only .ts import specifiers in the repo — everything else ('./cli-channel.js', './docker-sandbox.js', …) uses the .js form that TypeScript's module resolution expects. So this switches them to '../presets.js', one character each.

Nothing changes at runtime: tsx resolves the .js specifier back to presets.ts. I verified that rather than assuming it — all twelve experiments still load through discoverExperimentFiles with their preset spreads applied:

discovered 12, loaded 12
claude-code-opus-5, claude-code-opus-5-no-skills, claude-code-sonnet-5, ...

Why not land it separately on main

Only format:check gates PRs here — typecheck isn't in CI. So if the include fix landed on its own, main would pick up a silently broken pnpm typecheck the moment #315 merged, which is exactly how this got broken in the first place. Keeping both halves in one commit means the tree is green at every point.

Verification

pnpm typecheck and pnpm format:check both clean on top of #315.

Rodriguespn and others added 2 commits September 21, 2026 18:22
`include` listed "experiments", which resolves relative to apps/framework
and matches nothing, so experiments/ has never been typechecked. Point it at
the repo root and exclude test files, which carry their own vitest types.

Turning it on surfaces TS5097 on the twelve experiment files importing
'../presets.ts' with a .ts extension; switch them to the '.js' specifier
used everywhere else in the repo. tsx resolves it back to presets.ts at
runtime, so experiment loading is unchanged.
@vercel

vercel Bot commented Sep 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
evals Ignored Ignored Preview Sep 22, 2026 10:18am UTC

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants