Skip to content

The Dev database under load with a concurrent db update; the e2e job's database; the loop in the docs - #1242

Merged
borisno2 merged 2 commits into
prisma-8from
claude/issue-1238-concurrency-test-qa-gaps
Sep 6, 2026
Merged

The Dev database under load with a concurrent db update; the e2e job's database; the loop in the docs#1242
borisno2 merged 2 commits into
prisma-8from
claude/issue-1238-concurrency-test-qa-gaps

Conversation

@borisno2

@borisno2 borisno2 commented Sep 6, 2026

Copy link
Copy Markdown
Member

Implements #1238. Part of #1125.

Closes the three gaps the QA acceptance pass on #1125 returned FAIL on.

1. The concurrency test (blocking)

#1125's Testing Decisions require "the app under create/edit/delete load with a concurrent second-process db update on the single-connection binding." Nothing did this — staged-reconcile.test.ts runs the second process, but with no load, and the load is the whole point.

packages/cli/tests/dev-database-concurrency.test.ts runs a real opensaas dev loop over a new two-list fixture. Its app drives create/edit/delete through the generated bundle's own client — so the connection under test is the one resolveRuntimeConnection binds, not a hand-rolled one — in the mix ADR-0063's verification set used: context.transaction writing both sides of a relation, a .include() read back, an edit, a re-read, a delete, and a confirmation the row is gone. Every step verifies its own answer, so a corrupted session shows up as a wrong result and not only as a thrown error.

The load runs in batches that keep going until three second-process opensaas db update runs against the same sidecar have completed, so the two overlap whichever finishes first rather than relying on a fixed size landing in the right window.

It discriminates. With packages/core/src/db/client.ts's max: 1 raised to max: 10 and core rebuilt, the test fails: 7020 of 7488 cycles, reporting exactly the failures ADR-0063's amendment documents —

  • portal "" does not exist
  • unnamed prepared statement does not exist
  • the committed author was not readable (a committed row invisible to the next statement)
  • read back "note <another worker's tag>" (silent cross-talk between sessions)

— and all three db update runs exiting non-zero on CONTRACT.MARKER_READ_FAILED (bind message supplies 2 parameters, but prepared statement "" requires 1) and DRIVER.CONNECTION_FAILED (read ECONNRESET). A single sequential request stays clean under the raised pool, which is why the load — not a smoke check — is the test. client.ts is reverted; nothing in this PR touches it.

Runtime: ~12 s locally (~4 s of it load), alongside dev-loop.test.ts's ~24 s locally.

2. The e2e job's database

#1227 removed DATABASE_URL from the e2e job, which put CI on the Dev-database branch — the opposite of story 25 ("local e2e runs on the Dev database and CI on the container"). The job now resolves DATABASE_URL from a pgvector-capable Postgres 17 service container, using the same base_ref == 'main' expression and the same digest-pinned image the test job uses, so PRs into main keep the SQLite string that branch's examples still need.

On the if: github.base_ref == 'main' gate: it should not change yet, and this PR does not change it. Every step in the job stands on the SQLite examples and the Prisma 7 scaffold — the scaffold guard already skips itself against #1129, the rate-limit / OAuth-cascade / credential-deny guards run against SQLite, and pnpm test:e2e builds examples/starter-auth, which cannot build on this branch. Lifting the gate now would add a job that cannot pass. It should be lifted with #1129 / build spec 9, when the examples convert; the comment above the job already says so.

3. Stale docs

docs/content/how-to/installation.md now describes the shipped loop: install then generate (no db:push), pnpm dev bringing the database up, the watcher applying additive changes without a restart, pnpm db:update for a destructive one, and DATABASE_URL as the escape rather than "switching to PostgreSQL". The emitted-artifact list is the contract pair rather than schema.prisma.

The sweep of its neighbours found four more pnpm generate + pnpm db:push blocks, all of which now say what applies the change: how-to/authentication.md, how-to/rag.md, reference/tiptap.md (two) and reference/rag.md. No db:push remains in docs/content outside the Keystone-migration guides, which are about the reader's own existing Prisma.

Deliberately left alone: tutorials/quick-start.md and concepts/generators.md still carry a wider pre-Prisma-8 description (a prismaClientConstructor, a SQLite adapter, schema.prisma). Converting them is the examples' conversion, #1129 — half-converting them would leave an incoherent page.

Verification

  • pnpm lint — 0 errors (2 pre-existing warnings), pnpm format, pnpm manypkg fix clean
  • pnpm build green; CLI + core suites green both with DATABASE_URL unset (Dev database) and set to a real pgvector Postgres 17: core 78 files / 1564 passed / 1 skipped, CLI 36 files / 373 passed
  • No changeset: the change touches packages/cli/tests/ only, no src or package.json

🤖 Generated with Claude Code

… update

Spec #1125's Testing Decisions call for the app under create/edit/delete load
with a concurrent second-process `db update` on the single-connection binding.
The staged-reconcile test ran the second process but with no load, and the load
is what the binding exists for: `pglite-socket` multiplexes every connection
onto one PGlite backend, so a pool corrupts the session state a single backend
shares (ADR-0063, 2026-09-03 amendment).

`dev-database-concurrency.test.ts` runs a real `opensaas dev` loop whose app
drives transactions, `.include()` reads, edits and deletes through the generated
bundle's own client, in batches that keep going while three second-process
`opensaas db update` runs complete. Every cycle verifies what it reads back, so
a corrupted session shows up as a wrong answer and not only as a thrown error.

Raised to `max: 10`, the pool the runtime binds for the Dev database fails it:
7020 of 7488 cycles, with `portal "" does not exist`, `unnamed prepared
statement does not exist`, committed rows unreadable, and rows read back
carrying another worker's title — and all three `db update` runs exiting
non-zero on `CONTRACT.MARKER_READ_FAILED` / `ECONNRESET`. A single sequential
request stays clean either way, which is why the load is the test.

The `e2e` job resolves `DATABASE_URL` from a pgvector-capable Postgres 17
service container the way the `test` job does, so CI exercises the escape and a
local `pnpm test:e2e`, setting nothing, exercises the Dev database — story 25.
PRs into `main` keep the SQLite string that branch still uses. The job's
`base_ref == 'main'` gate is untouched: its steps stand on the SQLite examples
and the Prisma 7 scaffold, which cannot run here until #1129 converts them.

`installation.md` describes the shipped loop — install then generate, no
`db:push`, `pnpm dev` for the database, `pnpm db:update` for a destructive
change, `DATABASE_URL` as the escape. The four other how-to and reference pages
that still told the reader to `pnpm db:push` after `pnpm generate` follow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@changeset-bot

changeset-bot Bot commented Sep 6, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: f313b5c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@gitguardian

gitguardian Bot commented Sep 6, 2026

Copy link
Copy Markdown

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
32428770 Triggered Generic Password 0a03fed .github/workflows/test.yml View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@vercel

vercel Bot commented Sep 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
stack-docs Ready Ready Preview Sep 6, 2026 10:23pm UTC

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for Core Package Coverage (./packages/core)

Status Category Percentage Covered / Total
🟢 Lines 94.41% (🎯 65%) 2978 / 3154
🟢 Statements 93.19% (🎯 65%) 3274 / 3513
🟢 Functions 96.48% (🎯 62%) 604 / 626
🟢 Branches 88.13% (🎯 50%) 2281 / 2588
File CoverageNo changed files found.
Generated in workflow #2019 for commit f313b5c by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for UI Package Coverage (./packages/ui)

Status Category Percentage Covered / Total
🔵 Lines 78.45% 244 / 311
🔵 Statements 77.95% 251 / 322
🔵 Functions 69.81% 74 / 106
🔵 Branches 66.94% 160 / 239
File CoverageNo changed files found.
Generated in workflow #2019 for commit f313b5c by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for CLI Package Coverage (./packages/cli)

Status Category Percentage Covered / Total
🔵 Lines 73.64% 1646 / 2235
🔵 Statements 73.23% 1765 / 2410
🔵 Functions 82.42% 286 / 347
🔵 Branches 60.62% 819 / 1351
File CoverageNo changed files found.
Generated in workflow #2019 for commit f313b5c by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for Auth Package Coverage (./packages/auth)

Status Category Percentage Covered / Total
🔵 Lines 99.48% 195 / 196
🔵 Statements 98.13% 210 / 214
🔵 Functions 100% 44 / 44
🔵 Branches 90.77% 187 / 206
File CoverageNo changed files found.
Generated in workflow #2019 for commit f313b5c by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for Storage Package Coverage (./packages/storage)

Status Category Percentage Covered / Total
🔵 Lines 79.66% 235 / 295
🔵 Statements 81.17% 263 / 324
🔵 Functions 87.91% 80 / 91
🔵 Branches 77.46% 220 / 284
File CoverageNo changed files found.
Generated in workflow #2019 for commit f313b5c by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for RAG Package Coverage (./packages/rag)

Status Category Percentage Covered / Total
🔵 Lines 54.38% 397 / 730
🔵 Statements 53.85% 419 / 778
🔵 Functions 64.06% 82 / 128
🔵 Branches 47.25% 198 / 419
File CoverageNo changed files found.
Generated in workflow #2019 for commit f313b5c by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for Storage S3 Package Coverage (./packages/storage-s3)

Status Category Percentage Covered / Total
🔵 Lines 100% 40 / 40
🔵 Statements 100% 40 / 40
🔵 Functions 100% 9 / 9
🔵 Branches 100% 19 / 19
File CoverageNo changed files found.
Generated in workflow #2019 for commit f313b5c by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for Storage Vercel Package Coverage (./packages/storage-vercel)

Status Category Percentage Covered / Total
🔵 Lines 100% 68 / 68
🔵 Statements 100% 71 / 71
🔵 Functions 100% 15 / 15
🔵 Branches 97.87% 46 / 47
File CoverageNo changed files found.
Generated in workflow #2019 for commit f313b5c by the Vitest Coverage Report Action

Comment thread .github/workflows/test.yml Outdated
# into `main` keep the SQLite string that branch's examples still use, the
# same split the `test` job makes.
env:
DATABASE_URL: ${{ github.base_ref == 'main' && 'file:./dev.db' || 'postgres://postgres:postgres@localhost:5432/postgres' }}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ternary can only ever take the SQLite branch, so this doesn't put CI on the container.

The job itself is gated if: github.base_ref == 'main' (line 149). Every run that reaches this env: therefore has base_ref == 'main' true, so DATABASE_URL is always file:./dev.db and the postgres://... branch — and the service container added below — is unreachable.

Two consequences:

  1. Story 25's "CI on the container" is not achieved by this change; the job stays on the escape branch with a SQLite string.
  2. It reinstates exactly what CI on the escape for every job, the nightly cold clone, and e2e through the lookup #1227 removed. The comment this hunk replaces said a SQLite URL "is a value the Prisma 8 toolchain cannot use at all", and the run where that bites is the one case the gate lets through with this branch's code: the eventual prisma-8main integration PR, where base_ref == 'main', the job runs, Prisma 8 code is checked out, and DATABASE_URL is file:./dev.db. Before this PR that run used the in-process Dev database and worked.

If the intent is "container in CI, sidecar locally", the URL needs a condition that is not the same one that gates the job (e.g. set the Postgres URL unconditionally and let the main gate keep the SQLite-example steps out), otherwise the container and the || branch are dead until the gate moves.

Comment thread .github/workflows/test.yml Outdated
# pack-declaring contract's `CREATE EXTENSION vector` has something to
# install (ADR-0063, ADR-0065). Pinned by digest for the reason given
# there; bump both together.
services:

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This service starts on every e2e run and can never receive a connection.

services: are not conditional — GitHub starts them whenever the job runs. Because the job is gated on base_ref == 'main' and the DATABASE_URL above resolves to file:./dev.db under exactly that condition, no step in this job will ever dial localhost:5432. Net effect today: an extra image pull plus the pg_isready health wait added to every e2e run, for a database nothing opens.

Worth either deferring the service to the same commit that lifts the job gate, or fixing the URL expression so the container is actually used.

Comment thread docs/content/how-to/installation.md Outdated
```bash
pnpm generate # regenerate schema, types, and context
pnpm db:push # apply schema changes to the database
pnpm db:update

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pnpm db:update on its own will not apply the parked destructive change.

The section above says the loop parks a data-dropping change and this is how you approve it, but the approval needs a consent token:

  • examples/starter/package.json: "db:update": "opensaas db update" — no --confirm.
  • packages/cli/src/commands/db.ts forwards options.confirm ?? [] straight through.
  • planDatabaseUpdate() runs the Prisma CLI with --no-interactive, so with no token Prisma refuses the destructive plan; the loop replies The database is unchanged and nothing was promoted. and db update exits 1.

The loop's own parked-change message says it: run `pnpm db:update` (`opensaas db update --confirm postgres`) (packages/cli/src/commands/dev.ts:218). A reader following this page verbatim gets a failure at the one step it exists to explain.

Suggested change
pnpm db:update
pnpm db:update --confirm postgres

Comment thread docs/content/how-to/installation.md Outdated

- **`prisma/schema.prisma`** — the Prisma schema
- **`prisma.config.ts`** — Prisma CLI configuration (datasource URL for db push/migrations)
- **`prisma/contract.json`** and **`prisma/contract.d.ts`** — the emitted schema contract; commit both

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list leaves out prisma/contract.ts, and this bullet is what tells the reader what to commit.

opensaas generate emits the Contract module itself at prisma/contract.ts (DEFAULT_CONTRACT_MODULE in packages/cli/src/generator/output-paths.ts); contract.json / contract.d.ts are the two artifacts prisma contract emit writes beside it, and prisma.config.ts's contract field points at the module, not at the JSON.

Since the bullet says "commit both", a reader treating this list as the tracked set omits prisma/contract.ts — and then every prisma CLI command on a fresh clone fails on a missing contract module.

pnpm dev
```

`pnpm dev` is `opensaas dev`: it starts a local Postgres the stack runs

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale prerequisite one screen above this new paragraph: line 7 still says "Node.js 18+".

The loop this hunk describes depends on Node natively stripping types from the emitted .ts bundle (.opensaas/context.ts, imported with no loader — see packages/cli/tests/bundle-node-load.test.ts), and packages/cli/package.json declares engines.node: ">=22.18.0". A reader on Node 18 or 20 who follows this page fails at exactly the pnpm dev step being introduced here, with no hint from the prerequisites that their Node is the cause. Worth bumping the prerequisite in the same pass that brings the page up to date.

// overlap whichever finishes first — a sized load that ended before the
// first `db update` connected would assert nothing about the two together.
const observed: LoadResult = { cycles: 0, failures: [] }
while (!updatesFinished) {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only exit condition is updatesFinished, and nothing bounds it — a stalled db update degrades into a bare 600 s timeout with none of the diagnostics.

runDbUpdate never times out, and requestDatabaseUpdate (packages/cli/src/dev/control.ts) waits on the control socket with no deadline of its own. Two concrete ways this bites:

  1. The loop is alive but slow to reply. It is the process this test deliberately saturates, and dev-database.ts documents that "a connection holding an open transaction holds the whole query queue". While the update is pending the driver loop keeps issuing more load, which is precisely the traffic the reconcile has to queue behind — the load stops only when the update finishes, and the update is racing the load. Any real stall here surfaces as Test timed out in 600000ms with no loop.output(), no update output, and no failure list — the diagnostics every other assertion in this test is careful to attach.
  2. The spawn rejects (error without close). secondProcess rejects, updatesFinished stays false, await secondProcess on line 259 is never reached, and vitest reports an unhandled rejection instead of the failure.

Suggest a wall-clock deadline on the while loop (fail with loop.output() and the update outputs when it trips) plus a kill timer inside runDbUpdate, matching the SIGKILL timer startLoop and dev-loop.test.ts already use.

const CONNECTION_VARIABLES = ['DATABASE_URL', 'DIRECT_DATABASE_URL'] as const

const ROUNDS = 4
const CONCURRENCY = 16

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

16-wide against a max: 1 pool puts the batch's tail right up against the pool's own 20 s checkout ceiling.

resolveRuntimeConnection binds the Dev database to new Pool({ max: 1, connectionTimeoutMillis: 20_000 }) (packages/core/src/db/client.ts), and pg-pool applies connectionTimeoutMillis to queued acquisitions, not just to dialling. With CONCURRENCY = 16 every cycle's ~12 statements serialize through the one connection, so the last worker in a round waits roughly the whole round — and the round is competing with a db update whose reconcile holds PGlite's single query queue in the middle of it.

CI is the bad case: dev-loop.test.ts and staged-reconcile.test.ts each host their own PGlite in parallel vitest workers on the same 2-core runner, and this file drives batches continuously for as long as three db update runs take. A round that crosses 20 s makes the tail workers throw timeout exceeded when trying to connect, which cycle()'s catch records as a cycle failure — a red build that reads exactly like the session corruption this test exists to detect, on a machine where nothing was corrupted.

Same total load with a bounded tail: drop CONCURRENCY (4–8) and raise ROUNDS to compensate.

return {
output: () => output,
stop: async () => {
if (child.exitCode !== null) return

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stop() can never resolve for a child that died from a signal, so afterAll hangs to its hook timeout.

child.exitCode is null when a process was terminated by a signal (signalCode carries it instead), so this guard does not return early for a loop that was SIGKILLed — by the OOM killer, or by the 5 s timer on line 170 in an earlier stop(). close has already fired, so child.once('close', …) never runs; kill('SIGINT') on a dead child is a no-op; and the timer only issues another no-op SIGKILL without resolving. The promise stays pending, and afterAll fails on vitest's 10 s hook timeout — layered on top of whatever actually killed the loop, which is the failure you wanted to read.

Suggested change
if (child.exitCode !== null) return
if (child.exitCode !== null || child.signalCode !== null) return

Resolving from the timer as well (rather than only killing) would close the remaining case.

observed.failures.slice(0, 10),
`${observed.failures.length} of ${observed.cycles} cycles failed\n\n${loop.output()}`,
).toEqual([])
expect(observed.cycles).toBeGreaterThanOrEqual(ROUNDS * CONCURRENCY)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion cannot fail, and nothing else pins the property the test is named for.

The while loop always completes at least one full batch, and a batch is exactly ROUNDS * CONCURRENCY cycles, so observed.cycles >= ROUNDS * CONCURRENCY holds unconditionally — including in the degenerate case the batching comment above was written to rule out. If opensaas db update ever starts failing fast (a regression in the control channel, say), all three runs finish before the first batch returns, the exit-code assertions catch the failure — but had they been non-zero for a benign reason, the test would pass green having exercised no overlap at all.

Cheap fix: stamp a timestamp when the first db update completes and assert at least one batch's window spanned it, or count the batches issued while updatesFinished was false and assert > 1.

@borisno2 borisno2 left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review — high effort

Verdict: REQUEST CHANGES. (Same-account authorship blocks a formal review event, so this is a Comment review; the verdict is the one above.)

The concurrency test is a real test, not a vacuous one — the 7020/7488 failure rate at max: 10 is convincing evidence it discriminates, the fixture's ORM usage matches Prisma 8's surface, and tx.unsafe is correctly scoped so max: 1 can't self-deadlock. But it has three flakiness/vacuity holes, and the e2e workflow change does not do what its comment says it does. 9 inline comments posted; ordered by severity here.

Blocking

  1. .github/workflows/test.yml:160 — the postgres:// arm of the ternary is unreachable. The job is itself gated if: github.base_ref == 'main' (line 149), so github.base_ref == 'main' is always true inside it and DATABASE_URL is always file:./dev.db. Story 25's "CI on the container" is not achieved. Worse, the run where this bites is the eventual prisma-8main PR: the gate passes, Prisma 8 code checks out, and it gets handed a SQLite URL — exactly what #1227 removed. Either drop the gate and keep the ternary, or drop the ternary and set the container URL unconditionally; the two cannot both stand.

    On the ternary idiom itself: the &&/|| form is correct here, because the middle operand 'file:./dev.db' is a non-empty string and therefore truthy. The falsy-middle-operand pitfall does not apply. The service container is pinned by digest and its POSTGRES_PASSWORD/port match both the expression and the test job's service exactly — that part is clean.

  2. .github/workflows/test.yml:169 — consequence of (1): services: are unconditional, so every e2e run pulls the pgvector image and waits on its health check for a database nothing will ever dial. Pure added CI minutes.

  3. dev-database-concurrency.test.ts:264 — the headline assertion cannot fail, and nothing asserts overlap. observed.cycles >= ROUNDS * CONCURRENCY compares a batch against its own exact size. More importantly, nothing in the test asserts that any db update actually overlapped the load — the degenerate case where all three updates land before or after the driver loop would still pass green. Given this spec already shipped two tests that passed without testing anything, this needs a positive assertion: record the cycle counter at each db update start/finish and assert a non-trivial number of cycles fell inside those windows.

  4. dev-database-concurrency.test.ts:253 — no timeout on the load loop's only exit condition. The driver's sole exit is updatesFinished, and neither runDbUpdate nor requestDatabaseUpdate has a timeout. Since the load only stops when the update finishes, and the update queues behind that load on PGlite's single queue, a stall degrades into the bare 600s vitest timeout with no loop.output() — an opaque red build. A spawn rejection also leaves secondProcess unawaited (unhandled rejection).

  5. dev-database-concurrency.test.ts:166stop() can hang forever. The early return is on exitCode !== null; a signal-killed child has exitCode === null with close already fired, so the promise never settles and the 5s timer only re-kills. That hangs afterAll into vitest's 10s hook timeout, masking whatever actually failed — and is exactly the "leaves a sidecar behind" shape.

Flakiness risk (the timeout question)

  1. :36CONCURRENCY = 16 against a max: 1 pool with a 20s connectionTimeoutMillis. That timeout applies to queued acquisitions, so the 16th worker of a round waits nearly the whole round, and a db update holds the queue mid-round. On a CI runner also hosting two other PGlite test files, a round crossing 20s surfaces as timeout exceeded when trying to connect, recorded as a cycle failure — a red build that looks like the corruption the test hunts. This is precisely "a timeout that fails under CI load rather than under real corruption". Either raise connectionTimeoutMillis for the fixture or classify connect-timeouts separately from data-corruption failures.

On runtime and the constants: ~12s local → ~1min CI is a defensible extrapolation and proportionate for the risk this covers. But CONCURRENCY = 16 and the cycle counts read as arbitrary — the PR body justifies 16 by "it fails at max: 10", which argues the pool, not the concurrency. A sentence tying 16 to the queue depth it's meant to produce would make the number reviewable; as written a future reader has no basis to change it.

Docs

  1. how-to/installation.md:97 — factually wrong. pnpm db:update alone will not apply a parked destructive change. The starter script is opensaas db update with no --confirm, db.ts forwards an empty token list, and planDatabaseUpdate runs Prisma --no-interactive — so the loop prints "The database is unchanged and nothing was promoted" and exits 1. The loop's own message says --confirm postgres (dev.ts:218). The additive-vs-destructive story is the one thing these five files exist to tell, and this page tells it wrong.
  2. how-to/installation.md:60 — the emitted-artifact list omits prisma/contract.ts, which prisma.config.ts points at. The bullet says "commit both", so a reader treating this as the tracked set breaks the Prisma CLI on a fresh clone.
  3. how-to/installation.md:33 — prerequisites still say "Node.js 18+", but the loop being documented needs native type stripping for the .ts bundle; packages/cli declares engines.node >=22.18.0. A Node 18/20 reader fails at exactly the step this page adds.

On removing the pnpm db:studio line: removing rather than verifying was the wrong call. Studio is the reader's only way to see whether an additive change actually landed, which is the claim the surrounding prose now makes. Verify it and restore it, or say explicitly why it's gone.

On scope (quick-start.md, concepts/generators.md left to #1129): defensible, and I'd keep the boundary — a half-conversion is worse. But it is not free: a reader moving from installation.md (no db:push) to quick-start.md (still db:push) gets two stories. Drop a one-line "this page still describes the Prisma 7 loop; see #1129" marker on each un-converted page so the contradiction is signposted rather than discovered.

Checked and clean

Fixture ORM usage matches Prisma 8 (where().include().first(), where().update(), where().delete() — ADR-0041, #1068 spike); tx.unsafe correctly bound in runTransactionBody; packages/cli/tests/tmp-*/ is gitignored; promotion moves files so it cannot clobber .opensaas/dev-db; no db:push remains anywhere in docs/content; both pgvector services identically pinned and credentialed.

Repo rules: no any and no type casting in the diff. Comments are load-bearing (the pglite-socket single-backend multiplexing note is a legitimate external constraint) — except the e2e job comment at :153-158, which now describes behaviour the gate makes impossible; fix it with finding (1). No file under .changeset/ was added, modified, or deleted, and the no-changeset claim holds: packages/cli/tests/ ships nothing to consumers.

…e container

The completion assertion compared a batch count to its own size, so it held
unconditionally — and nothing asserted the load ever met a `db update`. The app
now keeps a counter of cycles that finished clean and serves it at `/completed`;
the test samples it either side of each `db update` child and requires a round's
worth of cycles to have completed while one was alive. Constructed without
overlap (updates awaited first, then one full batch) that assertion fails at 0
while the old `cycles >= ROUNDS * CONCURRENCY` still passes.

`CONCURRENCY` drops to 8 and `ROUNDS` rises to 8: the same total load, but the
tail worker of a round no longer sits near pg-pool's 20 s queued-checkout ceiling
on the `max: 1` binding. A checkout timeout is now recorded apart from the
failures, so a loaded runner cannot report itself as the session corruption this
test hunts.

Lifecycles: the load loop takes a wall-clock deadline and each batch an abort
signal, both failing with the loop output and every `db update`'s output;
`runDbUpdate` kills a stalled child and resolves on `error` rather than
rejecting; the second process cannot leave an unhandled rejection or a driver
loop with no exit; and `stop()` treats a signal-killed child (`exitCode === null`,
`signalCode` set) as already gone and resolves from its own timer.

The `e2e` job's `DATABASE_URL` ternary was keyed on the same condition that gates
the job, so it always yielded the SQLite string and the pgvector service started
for a database nothing dialled. Reverted to no `DATABASE_URL` — story 25's
container belongs in the commit that lifts the gate, which is #1129, and the
comment now says so rather than describing a split that cannot happen.

`installation.md`: `pnpm db:update` alone is refused on a parked destructive
change (`CLI.CONSENT_REQUIRED`, exit 1), so the page says `--confirm postgres`
and what the token is; `prisma/contract.ts` joins the emitted-artifact list; the
prerequisite follows `engines.node >= 22.18.0`. `prisma studio` is not a command
in Prisma 8 rc.12 (`CLI.UNKNOWN_COMMAND`), so the removed `db:studio` line is
replaced by the Admin UI, which is how a reader sees an additive change land.
`quick-start.md` and `concepts/generators.md` are signposted as still describing
the pre-Prisma-8 flow pending #1129.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@borisno2

borisno2 commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

Addressed in f313b5c. Per finding:

3 (blocking) — the completion assertion cannot fail, and nothing asserts overlap

The app now keeps a counter of cycles that finished clean and serves it at /completed. runDbUpdate is bracketed by a read of that counter, so overlappingCycles is the number of cycles that completed while a db update child was alive. The vacuous observed.cycles >= ROUNDS * CONCURRENCY is gone; the assertion is now overlappingCycles >= MIN_OVERLAPPING_CYCLES (one round's worth).

Proven, not assumed:

  • It discriminates. Temporarily raising the threshold to 1,000,000 reported only 1781 cycles completed while a db update child was alive — 1781 overlapping cycles locally against a bound of 8.
  • It fails without overlap. I built the degenerate case as a throwaway copy: await secondProcess first, then exactly one full batch. The old assertion expect(observed.cycles).toBeGreaterThanOrEqual(ROUNDS * CONCURRENCY) passed on that run (64 >= 64) and the new one failed with expected 0 to be greater than or equal to 8. That is precisely the case you described passing green before. Probe deleted.

1 + 2 (blocking) — the unreachable postgres:// arm and its service container

Reverted the whole hunk to #1227's state: no DATABASE_URL, no service container. You were right that the two cannot both stand, and of the two coherent options neither is available today — lifting the gate adds a job that cannot pass (the scaffold guard skips itself against #1129, the rate-limit / OAuth-cascade / credential-deny guards run against SQLite, and pnpm test:e2e builds examples/starter-auth, which cannot build here), and an unconditional container URL breaks the only runs the gate lets through. So the honest resolution is to stop claiming the coverage: the comment now says story 25's container belongs in the commit that lifts the gate, which is #1129, instead of describing a split the gate makes impossible. The pgvector service no longer starts on every e2e run.

4 + 5 (blocking) — lifecycle holes

  • :253 — the load loop takes a wall-clock deadline (LOAD_DEADLINE_MS), each batch an AbortSignal.timeout, and both fail through one diagnose() that attaches loop.output(), every db update's output, and the cycle/failure/timeout counts. runDbUpdate SIGKILLs a child that outlives DB_UPDATE_TIMEOUT_MS and notes it in the output.
  • :253 — the second process is now .catch(…).finally(() => { updatesFinished = true }). A spawn rejection can no longer go unhandled or leave the driver loop without an exit; runDbUpdate also resolves on error (with exitCode: null) rather than rejecting, so the existing expect(update.exitCode).toBe(0) catches it with the output attached.
  • :166stop() now returns early on child.exitCode !== null || child.signalCode !== null, and the 5 s timer resolves as well as killing.

6 (flakiness) — CONCURRENCY = 16 against a 20 s connectionTimeoutMillis

Two changes:

  • The two failure modes are now distinguishable. A cycle whose error contains timeout exceeded when trying to connect is recorded in a separate timeouts array, never in failures. The corruption assertion is expect(observed.failures).toEqual([]) and is untouched by pool starvation; the timeout count appears in every diagnostic message so a slow runner is legible as a slow runner.
  • The number is justified and changed. CONCURRENCY is the queue depth presented to the one connection: above 1 so cycles interleave on the shared session, low enough that a round's tail worker stays inside the 20 s ceiling. It is now 8, with ROUNDS raised to 8 — identical total load, bounded tail. The docblock says exactly that and points at DEV_CONNECTION_TIMEOUT_MS in packages/core/src/db/client.ts, so a future reader has a basis to change it.

Five consecutive local runs: 6.80s, 6.76s, 6.95s, 6.76s, 6.95s.

7, 8, 9 + db:studio + scope signposts (docs)

  • :97 — verified against the shipped code rather than reasoned about. A bare opensaas db update against a parked destructive change exits 1 with CLI.CONSENT_REQUIRED: "requires explicit consent, and the session is not interactive. Grant it by passing --confirm postgres", then The database is unchanged and nothing was promoted. The page now says pnpm db:update --confirm postgres, explains that the token is the database name Prisma asks for, and states that the bare form is refused and exits non-zero.
  • :60prisma/contract.ts added as its own bullet ("what prisma.config.ts points at"), with the two emitted artifacts beside it and "commit all three".
  • :33 — now "Node.js 22.18 or newer", with the reason (Node's own type stripping for the generated bundle, no loader).
  • pnpm db:studio: I could not restore it, because it cannot work. prisma studio is not a command in Prisma 8 rc.12 — pnpm exec prisma studio in examples/starter returns CLI.UNKNOWN_COMMAND: No command registered for 'studio', and prisma --help lists no studio. Documenting it would send the reader at a command that errors. Your underlying point stands, so the page now names the thing that does answer it: the Admin UI rebuilds from the same config, so reloading it is how you see an additive change land. Separately, examples/starter's "db:studio": "prisma studio" script is dead for the same reason — real, but outside this PR; flagged for its own change.
  • Scope signposts: tutorials/quick-start.md and concepts/generators.md each carry a one-line warning callout saying they still describe the pre-Prisma-8 flow, pointing at the relevant section of installation.md, and naming Spec: Examples, scaffolder, docs and agent guidance on Prisma 8 #1129. Neither is converted.

Verification

pnpm build clean. pnpm lint (0 errors, 2 pre-existing warnings), pnpm manypkg fix (no-op), pnpm format. tsc --noEmit on packages/cli clean. CLI (36 files / 373 tests) and core (78 files / 1564 tests) suites pass both with DATABASE_URL unset and with it pointed at a real local Postgres 14 — pglite-absence.test.ts (#1224) passed in all four runs. No file under .changeset/ touched; still no changeset needed, as the diff is packages/cli/tests/, docs and CI only.

@borisno2

borisno2 commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

Verification of f313b5c8 — all 9 findings

Verdict: APPROVE. (Same-account authorship blocks a formal Approve event, so this is a Comment review; the verdict is the one above.) Every finding is resolved, and the two that could have been resolved cosmetically — the overlap assertion and the e2e revert — were not. Two minor residuals noted at the end; neither blocks.

3 (blocking) — the completion assertion cannot fail, and nothing asserts overlap — RESOLVED

The vacuous observed.cycles >= ROUNDS * CONCURRENCY is gone; observed.cycles survives only as diagnostic text (:342). The replacement is overlappingCycles >= MIN_OVERLAPPING_CYCLES (:380-386).

I checked the counter against the three ways this kind of metric usually turns out to be co-occurrence rather than overlap:

  • Could a counted cycle have lived entirely outside an update's lifetime? No. completed += 1 (app :116) fires at the end of a cycle, after every verification, so a cycle inside the delta necessarily finished inside the sampled window. A cycle that started before the child and finished during it is still genuine overlap — it was in flight while the update ran. The window (:327-329) is readCompletedspawn → child lifetime → closereadCompleted, so it exceeds the child's life by two loopback HTTP round trips. That sliver cannot manufacture a pass: for the sliver to contribute cycles the driver loop must be running, and the driver loop is a continuous while of batches — if it is running during the sliver it was running throughout the child's life too. The only arrangement where the window is all sliver is a child that dies instantly, which the exitCode === 0 assertion (:389) catches.
  • Double counting or retry inflation? No. completed is a monotone counter incremented once per cycle with no retry path, and the three windows are strictly sequential (:326-330), so the deltas are disjoint.
  • Margin. MIN_OVERLAPPING_CYCLES = CONCURRENCY = 8 against the ~1781 you measured is ~220×, and the degenerate probe produced 0 — the assertion's two demonstrated points are 0 and 1781, with the bound near the floor. Low absolute bound, but that is the right direction: it is set to be unreachable-without-overlap rather than to police throughput.

One bonus property worth recording: because only clean cycles increment completed, a run where the pool starved and every cycle timed out cannot pass either — overlappingCycles would be 0. The overlap assertion doubles as the guard that the load did real work, which is what closes the hole finding 6's timeout bucket would otherwise have opened.

1 + 2 (blocking) — the unreachable postgres:// arm and its dead service container — RESOLVED

git diff origin/prisma-8...f313b5c8 -- .github/workflows/test.yml is now seven added comment lines and nothing else. No DATABASE_URL, no services:, no ternary — verified against the file, not just the diff (test.yml:145-163). Nothing dead remains.

The comment is honest about it: it says the container "cannot be added here yet and stay honest", gives the reason (the gate is the only condition the job runs under), names the cost ("a service container plus a URL nothing dials is cost that reads as coverage"), and hands story 25 to #1129. That is a claim of absence, not of coverage — which was the ask.

4 + 5 (blocking) — lifecycle holes — RESOLVED

  • stop() (:217) guards on exitCode !== null || signalCode !== null, and the 5 s timer resolves as well as killing (:221-224). Both halves of the hang are closed.
  • The load loop takes LOAD_DEADLINE_MS (:350-355) and each batch Math.min(remaining, BATCH_TIMEOUT_MS) (:356-360), both failing through diagnose() (:339-345), which attaches loop.output(), every update's output and the cycle/failure/timeout counts. No path left to a bare 600 s timeout with nothing to read.
  • runDbUpdate SIGKILLs at DB_UPDATE_TIMEOUT_MS and records it in the output (:251-254), and resolves on error with exitCode: null (:255-258) so a spawn failure lands on expect(update.exitCode).toBe(0) with the message attached rather than as an unhandled rejection.
  • The second process is .catch(…).finally(() => { updatesFinished = true }) (:332-337), so neither an unhandled rejection nor a driver loop without an exit is reachable, and the captured error is rethrown at :371.

6 (flakiness) — RESOLVED

Classification is on timeout exceeded when trying to connect (app :123), which is pg-pool's own literal for a checkout timeout, not a paraphrase — a stable signal for the driver this binding is pinned to. It cannot mask corruption: every corruption mode this test hunts arrives as a Postgres server error (portal "" does not exist, unnamed prepared statement…) or as one of the six value checks (:97-114), which never reach the catch at all and go to failures unconditionally. expect(observed.failures).toEqual([]) (:373-378) is untouched by the bucket, and the timeout count appears in both failure messages.

CONCURRENCY 8 / ROUNDS 8 keeps the total at 64 with a bounded tail, and the docblock (:35-45) now ties the number to the thing that constrains it and names DEV_CONNECTION_TIMEOUT_MS in packages/core/src/db/client.ts, so it is a reviewable number rather than an arbitrary one. That was the part of finding 6 I cared about most.

Confirmed independently of the local timings: the test job passes on f313b5c8 on a GitHub runner, which is the environment the finding was about.

7, 8, 9 — RESOLVED, each verified against the code rather than the PR body

  • :97--confirm postgres is right. db.ts:30 forwards options.confirm ?? [], dev.ts:219 prints exactly opensaas db update --confirm postgres, and dev-loop.test.ts:161-163 already asserts the bare form yields CONSENT_REQUIRED and a non-zero exit. The page now states the refusal and the exit status, which is more than the fix required.
  • :60prisma/contract.ts matches DEFAULT_CONTRACT_MODULE (output-paths.ts:9), with contract.json / contract.d.ts correctly described as emitted beside it (CONTRACT_ARTIFACTS, :35). "Commit all three" is now the tracked set.
  • :33 — Node 22.18 is corroborated twice: packages/cli engines.node >= 22.18.0, and prisma@8.0.0-rc.12's own engines.node: ">=22.18.0".
  • Signposts on quick-start.md and concepts/generators.md use {% callout type="warning" %}, which is in the Markdoc schema (docs/lib/markdoc.ts:22-31), and both anchors resolve to real headings in installation.md (## What just got generated, ## The development loop).

pnpm db:studiothe refusal was justified; my original comment was wrong

Verified without relying on the report: I pulled prisma@8.0.0-rc.12 and @prisma/cli@8.0.0-rc.12 from the registry and grepped the shipped bundles. studio occurs zero times in either package, while migrate (2), contract (11) and db (13) all appear in the same files — so the grep is finding command names, and studio is not one of them. prisma studio does not exist in Prisma 8 rc.12. Documenting it would have sent the reader at a command that errors, and the Admin UI substitution answers the underlying point (seeing an additive change land) with something that works. Restoring the line, as I asked, would have been the wrong change. Good call refusing it, and thank you for checking rather than complying.

examples/starter's dead "db:studio": "prisma studio" script is real and correctly scoped out.

Repo rules and scope

No any, no type casting, no @ts- directives in either new file — the only assertion is as const on a string-literal tuple (:33). error: unknown with instanceof narrowing throughout. No file under .changeset/ added, modified or deleted; the no-changeset claim still holds (packages/cli/tests/, docs and CI only). The full file list against origin/prisma-8 is the same ten files as before — no #1129 example conversions and nothing from #1223, #1224, #1226 or #1229.

Two residuals, neither blocking

  1. :339-345 — a db update child can outlive the test on the failure path. If the load loop throws through diagnose() while secondProcess is still pending, afterAll stops the loop but never awaits or kills the update child; only its own 180 s SIGKILL timer bounds it, and vitest may exit first. Narrow (failure path only, short-lived CLI, self-bounded) but it is the same shape as finding 5.
  2. :327-329 — the sampled window is the child's life plus two HTTP round trips. Analysed above as unable to produce a false pass. Sampling before after spawn returns would tighten it for free if you touch this again.

@borisno2
borisno2 merged commit 8d43426 into prisma-8 Sep 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant