Skip to content

Derive the report fields the model kept missing - #32

Merged
jserv merged 1 commit into
mainfrom
fix-gemini
Sep 8, 2026
Merged

Derive the report fields the model kept missing#32
jserv merged 1 commit into
mainfrom
fix-gemini

Conversation

@jserv

@jserv jserv commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

A report was refused whenever the model returned an improvement plan in the wrong order, or phase tags that disagreed with that plan. Both of those follow from the plan itself, so the server derives them now and the model answers only for the mapping it alone knows, which feedback improvement each plan item addresses. What used to reach the candidate as "no evaluation" is an assessment they had earned.

The call around it was measured rather than guessed. Generation lands in six seconds at the median and past twelve at the tail, so the eight-second attempt timeout was cancelling healthy calls; thinking tokens come out of maxOutputTokens, so a thinking response arrives as truncated JSON; and the call budget was a product of two guesses, which let a report die with four of its six calls unspent after two consecutive 503s. Attempts get twenty seconds, the thinking budget is pinned to zero, and the pool is one number the deadline can pay for, spent by whichever loop needs it.

Separately, sanitizeReport bounded the report summary at the generic 300-character field limit while the agent validates it at 1200, and real summaries run 400 to 450 characters, so every candidate has been reading a committee summary that stops mid-sentence. Both halves are named constants now, held together by a test in the style this repository already uses for the integrity bound.

Verified with scripts/test.sh green and 74 live report generations against gemini-3.1-flash-lite across three transcript shapes: a near-silent candidate, an ordinary session, and a 92KB transcript. No failures. Two runs in that set hit consecutive transport failures and recovered on a later call, which is the case the old fixed retry allotment lost.

Not addressed here: the ten-minute wait in #31 is far outside these bounds and nothing in this branch explains it.

Refs #31


Summary by cubic

Reports that were otherwise valid no longer become “no evaluation” when Gemini returns improvement-plan items out of order or phase tags inconsistent with that plan. The server now normalizes those derived fields and preserves the assessment the candidate earned.

Bug Fixes

  • Allows two semantic repairs and shares a five-call budget across repairs and transient retries.
  • Raises report call and overall deadlines, disables Gemini thinking tokens, and delays the browser escape hatch to avoid abandoning healthy reports.
  • Preserves summaries and fallback diagnostics up to 1200 characters instead of truncating them at 300.
  • Includes validation failures in the final error instead of returning a generic schema message.
  • Covers the report-generation timing and retry behavior related to 面試結束 Evaluation 產生時間過長 #31; the separate ten-minute wait remains unchanged.

Written for commit 51f4a12. Summary will update on new commits.

Review in cubic

cubic-dev-ai[bot]

This comment was marked as resolved.

A response was refused when its improvement plan was out of order or its
phase tags disagreed with that plan, and both of those follow from the
plan itself, so an assessment the model had otherwise earned reached the
candidate as no evaluation. The server derives them now and the model
answers only for the mapping it alone knows.

The call around it was measured rather than guessed: attempts were
cancelled under the model's own median latency, thinking was spending
the output budget the report had to fit in, and the deadline funded
fewer calls than the budget handed out. Separately, the browser bounded
a summary the agent validates at 1200 characters to 300, so every
candidate read one that stopped mid-sentence.
@jserv
jserv merged commit cb379c9 into main Sep 8, 2026
6 checks passed
@jserv
jserv deleted the fix-gemini branch September 8, 2026 17:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant