Handle transient worker-options file absence during parallel startup

[Shoryamishra61]

Problem

When parallel type checking is enabled (--num-workers > 0), build workers can intermittently fail during startup with:

FileNotFoundError: [Errno 2] No such file or directory: '.mypy_cache/.worker_options.<id>.data'
Failed to establish connection with worker: No status file found

The reported failures show that workers can intermittently observe the options file as absent during startup. The exact underlying filesystem/process timing mechanism has not been established, but the worker currently treats even a transient FileNotFoundError as fatal.

Root Cause & Context

PR #21306 serialized build options to .mypy_cache/.worker_options.<build_id>.data to avoid command-line argument limits with large configurations (#21305). The coordinator already uses WORKER_START_TIMEOUT / WORKER_START_INTERVAL while waiting for a worker to publish its status file, but the worker's initial options-file read currently has no tolerance for temporary absence.

Fix

• In mypy/build_worker/worker.py, introduce a read_options_data helper that adds a bounded retry only for FileNotFoundError using the existing WORKER_START_TIMEOUT and WORKER_START_INTERVAL defaults.
• Only FileNotFoundError is retried; unrelated OS errors (such as PermissionError) and deserialization corruption propagate immediately.
• The normal successful path performs no retry and no sleep; the file is read on the first attempt.
• If the retry deadline is exceeded, the original FileNotFoundError is re-raised.

Tests

• Added deterministic fault-injection unit tests in mypy/test/testworker.py covering:
  • Immediate success (verifying sleep is not called).
  • Single and multiple transient misses recovering within budget.
  • Timeout exhaustion (verifying the original exception instance propagates and no trailing sleep occurs).
  • Explicit timeout=0 failure without sleep.
  • Propagation of unrelated PermissionError without retry.
  • Corrupted data parsing failure.
  • Roundtrip Options serialization.
• Verified existing parallel suite:
  • python runtests.py testCheckingStubPackagesWorksInParallelMode
  • pytest -n0 mypy/test/testcmdline.py -k "Parallel"
• As supplementary validation, 20 concurrent invocations using four workers each completed without failure. The regression itself is covered deterministically by fault-injection tests because the original failure is intermittent.
• Passed self-check: python runtests.py self
• Passed linters: python runtests.py lint

Fixes #21974

[Copilot]

🟡 Changes recommended

The retry helper can sleep beyond its advertised timeout when the interval exceeds the remaining budget.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR adds bounded retries for transient worker-options file absence during parallel startup.

Changes:

• Retries FileNotFoundError while reading worker options.
• Preserves immediate propagation of other errors.
• Adds deterministic retry and serialization tests.

File summaries

File	Summary
mypy/test/testworker.py	Adds coverage for retry, timeout, error propagation, corruption, and serialization behavior.
mypy/build_worker/worker.py	Implements options-file retry handling. Moderate issue: sleep must be capped by the remaining timeout.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[ilevkivskyi]

Superseded by other PRs.