Skip to content

Serialize concurrent frontend dependency installs - #7051

Open
Alek99 wants to merge 3 commits into
mainfrom
codex/serialize-frontend-installs
Open

Serialize concurrent frontend dependency installs#7051
Alek99 wants to merge 3 commits into
mainfrom
codex/serialize-frontend-installs

Conversation

@Alek99

@Alek99 Alek99 commented Sep 6, 2026

Copy link
Copy Markdown
Member

Why

Two Reflex processes can currently enter the same frontend install transaction together. Both can mutate .web, invoke the package manager, overwrite the procedure cache, and persist root lockfiles concurrently. A simultaneous frontend reinitialization can also replace .web during an install.

What changed

  • Add a dependency-free per-project process lock at the stable, ignored app-root path .reflex.frontend.lock. POSIX uses flock; Windows uses a retrying one-byte msvcrt lock. The process-local layer is thread-safe and reentrant, and the OS releases ownership after crashes.
  • Hold that lock across the complete public install transaction: package-manager selection, root-to-web synchronization and cache invalidation, cached install execution, cache commit, and web-to-root persistence.
  • Coordinate initialize_web_directory with the same lock so it cannot replace .web during an active install.
  • Atomically replace procedure-cache and persisted package-manager files. Invalid/truncated pickles become cache misses; genuine I/O errors still surface. Existing modes and symlinks are preserved.
  • Keep the lock implementation lazily imported so ordinary CLI startup does not load platform lock modules or allocate lock state.

Atomic replacement guarantees complete-file visibility and preserves the prior file when commit fails; it does not claim power-loss durability without fsync.

Validation

  • uv run pytest -q tests/units/test_prerequisites.py tests/units/utils/test_utils.py tests/units/reflex_base/constants/test_base.py tests/units/reflex_base/constants/test_installer.py — 344 passed
  • uv run ruff check . — passed
  • uv run ruff format --check . — 1407 files already formatted
  • uv run pyright reflex tests — 0 errors, 0 warnings
  • uv run pre-commit run --all-files — all hooks passed, including Ruff, codespell, Pyright, ty, and Biome
  • Independent adversarial review: 104 prerequisite tests and 16 focused concurrency/atomic/cache tests passed; no release-blocking findings

Tests cover positive same-project OS contention, complete transaction scope, completed-cache reuse by a waiting process, separate-project concurrency, reentrancy, exceptions and abrupt process exits, Windows contention retries, initialization exclusion, atomic failure recovery, corrupt-cache recovery, permissions, and symlink compatibility.

Checklist

  • Bug fix (non-breaking)
  • Followed CONTRIBUTING.md
  • Checked for overlapping open PRs
  • Added regression tests for core behavior
  • Ran tests and linters locally

Review in cubic

@Alek99
Alek99 requested a review from a team as a code owner September 6, 2026 07:59
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread reflex/utils/frontend_lock.py
Comment thread packages/reflex-base/src/reflex_base/utils/decorator.py Outdated
@codspeed-hq

codspeed-hq Bot commented Sep 6, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 32 untouched benchmarks
⏩ 8 skipped benchmarks1


Comparing codex/serialize-frontend-installs (7c6ffe5) with main (c49a85d)

Open in CodSpeed

Footnotes

  1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@greptile-apps

greptile-apps Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR serializes frontend installation and web-directory initialization through a reentrant per-project process lock, while making generated package-manager files and procedure caches atomically replaceable.

  • Holds one project lock across package-manager selection, synchronization, installation, cache commit, and root persistence.
  • Resets inherited lock descriptors and process-local ownership after POSIX forks.
  • Treats malformed procedure caches as misses while preserving operational errors.
  • Adds focused concurrency, fork-safety, atomic-write, permission, symlink, and cache-recovery coverage.

Confidence Score: 5/5

The PR appears safe to merge; the changes since the previous review address the outstanding test-organization requirement without introducing a confirmed regression.

The fork-safety revision serializes descriptor transitions around forks, resets inherited ownership in children, and preserves cleanup across acquisition and release paths. Both previous review findings are now fully addressed.

Important Files Changed

Filename Overview
reflex/utils/frontend_lock.py Implements reentrant cross-process project locking with guarded raw-descriptor tracking and fork-safe child state reset.
reflex/utils/js_runtimes.py Extends the project lock across the complete frontend installation transaction and stabilizes cache inputs around resolved plugin dependencies.
reflex/utils/frontend_skeleton.py Coordinates web-directory initialization with installs and atomically replaces generated and persisted package-manager files.
packages/reflex-base/src/reflex_base/utils/decorator.py Atomically writes procedure caches and safely converts malformed cache contents into cache misses.
tests/units/utils/test_frontend_lock.py Provides focused coverage for contention, reentrancy, crash cleanup, Windows retries, and POSIX fork behavior.
tests/units/reflex_base/utils/test_decorator.py Moves decorator tests into the matching module and covers cache corruption, I/O errors, atomic replacement, and symlink preservation.
tests/units/test_prerequisites.py Tests complete installation transaction locking, cache reuse across processes, initialization exclusion, and atomic lockfile persistence.

Reviews (3): Last reviewed commit: "Make frontend lock cleanup fork-safe" | Re-trigger Greptile

Comment thread reflex/utils/frontend_lock.py
Comment thread tests/units/test_prerequisites.py Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread reflex/utils/frontend_lock.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant