refactor(bootstrapper): decompose monolithic bootstrapper into a subpackage#1226
refactor(bootstrapper): decompose monolithic bootstrapper into a subpackage#1226dhellmann wants to merge 19 commits into
Conversation
|
Warning Review limit reached
Next review available in: 36 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughChangesThe old Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant Bootstrapper
participant Resolve
participant Start
participant PrepareSource
participant PrepareBuild
participant Build
participant ProcessInstallDeps
participant Complete
Bootstrapper->>Resolve: run(bt)
Resolve->>Start: returns Start(work_item)
Start->>PrepareSource: returns PrepareSource(wi)
PrepareSource->>PrepareBuild: returns PrepareBuild(wi) + deps
PrepareBuild->>Build: returns Build(wi) + deps
Build->>ProcessInstallDeps: returns ProcessInstallDeps(wi)
ProcessInstallDeps->>Complete: returns Complete(wi) + deps
Complete->>Bootstrapper: returns []
Related PRs: None specified. Suggested labels: refactor, bootstrapper, tests Suggested reviewers: None specified. 🐰 one old module now split in parts, 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (5)
src/fromager/bootstrapper/_phase.py (1)
17-33: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
phaseClassVar has no default and isn't enforced byabc.A subclass that forgets to declare
phasewon't fail at class-definition time — it'll only blow up withAttributeErrorthe first time something accessesself.phase(e.g. inas_json()). Sincerun()is the onlyabstractmethod,phaseis easy to miss when adding a new concrete subclass.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/fromager/bootstrapper/_phase.py` around lines 17 - 33, The Phase base class currently relies on subclasses to remember to define the phase ClassVar, which can slip through until runtime. Make phase mandatory by enforcing it in the Phase hierarchy, either by turning it into an abstract class attribute pattern or by validating it when concrete subclasses are created, and ensure any subclass access points like as_json() can safely assume phase is present. Refer to the Phase class and its subclass implementations when applying the fix.src/fromager/bootstrapper/_cache.py (1)
212-227: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReuse
_create_unpack_dirinstead of duplicating its logic.Lines 224-225 duplicate the exact body of
_create_unpack_dir(lines 30-31) instead of calling it.♻️ Proposed fix
wheel_filename = wheels.download_wheel(req, wheel_url, ctx.wheels_prebuilt) - unpack_dir = ctx.work_dir / f"{req.name}-{resolved_version}" - unpack_dir.mkdir(parents=True, exist_ok=True) + unpack_dir = _create_unpack_dir(ctx.work_dir, req, resolved_version) server.update_wheel_mirror(ctx)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/fromager/bootstrapper/_cache.py` around lines 212 - 227, The `_bg_prepare_prebuilt` function duplicates unpack-directory creation logic that already exists in `_create_unpack_dir`. Replace the inline `unpack_dir = ctx.work_dir / f"{req.name}-{resolved_version}"` and `mkdir(..., exist_ok=True)` sequence with a call to `_create_unpack_dir(req, resolved_version, ctx.work_dir)` so the unpack path is built in one place and stays consistent with the existing helper.src/fromager/bootstrapper/_resolve.py (1)
88-126: 🚀 Performance & Scalability | 🔵 Trivial | 🏗️ Heavy liftSynchronous per-version cache-server lookups in
run().The
multiple_versionsfilter loop calls_cache._find_cached_wheelsynchronously once per candidate version._find_cached_wheelcan fall through to_download_wheel_from_cache, which does network I/O against the cache server — unlike other phases in this refactor, which push such I/O intobackground_work/bg_future. For packages with many candidate versions this can serialize several network round trips on the main dispatch path.Consider moving the cache lookup for each candidate version into background work (or parallelizing across the thread pool) to keep this consistent with the rest of the phase architecture.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/fromager/bootstrapper/_resolve.py` around lines 88 - 126, The multiple-versions filtering in `_resolve.py` is doing synchronous cache-server checks one version at a time via `_cache._find_cached_wheel`, which blocks the main `run()` path on network I/O. Move the per-candidate wheel cache lookup into background work or equivalent parallel execution, and then apply the same cached/non-cached filtering logic to the results before building `filtered` in the `multiple_versions` branch. Keep the existing behavior around retaining the highest version when everything is cached, but ensure the lookup work is scheduled consistently with the rest of the background-based phase handling.src/fromager/bootstrapper/_bootstrapper.py (2)
814-837: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMissing docstrings on public
Bootstrappermethods.
add_to_graph(814-837),has_been_seen(874-881), andadd_to_build_order(883-918) are public methods without docstrings, unlike their siblings (mark_as_seen,create_unresolved_work_items, etc.).📝 Suggested docstrings
+ def add_to_graph( self, req: Requirement, req_type: RequirementType, req_version: Version, download_url: str, parent: tuple[Requirement, Version] | None, ) -> None: + """Add a resolved dependency edge to the dependency graph and persist it.""" parent_req, parent_version = parent if parent else (None, None)def has_been_seen( self, req: Requirement, version: Version, sdist_only: bool = False, ) -> bool: + """Return True if this (req, version) has already been built (sdist or wheel).""" typ: typing.Literal["sdist", "wheel"] = "sdist" if sdist_only else "wheel"def add_to_build_order( self, req: Requirement, version: Version, source_url: str, source_type: SourceType, prebuilt: bool = False, constraint: Requirement | None = None, ) -> None: + """Record a built package in the build-order file, deduplicating by name+version.""" # We only care if this version of this package has been built,Based on learnings, as per coding guidelines:
**/*.py: "Add docstrings on all public functions and classes".Also applies to: 855-918
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/fromager/bootstrapper/_bootstrapper.py` around lines 814 - 837, Public Bootstrapper methods are missing required docstrings, specifically add documentation to add_to_graph, has_been_seen, and add_to_build_order so they match the style of neighboring public methods like mark_as_seen and create_unresolved_work_items. Add concise docstrings that describe each method’s purpose, inputs, and side effects, and ensure all public functions/classes in this area comply with the docstring guideline.Source: Coding guidelines
304-319: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExclusive-build log line issued outside
req_ctxvar_context.The "requires exclusive build, draining background pool" log (line 310-313) fires before entering the
req_ctxvar_contextblock below (316-319), so it won't get the automatic per-requirement log prefix used elsewhere in the codebase.♻️ Suggested fix
if item.requires_exclusive_run: - logger.info( - "%s requires exclusive build, draining background pool", - item.work_item.req, - ) - self._drain_background_pool() - - with ( req_ctxvar_context(item.work_item.req, item.work_item.resolved_version), self._track_why(item), ): + if item.requires_exclusive_run: + logger.info("requires exclusive build, draining background pool") + self._drain_background_pool() + try:As per coding guidelines: "Use
req_ctxvar_context()for per-requirement logging".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/fromager/bootstrapper/_bootstrapper.py` around lines 304 - 319, The exclusive-build log in the bootstrapper loop is emitted before entering req_ctxvar_context, so it misses the per-requirement logging prefix. Move the logger.info call in the stack-processing path inside the req_ctxvar_context block in _bootstrapper.py, near the existing _track_why context, so the "requires exclusive build, draining background pool" message is logged with the active requirement context.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/fromager/bootstrapper/_bootstrapper.py`:
- Around line 814-837: Public Bootstrapper methods are missing required
docstrings, specifically add documentation to add_to_graph, has_been_seen, and
add_to_build_order so they match the style of neighboring public methods like
mark_as_seen and create_unresolved_work_items. Add concise docstrings that
describe each method’s purpose, inputs, and side effects, and ensure all public
functions/classes in this area comply with the docstring guideline.
- Around line 304-319: The exclusive-build log in the bootstrapper loop is
emitted before entering req_ctxvar_context, so it misses the per-requirement
logging prefix. Move the logger.info call in the stack-processing path inside
the req_ctxvar_context block in _bootstrapper.py, near the existing _track_why
context, so the "requires exclusive build, draining background pool" message is
logged with the active requirement context.
In `@src/fromager/bootstrapper/_cache.py`:
- Around line 212-227: The `_bg_prepare_prebuilt` function duplicates
unpack-directory creation logic that already exists in `_create_unpack_dir`.
Replace the inline `unpack_dir = ctx.work_dir /
f"{req.name}-{resolved_version}"` and `mkdir(..., exist_ok=True)` sequence with
a call to `_create_unpack_dir(req, resolved_version, ctx.work_dir)` so the
unpack path is built in one place and stays consistent with the existing helper.
In `@src/fromager/bootstrapper/_phase.py`:
- Around line 17-33: The Phase base class currently relies on subclasses to
remember to define the phase ClassVar, which can slip through until runtime.
Make phase mandatory by enforcing it in the Phase hierarchy, either by turning
it into an abstract class attribute pattern or by validating it when concrete
subclasses are created, and ensure any subclass access points like as_json() can
safely assume phase is present. Refer to the Phase class and its subclass
implementations when applying the fix.
In `@src/fromager/bootstrapper/_resolve.py`:
- Around line 88-126: The multiple-versions filtering in `_resolve.py` is doing
synchronous cache-server checks one version at a time via
`_cache._find_cached_wheel`, which blocks the main `run()` path on network I/O.
Move the per-candidate wheel cache lookup into background work or equivalent
parallel execution, and then apply the same cached/non-cached filtering logic to
the results before building `filtered` in the `multiple_versions` branch. Keep
the existing behavior around retaining the highest version when everything is
cached, but ensure the lookup work is scheduled consistently with the rest of
the background-based phase handling.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 8d6d2213-e334-4785-96a4-204ebc0ac369
📒 Files selected for processing (17)
docs/proposals/phase-classes-plan.mdsrc/fromager/bootstrapper.pysrc/fromager/bootstrapper/__init__.pysrc/fromager/bootstrapper/_bootstrapper.pysrc/fromager/bootstrapper/_build.pysrc/fromager/bootstrapper/_cache.pysrc/fromager/bootstrapper/_complete.pysrc/fromager/bootstrapper/_phase.pysrc/fromager/bootstrapper/_prepare_build.pysrc/fromager/bootstrapper/_prepare_source.pysrc/fromager/bootstrapper/_process_install_deps.pysrc/fromager/bootstrapper/_resolve.pysrc/fromager/bootstrapper/_start.pysrc/fromager/bootstrapper/_types.pysrc/fromager/bootstrapper/_work_item.pytests/test_bootstrapper.pytests/test_bootstrapper_iterative.py
💤 Files with no reviewable changes (1)
- src/fromager/bootstrapper.py
rd4398
left a comment
There was a problem hiding this comment.
I like this refactoring. It makes code more readable and cleaner for sure. I have added few comments / sugegstions. Lets see what @tiran and / or @LalatenduMohanty has to say
|
I've pushed updates to respond to the coderabbit feedback. |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/fromager/bootstrapper/_phase.py (1)
28-48: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd docstrings for consistency with the rest of the class.
__init_subclass__(lines 28-44) and__init__(lines 45-48) rely on inline comments only, while every other method here (run,requires_exclusive_run,background_work,__str__,as_json) has a docstring. The__init_subclass__logic in particular is subtle enough (ABCMeta ordering workaround) to deserve a proper docstring rather than just a trailing comment block.📝 Suggested docstrings
def __init_subclass__(cls, **kwargs: typing.Any) -> None: + """Validate that concrete subclasses define the `phase` class attribute. + + Raises: + TypeError: If a non-abstract subclass omits `phase`. + """ super().__init_subclass__(**kwargs)def __init__(self, work_item: WorkItem) -> None: + """Wrap `work_item` and initialize the background-future slot.""" self.work_item = work_item self.bg_future: concurrent.futures.Future[typing.Any] | None = NoneAs per coding guidelines, "Add docstrings on all public functions and classes".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/fromager/bootstrapper/_phase.py` around lines 28 - 48, Add docstrings to the public methods __init_subclass__ and __init__ in the Bootstrapper base class for consistency with run, requires_exclusive_run, background_work, __str__, and as_json. Describe the subclass-phase enforcement behavior in __init_subclass__ (including the ABCMeta ordering workaround) and the initialization of work_item and bg_future in __init__, replacing the existing inline comment block with the docstring on __init_subclass__.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/fromager/bootstrapper/_phase.py`:
- Around line 28-48: Add docstrings to the public methods __init_subclass__ and
__init__ in the Bootstrapper base class for consistency with run,
requires_exclusive_run, background_work, __str__, and as_json. Describe the
subclass-phase enforcement behavior in __init_subclass__ (including the ABCMeta
ordering workaround) and the initialization of work_item and bg_future in
__init__, replacing the existing inline comment block with the docstring on
__init_subclass__.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 48feda80-1f47-4ded-9093-26426d03840d
📒 Files selected for processing (11)
src/fromager/bootstrapper/_bootstrapper.pysrc/fromager/bootstrapper/_build.pysrc/fromager/bootstrapper/_cache.pysrc/fromager/bootstrapper/_complete.pysrc/fromager/bootstrapper/_phase.pysrc/fromager/bootstrapper/_prepare_build.pysrc/fromager/bootstrapper/_prepare_source.pysrc/fromager/bootstrapper/_process_install_deps.pysrc/fromager/bootstrapper/_resolve.pysrc/fromager/bootstrapper/_start.pytests/test_bootstrapper.py
🚧 Files skipped from review as they are similar to previous changes (10)
- src/fromager/bootstrapper/_complete.py
- src/fromager/bootstrapper/_prepare_build.py
- src/fromager/bootstrapper/_process_install_deps.py
- src/fromager/bootstrapper/_build.py
- src/fromager/bootstrapper/_resolve.py
- src/fromager/bootstrapper/_start.py
- src/fromager/bootstrapper/_prepare_source.py
- src/fromager/bootstrapper/_cache.py
- src/fromager/bootstrapper/_bootstrapper.py
- tests/test_bootstrapper.py
This looks good! Thanks |
c136b16 to
147f8fb
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
src/fromager/bootstrapper/_bootstrapper.py (1)
51-52: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a class docstring for the public
BootstrapperAPI.As per coding guidelines,
**/*.py: “Add docstrings on all public functions and classes.”Suggested change
class Bootstrapper: + """Coordinate requirement resolution, dependency bootstrapping, and build phases.""" def __init__(🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/fromager/bootstrapper/_bootstrapper.py` around lines 51 - 52, The public Bootstrapper class is missing a docstring. Add a class-level docstring directly on Bootstrapper to describe its purpose and high-level role in the bootstrapping API, following the project’s public API documentation guideline for classes.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/fromager/bootstrapper/_bootstrapper.py`:
- Around line 1283-1285: The context-manager cleanup in the bootstrapper is
shutting down the background pool without waiting, so running tasks can outlive
`__exit__` and race with cleanup. Update the shutdown logic in the
bootstrapper’s background-pool cleanup path to wait for active work to finish
instead of returning immediately, and keep the pool reset logic in the same
`_bg_pool` handling block so cleanup only proceeds after background tasks have
completed.
- Around line 518-520: The install guard in the bootstrapper is too broad: using
build_dependencies.isdisjoint(build_system_dependencies) skips installing every
backend/sdist dependency whenever there is any overlap. Update the logic around
build_dependencies and build_env.install() so it filters out only the
dependencies already satisfied by build_system_dependencies and installs the
remaining ones, rather than skipping the whole set.
- Around line 760-761: Create the destination parent directory before calling
shutil.move in the bootstrapper’s git clone flow. In the code around
logger.info("moving cloned repo to %s", working_src_dir) and
shutil.move(clone_dir, str(working_src_dir)), ensure working_src_dir.parent (or
the relevant nested parent for the first-time versioned path) is created so
first-time git URL resolution does not fail with FileNotFoundError. Keep the fix
in the bootstrapper path handling near the clone/move logic.
In `@tests/test_bootstrapper.py`:
- Around line 1052-1063: The wheel-build test patches
fromager.server.update_wheel_mirror but never verifies the call, so the test
does not actually cover the mirror-update behavior. In the item._build_wheel
test, add an assertion on the mocked update_wheel_mirror alongside the existing
mock_build_sdist checks, and make sure the assertion matches the expected
arguments from _build_wheel(tmp_context) so the test fails if the wiring is
removed.
---
Nitpick comments:
In `@src/fromager/bootstrapper/_bootstrapper.py`:
- Around line 51-52: The public Bootstrapper class is missing a docstring. Add a
class-level docstring directly on Bootstrapper to describe its purpose and
high-level role in the bootstrapping API, following the project’s public API
documentation guideline for classes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: c79e8c11-99fd-4896-be7d-139abecfb41b
📒 Files selected for processing (16)
src/fromager/bootstrapper.pysrc/fromager/bootstrapper/__init__.pysrc/fromager/bootstrapper/_bootstrapper.pysrc/fromager/bootstrapper/_build.pysrc/fromager/bootstrapper/_cache.pysrc/fromager/bootstrapper/_complete.pysrc/fromager/bootstrapper/_phase.pysrc/fromager/bootstrapper/_prepare_build.pysrc/fromager/bootstrapper/_prepare_source.pysrc/fromager/bootstrapper/_process_install_deps.pysrc/fromager/bootstrapper/_resolve.pysrc/fromager/bootstrapper/_start.pysrc/fromager/bootstrapper/_types.pysrc/fromager/bootstrapper/_work_item.pytests/test_bootstrapper.pytests/test_bootstrapper_iterative.py
💤 Files with no reviewable changes (1)
- src/fromager/bootstrapper.py
🚧 Files skipped from review as they are similar to previous changes (13)
- src/fromager/bootstrapper/_work_item.py
- src/fromager/bootstrapper/_prepare_build.py
- src/fromager/bootstrapper/_complete.py
- src/fromager/bootstrapper/init.py
- src/fromager/bootstrapper/_phase.py
- src/fromager/bootstrapper/_types.py
- src/fromager/bootstrapper/_start.py
- src/fromager/bootstrapper/_prepare_source.py
- src/fromager/bootstrapper/_build.py
- src/fromager/bootstrapper/_resolve.py
- src/fromager/bootstrapper/_process_install_deps.py
- src/fromager/bootstrapper/_cache.py
- tests/test_bootstrapper_iterative.py
Move phase functions out of the bootstrapper into their own classes. Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
…rapper Move the seven `_phase_*` handler methods out of `Bootstrapper` and into a `PhaseItem` abstract base class hierarchy. Each concrete subclass (`ResolveItem`, `StartItem`, `PrepareSourceItem`, `PrepareBuildItem`, `BuildItem`, `ProcessInstallDepsItem`, `CompleteItem`) encodes one bootstrap phase and implements its own `run()` method, making each phase self-contained. Key changes: - Add `PhaseItem` ABC with `phase`/`tracks_why` ClassVars, `bg_future`, `background_work()`, `__str__()`, and `as_json()` - Add 7 concrete `PhaseItem` subclasses; move `_phase_*` bodies into their `run()` methods - Remove `phase` and `bg_future` fields from `WorkItem` - Remove `tracks_why` property from `BootstrapPhase` enum - Remove `_dispatch_phase` (was a one-liner: `item.run(self)`) and inline the call in `_run_bootstrap_loop` - Remove `_get_background_work`; replace with `PhaseItem.background_work()` - Remove all 7 `_phase_*` methods from `Bootstrapper` - Simplify `_push_items`, `_track_why`, `_record_stack_state`, `_handle_phase_error`, and related methods to use `PhaseItem` - Update tests to use `PhaseItem` subclass instances and `item.run(bt)` Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
Add a public `resolver` property to the Bootstrapper class to provide a public API for accessing the version resolver. This allows ResolveItem.background_work() to access the resolver through the public property instead of directly touching the private _resolver attribute, maintaining the boundary between PhaseItem and Bootstrapper. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
…rsion() Replace direct access to _failed_versions dict in ResolveItem.run() with a public has_failed_version() method. This maintains encapsulation of the private dict and provides a clean public API for checking if a package version has previously failed resolution. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
Remove leading underscores from Bootstrapper methods that are accessed from StartItem.run() to fix cross-boundary private-member access: - _add_to_graph → add_to_graph - _processing_build_requirement → processing_build_requirement - _has_been_seen → has_been_seen - _mark_as_seen → mark_as_seen - _explain → explain Also update internal callers within Bootstrapper: - _filter_deps_satisfied_by_build_system() calls self.add_to_graph() - _build_wheel() calls self.explain Update all references in tests/test_bootstrapper.py and tests/test_bootstrapper_iterative.py. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
…ved_work_items public Inline the thin wrapper _create_build_env() directly in PrepareSourceItem.run() since its req and resolved_version parameters were unused. This simplifies the code and removes an unnecessary layer of indirection. Rename _create_unresolved_work_items() to create_unresolved_work_items() to make it public, as it is called from three different PhaseItem subclasses (PrepareSourceItem, PrepareBuildItem, ProcessInstallDepsItem). Update all call sites in bootstrapper.py and test references in test_bootstrapper_iterative.py. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
Remove leading underscores from `_get_resolved_build_system_versions` and `_filter_deps_satisfied_by_build_system` to make them public methods. These methods are accessed from `PrepareBuildItem.run()` across module boundaries and should be part of the public API. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
Rename _drain_background_pool() to drain_background_pool() and _do_build() to do_build() to indicate these methods are accessed from BuildItem.run(). Internal methods _build_sdist() and _build_wheel() remain private as they are only called within Bootstrapper. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
…public Rename three private methods to public to support ProcessInstallDepsItem.run() accessing them without underscore prefix: - _record_test_mode_failure() -> record_test_mode_failure() - _get_install_dependencies() -> get_install_dependencies() - _add_to_build_order() -> add_to_build_order() All internal callers and test references have been updated accordingly. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
Each PhaseItem now declares whether it requires exclusive execution via a `requires_exclusive_run` property (default False). `_run_bootstrap_loop` drains the background pool before dispatching any item that returns True, rather than having `BuildItem.run()` reach into the Bootstrapper to do it. `BuildItem` overrides `requires_exclusive_run` to return `self.work_item.exclusive_build`, which is cached on `WorkItem` during `StartItem.run()` alongside `pbi_pre_built`. `drain_background_pool` is now private again since it's only called from within the Bootstrapper loop. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
…vel function The method only used `self.ctx` from the Bootstrapper; all other arguments were passed explicitly. It now follows the same pattern as `_bg_resolve`, `_bg_prepare_source`, and `_find_cached_wheel`: a module-level function that takes `ctx` directly. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
The method only needed `self.req_type` and `self.why` (which equals `wi.why_snapshot` at call time in the bootstrap loop). It is now `WorkItem.is_build_requirement_context()`, reading `self.req_type` and `self.why_snapshot` directly with no parameters. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
`do_build()`, `_build_sdist()`, and `_build_wheel()` on `Bootstrapper` are only called from `BuildItem.run()` and operate exclusively on `WorkItem` state. Moving them to `BuildItem` eliminates the need to pass `req`, `resolved_version`, and other `WorkItem` fields as parameters: the methods read them from `self.work_item` directly. `BuildItem.run()` now calls `self.do_build(bt.ctx, bt.explain)` instead of `bt.do_build(req=wi.req, ...)`. The only external state needed from `Bootstrapper` is: - `ctx` (WorkContext) — passed explicitly - `explain` (formatted why-stack string used in one log line) — passed explicitly Existing tests that patched `bt.do_build` or `bt._build_wheel` are updated to patch the corresponding `BuildItem` instance methods instead. New unit tests cover the three `BuildItem` methods directly. Co-Authored-By: Claude <claude@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
Split the 2,351-line `bootstrapper.py` into a subpackage `src/fromager/bootstrapper/` with one module per logical layer: - `_types.py`: type aliases, dataclasses, enums, and constants - `_cache.py`: shared wheel-cache and prebuilt-download helpers - `_work_item.py`: `WorkItem` dataclass - `_phase_item.py`: `PhaseItem` ABC - `_resolve_item.py`: `ResolveItem` + `_bg_resolve` - `_start_item.py`: `StartItem` - `_prepare_source_item.py`: `PrepareSourceItem` + `_bg_prepare_source` - `_prepare_build_item.py`: `PrepareBuildItem` - `_build_item.py`: `BuildItem` - `_process_install_deps_item.py`: `ProcessInstallDepsItem` + `_get_install_dependencies` - `_complete_item.py`: `CompleteItem` - `_bootstrapper.py`: `Bootstrapper` class - `__init__.py`: exports `Bootstrapper` and `_DEFAULT_BG_THREADS` only Phase modules that call `_cache` helpers use module-level imports (`from . import _cache`) so mock patch targets remain unambiguous (e.g. `fromager.bootstrapper._cache._find_cached_wheel`). Tests updated to import directly from submodules and use the new patch paths. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
…uffix Rename all phase subclasses of `PhaseItem` to drop the redundant "Item" suffix, and rename the abstract base `PhaseItem` to `Phase`. File names follow the class names. Class renames: - PhaseItem → Phase - ResolveItem → Resolve - StartItem → Start - PrepareSourceItem → PrepareSource - PrepareBuildItem → PrepareBuild - BuildItem → Build - ProcessInstallDepsItem → ProcessInstallDeps - CompleteItem → Complete File renames (in src/fromager/bootstrapper/): - _phase_item.py → _phase.py - _resolve_item.py → _resolve.py - _start_item.py → _start.py - _prepare_source_item.py → _prepare_source.py - _prepare_build_item.py → _prepare_build.py - _build_item.py → _build.py - _process_install_deps_item.py → _process_install_deps.py - _complete_item.py → _complete.py `WorkItem` and `_work_item.py` are unchanged. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
…og context - Add docstrings to `add_to_graph`, `has_been_seen`, and `add_to_build_order` in `Bootstrapper` - Move the "exclusive build, draining background pool" log and `_drain_background_pool()` call inside `req_ctxvar_context` so the message carries the per-requirement log prefix - Use the `_create_unpack_dir` helper in `_bg_prepare_prebuilt` instead of duplicating the inline path-and-mkdir logic - Add `Phase.__init_subclass__` to raise `TypeError` at class-definition time when a concrete subclass omits the required `phase` ClassVar; add two tests - Expand class docstrings on all seven `Phase` subclasses to describe each phase's role and its next-phase transitions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
The doc added to the proposals directory was the agent's plan, not really a proposal. Clean that up. Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
147f8fb to
9ece337
Compare
…n git clone flow On first-time git URL resolution, working_src_dir.parent did not exist yet, causing shutil.move to raise FileNotFoundError. The mkdir call was inside the `if working_src_dir.exists()` branch, so it was skipped when the destination was new. Move it unconditionally before shutil.move, and add an assert to satisfy the type checker. Adds a regression test that lets the real filesystem run (no shutil.move mock) to verify the parent directory is created before the move. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
…eel test The wheel-build test patched `fromager.server.update_wheel_mirror` but never asserted the call, so removing the wiring would go undetected. Capture the mock and assert it is called once with the expected context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
Summary
This branch refactors the monolithic
bootstrapper.py(2278 lines) into a well-structured subpackage with focused modules, making the code easier to navigate, test, and extend.What changed
Phaseand its subclasses) out ofBootstrapperinto individual modules, one per bootstrap phasebootstrapper.pyto a subpackage (bootstrapper/) with 10 focused modules:_phase.py,_resolve.py,_start.py,_prepare_source.py,_prepare_build.py,_build.py,_process_install_deps.py,_complete.py,_work_item.py,_cache.pydo_buildand build helpers →Build, install-dep extraction → module-level function in_process_install_deps,processing_build_requirementstate →WorkItemBootstrapper: inlined_create_build_env, moved exclusive-build pool drain into the loop, made internal members public where phase classes need them, exposed_resolveras a property, wrapped_failed_versionsbehindhas_failed_version()Itemsuffix (PhaseItem→Phase,BuildItem→Build,ResolveItem→Resolve, etc.) and renamed files to matchCommit sequence
design: phase classes— design doc / planrefactor: extract PhaseItem class hierarchy— initial extractionBootstrappermembers accessible to phase classesrefactor: convert bootstrapper.py to a subpackagerefactor: rename phase classes and files, drop "Item" suffixTest plan
hatch run test:test)hatch run mypy:check)hatch run lint:check)Advice to reviewers
There are no new features here, but there are a lot of code changes. It may be easier to review them by looking at the commit history individually instead of trying to understand the entire changeset as one big diff.
🤖 Generated with Claude Code