refactor(bootstrapper): simplify top-level requirement interface

[dhellmann]

Problem

The Bootstrapper class has an inconsistent public interface that forces callers to manage two separate loops: a pre-resolution loop calling resolve_and_add_top_level() and a bootstrap loop calling bootstrap(req, req_type). The caller also manages requirement_ctxvar context and tracks which requirements resolved successfully.

Proposed Change

Merge the two loops into a single public method bootstrap(requirements: list[Requirement]) -> None that handles resolution, graph population, and iterative DFS internally. The caller becomes:

bt.bootstrap(to_build)
exit_code = bt.finalize()

Implementation Notes

• Rename resolve_and_add_top_level() to _resolve_and_add_top_level() (private)
• Rename bootstrap(req, req_type) to _bootstrap_one(req, req_type) (private, for internal git-URL path only)
• Add bootstrap(requirements: list[Requirement]) -> None as the new public entry point
• Update commands/bootstrap.py to use the new interface
• Update affected tests

[dhellmann]

Bootstrapper Top-Level Requirements Interface Simplification

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Simplify the Bootstrapper interface so callers pass a list of top-level requirements to bootstrap() instead of managing two separate loops (resolve loop + bootstrap loop).

Architecture: Merge resolve_and_add_top_level() logic into bootstrap(requirements), rename the old single-req bootstrap(req, req_type) to _bootstrap_one() for internal use only, and update the CLI to call bt.bootstrap(to_build) as a single step.

Tech Stack: Python 3.10+, packaging, fromager internal modules.

---

Context

The current caller (CLI) must manage two separate loops:

# Current: caller manages two loops + context vars
resolved_reqs = []
for req in to_build:
    token = requirement_ctxvar.set(req)
    result = bt.resolve_and_add_top_level(req)
    if result is not None:
        resolved_reqs.append(req)
    requirement_ctxvar.reset(token)

for req in resolved_reqs:
    token = requirement_ctxvar.set(req)
    bt.bootstrap(req, requirements_file.RequirementType.TOP_LEVEL)
    progressbar.update()
    requirement_ctxvar.reset(token)

The desired caller is simply:

bt.bootstrap(to_build)
exit_code = bt.finalize()

---

File Map

File	Change
src/fromager/bootstrapper.py	Rename resolve_and_add_top_level → _resolve_and_add_top_level; rename bootstrap(req, req_type) → _bootstrap_one(req, req_type); add bootstrap(requirements); update _handle_build_requirements
src/fromager/commands/bootstrap.py	Replace two-loop pattern with bt.bootstrap(to_build)
tests/test_bootstrapper.py	Update calls to removed/renamed public methods
tests/test_bootstrapper_iterative.py	Update calls to removed/renamed public methods
tests/test_bootstrap_test_mode.py	Update calls to removed/renamed public methods

---

Task 1: File GitHub Issue

Files: none (shell commands only)

• Step 1: Create the GitHub issue

gh issue create \
  --title "refactor(bootstrapper): simplify top-level requirement interface" \
  --body "## Problem

The \`Bootstrapper\` class has an inconsistent public interface that forces callers to manage two separate loops: a pre-resolution loop calling \`resolve_and_add_top_level()\` and a bootstrap loop calling \`bootstrap(req, req_type)\`. The caller also manages \`requirement_ctxvar\` context and tracks which requirements resolved successfully.

## Proposed Change

Merge the two loops into a single public method \`bootstrap(requirements: list[Requirement]) -> None\` that handles resolution, graph population, and iterative DFS internally. The caller becomes:

\`\`\`python
bt.bootstrap(to_build)
exit_code = bt.finalize()
\`\`\`

## Implementation Notes

- Rename \`resolve_and_add_top_level()\` to \`_resolve_and_add_top_level()\` (private)
- Rename \`bootstrap(req, req_type)\` to \`_bootstrap_one(req, req_type)\` (private, for internal git-URL path only)
- Add \`bootstrap(requirements: list[Requirement]) -> None\` as the new public entry point
- Update \`commands/bootstrap.py\` to use the new interface
- Update affected tests
"

Record the issue number from the output (e.g., #42).

• Step 2: Assign the issue to dhellman

Replace <N> with the issue number from Step 1:

gh issue edit <N> --assignee dhellman

• Step 3: Post the implementation plan as a comment

Replace <N> with the issue number:

gh issue comment <N> --body-file /Users/dhellman/.claude/plans/delegated-percolating-bird.md

---

Task 2: Add new bootstrap(requirements) and make private renames

Files:

• Modify: src/fromager/bootstrapper.py

• Test: tests/test_bootstrapper.py

• Step 1: Write failing tests for the new interface

Add to tests/test_bootstrapper.py:

def test_bootstrap_with_empty_list(tmp_context: WorkContext) -> None:
    """bootstrap([]) completes without error and runs no phases."""
    bt = bootstrapper.Bootstrapper(tmp_context)
    with patch.object(bt, "_dispatch_phase") as mock_dispatch:
        bt.bootstrap([])
    mock_dispatch.assert_not_called()


def test_bootstrap_with_single_requirement(tmp_context: WorkContext) -> None:
    """bootstrap([req]) resolves and processes the requirement."""
    bt = bootstrapper.Bootstrapper(tmp_context)
    req = Requirement("testpkg==1.0")

    with (
        patch.object(
            bt,
            "_resolve_and_add_top_level",
            return_value=("http://example.test/testpkg-1.0.tar.gz", Version("1.0")),
        ),
        patch.object(bt, "_dispatch_phase", return_value=[]) as mock_dispatch,
        patch.object(bt, "_record_stack_state"),
    ):
        bt.bootstrap([req])

    mock_dispatch.assert_called_once()
    item = mock_dispatch.call_args[0][0]
    assert item.req == req
    assert item.req_type == RequirementType.TOP_LEVEL
    assert item.phase == bootstrapper.BootstrapPhase.RESOLVE


def test_bootstrap_skips_failed_resolution(tmp_context: WorkContext) -> None:
    """bootstrap() skips requirements whose resolution returns None."""
    bt = bootstrapper.Bootstrapper(tmp_context)
    req = Requirement("badpkg")

    with (
        patch.object(bt, "_resolve_and_add_top_level", return_value=None),
        patch.object(bt, "_dispatch_phase") as mock_dispatch,
        patch.object(bt, "_record_stack_state"),
    ):
        bt.bootstrap([req])

    mock_dispatch.assert_not_called()


def test_bootstrap_two_requirements_both_processed(tmp_context: WorkContext) -> None:
    """bootstrap() processes all requirements in the list."""
    bt = bootstrapper.Bootstrapper(tmp_context)
    req1 = Requirement("pkg1==1.0")
    req2 = Requirement("pkg2==2.0")

    dispatch_calls: list = []

    def fake_dispatch(item: bootstrapper.WorkItem) -> list:
        dispatch_calls.append(item.req.name)
        return []

    with (
        patch.object(
            bt,
            "_resolve_and_add_top_level",
            return_value=("http://example.test/pkg-1.0.tar.gz", Version("1.0")),
        ),
        patch.object(bt, "_dispatch_phase", side_effect=fake_dispatch),
        patch.object(bt, "_record_stack_state"),
    ):
        bt.bootstrap([req1, req2])

    assert set(dispatch_calls) == {"pkg1", "pkg2"}

• Step 2: Run the tests to verify they fail

hatch run test:test tests/test_bootstrapper.py::test_bootstrap_with_empty_list tests/test_bootstrapper.py::test_bootstrap_with_single_requirement tests/test_bootstrapper.py::test_bootstrap_skips_failed_resolution tests/test_bootstrapper.py::test_bootstrap_two_requirements_both_processed -v

Expected: FAILED or ERROR (methods don't exist yet).

• Step 3: In bootstrapper.py, rename resolve_and_add_top_level → _resolve_and_add_top_level

In src/fromager/bootstrapper.py:

Change line 203:

    def resolve_and_add_top_level(

to:

    def _resolve_and_add_top_level(

• Step 4: Rename bootstrap(req, req_type) → _bootstrap_one(req, req_type)

In src/fromager/bootstrapper.py:

Change line 354:

    def bootstrap(self, req: Requirement, req_type: RequirementType) -> None:
        """Bootstrap a package and its dependencies using an iterative loop.

        Uses an explicit LIFO stack instead of recursion to handle arbitrarily
        deep dependency graphs without hitting Python's recursion limit.

        In test mode, catches build exceptions, records package name, and continues.
        In normal mode, raises exceptions immediately (fail-fast).

        When multiple_versions is enabled, bootstraps all matching versions instead
        of just the highest version.
        """

to:

    def _bootstrap_one(self, req: Requirement, req_type: RequirementType) -> None:
        """Bootstrap a single requirement using an iterative DFS loop.

        Internal method used only by the git URL resolution path
        (``_handle_build_requirements``). All other callers should use
        ``bootstrap(requirements)`` instead.

        Uses an explicit LIFO stack instead of recursion to handle arbitrarily
        deep dependency graphs without hitting Python's recursion limit.

        In test mode, catches build exceptions, records package name, and continues.
        In normal mode, raises exceptions immediately (fail-fast).

        When multiple_versions is enabled, bootstraps all matching versions instead
        of just the highest version.
        """

Also remove the per-req failed-version reporting block at the end of _bootstrap_one (lines 418–430), since finalize() already handles summary reporting:

        # In multiple versions mode, report any failures for this requirement
        if self.multiple_versions and self._failed_versions:
            failed_for_req = [
                (name, ver, exc)
                for name, ver, exc in self._failed_versions
                if name == canonicalize_name(req.name)
            ]
            if failed_for_req:
                logger.warning(
                    f"{req.name}: {len(failed_for_req)} version(s) failed to bootstrap"
                )
                for name, ver, exc in failed_for_req:
                    logger.warning(f"  - {name}=={ver}: {type(exc).__name__}: {exc}")

Delete those lines entirely.

• Step 5: Update _handle_build_requirements to call _bootstrap_one

In src/fromager/bootstrapper.py, line 629, change:

                self.bootstrap(req=dep, req_type=build_type)

to:

                self._bootstrap_one(req=dep, req_type=build_type)

• Step 6: Add the new public bootstrap(requirements) method

Add the following method to src/fromager/bootstrapper.py after resolve_versions() and before _processing_build_requirement() (around line 327):

    def bootstrap(self, requirements: list[Requirement]) -> None:
        """Bootstrap all top-level requirements and their transitive dependencies.

        Resolves each requirement, adds it to the dependency graph, and processes
        the full dependency tree using an iterative DFS loop. Handles
        ``requirement_ctxvar`` context internally; callers do not need to manage it.

        In test mode, records failures and continues instead of raising. In
        ``multiple_versions`` mode, processes all matching versions per requirement.

        Args:
            requirements: Top-level requirements to resolve and bootstrap.
        """
        # Resolve all top-level reqs and build initial stack
        stack: list[WorkItem] = []
        for req in requirements:
            with req_ctxvar_context(req):
                result = self._resolve_and_add_top_level(req)
            if result is not None:
                stack.append(
                    WorkItem(
                        req=req,
                        req_type=RequirementType.TOP_LEVEL,
                        phase=BootstrapPhase.RESOLVE,
                        why_snapshot=[],
                        parent=None,
                    )
                )

        # Main iterative DFS loop
        while stack:
            self._record_stack_state(stack)
            item = stack.pop()
            self.why = list(item.why_snapshot)

            with req_ctxvar_context(item.req), self._track_why(item):
                try:
                    new_items = self._dispatch_phase(item)
                except Exception as err:
                    new_items = self._handle_phase_error(item, err)

            new_dep_count = sum(
                1 for it in new_items if it.phase == BootstrapPhase.RESOLVE
            )
            if new_dep_count > 0:
                self.progressbar.update_total(new_dep_count)
            if not new_items:
                self.progressbar.update()

            stack.extend(new_items)

• Step 7: Run the new tests to verify they pass

hatch run test:test tests/test_bootstrapper.py::test_bootstrap_with_empty_list tests/test_bootstrapper.py::test_bootstrap_with_single_requirement tests/test_bootstrapper.py::test_bootstrap_skips_failed_resolution tests/test_bootstrapper.py::test_bootstrap_two_requirements_both_processed -v

Expected: all PASSED.

• Step 8: Lint and type-check the modified file

hatch run lint:fix src/fromager/bootstrapper.py
hatch run mypy:check src/fromager/bootstrapper.py

Expected: no errors.

• Step 9: Commit

Write commit message to a file, then commit:

refactor(bootstrapper): add bootstrap(requirements) and rename internals

Add public `bootstrap(requirements: list[Requirement])` that combines
the resolution loop (previously `resolve_and_add_top_level()`) and the
DFS processing loop into a single method. Callers no longer need to
manage `requirement_ctxvar` context or track which requirements resolved
successfully.

Rename the old `bootstrap(req, req_type)` to `_bootstrap_one()` (private)
since it is only used by the git URL resolution path in
`_handle_build_requirements()`. Rename `resolve_and_add_top_level()` to
`_resolve_and_add_top_level()` (private) as it is now only called inside
`bootstrap()`.

Remove the per-requirement failed-version reporting block from
`_bootstrap_one()` since `finalize()` already produces a complete summary.

Co-Authored-By: Claude <claude@anthropic.com>

git add src/fromager/bootstrapper.py tests/test_bootstrapper.py
git commit --signoff -F /tmp/commit_msg.txt

---

Task 3: Update the CLI to use the new interface

Files:

• Modify: src/fromager/commands/bootstrap.py

• Step 1: Write a failing test for the simplified CLI flow

In tests/test_bootstrapper.py, add:

def test_bootstrap_records_stack_state_new_interface(tmp_context: WorkContext) -> None:
    """bootstrap(requirements) calls _record_stack_state for each DFS step."""
    bt = bootstrapper.Bootstrapper(tmp_context)
    req = Requirement("testpkg")

    with (
        patch.object(
            bt,
            "_resolve_and_add_top_level",
            return_value=("http://example.test/testpkg-1.0.tar.gz", Version("1.0")),
        ),
        patch.object(bt, "_record_stack_state") as mock_record,
        patch.object(bt._resolver, "resolve", side_effect=RuntimeError("forced")),
    ):
        bt.bootstrap([req])

    mock_record.assert_called()

Run to verify it fails or errors:

hatch run test:test tests/test_bootstrapper.py::test_bootstrap_records_stack_state_new_interface -v

• Step 2: Update commands/bootstrap.py

In src/fromager/commands/bootstrap.py, replace the entire pre-resolution + bootstrap section (roughly lines 203–229):

        logger.info("resolving top-level dependencies before building")
        resolved_reqs: list[Requirement] = []
        for req in to_build:
            token = requirement_ctxvar.set(req)
            result = bt.resolve_and_add_top_level(req)
            if result is not None:
                resolved_reqs.append(req)
            # If result is None, test_mode or multiple_versions recorded the failure
            requirement_ctxvar.reset(token)

        # Bootstrap only packages that were successfully resolved
        # Note: Same pattern - no try/finally to preserve context for error logging
        for req in resolved_reqs:
            token = requirement_ctxvar.set(req)
            bt.bootstrap(req, requirements_file.RequirementType.TOP_LEVEL)
            progressbar.update()
            requirement_ctxvar.reset(token)

with:

        bt.bootstrap(to_build)

Also remove the now-unused requirement_ctxvar import from the top of commands/bootstrap.py:

from ..log import requirement_ctxvar

(Delete that line if requirement_ctxvar is no longer referenced in the file.)

• Step 3: Run the new test to verify it passes

hatch run test:test tests/test_bootstrapper.py::test_bootstrap_records_stack_state_new_interface -v

Expected: PASSED.

• Step 4: Lint and type-check

hatch run lint:fix src/fromager/commands/bootstrap.py
hatch run mypy:check src/fromager/commands/bootstrap.py

Expected: no errors.

• Step 5: Commit

refactor(commands/bootstrap): use new Bootstrapper.bootstrap(requirements)

Replace the two-loop pattern (resolve loop + bootstrap loop) with a
single call to `bt.bootstrap(to_build)`. Context variable management
and resolution error handling are now internal to `Bootstrapper`.

Remove the no-longer-needed `requirement_ctxvar` import.

Co-Authored-By: Claude <claude@anthropic.com>

git add src/fromager/commands/bootstrap.py tests/test_bootstrapper.py
git commit --signoff -F /tmp/commit_msg.txt

---

Task 4: Fix broken existing tests

Files:

• Modify: tests/test_bootstrapper.py

• Modify: tests/test_bootstrapper_iterative.py

• Modify: tests/test_bootstrap_test_mode.py

• Step 1: Run the full test file to discover failures

hatch run test:test tests/test_bootstrapper.py tests/test_bootstrapper_iterative.py tests/test_bootstrap_test_mode.py -v 2>&1 | grep -E "FAILED|ERROR"

• Step 2: Update tests calling the old bootstrap(req, req_type) public signature

Any test calling bt.bootstrap(req=..., req_type=...) must change to one of:

• If testing top-level flow: wrap in the new public API:

  with patch.object(bt, "_resolve_and_add_top_level", return_value=("http://example.test/pkg-1.0.tar.gz", Version("1.0"))):
      bt.bootstrap([req])

• If testing internal iterative phases directly: call the private method:

  bt._bootstrap_one(req=req, req_type=RequirementType.INSTALL)

For tests in test_bootstrapper_iterative.py that test phase handlers in isolation (e.g., _phase_resolve, _phase_start, etc.), these call the private methods directly already and should need no change. Only calls to the old public bootstrap() need updating.

• Step 3: Update tests calling resolve_and_add_top_level

Any test calling bt.resolve_and_add_top_level(req) must change to bt._resolve_and_add_top_level(req).

• Step 4: Run the affected test files to verify all pass

hatch run test:test tests/test_bootstrapper.py tests/test_bootstrapper_iterative.py tests/test_bootstrap_test_mode.py -v

Expected: all PASSED.

• Step 5: Commit

test(bootstrapper): update tests for new bootstrap(requirements) interface

Update calls to the old public `bootstrap(req, req_type)` and
`resolve_and_add_top_level()` to use the renamed private methods
`_bootstrap_one()` and `_resolve_and_add_top_level()`, or to use the
new public `bootstrap(requirements)` entry point as appropriate.

Co-Authored-By: Claude <claude@anthropic.com>

git add tests/
git commit --signoff -F /tmp/commit_msg.txt

---

Task 5: Final quality checks and PR

• Step 1: Run the full project-wide quality checks

hatch run lint:fix && hatch run test:test && hatch run mypy:check && hatch run lint:check

Expected: all pass.

• Step 2: Create a pull request

gh pr create \
  --title "refactor(bootstrapper): simplify top-level requirement interface" \
  --body "Closes #<N>

## Summary

Replace the two-step caller pattern:
\`\`\`python
# Old: caller manages two loops + context vars
resolved_reqs = []
for req in to_build:
    token = requirement_ctxvar.set(req)
    result = bt.resolve_and_add_top_level(req)
    if result is not None:
        resolved_reqs.append(req)
    requirement_ctxvar.reset(token)

for req in resolved_reqs:
    token = requirement_ctxvar.set(req)
    bt.bootstrap(req, RequirementType.TOP_LEVEL)
    progressbar.update()
    requirement_ctxvar.reset(token)
\`\`\`

with a single method call:
\`\`\`python
bt.bootstrap(to_build)
\`\`\`

## Changes

- \`bootstrapper.py\`: Add \`bootstrap(requirements)\`, rename \`resolve_and_add_top_level\` → \`_resolve_and_add_top_level\` (private), rename \`bootstrap(req, req_type)\` → \`_bootstrap_one\` (private)
- \`commands/bootstrap.py\`: Replace two-loop pattern with single \`bt.bootstrap(to_build)\` call
- Tests: update to use new interface
"

---

Verification

End-to-end: after all tasks complete, a quick smoke test is:

# Verify the CLI still parses and shows help (no import errors)
fromager bootstrap --help

# If you have a test package available:
# fromager --work-dir /tmp/test-bootstrap bootstrap "requests"

Regression check:

hatch run test:test tests/test_bootstrapper.py tests/test_bootstrapper_iterative.py tests/test_bootstrap_test_mode.py tests/test_commands_bootstrap.py -v