Skip to content

Add global hook point for get_build_system_dependencies #1263

Description

@vshawrh

Problem

setuptools 81 removed distutils.spawn(dry_run=...) and distutils.dir_util.remove_tree(dry_run=...). setuptools 82 removed pkg_resources entirely. Many packages on PyPI still use these in their setup.py, causing build failures when Fromager resolves an uncapped setuptools as a build dependency.

In the AIPCC wheels builder (fondue), 22 identical per-package plugins exist solely to add a setuptools version cap via get_build_system_dependencies. Each plugin is a copy-paste of the same logic. Additionally, ~127 packages with settings overrides have no protection at all.

Proposed Solution

Instead of putting setuptools-specific detection logic into Fromager core, extend fromager.hooks to support get_build_system_dependencies as a global hook point. This lets downstream projects register hooks that post-process build dependencies for all packages.

Changes in Fromager (~20 lines across two files)

hooks.py:

  • Add get_build_system_dependencies to GLOBAL_HOOK_NAMES
  • Add run_get_build_system_dependencies_hooks() following the same pattern as run_post_build_hooks()
  • The hook receives the resolved requirements list and returns a modified list

dependencies.py:

  • Inside get_build_system_dependencies(), after overrides.find_and_invoke() returns the dependency list (from a per-package plugin or the default), pass it through the global hook before _filter_requirements()

How downstream projects use it

Downstream projects register hooks via fromager.hooks entry points, same as post_build today:

[project.entry-points."fromager.hooks"]
get_build_system_dependencies = "my_plugins.hooks.setuptools_cap:get_build_system_dependencies"

The hook signature:

def get_build_system_dependencies(
    ctx: context.WorkContext,
    req: Requirement,
    sdist_root_dir: pathlib.Path,
    build_dir: pathlib.Path,
    requirements: list[str],
) -> list[str]:
    # post-process and return modified requirements
    ...

Why this approach

  • Fromager stays generic -- no setuptools version numbers or AST scanning in core
  • Opt-in -- users register hooks or don't. No forced behavior
  • Extensible -- downstream projects can add any build dependency fixups (setuptools caps, cmake constraints, custom build deps) without Fromager core changes
  • Follows existing patterns -- same mechanism as post_build / post_bootstrap / prebuilt_wheel hooks

Impact

In the fondue builder:

  • 22 plugins eliminated -- one global hook replaces all of them
  • 3 plugins simplified (llvmlite, torchvision, pyarrow) -- can remove setuptools constraint handling from their get_build_system_dependencies override
  • ~127 packages gain protection -- currently have no setuptools cap

Previous approach

PR #1264 (now closed) put the setuptools detection logic directly in default_get_build_system_dependencies. Per feedback from @tiran, this was too specific to put in core. The global hooks approach keeps Fromager generic while giving downstream projects the extension point they need.

Relationship to existing issues

Related to #928 (Reduce downstream plugin needs through YAML-driven configuration enhancements).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions