Skip to content

feat(marketplace): inherit description/version from local-path apm.yml#1581

Closed
imbabamba wants to merge 3 commits into
microsoft:mainfrom
imbabamba:feat/local-metadata-inherit-description
Closed

feat(marketplace): inherit description/version from local-path apm.yml#1581
imbabamba wants to merge 3 commits into
microsoft:mainfrom
imbabamba:feat/local-metadata-inherit-description

Conversation

@imbabamba

@imbabamba imbabamba commented Jun 2, 2026

Copy link
Copy Markdown

TL;DR

apm pack now auto-fills description and version in
marketplace.json from each local-path package's own apm.yml --
the same fallback the remote source path has used since #1061.
A curator-side value under marketplace.packages still wins when
set, and the local read is path-traversal-guarded against the
project root. Validated against the full unit suite (15837 tests)
with 13 new tests covering the local fallback path.

Problem (WHY)

The remote branch in ClaudeMarketplaceMapper.compose already reads
description and version from each resolved package's own
apm.yml when the curator entry omits them; _fetch_remote_metadata
populates that lookup over HTTPS.

The local branch had no such fallback. It emitted only what the
curator wrote in root apm.yml's marketplace.packages[]. A stale
comment in _prefetch_metadata even claimed "Local-path packages are
skipped (they carry their own metadata)" -- but the local code path
never read any metadata at all.

For a monorepo marketplace using source: ./plugins/<name> entries,
that gap leaves producers with two options:

  • Duplicate the description between root apm.yml's
    marketplace.packages[] and each plugins/<name>/apm.yml. The
    two copies drift over time -- the producer edits one and forgets
    the other.
  • Run custom sync tooling outside apm pack to keep them aligned.

Remote sources do not have this gap. This PR closes it for local
sources by mirroring the existing remote behavior exactly.

Approach (WHAT)

Layer Before After
builder._prefetch_metadata Filtered out local packages. Returned {} under --offline. Reads local packages from disk first (always), then remote concurrently (still skipped under --offline). Returns one dict the mapper consumes the same way regardless of source kind.
builder._fetch_local_metadata Did not exist. New method. Reads <project_root>/<subdir>/apm.yml, extracts description and version. Path-traversal-guarded via ensure_path_within. Skips a source that resolves to the project root itself.
output_mappers.ClaudeMarketplaceMapper.compose (is_local branch) Emitted curator-side fields only. No fallback. Same hierarchy as the remote branch: curator wins; otherwise fall back to package-manifest meta. Verbose diagnostic on divergence.
docs/reference/manifest-schema.md description row read "Pass-through to marketplace.json" with no mention of the fallback. One-paragraph note after the marketplace.packages table covers the fallback for both source kinds.

Remote-source override semantics are unchanged, including the
verbose diagnostic that logs divergence between curator and package
manifest. Local sources log the equivalent diagnostic with
(package: ...) instead of (remote: ...).

Implementation (HOW)

File What changed
src/apm_cli/marketplace/builder.py New _fetch_local_metadata(pkg), mirroring the _fetch_remote_metadata shape -- reads pkg.subdir from disk, validates path containment, returns dict or None. _prefetch_metadata rewritten to process local packages serially first (no thread pool needed), then remote concurrently. Stale "Local-path packages are skipped" comment removed.
src/apm_cli/marketplace/output_mappers.py is_local branch in ClaudeMarketplaceMapper.compose reads meta = remote_metadata.get(pkg.name, {}) and applies the same curator-wins-else-meta hierarchy as the remote branch. Verbose diagnostic on divergence.
tests/unit/marketplace/test_builder.py New TestFetchLocalMetadata class -- 8 tests: happy path (description + version), description-only, missing apm.yml, missing subdir, path-escape, project-root edge, malformed YAML, empty subdir field.
tests/unit/marketplace/test_local_path_compose.py 5 new compose-level tests: description inherited from package manifest, version inherited, curator override wins, missing per-package apm.yml omits both fields, project-root edge does not read the marketplace's own apm.yml.
docs/src/content/docs/reference/manifest-schema.md One-paragraph note after the marketplace.packages table covering the fallback rule.

Edge cases handled

  • Missing per-package apm.yml: returns None; no key emitted.
  • Malformed YAML: returns None; error logged at debug level;
    the build continues.
  • Source escapes project root: ensure_path_within raises;
    _fetch_local_metadata catches and returns None.
  • Source resolves to the project root itself: explicit check;
    returns None so the marketplace's own apm.yml is never read
    as a package manifest.
  • Empty subdir on the ResolvedPackage: defensive early
    return.
  • --offline: local reads always run (filesystem, no network);
    remote reads are still skipped.

Validation evidence

$ uv run --extra dev ruff check src/ tests/            # All checks passed!
$ uv run --extra dev ruff format --check src/ tests/   # 1158 files already formatted

Tests: 15837 passed, 1 skipped, 21 xfailed (full tests/unit +
tests/test_console.py, ~1:25). 1418 passed across
tests/unit/marketplace, including the 13 new tests added in this
PR.

Follow-up to #1061

#1061 added the remote-source fallback (_fetch_remote_metadata)
to close the "remote-source pass-through metadata dropped" gap.
The same gap existed for local sources. This PR closes it.

Local-path packages (`source: ./...`) now use the same fallback as
remote sources when the curator entry under `marketplace.packages`
omits `description` or `version`: `apm pack` reads the field from
the package's own `apm.yml` and writes it to `marketplace.json`. A
curator-side value still wins when set. Path resolution is constrained
to the project root, and a source that resolves to the marketplace's
own `apm.yml` is skipped.

Follow-up to microsoft#1061, which added the same behavior for remote sources.
@imbabamba imbabamba force-pushed the feat/local-metadata-inherit-description branch from 246ac18 to 023c1be Compare June 2, 2026 10:24
danielmeppiel added a commit to imbabamba/apm that referenced this pull request Jun 11, 2026
Tighten the local metadata inheritance docs and diagnostics, add the missing curator-side version precedence regression test, and record the bug fix in CHANGELOG. Addresses apm-review-panel follow-ups for PR microsoft#1581.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danielmeppiel danielmeppiel marked this pull request as ready for review June 11, 2026 16:44
Copilot AI review requested due to automatic review settings June 11, 2026 16:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aligns local-path marketplace packages with the existing remote-source behavior by allowing apm pack to auto-fill description and version in marketplace.json from each referenced package’s own apm.yml when the curator entry omits them.

Changes:

  • Add best-effort local metadata loading (description/version) from <project_root>/<local source>/apm.yml during metadata prefetch.
  • Apply the same “curator wins, otherwise manifest fallback” mapping logic for local packages in ClaudeMarketplaceMapper.compose, including verbose diagnostics for divergence.
  • Add unit tests + compose-level tests for the local fallback behavior, and document the fallback rule in the manifest schema reference.
Show a summary per file
File Description
src/apm_cli/marketplace/builder.py Adds _fetch_local_metadata() and updates _prefetch_metadata() to enrich local packages from on-disk apm.yml before optional remote fetches.
src/apm_cli/marketplace/output_mappers.py Extends the local compose branch to inherit description/version from prefetched metadata when curator values are absent; adds _diagnostic_preview().
tests/unit/marketplace/test_builder.py Adds focused unit tests for _fetch_local_metadata() covering happy path + edge cases.
tests/unit/marketplace/test_local_path_compose.py Adds compose-level tests verifying local inheritance and curator-override precedence.
docs/src/content/docs/reference/manifest-schema.md Documents the new fallback behavior for description/version when omitted in marketplace.packages[].
CHANGELOG.md Adds an Unreleased “Fixed” entry describing the new local fallback behavior.

Copilot's findings

  • Files reviewed: 6/6 changed files
  • Comments generated: 1

Comment on lines +759 to +765
package_root = ensure_path_within(self._project_root / pkg.subdir, self._project_root)
if package_root == self._project_root.resolve():
return None
file_path = package_root / "apm.yml"
if not file_path.is_file():
return None
data = yaml.safe_load(file_path.read_text(encoding="utf-8"))
Tighten the local metadata inheritance docs and diagnostics, add the missing curator-side version precedence regression test, and record the bug fix in CHANGELOG. Addresses apm-review-panel follow-ups for PR microsoft#1581.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danielmeppiel danielmeppiel force-pushed the feat/local-metadata-inherit-description branch from 54b298d to 734f4ed Compare June 11, 2026 16:59
@danielmeppiel

Copy link
Copy Markdown
Collaborator

APM Review Panel: ship_now

Marketplace virtual-subdirectory packages now inherit description/version from their own apm.yml, eliminating silent monorepo metadata gaps.

cc @imbabamba -- a fresh advisory pass is ready for your review.

All panel signals converge cleanly from a code-quality perspective: zero correctness, security, auth, CLI UX, or coverage concerns remain after the shepherd fold commit. The PR preserves the established curator-entry-wins precedence, reads local metadata under the project root, keeps remote auth/token behavior unchanged, and now carries docs, changelog, and regression coverage for the missing version-precedence case.

Aligned with: portable_by_manifest: package-level apm.yml metadata now flows into marketplace output without root-level duplication; pragmatic_as_npm: workspace-style metadata inheritance reduces surprise for monorepo producers; oss_community_driven: fixes community-reported #1725 with all in-scope review feedback folded.

Growth signal. Monorepo packages now show rich marketplace browse rows (description, version) without duplicating metadata in the root manifest.

Panel summary

Persona B R N Takeaway
Python Architect 0 0 0 Shared precedence logic was extracted into a helper; no architecture concerns remain.
CLI Logging Expert 0 0 0 Verbose diagnostics now use source-neutral summary wording and explicit truncation markers.
DevX UX Expert 0 0 0 No CLI surface change; local metadata inheritance is automatic on the happy path.
Supply Chain Security Expert 0 0 0 Path containment, safe YAML loading, fail-closed handling, and project-root skip are sound.
OSS Growth Hacker 0 0 0 Changelog now frames this as a monorepo marketplace quality-of-life fix.
Doc Writer 0 0 0 Manifest docs now explain package apm.yml fallback and packages[] entry precedence.
Test Coverage Expert 0 0 0 Regression coverage includes curator-side version precedence and mutation-break evidence.

B = blocking-severity findings, R = recommended, N = nits.
Counts are signal strength, not gates. The maintainer ships.

Architecture

classDiagram
    class MarketplaceBuilder {
      +_fetch_local_metadata(pkg) dict or None
      +_fetch_remote_metadata(pkg) dict or None
      +_prefetch_metadata(resolved) dict
    }
    class ClaudeMarketplaceMapper {
      +compose(config, resolved, remote_metadata) MapperResult
    }
    class ResolvedPackage {
      +name str
      +source_repo str
      +subdir str
    }
    MarketplaceBuilder ..> ResolvedPackage : reads
    MarketplaceBuilder ..> ClaudeMarketplaceMapper : passes metadata
    ClaudeMarketplaceMapper ..> ResolvedPackage : emits plugin
Loading
flowchart TD
    A[apm pack] --> B[_prefetch_metadata]
    B --> C{local package?}
    C -- yes --> D[read package apm.yml under project root]
    C -- no --> E[fetch remote apm.yml unless offline]
    D --> F[metadata map]
    E --> F
    F --> G[ClaudeMarketplaceMapper]
    G --> H{packages entry field set?}
    H -- yes --> I[use entry value]
    H -- no --> J[inherit package apm.yml field]
Loading

Recommendation

Code-side recommendation is ship. One required GitHub check (license/cla) is still QUEUED after repeated polling, so this PR is not yet reporting ready-to-merge from the shepherd loop. Once that external check completes successfully, no code follow-up remains from this advisory pass.

Folded in this run

  • (panel) Added CHANGELOG.md entry for the monorepo marketplace fix -- resolved in 734f4ed.
  • (panel) Tightened manifest-schema docs to avoid new curator jargon and mention package apm.yml fallback in the field rows -- resolved in 734f4ed.
  • (panel) Made verbose override diagnostics source-neutral and explicit about truncated previews -- resolved in 734f4ed.
  • (panel) Added curator-side version precedence regression coverage and proved it with a mutation-break gate -- resolved in 734f4ed.
  • (panel) Extracted the shared curator-wins precedence/diagnostic logic into _apply_field_with_precedence -- resolved in 734f4ed.

Copilot signals reviewed

No copilot-pull-request-reviewer[bot] inline comments were present in either fetch round.

Regression-trap evidence (mutation-break gate)

  • tests/unit/marketplace/test_local_path_compose.py::test_compose_local_curator_version_wins_over_package -- deleted _apply_field_with_precedence entry-value guard; test FAILED as expected; guard restored.

Lint contract

uv run --extra dev ruff check src/ tests/ and
uv run --extra dev ruff format --check src/ tests/ both silent.

CI

license/cla is still QUEUED at https://github.com/apps/microsoft-github-policy-service after 0 CI fix iteration(s). Local validation passed: ruff pair silent; uv run --extra dev pytest tests/unit/marketplace/test_local_path_compose.py tests/unit/marketplace/test_builder.py -q => 150 passed.

Mergeability status

Captured from gh pr view 1581 --json mergeable,mergeStateStatus,statusCheckRollup after the last push of this run.

PR head SHA CEO stance iters folds defers Copilot rounds CI mergeable mergeStateStatus notes
#1581 734f4ed ship_now 1 5 0 2 yellow MERGEABLE BLOCKED license/cla queued

Convergence

1 outer iteration(s); 2 Copilot round(s). Final panel verdict: ship_now.

Not ready-to-merge until the queued license/cla check completes successfully.


Full per-persona findings

All in-scope findings from the initial panel pass were folded in commit 734f4ed. The only remaining external item is the queued license/cla check.

This panel is advisory. It does not block merge. Re-apply the panel-review label after addressing feedback to re-run.

@danielmeppiel

Copy link
Copy Markdown
Collaborator

Thanks for this fix, @imbabamba -- the approach was sound and went through full review. Unfortunately we can't merge it because the CLA hasn't been signed, so it's blocked indefinitely on our side.

To get the fix to users we've opened an equivalent maintainer-authored PR (#1755) that mirrors your approach (local-path apm.yml description/version fallback, curator-wins precedence, path-traversal guard). Your original work here is credited in that PR's description. Really appreciate you surfacing and solving this -- if you're able to sign the CLA in the future we'd be glad to take direct contributions.

Closing in favor of #1755.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants