Chore/backlog sweep 2026 07

[macanderson]

Introduce a full prompt-composition and budgeting layer on top of the host fan-out, add MCP interop and in-repo reference providers, harden stdio and HTTP transports, and broaden conformance, CI, and documentation around provider and host behavior.

New Features:

Add a reference prompt-composition module that splits global budgets across providers, deduplicates cross-provider evidence, orders frames by value, and produces a fenced prompt plus citation map and audit record
Introduce MCP interop: an MCP→CGP bridge that wraps MCP resource servers as CGP providers, and a CGP→MCP server that exposes a host fan-out as an MCP query_context tool
Enhancements:

Refactor stdio transport to support correlated, pipelined queries via a dedicated reader task while keeping non-correlating and verify exchanges lock-step
Extend the HTTP transport with bearer credential support and transport security checks that reject insecure non-loopback HTTP endpoints before any bytes leave the host
Add host-side budgeted fan-out and usage reporting improvements so honest legs collectively respect a global token budget
Strengthen host and provider conformance checks with new scenarios for version rejection, crash isolation, composition audit, malformed input labelling, and provenance–fixture consistency
Add shared reference-provider skeleton and two conformant in-repo providers (ripgrep and treesitter) backed by real files and symbol graphs
Build:

Wire new reference providers, MCP bridge/server, and create-contextgraph-provider scaffold into the workspace and CI, including external conformance gates for all SDKs and HTTP adapters and a downstream canary against stella
Add release and SDK publish workflows, including crates.io dry-run and guarded tag-triggered publishing steps
CI:

Extend CI to run host-conformance, provider conformance for reference and MCP-based providers, HTTP conformance for SDK examples, scaffold-generated project conformance, schema validation, and crates.io dry-run packaging
Documentation:

Expand provider implementation docs with SDK quick-starts, HTTP hosting guidance, scaffold usage, and conformance registry information, plus new pages on composing frames into prompts, reference providers, MCP composition, and protocol surface/stability clarifications
Tests:

Add extensive unit and property tests for the composition module, pipelined stdio transport, HTTP credential handling and transport security, conformance suite behavior, and MCP bridge/server flows
Chores:

Update SPEC, governance, ADRs, changelog, PR template, and schema tooling to reflect new host checks, HTTP rules, composition behavior, usage-report naming, and downstream canary setup

[sourcery-ai]

Sorry @macanderson, your pull request is larger than the review limit of 150000 diff characters

[sourcery-ai]

Reviewer's Guide

Introduces a full reference prompt-composition module and global-budgeted fan-out on the host; pipelines the stdio transport with correlation-aware demux; hardens HTTP transport with TLS/credential rules; extends provider and host conformance (including provenance byte checks and composition audit); adds MCP bridges, reference providers, SDK HTTP adapters and scaffolding; and wires new CI and publishing workflows and documentation updates around these features.

Sequence diagram for the pipelined stdio provider with correlation-aware demux

sequenceDiagram
  participant Host
  participant StdioProvider
  participant ReaderTask
  participant Child as ChildProcess

  Host->>StdioProvider: query(query)
  alt capabilities.correlation == false
    StdioProvider->>StdioProvider: exchange_lockstep(Envelope::Query)
    StdioProvider->>Child: write_envelope
    Child-->>ReaderTask: Frames (no id)
    ReaderTask->>StdioProvider: Reply via no_id_slot
    StdioProvider-->>Host: ContextQueryResult
  else correlation enabled
    StdioProvider->>StdioProvider: next_correlation_id
    StdioProvider->>StdioProvider: pending.insert(id, oneshot::Sender)
    StdioProvider->>Child: write_envelope(Envelope::Query{id})
    par reader loop
      loop run_reader
        ReaderTask->>ReaderTask: read_framed_line
        ReaderTask->>ReaderTask: decode_line
        alt Envelope::Frames{id}
          ReaderTask->>StdioProvider: fulfill oneshot::Sender(Ok(Envelope::Frames))
        else error / EOF
          ReaderTask->>StdioProvider: drain_waiters(Err(HostError))
        end
      end
    and reply handling
      StdioProvider-->>Host: ContextQueryResult
    end
  end

Loading

Flow diagram for the new reference prompt composition pipeline

flowchart LR
  Host[Host::query_all_budgeted]
  Budget[compose::budget_split]
  Providers[ContextProvider implementations]
  FanOut[FanOut]
  Compose[compose_for_prompt]
  Dedup[dedup_cross_provider]
  Order[order_by_value]
  Render[render_frame + EVIDENCE_PREAMBLE]
  Output[ComposedPrompt]

  Host --> Budget
  Budget --> Providers
  Providers --> FanOut
  FanOut --> Compose
  Compose --> Dedup
  Dedup --> Order
  Order --> Render
  Render --> Output

Loading

File-Level Changes

Change	Details	Files
Add reference prompt-composition module and global-budgeted fan-out with a detailed composition audit.	Refactor render_frame to use a shared citation_label_for helper and import Provenance. Introduce budget_split to divide a global token budget into per-provider shares for query_all_budgeted. Add dedup_cross_provider and supporting types (Deduped, DedupDrop, Citation, VerificationState, FrameDisposition, CompositionAudit) to collapse cross-provider duplicates while merging provenance. Implement order_by_value and fold_to_edges to place highest-value frames at the top/bottom of the prompt (Lost-in-the-Middle). Add compose_for_prompt to produce a preambled, fenced prompt, citations, and audit over accepted frames and expose these APIs from contextgraph-host. Extend Host and FanOut with query_all_budgeted and compose_for_prompt, plus tests to assert share-splitting and per-leg budget enforcement.	contextgraph-host/src/compose.rs contextgraph-host/src/host.rs contextgraph-host/src/lib.rs
Pipeline the stdio transport with correlation-aware demultiplexing and shared framing helpers.	Factor shared read_framed_line and write_framed_line helpers and use them from RawStdioConnection and the new StdioProvider reader/writer paths. Split RawStdioConnection into stdin/stdout plus a StdioControl handle via into_parts, moving process-group teardown out of the connection. Redesign StdioProvider to own stdin, a pending table keyed by correlation id, an id-less slot plus lockstep mutex, a control handle, and a reader task that demuxes replies on id. Implement exchange_lockstep for non-correlating queries and verify, preserving strict round-trip semantics for those cases. Add an ADR-0002 witness test that two correlated queries answered out of order are demuxed back to their caller correctly.	contextgraph-host/src/stdio.rs
Harden HTTP transport with TLS/credential rules and bearer auth support, including new HostError variants and unit tests.	Introduce a Credential type with redacted Debug/Display and expose-bearer-token helper, and wire it into HttpProvider and Host::add_http. Add is_loopback_host and refuse_insecure_transport to reject plaintext http:// to non-loopback hosts with HostError::InsecureTransport. Extend post_envelope to attach Authorization: Bearer headers and surface HTTP 401 as HostError::Unauthorized, and propagate provider error codes via HostError::Provider.code. Add tests for insecure transport refusal, loopback allowance, bearer header attachment, and credential redaction/HostError string hygiene.	contextgraph-host/src/http.rs contextgraph-host/src/error.rs contextgraph-host/src/host.rs
Extend provider and host conformance suites with version-reject, provenance byte-consistency, composition audit, crash isolation, and malformed-input code checks.	Add HCHECK_VERSION_REJECT, HCHECK_CRASH_ISOLATION, and HCHECK_COMPOSITION_AUDIT to host_conformance, including fixture-based tests that drive version mismatch and crash-isolation scenarios via StdioProvider and bash scripts. Extend the provider conformance suite with CHECK_PROVENANCE_FIXTURE_CONSISTENCY that re-reads fixture files and re-hashes file provenance via verify_file_provenance, and add a stale-digest misbehave mode to the example provider. Tighten malformed-input-tolerance to require bad_request error codes (not just any error) and add a mislabel-malformed mode and corresponding tests. Update SPEC.md and protocol-surface.md to reflect UR1 naming, H3 host-side check, R3 composition module reference, deferred neighbors operation, and clarified HTTP transport rules. Adjust host_conformance and conformance tests to assert expanded check counts (host: 9, provider: 13) and new check names.	contextgraph-conformance/src/host_conformance.rs contextgraph-conformance/src/lib.rs contextgraph-conformance/src/bin/contextgraph-example-docs.rs contextgraph-conformance/tests/conformance_suite.rs contextgraph-conformance/tests/host_conformance_suite.rs contextgraph-conformance/tests/verify_wire.rs contextgraph-conformance/src/bin/contextgraph-inspect.rs SPEC.md docs/protocol-surface.md docs/context-reuse.md docs/running-conformance.md
Introduce MCP interop (bridge and server) plus reference providers and shared refprov crate, and wire them into CI.	Add contextgraph-refprov crate exposing a shared stdio loop, honest file-backed/derived frame builders, and utilities (sha256, line_range_bytes, walk) for reference providers. Implement contextgraph-ripgrep and contextgraph-treesitter binaries using refprov::FrameSource to serve Snippet and Symbol/Graph frames from fixture/workspace files, with real file provenance and graph relations. Add contextgraph-mcp-bridge (MCP client → CGP provider) and contextgraph-mcp-fixture (local MCP server) crates, plus an end-to-end host integration test that fans out against the bridge and asserts consent gating for remote configuration. Add contextgraph-mcp-server crate that wraps a Host as an MCP server exposing a query_context tool and tests it via JSON-RPC over stdio. Extend CI to run conformance-external against ripgrep/treesitter, the MCP bridge (wrapping the fixture server), and HTTP adapters for all SDKs, and to add downstream-canary and publish-dry-run jobs.	contextgraph-refprov/src/lib.rs contextgraph-refprov/Cargo.toml contextgraph-ripgrep/src/main.rs contextgraph-ripgrep/Cargo.toml contextgraph-ripgrep/fixtures/reference.md contextgraph-treesitter/src/main.rs contextgraph-treesitter/Cargo.toml contextgraph-treesitter/fixtures/sample.rs contextgraph-mcp-bridge/src/lib.rs contextgraph-mcp-bridge/src/bin/contextgraph-mcp-bridge.rs contextgraph-mcp-bridge/src/bin/contextgraph-mcp-fixture.rs contextgraph-mcp-bridge/tests/bridge_via_host.rs contextgraph-mcp-bridge/Cargo.toml contextgraph-mcp-server/src/lib.rs contextgraph-mcp-server/src/bin/contextgraph-mcp-server.rs contextgraph-mcp-server/tests/smoke.rs contextgraph-mcp-server/Cargo.toml .github/workflows/ci.yml .github/workflows/downstream-canary.yml .github/workflows/release.yml .github/scripts/downstream-canary-stella.sh .github/scripts/wait-for-crate.sh
Enhance SDKs and scaffolding with HTTP adapters, CLI scaffolder, and README updates.	Export HTTP helpers from the TypeScript SDK (handleEnvelope, respondToEnvelopeBody, createHttpHandler) and add an HTTP example provider, plus documentation in README and implementing-a-provider docs. Add HTTP adapter functions to the Python SDK (handle_envelope, respond_to_body, make_wsgi_app) and an example_docs_http WSGI server, with README updates mirroring TS. Expose an HTTP handler in the Go SDK and add an example HTTP provider, with README and CI wiring to run contextgraph-inspect http against it. Introduce the create-contextgraph-provider scaffold CLI with TypeScript/Python templates, scripts, and GitHub Actions workflows that run contextgraph-inspect in generated projects; wire a CI job to scaffold and test both languages. Update docs (implementing-a-provider, site docs index, sdk/README) to advertise SDKs, HTTP adapters, and scaffolding, and clarify publishing status in SDK READMEs.	sdk/typescript/src/http.ts sdk/typescript/src/index.ts sdk/typescript/examples/example-docs-http.ts sdk/typescript/README.md sdk/python/contextgraph_sdk/http.py sdk/python/contextgraph_sdk/__init__.py sdk/python/examples/example_docs_http.py sdk/python/README.md sdk/go/contextgraph/http.go sdk/go/examples/example-docs-http/main.go sdk/go/README.md sdk/create-contextgraph-provider/index.js sdk/create-contextgraph-provider/package.json sdk/create-contextgraph-provider/templates/** sdk/README.md docs/implementing-a-provider.md site/content/docs/implementing-a-provider.mdx .github/workflows/ci.yml sdk/PUBLISHING.md
Add conformance registry, publishing prep, badges, and documentation cross-links.	Introduce docs/registry.md and site/content/docs/registry.mdx plus a static conformant.svg badge and PULL_REQUEST_TEMPLATE registry submission checklist. Add crates.io and docs.rs badges to the root README and crate READMEs, and clarify that they read 'not found' until the first publish. Add PUBLISHING.md for crates and sdk/PUBLISHING.md for SDKs, plus a publish-dry-run CI job and a tag-triggered release workflow guarded by a crates-io environment. Update docs index pages to link to new composition, reference providers, composition/MCP walkthrough, and registry docs, and add sketches/context-neighbors.md. Clarify downstream canary behavior in ADR 0007 and adaptive-context-reconciliation docs, making it advisory, not gating.	docs/registry.md site/content/docs/registry.mdx site/public/badges/conformant.svg .github/PULL_REQUEST_TEMPLATE.md README.md contextgraph-types/README.md contextgraph-host/README.md contextgraph-conformance/README.md PUBLISHING.md sdk/PUBLISHING.md docs/index.md site/content/docs/index.mdx docs/composing-frames-into-a-prompt.md docs/composition-walkthrough.md docs/sketches/context-neighbors.md docs/adaptive-context-reconciliation.md docs/adr/0007-protocol-product-boundary.md .github/workflows/ci.yml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.