Close 2026-07-06 spec maintenance gaps for parser vectors, security notes, AI Credits sync gating, and SDK driver sync guidance

[Copilot]

This PR closes the outstanding SPDD maintenance items across four specs: the missing bash parser conformance artifacts, unresolved security-architecture follow-ups, the missing AI Credits mirror-consistency gate, and the Copilot SDK driver sync/normative cleanup. It also updates the associated tracking docs so the spec set and maintenance state stay aligned.

• Bash command parser conformance artifacts

  • Adds the missing canonical vector directory and seed files:
    • specs/test-vectors/bash-command-parser/v1.1.0-model-based.json
    • specs/test-vectors/bash-command-parser/v1.1.0-verification.json
  • Covers the required S-CORE, E-CORE, P-CORE, and R-CORE cases from §6.1.
  • Adds Sync Notes to the spec and records the canonical vector path plus revalidation triggers.
  • Updates the spec change log to reflect the new versioned vector corpus.

• Security architecture spec maintenance

  • Makes the pre_activation → runtime role-validation linkage explicit under PM-11.
  • Names the compiled detection job explicitly as the runtime threat-detection layer in Appendix D guidance.
  • Documents conclusion as optional, non-normative post-processing.
  • Adds an explicit forward reference for trusted-users runtime enforcement to the companion MCP access-control specs.
  • Marks the four previously pending summary items as done and updates the validation document to reflect the maintenance pass.

• AI Credits mirror consistency gate

  • Strengthens §5.4 so mirror divergence is a normative CI failure condition.
  • Adds make validate-models-json-sync as the repository gate for pkg/cli/data/models.json and actions/setup/js/models.json.
  • Compares normalized JSON content rather than raw file ordering, so the gate enforces dataset consistency without false positives from key ordering.
  • Adds Sync Notes with concrete revalidation triggers for model additions, pricing changes, and spec minor bumps.

• Copilot SDK driver spec cleanup

  • Adds Sync Notes tied to the harness/driver implementation files and revalidation conditions.
  • Resolves the §4.5 ambiguity by making denial counting mandatory in the spec, with an explicit note that this matches existing gh-aw reference behavior rather than introducing new runtime behavior in this repo.

• Example: new mirror-consistency gate

  .PHONY: validate-models-json-sync
  validate-models-json-sync:
  	@tmp1=$$(mktemp); tmp2=$$(mktemp); \
  	trap 'rm -f "$$tmp1" "$$tmp2"' EXIT; \
  	jq -S . pkg/cli/data/models.json > "$$tmp1"; \
  	jq -S . actions/setup/js/models.json > "$$tmp2"; \
  	cmp -s "$$tmp1" "$$tmp2" || { \
  		echo "models.json mirrors diverged:"; \
  		echo "  pkg/cli/data/models.json and actions/setup/js/models.json must remain semantically identical after key sorting."; \
  		echo "  Run 'make refresh-models-json' to synchronize the mirrors."; \
  		exit 1; \
  	}

---

Generated by 👨‍🍳 PR Sous Chef · 6.99 AIC · ⌖ 5.55 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

[github-actions]

Hey @app/copilot-swe-agent 👋 — thanks for the thorough spec maintenance sweep! The detailed PR body makes it easy to follow all four areas being addressed.

• Split the PR — this bundles 4 separate spec areas into one change (bash parser conformance vectors, security architecture follow-ups, AI Credits mirror-consistency gate, and Copilot SDK driver sync). Splitting into focused per-spec PRs would make it easier for reviewers to evaluate each area independently, and safer for partial reverts if one spec needs further revision.

If you’d like a hand, you can assign this prompt to your coding agent:

Split PR #43839 into 4 focused per-spec PRs:
1. Bash command parser conformance artifacts (specs/test-vectors/bash-command-parser/, bash-command-parser-specification.md)
2. Security architecture spec maintenance (specs/security-architecture-spec*.md)
3. AI Credits mirror consistency gate (Makefile + docs/src/content/docs/specs/ai-credits-specification.md)
4. Copilot SDK driver spec cleanup (docs/src/content/docs/specs/copilot-sdk-driver-specification.md)
Each PR should touch only its own spec area and have a focused title and description.

Generated by ✅ Contribution Check · 78.3 AIC · ⌖ 17.7 AIC · ⊞ 6.2K · ◷

[github-actions]

🤖 PR Triage

Field	Value
Category	docs
Risk	🟢 low
Score	37 / 100 (impact 15 · urgency 10 · quality 12)
Action	🕐 defer

Summary: DRAFT. Spec maintenance across 4 areas: bash parser test vectors, security-architecture follow-ups, AI Credits mirror-consistency gate, Copilot SDK driver sync. 9 files. No CI.

Review when promoted from draft.

Generated by 🔧 PR Triage Agent · 136.6 AIC · ⌖ 9.03 AIC · ⊞ 5.5K · ◷

[github-actions]

✅ Test Quality Sentinel completed test quality analysis.

No test files were added or modified in this PR. Test Quality Sentinel skipped.

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

[github-actions]

✅ PR Code Quality Reviewer completed the code quality review.

[github-actions]

✅ Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #43839 does not have the 'implementation' label and has 0 new lines of code in business logic directories (threshold: 100).

[Copilot]

Pull request overview

This PR updates several gh-aw specifications and maintenance artifacts: it adds canonical bash command parser conformance vectors, clarifies/aligns the security architecture spec with current compiled workflow structure, introduces a models.json mirror-consistency validation target, and adds sync guidance + normative tightening to the Copilot SDK driver spec.

Changes:

• Added versioned bash command parser conformance vectors (v1.1.0) and documented sync/revalidation triggers in the spec.
• Clarified security-architecture runtime gating semantics (pre_activation, detection, optional conclusion) and updated the validation + summary tracking docs accordingly.
• Added make validate-models-json-sync and documented mirror-consistency requirements in the AI credits spec; added Sync Notes to multiple specs.

Show a summary per file

File	Description
specs/test-vectors/bash-command-parser/v1.1.0-verification.json	Adds verification-focused parser vectors (robustness + metamorphic relations).
specs/test-vectors/bash-command-parser/v1.1.0-model-based.json	Adds model-based canonical vectors covering S/E/P core requirements.
specs/security-architecture-spec.md	Clarifies runtime role validation, threat-detection job role, optional conclusion, and companion spec references.
specs/security-architecture-spec-validation.md	Updates validation date/results and records the completed maintenance follow-ups.
specs/security-architecture-spec-summary.md	Marks previously pending maintenance follow-ups as done.
Makefile	Adds validate-models-json-sync to compare normalized models.json mirrors.
docs/src/content/docs/specs/copilot-sdk-driver-specification.md	Adds Sync Notes and makes denial counting mandatory to match reference behavior.
docs/src/content/docs/specs/bash-command-parser-specification.md	Adds Sync Notes + changelog entry for canonical vector corpus.
docs/src/content/docs/specs/ai-credits-specification.md	Strengthens mirror-consistency requirement and adds Sync Notes/revalidation triggers.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 9/9 changed files
• Comments generated: 2
• Review effort level: Low

[github-actions]

REQUEST_CHANGES — Two blocking correctness issues must be resolved before merge

The spec maintenance is well-intentioned but introduces a normative CI requirement with no enforcement, and a new Makefile gate that silently passes on the exact failure case it is supposed to catch. The test vector files also have structural issues.

### Blocking issues (2)

1. CI gate not wired ( line 49): The spec now states CI MUST run make validate-models-json-sync, but no workflow file was changed in this PR and no existing workflow calls this target. A MUST without an enforcement point is misleading spec text.

2. Makefile silent false-positive ( line 16): If jq is not installed or both source files are missing, both temp files are left empty, cmp -s considers them identical, and the gate exits 0. The existing refresh-models-json target uses set -e; this one does not. Non-empty guards on the jq output are also needed.

### High-severity structural issues (2)

3. Test vectors inert (): The new files use a flat "function" field schema that is incompatible with the function-keyed schema of actions/setup/js/bash_command_parser_spec_vectors.json. No test harness, workflow, or Makefile target loads them. They are documentation, not executed tests.

4. Duplicate metamorphic vectors (): BP-VF-MR-001/002/003 are character-for-character duplicates of BP-MR-001/005/004 in the existing spec_vectors file. Once a harness is wired up, two authoritative sources will silently diverge.

### Medium issue (1)

5. Rationale in normative body ( line 146): The paragraph explaining the SHOULD→MUST change is editorial prose embedded between normative MUST clauses. Use a > Note (non-normative): block or move it to the Change Log.

🔎 Code quality review by PR Code Quality Reviewer · 152.8 AIC · ⌖ 5.81 AIC · ⊞ 5.5K
_{Comment /review to run again}

Comments that could not be inline-anchored

docs/src/content/docs/specs/ai-credits-specification.md:49

Spec mandates CI MUST run make validate-models-json-sync but no workflow file runs it. This MUST-level requirement is unenforceable dead text on merge.

<details>
<summary>💡 Details</summary>

The diff adds this normative text to §5.4:

> CI MUST run make validate-models-json-sync (or an equivalent normalized-content comparison gate).

No .github/workflows/*.yml file was changed in this PR, and a grep of the entire .github/ tree confirms no workflow calls this target. A divergence …

Makefile:16

validate-models-json-sync silently passes when jq is absent or when both source files are missing: both temp files are empty, cmp reports them identical, and the gate exits 0.

<details>
<summary>💡 Details and suggested fix</summary>

The current logic:

jq -S . pkg/cli/data/models.json > "$$tmp1"; jq -S . actions/setup/js/models.json > "$$tmp2"; cmp -s "$$tmp1" "$$tmp2" || { ... exit 1; }

If jq is not installed, both invocations silently fail (exit non-zero, write…

specs/test-vectors/bash-command-parser/v1.1.0-model-based.json:5

These vector files use a different schema from the existing bash_command_parser_spec_vectors.json and no test runner loads them. They are inert documentation.

<details>
<summary>💡 Details</summary>

The authoritative vector file is actions/setup/js/bash_command_parser_spec_vectors.json, loaded by actions/setup/js/bash_command_parser_spec_vectors.test.cjs. That file uses a function-keyed schema:

{ "vectors": { "splitOnPipelineOperators": [...], "extractCommandName": [...] },…

</details>

<details><summary>specs/test-vectors/bash-command-parser/v1.1.0-verification.json:18</summary>

**BP-VF-MR-001, BP-VF-MR-002, and BP-VF-MR-003 are semantic duplicates of entries already in `bash_command_parser_spec_vectors.json`**, creating a silent-drift trap.

&lt;details&gt;
&lt;summary&gt;💡 Details&lt;/summary&gt;

The existing `actions/setup/js/bash_command_parser_spec_vectors.json` already contains these metamorphic relations:

| New vector | Existing entry | Relation |
|---|---|---|
| BP-VF-MR-001 | BP-MR-001 | whitespace-invariance — identical `left`/`right`/`expected` |
| BP-VF-MR-002 | BP-MR-005…

</details>

<details><summary>docs/src/content/docs/specs/copilot-sdk-driver-specification.md:146</summary>

**Rationale paragraph is embedded in the normative spec body** — future readers cannot distinguish this prose from binding conformance text.

&lt;details&gt;
&lt;summary&gt;💡 Details&lt;/summary&gt;

The added paragraph reads:

&gt; This counting requirement tightens the specification to match the existing gh-aw reference implementation... It does not change gh-aw runtime behavior, but implementations that previously treated denial counting as optional would need to align...

This is rationale/editorial commentary…

</details>

[github-actions]

Skills-Based Review 🧠

Applied /grill-with-docs and /tdd — requesting changes on three issues: a normative-but-never-run CI gate, a SHOULD/MUST inconsistency in the SDK spec table, and a gap between the new test vectors and the live test runner.

📋 Key Themes & Highlights

Blocking Issues

1. validate-models-json-sync is never called (Makefile line 909) — the AI Credits spec §5.4 now declares CI MUST run make validate-models-json-sync, but the target is not wired into lint, agent-report-progress, or any GitHub Actions workflow. The normative claim is unsatisfied.

2. Test runner does not consume the new vectors — spec §9 says the native test runner MUST consume vectors from specs/test-vectors/bash-command-parser/ once the collection exists. bash_command_parser.test.cjs has no such loading logic; the 20 new vectors are dead spec artifacts unless this is added.

3. SHOULD/MUST split in SDK driver spec table — the env-var reference table still says SHOULD for GH_AW_MAX_TOOL_DENIALS input while §4.5 prose now mandates MUST count. The ambiguity will confuse conformance authors.

Secondary Issues

4. Incomplete metamorphic coverage — the verification file covers 3 of the 6 relations listed in §8. Missing: env-prefix-invariance (§8.3) and redirection-suffix-invariance (§8.4).

5. Vector count alignment is coincidental, not spec-faithful — model-based.json has 15 vectors by expanding E-CORE and P-CORE into branch cases, not by including R-CORE. A conformance note would prevent misreading.

Positive Highlights

• ✅ All 15 model-based vectors verified correct against the live bash_command_parser.cjs implementation.
• ✅ All MR and R-CORE vectors in the verification file produce correct expected outputs.
• ✅ jq -S normalization in the Makefile target is the right approach for JSON mirror comparison.
• ✅ Security spec maintenance items are well-scoped and the PM-11/trusted-users forward references are precise.
• ✅ The SHOULD→MUST change in §4.5 is correctly annotated with a non-breaking note.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 125.1 AIC · ⌖ 8.53 AIC · ⊞ 6.7K
_{Comment /matt to run again}

Comments that could not be inline-anchored

Makefile:909

[/grill-with-docs] validate-models-json-sync is not wired into any CI chain — the spec it enforces is now normative but the gate will never run automatically.

The AI Credits spec (§5.4) now states: "CI MUST run make validate-models-json-sync (or an equivalent normalized-content comparison gate)." But validate-models-json-sync is absent from lint, agent-report-progress, and no GitHub Actions workflow calls it.

<details>
<summary>💡 Suggested fix</summary>

Add it to the lint …

specs/test-vectors/bash-command-parser/v1.1.0-verification.json:57

[/tdd] The verification file covers only 3 of the 6 metamorphic relations defined in §8 — missing env-prefix-invariance (§8.3) and redirection-suffix-invariance (§8.4).

§8 requires verification vectors for: whitespace-invariance ✅, quoted-operator-shielding ✅, env-prefix-invariance ❌, redirection-suffix-invariance ❌, duplicate-collapse-invariance ✅, no-throw robustness ✅. The spec says verification MUST include metamorphic/property-derived vectors for all listed relations.

<details>
<…

docs/src/content/docs/specs/bash-command-parser-specification.md:363

[/tdd] The spec (§9) says the native test runner MUST consume vectors from specs/test-vectors/bash-command-parser/ when the collection is present — but bash_command_parser.test.cjs does not load these files at all.

The canonical vector files are now present in the repo, which activates the MUST requirement, yet the existing test file uses only inline hardcoded cases. No test connects the new v1.1.0-model-based.json or v1.1.0-verification.json to the runner.

<details>
<summary>💡 S…

docs/src/content/docs/specs/copilot-sdk-driver-specification.md:146

[/grill-with-docs] The reference table at line 146 still says SHOULD for GH_AW_MAX_TOOL_DENIALS, while §4.5 prose (line 185) now says MUST count — the same env var carries two different RFC 2119 levels depending on where you read.

This is confusing for conformance authors: the table implies optional input, but the prose makes counting mandatory. The table describes input validation, not counting behaviour, so the conflict may be unintentional — but it should be resolved explicitly.

…

specs/test-vectors/bash-command-parser/v1.1.0-model-based.json:129

[/tdd] The model-based file has 15 total vectors (S×4 + E×6 + P×5), but §6.1 counts the mandatory suite as S-CORE (4) + E-CORE (5) + P-CORE (4) + R-CORE (2) = 15. The model-based file has no R-CORE and expands E to 6 and P to 5 by splitting individual requirements into two branches.

The spec counts branches as single tests: e.g., E-CORE-4 is one test ("clause keywords continue extraction while structural keywords return null") with two sub-cases. Encoding these as separate vectors is fine,…

[github-actions]

Review Summary

The PR cleanly closes four maintenance gaps. Most changes are documentation-only clarifications that accurately reflect existing implementation behavior. One blocking issue was found:

🔴 Blocking: Spec claims normative CI gate that does not exist

File: docs/src/content/docs/specs/ai-credits-specification.md, line 254

The new §5.4 text contains: "CI MUST run make validate-models-json-sync". However, make validate-models-json-sync is not wired into any CI workflow file (.github/workflows/*.yml). The spec is asserting a MUST-level conformance requirement that the repository itself does not currently satisfy.

Fix options:

1. Add a CI step (e.g. in cjs.yml or a new workflow) that runs make validate-models-json-sync
2. Change the MUST to SHOULD in the spec until the CI job is added

All other changes look correct:

• ✅ Test vectors for bash-command-parser v1.1.0 are accurate (spot-checked S-CORE, E-CORE, P-CORE, R-CORE, MR cases against implementation)
• ✅ Security spec PM-11 forward reference to pre_activation is consistent with implementation
• ✅ Appendix D detection job naming is accurate
• ✅ SDK driver §4.5 SHOULD→MUST tightening matches existing reference implementation
• ✅ Companion spec references (scratchpad/ files) exist in the repository

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · 104.6 AIC · ⌖ 6.03 AIC · ⊞ 4.9K

Done

[gh-aw-bot]

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

Generated by 👨‍🍳 PR Sous Chef · 8.4 AIC · ⌖ 5.03 AIC · ⊞ 4.6K · ◷
_{Comment /souschef to run again}

[Copilot]

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

...

Addressed the unresolved review feedback in a6f2531:

• hardened make validate-models-json-sync to fail fast if jq is unavailable
• wired make validate-models-json-sync into CI via .github/workflows/cjs.yml and added pkg/cli/data/models.json to CJS path triggers

Reran local checks on the updated branch: make fmt, make lint, make test-unit, make test, make recompile, and make agent-report-progress all pass.

I can’t reliably rerun/verify fresh GitHub checks from this agent context after pushing; please re-trigger CI from GitHub (close/reopen PR, workflow_dispatch, or maintainer push) to validate the new HEAD.

[gh-aw-bot]

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

Unresolved review feedback:

• @github-actions: Close 2026-07-06 spec maintenance gaps for parser vectors, security notes, AI Credits sync gating, and SDK driver sync guidance #43839
• @github-actions: Close 2026-07-06 spec maintenance gaps for parser vectors, security notes, AI Credits sync gating, and SDK driver sync guidance #43839

Generated by 👨‍🍳 PR Sous Chef · 6.99 AIC · ⌖ 5.55 AIC · ⊞ 7.1K · ◷
_{Comment /souschef to run again}

[Copilot]

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

Unresolved review feedback:...

Done. I ran the pr-finisher flow on the current branch head (96c6943), confirmed the unresolved mirror-gate feedback is already addressed by prior commits (a6f2531 + merge update), and reran local checks successfully: make fmt, make lint, make test-unit, make test, make recompile, and make agent-report-progress.

For CI: I reviewed recent workflow runs/logs on this branch. The latest PR workflows are in action_required/pending state with no failed jobs exposed to this agent context, so fresh CI on the current HEAD still needs a maintainer-triggered rerun from GitHub (close/reopen PR, workflow_dispatch, or maintainer push).

[pelikhan]

@copilot this document must be self contained

[Copilot]

Updated in bbc2031: the summary now states the trusted-users runtime enforcement scope directly in this document instead of forwarding readers to companion specs.

[github-actions]

🎉 This pull request is included in a new release.

Release: v0.82.3