SPDD 2026-06-30: tighten normative requirements across 5 specs

docs/src/content/docs/specs/ai-credits-specification.md

@@ -230,6 +230,18 @@ Catalog refresh processes SHOULD use normalized upstream model inventories and S
 
 Catalog updates MUST preserve JSON validity and MUST maintain backward-safe handling for historical model IDs that remain in the catalog but are absent from current live inventories.
 
+### 5.4 Catalog Sync Failure Handling
+
+A conforming implementation MUST define and enforce behavior for catalog synchronization failures. Specifically:
+
+1. **Unreachable catalog source**: When the upstream pricing catalog source is unreachable during a sync operation, the implementation MUST fall back to the last successfully synchronized catalog version. The implementation MUST NOT silently use an empty or zero-cost catalog. The implementation MUST emit a diagnostic warning identifying the sync failure and the fallback version in use.
+
+2. **Malformed catalog**: When a fetched catalog update fails JSON schema validation or contains non-numeric cost fields, the implementation MUST reject the update entirely, retain the previous valid catalog, and emit a validation error identifying the malformed field(s). The implementation MUST NOT partially apply a malformed update.
+
+3. **Missing required fields**: When a catalog entry is missing required fields (`input` or `output` cost values), the implementation MUST treat that entry as invalid and MUST NOT compute AIC using zero or undefined costs for that model. The entry MUST be skipped or flagged, and a diagnostic MUST be emitted.
+
+4. **Catalog version mismatch**: When the two required catalog paths (`pkg/cli/data/models.json` and `actions/setup/js/models.json`) diverge in content after a sync operation, the sync tooling/CI gate MUST treat this as a sync failure and MUST fail the refresh operation until consistency is restored.
+
 ---
 
 ## 6. Copilot Billing Reference Requirements
@@ -495,11 +507,12 @@ Conforming implementations SHOULD surface explicit validation errors for:
 
 ### Appendix C: Security and Integrity Considerations
 
-Pricing catalogs are configuration inputs. Implementations SHOULD:
+Pricing catalogs are configuration inputs. Implementations MUST:
 
-- Treat catalog updates as controlled changes.
-- Validate and review catalog source provenance.
+- Treat catalog updates as controlled changes and gate all updates through the project's standard change-control workflow.
+- Validate catalog source provenance before applying any update; the source origin MUST be confirmed as an authorized upstream before the catalog is consumed.
 - Avoid silently mutating cost values at runtime.
+- Review catalog changes for correctness and billing alignment before merging; reviews MUST be performed by at least one team member with knowledge of the active model pricing.
 
 ---
 

docs/src/content/docs/specs/bash-command-parser-specification.md

@@ -73,6 +73,8 @@ The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**, **
 
 A conforming implementation MUST satisfy all applicable MUST-level requirements for its class.
 
+A Class I (Integration Consumer) conforming implementation MUST apply default-deny behavior when `extractCommandNamesFromPipeline` (§4.3) returns no commands (`length === 0`). An empty list result MUST NOT be treated as implicit authorization to proceed with command execution.
+
 ### 2.3 Language-Neutral Type Contract
 
 Conforming implementations MUST expose semantics equivalent to the following total functions:
@@ -277,7 +279,7 @@ Conforming projects SHOULD publish vectors in JSON (or equivalent structured dat
 - `source = "model-based"` for state-model-derived vectors
 - `source = "verification"` for metamorphic/property-derived vectors
 
-Reference vectors SHOULD be stored in repository-owned, language-neutral artifacts and consumed by each implementation's native test runner.
+Reference vectors MUST be stored at `specs/test-vectors/bash-command-parser/` within the repository root and MUST be addressable via stable, version-qualified filenames following the pattern `<version>-<source>.json` (for example, `v1.1.0-model-based.json` and `v1.1.0-verification.json`). The canonical path for the test vector collection is `specs/test-vectors/bash-command-parser/` relative to the repository root. Each implementation's native test runner MUST consume vectors from this canonical path when this collection is present in the repository.
 
 ---
 
@@ -302,7 +304,21 @@ Conforming projects SHOULD apply all of the following:
 
 ## 11. Security Considerations
 
-This parser is not a shell sandbox and MUST NOT be treated as proof of command safety. Consumers MUST keep permission checks default-deny when command identification fails. Ambiguous/unparseable input SHOULD result in deny behavior at integration layer.
+This parser is not a shell sandbox and MUST NOT be treated as proof of command safety. Consumers MUST keep permission checks default-deny when command identification fails. Ambiguous/unparseable input MUST result in deny behavior at the integration layer.
+
+### 11.1 Misuse Examples
+
+The following examples illustrate security misuse patterns that conforming consumers MUST NOT adopt:
+
+**Misuse Example 1 — Using parser output as sandbox proof**: An integration that allows a shell command to execute solely because the parser returned a non-null command name is incorrect. The parser identifies command tokens; it does not validate that the command is safe, permitted, or confined. A consumer MUST NOT treat a non-null extraction result as proof that execution is authorized or sandbox-contained.
+
+**Misuse Example 2 — Allowing execution on empty parse result**: An integration that permits shell execution when `extractCommandNamesFromPipeline` returns an empty list (for example, because the input contained only redirections or structural keywords) violates the default-deny contract. An empty result MUST NOT be treated as implicit permission; the integration MUST apply its default-deny fallback.
+
+**Misuse Example 3 — Bypassing permission checks on single-command pipelines**: An integration that skips scoped allow-list checks for inputs the parser classifies as "simple" (single command, no operators) misuses the parser's output. Parser-identified simplicity is a syntactic property only. All permission-check logic MUST be applied regardless of pipeline complexity as determined by the parser.
+
+### 11.2 Normative Security Requirements
+
+A Class I (Integration Consumer) conforming implementation MUST NOT treat any parser output — including non-empty command-name results and empty lists — as proof that the corresponding shell invocation is safe or permitted. Permission-enforcement logic MUST remain fully operative independent of what the parser returns.
 
 ---
 

docs/src/content/docs/specs/copilot-sdk-driver-specification.md

@@ -332,6 +332,33 @@ When a permission request is denied, the implementation MUST:
 
 In standalone mode, missing required environment variables or unreadable prompt input MUST be reported to standard error with a driver-specific prefix.
 
+### 6.5 Structured Event Serialization Schema
+
+All structured log entries emitted by a conforming implementation MUST conform to the following field schema. Implementations MUST include all required fields and MAY include additional fields.
+
+#### 6.5.1 Lifecycle Event Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `event` | string | MUST | Event type identifier (e.g., `"connection.start"`, `"session.created"`, `"prompt.dispatched"`, `"session.completed"`, `"error"`) |
+| `timestamp` | string (ISO 8601) | MUST | UTC timestamp of the event |
+| `sessionId` | string | MUST when available | Session identifier assigned at session creation; MUST be omitted before session is established |
+| `durationMs` | number | MUST for completion events | Elapsed time in milliseconds from prompt dispatch to completion |
+| `hasOutput` | boolean | MUST for completion events | Indicates whether the session produced a non-empty output |
+| `level` | string | MUST | Log severity level: one of `"info"`, `"warning"`, `"error"` |
+| `message` | string | MUST | Human-readable description of the event |
+
+#### 6.5.2 Policy-Denial Log Entry Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `event` | string | MUST | MUST be `"permission.denied"` |
+| `timestamp` | string (ISO 8601) | MUST | UTC timestamp of the denial |
+| `sessionId` | string | MUST | Session identifier in which the denial occurred |
+| `requestKind` | string | MUST | Permission request kind: one of `"read"`, `"write"`, `"url"`, `"shell"`, `"mcp"`, `"custom-tool"` |
+| `requestSummary` | string | MUST | Compact human-readable summary of the denied request (MUST NOT include secret values or raw token content) |
+| `level` | string | MUST | MUST be `"warning"` or `"error"` |
+
 ---
 
 ## 7. Error Handling and Exit Behavior
@@ -436,12 +463,18 @@ Implementations MUST provide automated tests for all Level 1 and Level 2 require
 
 ### Appendix C: Security Considerations
 
+#### Glossary
+
+**Non-ephemeral events**: Events that persist beyond the lifetime of a single session or request and are written to durable storage (for example, audit logs, telemetry sinks, or artifact uploads). Ephemeral events, by contrast, exist only in process memory or transient I/O buffers and are discarded when the driver process exits.
+
+#### Requirements
+
 A conforming implementation SHOULD:
 
 - Treat connection tokens as secrets and avoid logging raw token values.
 - Apply least-privilege permission rules and avoid broad allow-all configurations unless operationally justified.
 - Preserve auditable denial logs for policy and incident review.
-- Restrict event persistence to non-ephemeral events and avoid writing sensitive transient state.
+- Restrict event persistence to non-ephemeral events (as defined above) and avoid writing sensitive transient state to durable storage.
 - Not attempt to read, fall back to, or check for platform authentication tokens (`GITHUB_TOKEN`, `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`). These tokens are not present in the driver subprocess environment by design. Attempting to use them would fail silently or cause unexpected behavior.
 
 ---

specs/security-architecture-spec-validation.md

@@ -472,16 +472,81 @@ concurrency:
 
 ---
 
-## Revalidation Cadence
+## Re-validation Triggers
 
-Whenever any of the following occur, re-run this validation:
+### Normative Triggers
+
+A conforming maintainer MUST re-run this validation when any of the following occur:
 
 1. `specs/security-architecture-spec.md` changes any MUST/SHALL-level requirement, compliance-test mapping, or security guarantee wording.
 2. Compiler or runtime changes alter compiled job structure, permission separation, threat-detection placement, timestamp validation, or other `.lock.yml` security behaviors described in this report.
 3. Companion security specifications (`scratchpad/guard-policies-specification.md`, `scratchpad/github-mcp-access-control-specification.md`) add or revise GitHub guard-policy or runtime-access-control behaviors that the top-level security architecture depends on.
 4. Validation evidence files, example workflows, or implementation locations cited in the Detailed Validation table move or change substantially.
 
-For each revalidation pass, reviewers SHOULD rerun the Detailed Validation procedure above, refresh the evidence-location table, update the Minor Discrepancies section, and revise the validation grade if any claim is no longer fully verified.
+For each re-validation pass, reviewers MUST rerun the Detailed Validation procedure above, refresh the evidence-location table, update the Minor Discrepancies section, and revise the validation grade if any claim is no longer fully verified.
+
+### Failure Escalation
+
+When a re-validation pass identifies a specification claim that is no longer verifiable against the implementation:
+
+1. The validation grade MUST be downgraded to reflect the unverified claim (e.g., from **A** to **B** or lower).
+2. A tracking issue MUST be opened in the repository describing the gap between specification claim and implementation evidence, referencing the affected requirement identifier (e.g., `OI-01`, `IS-04`).
+3. The affected row in the Specification Accuracy Summary table MUST be updated to ❌ **UNVERIFIED** with a link to the tracking issue.
+4. No new security-impacting features that depend on the unverified claim MUST be merged until the claim is re-verified or the specification is amended to match the implementation.
+5. A re-validation MUST be performed and the validation grade MUST be restored before the tracking issue is closed.
+
+---
+
+## Automation Approach
+
+### Proposed CI Mechanism
+
+To reduce manual re-validation burden and catch specification drift early, the following CI mechanism is proposed for adoption:
+
+**Approach 1 — Compiled YAML structural assertions (recommended)**: Add a CI job that runs after `make recompile` and asserts structural properties of compiled `.lock.yml` files against the security architecture requirements documented in this report. Assertions MUST include:
+
+- Presence and ordering of `pre_activation`, `activation`, `agent`, `detection`, and `safe_outputs` jobs.
+- Permission set on `agent` job is read-only (`contents: read`, no write permissions).
+- Permission set on `safe_outputs` job contains at least one write permission.
+- All non-local `uses:` action references (excluding local paths such as `./actions/...`) match the SHA-pinned format (`owner/action@<40-hex-chars>`).
+- `concurrency.group` contains at least one dynamic expression (`${{ ... }}`).
+
+This job SHOULD be implemented as a lightweight script (e.g., a YAML-parsing shell or Node.js script) that exits non-zero when any assertion fails, blocking merge.
+
+**Approach 2 — Spec-to-implementation cross-reference linter**: Extend the existing `gh aw compile` linter to emit warnings when compiled output omits patterns required by the security architecture specification (e.g., missing timestamp validation step in activation job, missing fork-protection condition in `if:` expressions).
+
+A conforming CI pipeline SHOULD implement at least Approach 1 and SHOULD run the assertion suite on every pull request that modifies `.github/workflows/*.md` files or `specs/security-architecture-spec.md`.
+
+---
+
+## §12 Compliance Test Matrix Gap Analysis
+
+This section audits the compliance test matrix defined in `security-architecture-spec.md §12` against the evidence documented in this validation report. Each test category is classified as **EVIDENCED**, **PARTIALLY EVIDENCED**, or **GAP** based on whether this document provides concrete implementation evidence.
+
+| Test Category | Test IDs | Status | Notes |
+|---------------|----------|--------|-------|
+| Input Sanitization | T-IS-001 to T-IS-008 | ✅ EVIDENCED | Covered in §1a (IS-04 to IS-09); all sanitization functions verified in `sanitize_content_core.cjs` |
+| Output Isolation | T-OI-001 to T-OI-007 | ⚠️ PARTIALLY EVIDENCED | OI-01 (job architecture) and OI-06 (output validation) verified; OI-02 (agent read-only) implicitly verified via permission blocks; T-OI-003 (safe output type support), T-OI-004 (output validation rules), T-OI-005 (token precedence), T-OI-006 (token secret expression), T-OI-007 (write operation isolation) lack dedicated evidence entries |
+| Network Isolation | T-NI-001 to T-NI-009 | ⚠️ PARTIALLY EVIDENCED | AWF binary installation (T-NI-007) verified; T-NI-001 (network mode support), T-NI-002 (ecosystem expansion), T-NI-003 (domain matching), T-NI-004 (protocol filtering), T-NI-005 (invalid protocol), T-NI-006 (blocked domain precedence), T-NI-008 (MCP isolation), T-NI-009 (content sanitization integration) lack dedicated evidence entries |
+| Permission Management | T-PM-001 to T-PM-007 | ⚠️ PARTIALLY EVIDENCED | PM-01/PM-02 (permission defaults), PM-08 (fork protection), PM-10/PM-11 (RBAC) verified; T-PM-003 (strict mode), T-PM-005 (repository validation for `workflow_run`), T-PM-007 (token validation) lack dedicated evidence entries |
+| Sandbox Isolation | T-SI-001 to T-SI-007 | ❌ GAP | No sandbox isolation evidence entries are present in this validation document; T-SI-001 through T-SI-007 (AWF chroot, Docker socket, `--env-all`, GOROOT, MCP container, network isolation independence) are not covered |
+| Threat Detection | T-TD-001 to T-TD-007 | ⚠️ PARTIALLY EVIDENCED | TD-01 (automatic threat detection) verified via `detection:` job; T-TD-002 (prompt injection), T-TD-003 (secret leaks), T-TD-004 (malicious patches), T-TD-005 (custom prompt), T-TD-006 (engine override), T-TD-007 (workflow failure on detection) lack dedicated evidence entries |
+| Compilation-Time Security | T-CS-001 to T-CS-006, T-SG07-001, T-SG07-002 | ⚠️ PARTIALLY EVIDENCED | CS-10 (action pinning, T-CS-005) verified; T-CS-001 (schema validation), T-CS-002 (expression safety), T-CS-003 (permission validation), T-CS-004 (network config validation), T-CS-006 (deprecated feature rejection), T-SG07-001 and T-SG07-002 (fail-secure behaviors) lack dedicated evidence entries |
+| Runtime Security | T-RS-001 to T-RS-011 | ⚠️ PARTIALLY EVIDENCED | RS-01/RS-02 (timestamp validation) and RS-16 to RS-22 (concurrency control) verified; T-RS-003 through T-RS-008 (repository validation for `workflow_run`, role validation, token validation, AWF/MCP network enforcement, output validation) lack dedicated evidence entries |
+| Companion MCP Access-Control | T-GH-047 to T-GH-060 | ⚠️ PARTIALLY EVIDENCED | Deferred to companion specifications (`scratchpad/github-mcp-access-control-specification.md`, `scratchpad/guard-policies-specification.md`); not directly evidenced in this document |
+
+### Gap Summary
+
+The following test categories require dedicated evidence entries to achieve full coverage in this validation document:
+
+1. **Sandbox Isolation (T-SI-001 to T-SI-007)** — No evidence present. A re-validation pass MUST document how AWF chroot behavior, Docker socket visibility, and MCP container isolation are reflected in compiled workflow behavior.
+2. **Output Isolation (T-OI-003 to T-OI-007)** — Partial evidence. Missing evidence for token precedence, secret expression validation, and write operation isolation.
+3. **Network Isolation (T-NI-001 to T-NI-009)** — Partial evidence. Network mode, domain matching, protocol filtering, and MCP isolation behavior are unverified in this document.
+4. **Threat Detection (T-TD-002 to T-TD-007)** — Partial evidence. Only TD-01 (job presence) verified; detection capability assertions unverified.
+5. **Compilation-Time Security (T-CS-001 to T-CS-004, T-CS-006, T-SG07)** — Partial evidence. Only action pinning (T-CS-005) verified.
+6. **Runtime Security (T-RS-003 to T-RS-008)** — Partial evidence. Only timestamp and concurrency verified.
+
+Maintainers SHOULD address gaps in order of risk: Sandbox Isolation and Network Isolation are the highest-priority gaps given their role in containing agent execution.
 
 ---