Skip to content

Add flexible intake and working briefs - #34

Merged
samtay32 merged 12 commits into
mainfrom
codex/flexible-intake-brief
Jul 30, 2026
Merged

Add flexible intake and working briefs#34
samtay32 merged 12 commits into
mainfrom
codex/flexible-intake-brief

Conversation

@samtay32

@samtay32 samtay32 commented Jul 29, 2026

Copy link
Copy Markdown
Owner

What changed

  • Adds develop-project-brief and an unlocked BRIEF.md working artifact for DISCOVER and EXTERNAL intake.
  • Routes intake in deterministic RESUMEEXTERNALDISCOVERDIRECT order. Valid unfinished work resumes at its first unmet condition; substantial supplied plans are audited and reconciled; unclear intent is developed; clear bounded work keeps the existing micro-brief path.
  • Preserves supplied sources and provenance, reconciles them with repository reality, records contradictions and claim dispositions, preserves closed decisions, and promotes intent into the existing lockable contracts only after explicit approval and conflict closure.
  • Separates explanation, brief-only, delivery, implementation, verification, PR-review, and work/evidence routes so an unrelated skill or end-to-end workflow is not activated falsely.
  • Gives no-coders one repository-first recommendation, keeps recommendation separate from approval, and exposes advanced providers only when evidence or an explicit request makes them relevant.
  • Hardens artifact locking and behavioral evidence: DRAFT, unresolved, duplicated, invalid, fenced, or open-conflict states fail closed; schema-version-2 run records and deterministic project fixtures cover the full observable behavior surface.
  • Keeps harness support capability-based and provider-neutral. A harness that cannot safely load or execute the workflow is reported as limited or untested and falls back to the safest supported mode; no named harness is forced.

Why

The delivery back half was strong, but the front door assumed a short idea. This milestone lets one Project Steward handle vague ideas, detailed outside plans, clear bounded requests, existing-project reconciliation, and valid continuity without adding a second delivery state machine or weakening authority and release controls.

Validation

  • Exact head: d33a2dcc73becf516ac86c762f64a5d63da10651.
  • Behavioral contracts: 27 scenarios across 13 skills and 8 categories, including one false-activation case.
  • Behavior surface: sha256:cdd2cf67d0a7512ab02051f4fcb84ad0ed3a1890f5c89cec164624f4c0aca4ec.
  • Local Node 24.17.0 npm run release:check: 191/191 tests, all 27 contracts, package dry-run, and packed-install smoke passed.
  • Packed-install smoke: 115 installed files.
  • Protected fixture baselines reproduce exactly.
  • All 63 tracked Markdown files pass markdownlint; git diff --check passes.
  • GitHub compatibility passes on Ubuntu Node 22, Windows Node 22, and Windows current; verify and CodeQL pass.
  • Qodo's unified review is updated through the exact head. Every production-grade finding was fixed or evidence-backed and all six review threads are resolved. The packaged-fixture observation was rebutted: those files intentionally implement the public installed evaluation commands, are explicitly allowlisted and smoke-tested, while test/ remains excluded.

Harness evidence and claim boundary

Deterministic contracts prove catalog and evaluator behavior; they do not prove that every model or harness follows the instructions.

  • Gemini CLI 0.29.5 / gemini-2.5-pro: direct-setup completed on the exact surface. direct-delivery was interrupted by provider HTTP 429 quota, and the remaining 25 cases were not attempted. This incomplete run does not count as behavioral evidence and is not a product or general Gemini compatibility failure.
  • Codex CLI 0.145.0: the exact-surface collector was prepared, but no live run was performed because a safely isolated API credential was unavailable. Desktop auth.json was not copied, and evaluator state was not exposed to the model.
  • Claude Code: no safely isolated live run was counted because bare mode cannot use the available subscription OAuth while normal mode loads unrelated user plugins, hooks, and memory.
  • Cursor: not evaluated on this surface.

This PR therefore makes no broad cross-harness behavior claim. No individual harness is a merge veto for this feature PR. Complete live runs remain a separate evidence track; release claims must follow the repository's evidence policy.

Authority

This PR does not authorize merge, npm publication, deployment, or release. Those remain separate human-authorized actions.

Summary by CodeRabbit

  • New Features

    • Added flexible intake for direct work, project discovery, external briefs, and checkpoint resumption.
    • Added working project briefs with source tracking, conflict handling, review, and promotion into delivery plans.
    • Added clearer onboarding recommendations and route-aware workflow guidance.
    • Added stronger approval, security, source-preservation, and secret-redaction safeguards.
  • Documentation

    • Updated guides, templates, changelog, and architecture documentation to explain intake, brief lifecycle, approvals, and upgrade impact.
  • Tests

    • Expanded behavioral coverage, deterministic evaluation fixtures, packaging checks, and multi-harness release evidence requirements.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: bf5321a6-5368-44c5-a819-b1e72efcfb05

📥 Commits

Reviewing files that changed from the base of the PR and between 5ae7ac8 and d33a2dc.

⛔ Files ignored due to path filters (1)
  • docs/assets/synthesis-map.svg is excluded by !**/*.svg
📒 Files selected for processing (47)
  • .codex-plugin/plugin.json
  • .gitattributes
  • CHANGELOG.md
  • README.md
  • STARTER_PROMPT.md
  • assets/project-template/.agent-stack/HANDOFF.md
  • assets/project-template/.agent-stack/artifacts/ARCHITECTURE.md
  • assets/project-template/.agent-stack/artifacts/BRIEF.md
  • assets/project-template/.agent-stack/artifacts/DECISIONS.md
  • assets/project-template/.agent-stack/artifacts/DELEGATION.md
  • assets/project-template/.agent-stack/artifacts/DELIVERY.md
  • assets/project-template/.agent-stack/artifacts/SECURITY.md
  • assets/project-template/.agent-stack/artifacts/VERIFICATION.md
  • assets/project-template/AGENTS.md
  • assets/project-template/GEMINI.md
  • bin/ultimate-agent-stack.mjs
  • docs/ARCHITECTURE.md
  • docs/BEHAVIORAL_EVALS.md
  • docs/OPERATING_MANUAL.md
  • docs/RELEASE.md
  • docs/SKILL_STACK.md
  • docs/SOURCES_AND_TRADEOFFS.md
  • docs/TRUST.md
  • evals/fixture-baselines.json
  • evals/fixtures.json
  • evals/scenarios.json
  • package.json
  • scripts/packed-smoke.mjs
  • scripts/skill-eval.mjs
  • scripts/skill-fixture.mjs
  • skills/build-vertical-slice/SKILL.md
  • skills/close-review-loop/SKILL.md
  • skills/develop-project-brief/SKILL.md
  • skills/develop-project-brief/agents/openai.yaml
  • skills/develop-project-brief/references/brief-contract.md
  • skills/develop-project-brief/references/intake-and-reconciliation.md
  • skills/manage-project-work/SKILL.md
  • skills/run-autonomous-delivery/SKILL.md
  • skills/run-autonomous-delivery/references/delivery-policy.md
  • skills/setup-autonomous-project/SKILL.md
  • skills/shape-project/SKILL.md
  • skills/shape-project/references/shaping-contract.md
  • skills/verify-change/SKILL.md
  • test/agent-stack.test.mjs
  • test/maintenance.test.mjs
  • test/skill-eval.test.mjs
  • test/skill-fixture.test.mjs

📝 Walkthrough

Walkthrough

This PR adds flexible intake routing with working-brief artifacts, strengthens approval-lock validation, introduces deterministic fixture materialization and baseline receipts, expands behavioral scenario evidence to schema version 2, and updates packaging, documentation, adapters, and tests.

Changes

Flexible intake and delivery contracts

Layer / File(s) Summary
Working brief and intake contracts
assets/project-template/.agent-stack/artifacts/*, skills/develop-project-brief/*, skills/shape-project/*, docs/ARCHITECTURE.md
Adds BRIEF.md, source-claim and closed-decision structures, promotion rules, and explicit conflict/status semantics.
Route-aware delivery and onboarding
STARTER_PROMPT.md, skills/run-autonomous-delivery/*, skills/setup-autonomous-project/*, README.md, docs/OPERATING_MANUAL.md, docs/SKILL_STACK.md, assets/project-template/*
Defines RESUME, EXTERNAL, DISCOVER, and DIRECT routing, condensed onboarding recommendations, provider boundaries, and continuity behavior.
Visible declaration locking
bin/ultimate-agent-stack.mjs, test/agent-stack.test.mjs
Validates visible Markdown declarations outside fenced blocks, rejects invalid locks and placeholders, and extracts structured start prompts.
Supporting skills and trust guidance
skills/build-vertical-slice/SKILL.md, skills/close-review-loop/SKILL.md, skills/manage-project-work/SKILL.md, skills/verify-change/SKILL.md, docs/TRUST.md
Updates closed-decision handling, review closure scope, evidence receipts, provider readiness, and lock guarantees.

Deterministic fixture and evidence evaluation

Layer / File(s) Summary
Fixture catalogs and materialization
evals/fixtures.json, evals/fixture-baselines.json, scripts/skill-fixture.mjs
Adds canonical fixtures, protected Git/tree baselines, deterministic project materialization, external-input handling, receipt hashing, and inspection commands.
Scenario and run-record contracts
evals/scenarios.json, scripts/skill-eval.mjs
Adds flexible-intake scenarios and validates question bounds, actions, writes, artifact states, source dispositions, receipts, harness isolation, and schema version 2 records.
Evaluation and release validation
docs/BEHAVIORAL_EVALS.md, docs/RELEASE.md, scripts/packed-smoke.mjs, test/*, package.json
Extends release evidence requirements, validates packed fixtures and lock behavior, and adds deterministic materialization and behavioral-evaluation coverage.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/flexible-intake-brief

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Owner Author

/review

@qodo-code-review

qodo-code-review Bot commented Jul 29, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (1)

Context used
✅ Compliance rules (platform): 98 rules
✅ Skills: 4 invoked
  verify-change
  maintain-agent-stack
  build-vertical-slice
  secure-launch

Grey Divider


Action required

1. Lock fence detection bypass ✓ Resolved 🐞 Bug ⛨ Security
Description
commandLock uses markdownOutsideFencedCode to exclude fenced-code content before scanning for
visible Status: / Material open conflicts: declarations, but markdownOutsideFencedCode only
detects fences that begin with up to 3 leading spaces. A fence opened with a list marker (e.g., `-
``...) is not detected, so indented lines inside that code block (e.g.,   Status: APPROVED) can
be counted as “visible” declarations and allow locking artifacts without real visible approvals.
Code

bin/ultimate-agent-stack.mjs[R8520-8534]

+  for (const line of content.split(/\r?\n/)) {
+    if (fence) {
+      const closing = line.match(/^ {0,3}(`+|~+)[ \t]*$/);
+      if (
+        closing &&
+        closing[1][0] === fence.character &&
+        closing[1].length >= fence.length
+      ) {
+        fence = null;
+      }
+      visible.push("");
+      continue;
+    }
+    const opening = line.match(/^ {0,3}(`{3,}|~{3,})(.*)$/);
+    if (opening) {
Relevance

●●● Strong

Team repeatedly accepts security hardening and fail-closed parsing/regex fixes
(indentation/TOCTOU/symlink patterns).

PR-#21
PR-#30
PR-#23

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The fence detector only matches lines starting with up to three spaces before the fence, which does
not match list-introduced fences like - `...; commandLock() then scans the resulting text with
declaration regexes that allow up to three leading spaces and treats matches as “visible”
declarations.

bin/ultimate-agent-stack.mjs[8517-8555]
bin/ultimate-agent-stack.mjs[8576-8605]
bin/ultimate-agent-stack.mjs[109-113]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`commandLock()` intends to validate exactly one **visible** `Status: APPROVED` and `Material open conflicts: NO` declaration by first stripping fenced code via `markdownOutsideFencedCode()`. The current fence detection only matches fences that start at the beginning of the line (up to 3 spaces), so a valid fence opened under a list item (e.g., `- ````) is not recognized and its contents remain “visible” to the declaration regexes.

This can allow artifacts to be locked by placing `Status: APPROVED` / `Material open conflicts: NO` inside a list-contained fenced code block with <=3-space indentation on the inner lines.

### Issue Context
- `markdownOutsideFencedCode()` opening fence detection is `^ {0,3}(`{3,}|~{3,})(.*)$`, which misses `- ```...`.
- Declaration matchers allow leading spaces (`^ {0,3}Status:`), so indented code-block lines can still match.

A concrete bypass-shaped example:
```md
- ```
 Status: APPROVED
 Material open conflicts: NO
 ```
```

### Fix Focus Areas
- bin/ultimate-agent-stack.mjs[8517-8555]
- bin/ultimate-agent-stack.mjs[8576-8605]

### Suggested fix
1. Make `markdownOutsideFencedCode()` recognize fenced code blocks that are preceded by common block prefixes (at least list markers like `- `, `* `, `+ `, and ordered list markers like `1. ` / `1) `) and/or blockquote markers.
  - One approach: derive a `probeLine` for fence detection by stripping an optional list prefix (and optional blockquote prefix) before applying the fence regex.
  - Ensure closing fences are detected using the same prefix-stripping logic.
2. Add a regression test that constructs an artifact containing the example above and asserts `lock` rejects it unless there is a real top-level `Status: APPROVED` and `Material open conflicts: NO` outside any fenced code blocks.
3. Keep the current “invalid backtick in info string” safety behavior, but apply it after prefix-stripping so it also protects list-contained fences.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Test fixtures shipped in tarball 📜 Skill insight ⛨ Security
Description
The PR explicitly ships evaluation fixtures/materializer code in the npm package, which constitutes
test artifacts in the published tarball. This increases the risk of publishing unintended
internal/testing content and violates the tarball-content restriction.
Code

package.json[29]

+    "scripts/skill-fixture.mjs",
Relevance

●● Moderate

Repo enforces strict tarball allowlist/packed-smoke, but no precedent rejecting shipped fixture
tooling specifically.

PR-#15
PR-#9
PR-#10

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
PR Compliance ID 2395845 forbids publishing test artifacts in the npm tarball. The change adds/ships
fixture tooling (scripts/skill-fixture.mjs) via the files whitelist and enforces its inclusion
(along with evals/fixture-baselines.json and evals/fixtures.json) in the packed tarball via
packed-smoke checks.

package.json[29-29]
scripts/packed-smoke.mjs[77-89]
Skill: maintain-agent-stack

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The npm package tarball is being forced to include evaluation/test artifacts (fixtures + fixture materializer), which violates the requirement that published tarballs must not include test artifacts or unrelated files.

## Issue Context
`package.json` includes `scripts/skill-fixture.mjs` in the published `files` list, and `scripts/packed-smoke.mjs` asserts that `npm pack` must include `evals/fixture-baselines.json`, `evals/fixtures.json`, and `scripts/skill-fixture.mjs`.

## Fix Focus Areas
- package.json[9-49]
- scripts/packed-smoke.mjs[74-90]
- evals/fixture-baselines.json[1-135]
- evals/fixtures.json[1-40]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

3. INVALID status semantic conflict ✓ Resolved 🐞 Bug ≡ Correctness ⭐ New
Description
scripts/skill-eval.mjs allows scenario expectations to specify artifact status "INVALID", but
validateRunRecord always emits a finding when any observed artifact has status "INVALID" and any
finding makes the scenario fail. This makes "INVALID" simultaneously an allowed schema value and an
unconditional validation failure, creating confusing and potentially unsatisfiable scenario
contracts.
Code

scripts/skill-eval.mjs[R872-878]

+        for (const artifact of asArray(observed.artifacts)) {
+          if (artifact?.status === "INVALID") {
+            findings.push(
+              `noncanonical artifact status was observed: ${artifact.path}`,
+            );
+          }
+        }
Relevance

●●● Strong

Team fixes contract/validator mismatches; aligned schema/docs to runtime in PR #27 and tightened
evaluator tests in PR #24.

PR-#27
PR-#24

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The evaluator’s allowed status set includes INVALID and is reused to validate scenario expectations,
but the run-record validator always treats observed INVALID as a finding; since scenario success is
defined as having zero findings, this turns an “allowed” value into an unconditional failure state.

scripts/skill-eval.mjs[37-51]
scripts/skill-eval.mjs[521-535]
scripts/skill-eval.mjs[860-878]
scripts/skill-eval.mjs[990-1011]
scripts/skill-eval.mjs[1044-1049]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`validateScenarioCatalog()` validates `expected.required_artifact_states` via `validateArtifactStates()`, which accepts `status: "INVALID"` because `ARTIFACT_STATUSES` includes it. Later, `validateRunRecord()` unconditionally adds a finding whenever an observed artifact has `status === "INVALID"`, and any finding causes the scenario to fail. This creates contradictory semantics for `INVALID` and makes any scenario that might want to expect `INVALID` unsatisfiable.

### Issue Context
- `INVALID` appears to represent an error/noncanonical artifact state.
- The schema/validation should make it clear whether `INVALID` is:
 - a valid *expected* state, or
 - an internal/error state that can be *observed* but should always fail.

### Fix Focus Areas
- scripts/skill-eval.mjs[40-51]
- scripts/skill-eval.mjs[530-535]
- scripts/skill-eval.mjs[872-878]
- scripts/skill-eval.mjs[996-1011]
- scripts/skill-eval.mjs[1044-1049]

### Suggested fix
Choose one of these consistent policies:
1) **Recommended:** Keep allowing `INVALID` in observed run records (so parsing can fail closed with a clear reason), but **reject `INVALID` inside `expected.required_artifact_states`** during scenario catalog validation (e.g., add a check after `validateArtifactStates()` to error if any expected artifact has status `INVALID`).

2) Alternatively, if `INVALID` should be a fully supported status:
- Remove the unconditional finding for observed `INVALID`, or only add it when the scenario did *not* explicitly expect `INVALID` for that artifact path.

Implementing (1) is simplest and makes the contract unambiguous.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


4. Run record schema drift ✓ Resolved 🐞 Bug ☼ Reliability
Description
The evaluator now unconditionally requires observed.source_claim_dispositions to be an array, but
run records still declare schema_version: 1, so older schema-v1 run records that omit this new
field will fail validation. This is a backwards-incompatible schema change without a version bump or
compatibility handling.
Code

scripts/skill-eval.mjs[R593-597]

+        validateSourceClaimDispositions(
+          observed.source_claim_dispositions,
+          "source_claim_dispositions",
+          findings,
+        );
Relevance

●●● Strong

They enforce strict validation/contract consistency in skill-eval; likely bump schema or allow
missing field.

PR-#24

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
validateRunRecord still requires schema_version 1, but it now calls
validateSourceClaimDispositions unconditionally, and that validator errors when the field is
missing because it requires an array. The scaffold remains schema_version 1 even though it now
includes the new field, confirming the schema contract changed without versioning.

scripts/skill-eval.mjs[475-482]
scripts/skill-eval.mjs[281-306]
scripts/skill-eval.mjs[588-597]
scripts/skill-eval.mjs[773-797]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`scripts/skill-eval.mjs` now treats `observed.source_claim_dispositions` as required (must be an array) during `validateRunRecord`, but `record.schema_version` is still required to equal `1`. This breaks existing schema-v1 run records that were valid before this PR.

## Issue Context
- `validateRunRecord` enforces `schema_version === 1`.
- `validateSourceClaimDispositions` rejects `undefined`.
- The new validation call is unconditional.

## Fix Focus Areas
- scripts/skill-eval.mjs[475-482]
- scripts/skill-eval.mjs[281-306]
- scripts/skill-eval.mjs[588-597]
- scripts/skill-eval.mjs[773-797]

### Implementation direction
Choose one compatible strategy:
1) **Bump schema version** (e.g., to 2) and:
  - Update scaffold to emit `schema_version: 2`.
  - Validate `source_claim_dispositions` only for v2.
  - Optionally provide a small migration path for v1 → v2.

2) **Keep schema_version 1** but make the new field **optional** for v1:
  - Treat missing `observed.source_claim_dispositions` as `[]`.
  - Only enforce claim disposition constraints when the field is present.

Either approach restores compatibility for existing v1 run records while allowing the new checks for updated harness output.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


5. Lock checks preamble-only ✓ Resolved 🐞 Bug ≡ Correctness
Description
commandLock only checks for Status: DRAFT and Material open conflicts: YES in the text before
the first ## heading, so moving/misplacing those declarations below the first section header will
allow locking artifacts that should be rejected. This makes the new “reject drafts/conflicts before
lock mutation” behavior depend on a formatting convention rather than being enforced reliably.
Code

bin/ultimate-agent-stack.mjs[R8529-8542]

+    const content = readFileSync(file, "utf8");
+    const firstSection = content.search(/^##(?:\s|$)/m);
+    const declarations =
+      firstSection === -1 ? content : content.slice(0, firstSection);
+    if (DRAFT_ARTIFACT_STATUS.test(declarations)) {
+      throw new StackError(
+        `Cannot lock ${artifact}; artifact status is DRAFT`,
+      );
+    }
+    if (OPEN_MATERIAL_CONFLICTS.test(declarations)) {
+      throw new StackError(
+        `Cannot lock ${artifact}; material open conflicts remain`,
+      );
+    }
Relevance

●● Moderate

Repo likes fail-closed parsing, but no history on lock preamble/heading conventions; might be
intentional.

PR-#21
PR-#33

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The lock logic explicitly slices content to declarations (everything before the first ##) and
only runs the DRAFT/open-conflict regexes on that slice. The added test demonstrates that a later
occurrence of Status: DRAFT after a ## heading is intentionally ignored and lock succeeds,
proving that post-heading markers are not checked at all.

bin/ultimate-agent-stack.mjs[8529-8542]
test/agent-stack.test.mjs[841-863]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`commandLock` only scans the artifact “declarations” preamble (content before the first `##`) for `Status: DRAFT` and `Material open conflicts: YES`. If those declarations are moved below the first section header (or otherwise misplaced), the lock will not reject drafts/conflicts.

## Issue Context
This PR adds lock-time rejection for draft artifacts and material open conflicts. The implementation currently slices `declarations` based on the first `##` heading, which means the rejection is not enforced if the declarations are not located in the preamble.

## Fix Focus Areas
- bin/ultimate-agent-stack.mjs[8529-8542]

### Implementation direction
Pick one:
1) **Enforce header placement**: require that (for artifacts that contain these declarations) `Status:` and `Material open conflicts:` appear in the preamble before the first `##` (exactly once). If missing from the preamble, fail the lock with an explicit error telling the user to move them back to the top.
2) **Whole-file scan with exclusions**: scan the whole file for these markers but ignore fenced code blocks (```…```), so historical examples don’t trip the guard.

Either approach prevents accidental/malicious bypass while preserving the test intent of not matching historical examples.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Informational

6. Brittle doc phrase assertions 🐞 Bug ⚙ Maintainability
Description
test/maintenance.test.mjs now enforces exact, case-sensitive substrings for the “two harnesses”
release policy across multiple Markdown files, so harmless copy edits (capitalization/wording
tweaks) will fail CI and require test updates. If the intent is to make that wording contractual,
the test should say so explicitly; otherwise it should normalize case or relax the match.
Code

test/maintenance.test.mjs[R629-638]

+  for (const source of [readme, behavioralEvals, releaseGuide, trustGuide]) {
+    const normalized = source.replace(/\s+/g, " ");
+    assert.match(
+      normalized,
+      /at least two distinct primary supported harnesses/,
+    );
+    assert.match(
+      normalized,
+      /No named harness is privileged by this (?:release )?rule/,
+    );
Relevance

●●● Strong

Team previously accepted loosening brittle maintenance-test doc/prose regexes to avoid harmless
rewording failures (PR #29, #31).

PR-#29
PR-#31

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The new test normalizes whitespace but then uses literal, case-sensitive regexes that require exact
substrings to be present in each referenced doc; any capitalization/wording change will fail these
assert.match checks.

test/maintenance.test.mjs[619-645]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`test/maintenance.test.mjs` asserts specific, case-sensitive policy phrases across multiple docs. This makes documentation edits unnecessarily coupled to test updates.

### Issue Context
These assertions are intended to ensure policy consistency, but as written they fail for benign capitalization/wording changes.

### Fix Focus Areas
- test/maintenance.test.mjs[629-642]

### Suggested fix
- If the exact wording is *not* meant to be contractual: normalize case (e.g., `normalized.toLowerCase()`) and/or use case-insensitive regexes (add `/i`) and match more semantically stable tokens.
- If the exact wording *is* meant to be contractual: add an inline comment explaining that these phrases are intentionally required verbatim so future edits don’t look like accidental brittleness.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

To customize comments, go to the Qodo configuration screen, or learn more in the docs.

Previous review results

Review updated until commit d33a2dc

Results up to commit 9bcfaa2 ⚖️ Balanced


🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)


Remediation recommended
1. Run record schema drift ✓ Resolved 🐞 Bug ☼ Reliability
Description
The evaluator now unconditionally requires observed.source_claim_dispositions to be an array, but
run records still declare schema_version: 1, so older schema-v1 run records that omit this new
field will fail validation. This is a backwards-incompatible schema change without a version bump or
compatibility handling.
Code

scripts/skill-eval.mjs[R593-597]

+        validateSourceClaimDispositions(
+          observed.source_claim_dispositions,
+          "source_claim_dispositions",
+          findings,
+        );
Relevance

●●● Strong

They enforce strict validation/contract consistency in skill-eval; likely bump schema or allow
missing field.

PR-#24

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
validateRunRecord still requires schema_version 1, but it now calls
validateSourceClaimDispositions unconditionally, and that validator errors when the field is
missing because it requires an array. The scaffold remains schema_version 1 even though it now
includes the new field, confirming the schema contract changed without versioning.

scripts/skill-eval.mjs[475-482]
scripts/skill-eval.mjs[281-306]
scripts/skill-eval.mjs[588-597]
scripts/skill-eval.mjs[773-797]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`scripts/skill-eval.mjs` now treats `observed.source_claim_dispositions` as required (must be an array) during `validateRunRecord`, but `record.schema_version` is still required to equal `1`. This breaks existing schema-v1 run records that were valid before this PR.

## Issue Context
- `validateRunRecord` enforces `schema_version === 1`.
- `validateSourceClaimDispositions` rejects `undefined`.
- The new validation call is unconditional.

## Fix Focus Areas
- scripts/skill-eval.mjs[475-482]
- scripts/skill-eval.mjs[281-306]
- scripts/skill-eval.mjs[588-597]
- scripts/skill-eval.mjs[773-797]

### Implementation direction
Choose one compatible strategy:
1) **Bump schema version** (e.g., to 2) and:
  - Update scaffold to emit `schema_version: 2`.
  - Validate `source_claim_dispositions` only for v2.
  - Optionally provide a small migration path for v1 → v2.

2) **Keep schema_version 1** but make the new field **optional** for v1:
  - Treat missing `observed.source_claim_dispositions` as `[]`.
  - Only enforce claim disposition constraints when the field is present.

Either approach restores compatibility for existing v1 run records while allowing the new checks for updated harness output.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Lock checks preamble-only ✓ Resolved 🐞 Bug ≡ Correctness
Description
commandLock only checks for Status: DRAFT and Material open conflicts: YES in the text before
the first ## heading, so moving/misplacing those declarations below the first section header will
allow locking artifacts that should be rejected. This makes the new “reject drafts/conflicts before
lock mutation” behavior depend on a formatting convention rather than being enforced reliably.
Code

bin/ultimate-agent-stack.mjs[R8529-8542]

+    const content = readFileSync(file, "utf8");
+    const firstSection = content.search(/^##(?:\s|$)/m);
+    const declarations =
+      firstSection === -1 ? content : content.slice(0, firstSection);
+    if (DRAFT_ARTIFACT_STATUS.test(declarations)) {
+      throw new StackError(
+        `Cannot lock ${artifact}; artifact status is DRAFT`,
+      );
+    }
+    if (OPEN_MATERIAL_CONFLICTS.test(declarations)) {
+      throw new StackError(
+        `Cannot lock ${artifact}; material open conflicts remain`,
+      );
+    }
Relevance

●● Moderate

Repo likes fail-closed parsing, but no history on lock preamble/heading conventions; might be
intentional.

PR-#21
PR-#33

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The lock logic explicitly slices content to declarations (everything before the first ##) and
only runs the DRAFT/open-conflict regexes on that slice. The added test demonstrates that a later
occurrence of Status: DRAFT after a ## heading is intentionally ignored and lock succeeds,
proving that post-heading markers are not checked at all.

bin/ultimate-agent-stack.mjs[8529-8542]
test/agent-stack.test.mjs[841-863]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`commandLock` only scans the artifact “declarations” preamble (content before the first `##`) for `Status: DRAFT` and `Material open conflicts: YES`. If those declarations are moved below the first section header (or otherwise misplaced), the lock will not reject drafts/conflicts.

## Issue Context
This PR adds lock-time rejection for draft artifacts and material open conflicts. The implementation currently slices `declarations` based on the first `##` heading, which means the rejection is not enforced if the declarations are not located in the preamble.

## Fix Focus Areas
- bin/ultimate-agent-stack.mjs[8529-8542]

### Implementation direction
Pick one:
1) **Enforce header placement**: require that (for artifacts that contain these declarations) `Status:` and `Material open conflicts:` appear in the preamble before the first `##` (exactly once). If missing from the preamble, fail the lock with an explicit error telling the user to move them back to the top.
2) **Whole-file scan with exclusions**: scan the whole file for these markers but ignore fenced code blocks (```…```), so historical examples don’t trip the guard.

Either approach prevents accidental/malicious bypass while preserving the test intent of not matching historical examples.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Results up to commit a31f58c ⚖️ Balanced


🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (1)


Action required
1. Lock fence detection bypass ✓ Resolved 🐞 Bug ⛨ Security
Description
commandLock uses markdownOutsideFencedCode to exclude fenced-code content before scanning for
visible Status: / Material open conflicts: declarations, but markdownOutsideFencedCode only
detects fences that begin with up to 3 leading spaces. A fence opened with a list marker (e.g., `-
``...) is not detected, so indented lines inside that code block (e.g.,   Status: APPROVED) can
be counted as “visible” declarations and allow locking artifacts without real visible approvals.
Code

bin/ultimate-agent-stack.mjs[R8520-8534]

+  for (const line of content.split(/\r?\n/)) {
+    if (fence) {
+      const closing = line.match(/^ {0,3}(`+|~+)[ \t]*$/);
+      if (
+        closing &&
+        closing[1][0] === fence.character &&
+        closing[1].length >= fence.length
+      ) {
+        fence = null;
+      }
+      visible.push("");
+      continue;
+    }
+    const opening = line.match(/^ {0,3}(`{3,}|~{3,})(.*)$/);
+    if (opening) {
Relevance

●●● Strong

Team repeatedly accepts security hardening and fail-closed parsing/regex fixes
(indentation/TOCTOU/symlink patterns).

PR-#21
PR-#30
PR-#23

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The fence detector only matches lines starting with up to three spaces before the fence, which does
not match list-introduced fences like - `...; commandLock() then scans the resulting text with
declaration regexes that allow up to three leading spaces and treats matches as “visible”
declarations.

bin/ultimate-agent-stack.mjs[8517-8555]
bin/ultimate-agent-stack.mjs[8576-8605]
bin/ultimate-agent-stack.mjs[109-113]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`commandLock()` intends to validate exactly one **visible** `Status: APPROVED` and `Material open conflicts: NO` declaration by first stripping fenced code via `markdownOutsideFencedCode()`. The current fence detection only matches fences that start at the beginning of the line (up to 3 spaces), so a valid fence opened under a list item (e.g., `- ````) is not recognized and its contents remain “visible” to the declaration regexes.

This can allow artifacts to be locked by placing `Status: APPROVED` / `Material open conflicts: NO` inside a list-contained fenced code block with <=3-space indentation on the inner lines.

### Issue Context
- `markdownOutsideFencedCode()` opening fence detection is `^ {0,3}(`{3,}|~{3,})(.*)$`, which misses `- ```...`.
- Declaration matchers allow leading spaces (`^ {0,3}Status:`), so indented code-block lines can still match.

A concrete bypass-shaped example:
```md
- ```
 Status: APPROVED
 Material open conflicts: NO
 ```
```

### Fix Focus Areas
- bin/ultimate-agent-stack.mjs[8517-8555]
- bin/ultimate-agent-stack.mjs[8576-8605]

### Suggested fix
1. Make `markdownOutsideFencedCode()` recognize fenced code blocks that are preceded by common block prefixes (at least list markers like `- `, `* `, `+ `, and ordered list markers like `1. ` / `1) `) and/or blockquote markers.
  - One approach: derive a `probeLine` for fence detection by stripping an optional list prefix (and optional blockquote prefix) before applying the fence regex.
  - Ensure closing fences are detected using the same prefix-stripping logic.
2. Add a regression test that constructs an artifact containing the example above and asserts `lock` rejects it unless there is a real top-level `Status: APPROVED` and `Material open conflicts: NO` outside any fenced code blocks.
3. Keep the current “invalid backtick in info string” safety behavior, but apply it after prefix-stripping so it also protects list-contained fences.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Test fixtures shipped in tarball 📜 Skill insight ⛨ Security
Description
The PR explicitly ships evaluation fixtures/materializer code in the npm package, which constitutes
test artifacts in the published tarball. This increases the risk of publishing unintended
internal/testing content and violates the tarball-content restriction.
Code

package.json[29]

+    "scripts/skill-fixture.mjs",
Relevance

●● Moderate

Repo enforces strict tarball allowlist/packed-smoke, but no precedent rejecting shipped fixture
tooling specifically.

PR-#15
PR-#9
PR-#10

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
PR Compliance ID 2395845 forbids publishing test artifacts in the npm tarball. The change adds/ships
fixture tooling (scripts/skill-fixture.mjs) via the files whitelist and enforces its inclusion
(along with evals/fixture-baselines.json and evals/fixtures.json) in the packed tarball via
packed-smoke checks.

package.json[29-29]
scripts/packed-smoke.mjs[77-89]
Skill: maintain-agent-stack

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The npm package tarball is being forced to include evaluation/test artifacts (fixtures + fixture materializer), which violates the requirement that published tarballs must not include test artifacts or unrelated files.

## Issue Context
`package.json` includes `scripts/skill-fixture.mjs` in the published `files` list, and `scripts/packed-smoke.mjs` asserts that `npm pack` must include `evals/fixture-baselines.json`, `evals/fixtures.json`, and `scripts/skill-fixture.mjs`.

## Fix Focus Areas
- package.json[9-49]
- scripts/packed-smoke.mjs[74-90]
- evals/fixture-baselines.json[1-135]
- evals/fixtures.json[1-40]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Results up to commit a78b870 ⚖️ Balanced


🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)


Informational
1. Brittle doc phrase assertions 🐞 Bug ⚙ Maintainability
Description
test/maintenance.test.mjs now enforces exact, case-sensitive substrings for the “two harnesses”
release policy across multiple Markdown files, so harmless copy edits (capitalization/wording
tweaks) will fail CI and require test updates. If the intent is to make that wording contractual,
the test should say so explicitly; otherwise it should normalize case or relax the match.
Code

test/maintenance.test.mjs[R629-638]

+  for (const source of [readme, behavioralEvals, releaseGuide, trustGuide]) {
+    const normalized = source.replace(/\s+/g, " ");
+    assert.match(
+      normalized,
+      /at least two distinct primary supported harnesses/,
+    );
+    assert.match(
+      normalized,
+      /No named harness is privileged by this (?:release )?rule/,
+    );
Relevance

●●● Strong

Team previously accepted loosening brittle maintenance-test doc/prose regexes to avoid harmless
rewording failures (PR #29, #31).

PR-#29
PR-#31

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The new test normalizes whitespace but then uses literal, case-sensitive regexes that require exact
substrings to be present in each referenced doc; any capitalization/wording change will fail these
assert.match checks.

test/maintenance.test.mjs[619-645]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`test/maintenance.test.mjs` asserts specific, case-sensitive policy phrases across multiple docs. This makes documentation edits unnecessarily coupled to test updates.

### Issue Context
These assertions are intended to ensure policy consistency, but as written they fail for benign capitalization/wording changes.

### Fix Focus Areas
- test/maintenance.test.mjs[629-642]

### Suggested fix
- If the exact wording is *not* meant to be contractual: normalize case (e.g., `normalized.toLowerCase()`) and/or use case-insensitive regexes (add `/i`) and match more semantically stable tokens.
- If the exact wording *is* meant to be contractual: add an inline comment explaining that these phrases are intentionally required verbatim so future edits don’t look like accidental brittleness.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Qodo Logo

Comment thread bin/ultimate-agent-stack.mjs
Comment thread scripts/skill-eval.mjs
@samtay32

Copy link
Copy Markdown
Owner Author

/improve

Comment thread package.json
Comment thread bin/ultimate-agent-stack.mjs
@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit a31f58c

@samtay32

Copy link
Copy Markdown
Owner Author

Qodo disposition evidence for the current head:

  • fixedLock fence detection bypass. markdownFenceProbeLine() now normalizes nested blockquote, unordered-list, and ordered-list container prefixes before both opening- and closing-fence detection. Adversarial list and blockquote fixtures prove declarations inside those fences cannot satisfy lock approval. The lock test, all 185 tests on Node 22 and Node 24, release packaging, packed smoke, baseline regeneration, Markdown lint, and git diff --check pass.
  • rebuttedTest fixtures shipped in tarball. The packaged files are not incidental test output or private fixtures. They are the bounded maintainer-facing behavioral-evidence interface: the installed package exposes eval:fixture, eval:contracts, eval:scaffold, and eval:behavior; canonical synthetic fixtures and the materializer are required so an installed release can reproduce, inspect, and validate live harness evidence. The explicit files allowlist and packed-smoke assertions prevent accidental contents, and the actual test/ directory remains excluded. Removing these files would make the advertised installed evaluation commands incomplete rather than reduce a runtime risk.

Final behavior surface: sha256:a1456002b770f46aa1f3cbc9b700d5757e1698d009a7d98a7bbcf313fb08be1e.

@samtay32

Copy link
Copy Markdown
Owner Author

/improve

@qodo-code-review

Copy link
Copy Markdown

Qodo disposition evidence for the current head:

  • fixedLock fence detection bypass. markdownFenceProbeLine() now normalizes nested blockquote, unordered-list, and ordered-list container prefixes before both opening- and closing-fence detection. Adversarial list and blockquote fixtures prove declarations inside those fences cannot satisfy lock approval. The lock test, all 185 tests on Node 22 and Node 24, release packaging, packed smoke, baseline regeneration, Markdown lint, and git diff --check pass.
  • rebuttedTest fixtures shipped in tarball. The packaged files are not incidental test output or private fixtures. They are the bounded maintainer-facing behavioral-evidence interface: the installed package exposes eval:fixture, eval:contracts, eval:scaffold, and eval:behavior; canonical synthetic fixtures and the materializer are required so an installed release can reproduce, inspect, and validate live harness evidence. The explicit files allowlist and packed-smoke assertions prevent accidental contents, and the actual test/ directory remains excluded. Removing these files would make the advertised installed evaluation commands incomplete rather than reduce a runtime risk.

Final behavior surface: sha256:a1456002b770f46aa1f3cbc9b700d5757e1698d009a7d98a7bbcf313fb08be1e.

Noted. The evidence supports the fix for finding 1, and the reported validation results are comprehensive. The rationale for rebutting finding 2 is clear; it remains active because no explicit dismissal was requested.

@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 0426f6d

@samtay32

Copy link
Copy Markdown
Owner Author

Disposition: fixed

Claim validation: the prior nested-container fence repair still allowed an apparent list/blockquote fence marker inside an already-open outer fence to be mistaken for the outer closing fence. That could expose later declarations to the lock scanner.

Evidence on exact head 0c1870d25f6d9647d40b22bccdf29bb907f16121:

  • closing fences are now recognized only from the raw line once a fence is open; newly appearing list/blockquote markers are never stripped inside that fence;
  • list- and blockquote-contained fence openings fail closed with an explicit nested-fence error;
  • adversarial regressions cover both nested openings and - ````` / > ````` fake-closing patterns inside an unclosed outer fence;
  • docs/TRUST.md states the nested-fence compatibility boundary;
  • Node 22 release:check passed 185 tests, behavioral contracts, dry pack, and packed smoke (115 files);
  • Node 24 passed all 185 tests;
  • all 63 Markdown files passed markdownlint;
  • protected fixture baselines reproduce exactly and git diff --check passes.

The prior optional observation about packaged fixture tooling remains rebutted: these files intentionally implement the shipped maintainer/evidence runtime commands and are protected by packed-smoke and baseline tests; package-internal test/ files remain excluded.

@samtay32

Copy link
Copy Markdown
Owner Author

/improve

@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 0c1870d

@samtay32

Copy link
Copy Markdown
Owner Author

Disposition: fixed

Exact-head follow-up for a78b870225e3b417075af6691efadc0c7082eb5f:

  • release evidence now requires any two distinct primary supported harnesses; no named harness is privileged;
  • README, release guide, trust guide, and behavioral-evaluation guide use the same rule;
  • a maintenance regression mechanically checks all four sources and scopes the untested list to supported harnesses;
  • the behavior surface remains sha256:133bbe7b0d1c506e148ba2ca3798246e106a7dfd87fbb321926a93d020be265f;
  • Node 22 full release:check remains clean at 185 tests plus contracts, pack, and packed smoke; the final policy regression and Markdown checks pass.

Disposition: rebutted — please dismiss the remaining “Test fixtures shipped in tarball” skill insight. The packaged fixture catalog and materializer are deliberate installed maintainer/evidence runtime commands, covered by the explicit package allowlist and packed smoke; the actual test/ directory is excluded. This is not an unintended test artifact or production security risk.

@samtay32

Copy link
Copy Markdown
Owner Author

/improve

Comment thread test/maintenance.test.mjs
@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit a78b870

Copy link
Copy Markdown
Owner Author

/improve

Comment thread scripts/skill-eval.mjs
@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 32bc89f

@samtay32

Copy link
Copy Markdown
Owner Author

/improve

@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit d33a2dc

@samtay32
samtay32 marked this pull request as ready for review July 30, 2026 01:15
@samtay32
samtay32 merged commit 6954433 into main Jul 30, 2026
11 of 12 checks passed
@samtay32
samtay32 deleted the codex/flexible-intake-brief branch July 30, 2026 01:18
@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit d33a2dc

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Add flexible intake flow and working project briefs

✨ Enhancement 🧪 Tests 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

Grey Divider

AI Description

• Add develop-project-brief skill and unlocked BRIEF.md for EXTERNAL/DISCOVER intake.
• Update start prompts to route RESUME→EXTERNAL→DISCOVER→DIRECT and avoid false activation.
• Harden artifact locking and add deterministic fixture-based behavioral contracts.
Diagram

graph TD
  U["User request"] --> CLI["ultimate-agent-stack CLI\n(start/lock)"] --> R["Intake routing\nRESUME/EXTERNAL/DISCOVER/DIRECT"] --> B["Working BRIEF.md\n+ develop-project-brief"] --> P["shape-project\n(promotion)"] --> D["Delivery skills\n(build/verify/review)"]
  H[("Evals harness\nfixtures + contracts")] --> CLI
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Fold brief development into run-autonomous-delivery only
  • ➕ Fewer top-level skills and entry points
  • ➕ Single controller owns all routing decisions
  • ➖ Harder to guarantee DIRECT/explanation-only paths don’t trigger brief workflows
  • ➖ Increases risk of creating a second delivery state machine instead of a bounded pre-step
2. Use a structured brief format (JSON/YAML) instead of BRIEF.md
  • ➕ Machine-validated schema for claim disposition/conflict tracking
  • ➕ Less ambiguity around parsing vs Markdown formatting
  • ➖ Worse ergonomics for non-coders and manual review
  • ➖ Requires additional tooling/migration in a Markdown-first artifact ecosystem
3. Soft-lock with warnings instead of fail-closed guards
  • ➕ Lower friction for teams iterating quickly
  • ➕ Allows locking even with known gaps
  • ➖ Undermines the PR’s explicit authority model (DRAFT/conflicts/placeholders must block locks)
  • ➖ Higher likelihood of promoting contradictory or incomplete intent as “locked”

Recommendation: The PR’s approach—explicit develop-project-brief skill + unlocked BRIEF.md, followed by an explicit promotion step—is the right separation to keep intake flexible without expanding delivery into a second state machine. The stricter lock guards and deterministic fixtures are appropriate given the stated goal of fail-closed authority/evidence handling; the main alternative (folding into run-autonomous-delivery) would make false-activation avoidance materially harder.

Files changed (48) +7677 / -250

Enhancement (3) +387 / -10
BRIEF.mdAdd unlocked working BRIEF.md template +103/-0

Add unlocked working BRIEF.md template

• Introduces a new working brief artifact with required provenance, claim ledger, contradictions/gaps, and promotion checklist sections. Defaults to 'Status: DRAFT' and 'Material open conflicts: YES'.

assets/project-template/.agent-stack/artifacts/BRIEF.md

ultimate-agent-stack.mjsAdd deterministic intake policy prompts and stronger lock guards +196/-10

Add deterministic intake policy prompts and stronger lock guards

• Refactors start prompts into dedicated builders and adds deterministic routing guidance (RESUME→EXTERNAL→DISCOVER→DIRECT), including brief-only and false-activation boundaries. Hardens artifact locking by enforcing visible 'Status: APPROVED' and 'Material open conflicts: NO', rejecting DRAFT/conflicts, and failing closed on nested/invalid/unclosed Markdown fences that would make declarations ambiguous.

bin/ultimate-agent-stack.mjs

SKILL.mdAdd develop-project-brief skill definition +88/-0

Add develop-project-brief skill definition

• Adds a new skill that creates/audits an unlocked working BRIEF.md for EXTERNAL/DISCOVER intake, including provenance, claim disposition, conflict closure, and explicit approval requirements before promotion.

skills/develop-project-brief/SKILL.md

Tests (3) +2283 / -4
packed-smoke.mjsValidate packed tarball includes fixtures and can run contracts +216/-1

Validate packed tarball includes fixtures and can run contracts

• Extends packed-install smoke tests to ensure required new files are included and to validate that contracts and fixture catalogs are runnable/consistent from the packed distribution.

scripts/packed-smoke.mjs

skill-eval.mjsBind behavior surface to prompt policy and fixture catalogs +685/-3

Bind behavior surface to prompt policy and fixture catalogs

• Extends the behavior surface inputs (including new templates/fixtures/tooling) and adds start-prompt policy surface capture used in hashing and validation. Tightens evaluation validation around receipts and observable policy state.

scripts/skill-eval.mjs

skill-fixture.mjsAdd deterministic fixture generator CLI +1382/-0

Add deterministic fixture generator CLI

• Introduces a new CLI to materialize fixtures for scenarios deterministically, scrub sensitive environment variables, compute receipts/hashes, and propose baselines for fixture verification.

scripts/skill-fixture.mjs

Documentation (27) +1190 / -184
CHANGELOG.mdChangelog entry for flexible intake + evidence/locking +75/-0

Changelog entry for flexible intake + evidence/locking

• Adds a detailed release note documenting working briefs, deterministic routing, lock hardening, and fixture-backed behavioral validation.

CHANGELOG.md

README.mdDocument the working brief and intake routing behavior +82/-25

Document the working brief and intake routing behavior

• Updates user-facing documentation to explain the new intake modes, when BRIEF.md is used, and how promotion/approval interacts with lockable artifacts.

README.md

STARTER_PROMPT.mdAlign starter prompt with new intake and promotion model +29/-20

Align starter prompt with new intake and promotion model

• Updates the starter prompt to match the RESUME→EXTERNAL→DISCOVER→DIRECT routing guidance and the brief-only vs delivery separation.

STARTER_PROMPT.md

HANDOFF.mdUpdate template handoff for working-brief workflows +29/-19

Update template handoff for working-brief workflows

• Adjusts the handoff instructions to incorporate the working brief, provenance handling, and explicit approval/promotion boundaries.

assets/project-template/.agent-stack/HANDOFF.md

DECISIONS.mdMake decisions log promotion-aware and lock-gatable +21/-5

Make decisions log promotion-aware and lock-gatable

• Adds Status/conflict declarations and restructures the file to clearly record closed decisions and later intent changes. Positions DECISIONS.md as the canonical post-promotion record.

assets/project-template/.agent-stack/artifacts/DECISIONS.md

DELIVERY.mdTrack working brief linkage and conflict state +3/-0

Track working brief linkage and conflict state

• Adds 'Material open conflicts' and a 'Working brief' field to make the delivery contract explicitly reference the working-brief path when applicable.

assets/project-template/.agent-stack/artifacts/DELIVERY.md

AGENTS.mdUpdate template agent guidance for flexible intake +66/-2

Update template agent guidance for flexible intake

• Expands the project template instructions to reflect the new intake routing, when to use develop-project-brief, and how approval/promotion relates to locked artifacts.

assets/project-template/AGENTS.md

GEMINI.mdAlign Gemini template guidance with provider-neutral intake +8/-3

Align Gemini template guidance with provider-neutral intake

• Updates the Gemini template docs to match the revised intake flow and provider-neutral onboarding posture.

assets/project-template/GEMINI.md

ARCHITECTURE.mdDescribe intake/brief/promotion architecture +68/-10

Describe intake/brief/promotion architecture

• Updates architecture docs to explain how the working brief fits into shaping and delivery, and how routing decisions avoid false activation.

docs/ARCHITECTURE.md

BEHAVIORAL_EVALS.mdExpand behavioral eval documentation for the new surface +246/-14

Expand behavioral eval documentation for the new surface

• Documents the expanded behavioral contracts and deterministic fixtures/baselines that validate intake routing, evidence handling, and provider neutrality.

docs/BEHAVIORAL_EVALS.md

OPERATING_MANUAL.mdUpdate operating guidance for intake and approval boundaries +92/-10

Update operating guidance for intake and approval boundaries

• Clarifies operator workflow for flexible intake, evidence expectations, and the separation between recommendation and approval/promotion.

docs/OPERATING_MANUAL.md

RELEASE.mdUpdate release guidance for fixture-backed behavior checks +25/-0

Update release guidance for fixture-backed behavior checks

• Adjusts release documentation to reflect the expanded behavioral surface and packed-install validations.

docs/RELEASE.md

SKILL_STACK.mdAdd develop-project-brief to the documented skill stack +44/-8

Add develop-project-brief to the documented skill stack

• Updates the skill stack docs so reviewers/operators understand when brief development is invoked vs direct delivery.

docs/SKILL_STACK.md

SOURCES_AND_TRADEOFFS.mdClarify provenance, reconciliation, and contradiction handling tradeoffs +33/-0

Clarify provenance, reconciliation, and contradiction handling tradeoffs

• Adds/updates guidance on preserving supplied sources, auditing against repository reality, and recording claim dispositions and contradictions.

docs/SOURCES_AND_TRADEOFFS.md

TRUST.mdReinforce EXTERNAL intake trust and authority constraints +39/-2

Reinforce EXTERNAL intake trust and authority constraints

• Updates trust guidance to treat supplied documents as untrusted data, keep provider usage neutral, and require fail-closed behavior on unresolved conflicts.

docs/TRUST.md

synthesis-map.svgRefresh synthesis-map diagram asset +2/-2

Refresh synthesis-map diagram asset

• Minor updates to the SVG asset to reflect the revised intake/synthesis flow.

docs/assets/synthesis-map.svg

SKILL.mdAlign build skill docs with new routing gates +4/-1

Align build skill docs with new routing gates

• Small doc updates to match the revised flow where implementation is gated and explicitly entered.

skills/build-vertical-slice/SKILL.md

SKILL.mdTighten close-review-loop activation boundaries +5/-2

Tighten close-review-loop activation boundaries

• Updates documentation to emphasize PR/review-thread preconditions, reducing false activation during non-review workflows.

skills/close-review-loop/SKILL.md

brief-contract.mdAdd brief contract reference documentation +90/-0

Add brief contract reference documentation

• Adds a reference describing required brief structure, approval semantics, and promotion expectations into lockable artifacts.

skills/develop-project-brief/references/brief-contract.md

intake-and-reconciliation.mdAdd intake and reconciliation reference documentation +68/-0

Add intake and reconciliation reference documentation

• Adds reference guidance for auditing supplied sources, reconciling with repository reality, tracking contradictions, and preventing tool-authority injection from external material.

skills/develop-project-brief/references/intake-and-reconciliation.md

SKILL.mdMinor documentation alignment for work/evidence routing +1/-1

Minor documentation alignment for work/evidence routing

• Small wording change to align with revised start prompt policy and evidence handling expectations.

skills/manage-project-work/SKILL.md

SKILL.mdUpdate delivery controller docs for intake routing and brief-only paths +51/-8

Update delivery controller docs for intake routing and brief-only paths

• Updates documentation to reflect deterministic routing and when to invoke develop-project-brief vs proceed directly to delivery.

skills/run-autonomous-delivery/SKILL.md

delivery-policy.mdRefine delivery policy for approval and routing gates +20/-0

Refine delivery policy for approval and routing gates

• Updates policy reference text to align with the new brief/promotion gates and evidence/authority constraints.

skills/run-autonomous-delivery/references/delivery-policy.md

SKILL.mdUpdate onboarding docs for repository-first recommendation policy +36/-43

Update onboarding docs for repository-first recommendation policy

• Updates onboarding documentation to reflect the repository-first recommendation and conditional reveal of advanced providers.

skills/setup-autonomous-project/SKILL.md

SKILL.mdClarify BRIEF.md promotion into canonical artifacts +23/-7

Clarify BRIEF.md promotion into canonical artifacts

• Updates shaping documentation to treat BRIEF.md as a working input and to promote intent/decisions only after explicit approval and conflict closure.

skills/shape-project/SKILL.md

shaping-contract.mdAlign shaping contract reference with working-brief promotion +19/-0

Align shaping contract reference with working-brief promotion

• Updates shaping reference to incorporate provenance/claim disposition expectations from the working brief.

skills/shape-project/references/shaping-contract.md

SKILL.mdAlign verify skill docs with explicit verification gating +11/-2

Align verify skill docs with explicit verification gating

• Updates documentation to match the explicit start prompt guidance for entering verification via verify-change.

skills/verify-change/SKILL.md

Other (15) +3817 / -52
plugin.jsonUpdate plugin metadata to reflect flexible intake +2/-1

Update plugin metadata to reflect flexible intake

• Rewrites the long description to emphasize flexible routing across vague ideas, supplied briefs, bounded requests, and resumable work. Adds a new capability entry for flexible project intake.

.codex-plugin/plugin.json

.gitattributesAdd LF normalization for deterministic fixtures +1/-0

Add LF normalization for deterministic fixtures

• Adds a repo-wide text normalization rule ('* text=auto eol=lf') to reduce cross-platform diffs and stabilize hashing/fixtures.

.gitattributes

ARCHITECTURE.mdAdd conflict declaration header to artifact template +1/-0

Add conflict declaration header to artifact template

• Adds 'Material open conflicts' to the architecture artifact template so lock gating can verify conflict closure.

assets/project-template/.agent-stack/artifacts/ARCHITECTURE.md

DELEGATION.mdAdd status/conflict declarations to delegation template +3/-0

Add status/conflict declarations to delegation template

• Adds 'Status: DRAFT' and 'Material open conflicts: YES' to align delegation artifacts with lock gating requirements.

assets/project-template/.agent-stack/artifacts/DELEGATION.md

SECURITY.mdAdd conflict declaration header to security template +1/-0

Add conflict declaration header to security template

• Adds 'Material open conflicts' to support fail-closed locking semantics for the security artifact.

assets/project-template/.agent-stack/artifacts/SECURITY.md

VERIFICATION.mdAdd conflict declaration header to verification template +1/-0

Add conflict declaration header to verification template

• Adds 'Material open conflicts' to support fail-closed locking semantics for the verification artifact.

assets/project-template/.agent-stack/artifacts/VERIFICATION.md

fixture-baselines.jsonAdd fixture baseline catalog (git head + tree hash) +140/-0

Add fixture baseline catalog (git head + tree hash)

• Introduces a committed baseline catalog mapping each scenario to an expected git head and project tree SHA-256 for determinism checks.

evals/fixture-baselines.json

fixtures.jsonAdd deterministic fixture definitions for scenarios +449/-0

Add deterministic fixture definitions for scenarios

• Adds a fixture catalog describing per-scenario project files, configuration modes, and state knobs used by behavioral evaluation.

evals/fixtures.json

scenarios.jsonUpdate scenarios to test flexible intake and evidence policies +755/-25

Update scenarios to test flexible intake and evidence policies

• Expands and adjusts behavioral scenarios to assert the new routing behavior, prevent false activation, and validate evidence/authority constraints (including provider unavailability fallbacks).

evals/scenarios.json

package.jsonShip fixture tooling and include new files in package output +4/-1

Ship fixture tooling and include new files in package output

• Adds .gitattributes and scripts/skill-fixture.mjs to the published files list, includes it in lint checks, and adds an eval:fixture command entrypoint.

package.json

openai.yamlAdd agent config for develop-project-brief +6/-0

Add agent config for develop-project-brief

• Adds OpenAI agent configuration wiring for the new skill.

skills/develop-project-brief/agents/openai.yaml

agent-stack.test.mjsUpdate prompt tests and add artifact lock rejection coverage +432/-7

Update prompt tests and add artifact lock rejection coverage

• Updates test expectations for the new repository-first onboarding recommendation and intake routing guidance. Adds coverage for stricter lock preconditions (status/conflicts/placeholders/fence safety).

test/agent-stack.test.mjs

maintenance.test.mjsUpdate maintenance tests for expanded behavior surface +355/-0

Update maintenance tests for expanded behavior surface

• Adjusts maintenance tests to account for the expanded behavior surface and new evaluation artifacts.

test/maintenance.test.mjs

skill-eval.test.mjsUpdate eval contract tests for expanded receipts/surface +665/-18

Update eval contract tests for expanded receipts/surface

• Updates tests to reflect the expanded evaluation surface and stricter validation expectations around recorded evidence and deterministic inputs.

test/skill-eval.test.mjs

skill-fixture.test.mjsAdd tests for deterministic fixture generation +1002/-0

Add tests for deterministic fixture generation

• Adds coverage for the new fixture generator, including catalog integrity, determinism/baseline matching, and safety constraints like credential scrubbing.

test/skill-fixture.test.mjs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant