feat: add bounded evidence graph reports

[samtay32]

Summary

• derive provider-neutral JSON evidence coverage from validated repository state
• render bounded Mermaid diagrams with generated aliases and sanitized labels
• constrain report writes to .agent-stack/reports and add behavioral/packed coverage

Verification

• npm run release:check (131 tests, behavioral contracts, dry pack, packed smoke)
• npx -y node@22.22.0 --test (131 tests)
• npx -y markdownlint-cli2@0.18.1 "**/*.md" "#node_modules" (58 files)
• git diff --check

[samtay32]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 15 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 87168395-74df-45c5-b20b-d86b73e51277

📥 Commits

Reviewing files that changed from the base of the PR and between 5a7f44b and 9e0c090.

📒 Files selected for processing (11)

• .codex-plugin/plugin.json
• README.md
• bin/ultimate-agent-stack.mjs
• docs/ARCHITECTURE.md
• docs/OPERATING_MANUAL.md
• docs/TRUST.md
• evals/scenarios.json
• scripts/packed-smoke.mjs
• skills/manage-project-work/SKILL.md
• test/agent-stack.test.mjs
• test/skill-eval.test.mjs

---

_{Comment @coderabbitai help to get the list of available commands.}

[samtay32]

@coderabbitai full review

[coderabbitai]

✅ Action performed

Full review finished.

---

Your included review limit is currently reached under our Fair Usage Limits Policy. Your recent PR review activity is in the 95th percentile or higher among CodeRabbit users, so adaptive limits apply. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 28 minutes.

[samtay32]

/review

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (2)

Context used

✅ Compliance rules (platform): 79 rules

✅ Skills: 4 invoked

  verify-change

  maintain-agent-stack

  build-vertical-slice

  secure-launch

1. Report directory symlink bypass ✓ Resolved 🐞 Bug ⛨ Security

Description

The output allowlist checks the lexical path, so a project-local .agent-stack/reports symlink can
redirect report writes elsewhere inside the repository. Linking that directory to the project root
allows .agent-stack/reports/package.json to pass validation and replace the real package.json.

Code

bin/ultimate-agent-stack.mjs[R3679-3681]

+    normalizedOutput = relative(realpathSync(target), outputFile)
+      .split(sep)
+      .join("/");

Evidence

projectFile rejects only symlinks resolving outside the project and returns the unresolved
candidate, while report validation derives its allowlisted path from that lexical candidate and
checks only the final file with lstatSync. The atomic helpers create and rename files through the
destination parent, so a symlinked reports directory targeting another in-project directory is
followed despite the documented reports-only boundary.

bin/ultimate-agent-stack.mjs[602-646]
bin/ultimate-agent-stack.mjs[528-551]
bin/ultimate-agent-stack.mjs[3677-3699]
README.md[279-290]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Evidence report output validation uses a lexical path and checks only whether the final file is a symlink. A symlinked `.agent-stack/reports` ancestor can therefore redirect an allowed report write onto another file inside the repository.

## Issue Context
Resolve and validate the actual destination against a canonical `.agent-stack/reports` directory immediately before writing. Reject symlinked report-directory ancestors or otherwise guarantee that atomic temporary-file creation and rename remain physically inside that directory; add a regression test using a reports-directory symlink to the project root.

## Fix Focus Areas
- bin/ultimate-agent-stack.mjs[3677-3699]
- bin/ultimate-agent-stack.mjs[602-646]
- test/agent-stack.test.mjs[1749-1775]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Unvalidated second read ✓ Resolved 🐞 Bug ☼ Reliability

Description

commandEvidenceReport validates the work ledger/evidence graph, then re-reads both JSON files and
generates the report from that second read without validating that snapshot. If the files change
between validation and the second read, the report can be inconsistent with the validated state or
throw during report/Mermaid generation.

Code

bin/ultimate-agent-stack.mjs[R3689-3710]

+  const work = commandWorkValidate(target);
+  const evidence = commandEvidenceValidate(target);
+  if (!work.ok || !evidence.ok) {
+    throw new StackError(
+      "Evidence reporting requires valid repository work and evidence.",
+      2,
+      [...work.errors, ...evidence.errors],
+    );
+  }
+  const ledger = readJson(
+    projectFile(target, WORK_LEDGER_PATH, "work ledger"),
+    "work ledger",
+  );
+  const graph = readJson(
+    projectFile(target, EVIDENCE_GRAPH_PATH, "evidence graph"),
+    "evidence graph",
+  );
+  const report = evidenceReportData(ledger, graph);
+  const visualization =
+    format === "mermaid"
+      ? evidenceMermaid(graph, numericMaxNodes)
+      : null;

Evidence

The report command performs validation but discards the validated objects and re-reads JSON for
rendering; Mermaid output interpolates node.state and edge.relation directly, which is only safe
when those fields come from validated input. The validation schema does constrain these fields,
demonstrating the safety assumptions that the second read bypasses.

bin/ultimate-agent-stack.mjs[3689-3710]
bin/ultimate-agent-stack.mjs[3605-3642]
bin/ultimate-agent-stack.mjs[2677-2775]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`commandEvidenceReport` runs validation (`commandWorkValidate` / `commandEvidenceValidate`) but then calls `readJson(...)` again and uses those objects to build the report and Mermaid output without validating that second-read snapshot.

This creates a time-of-check/time-of-use gap: if `.agent-stack/work-items.json` or `.agent-stack/evidence-graph.json` changes between the validation step and the second read, the generated output is no longer guaranteed to reflect a validated repository state and may crash while building counts/Mermaid.

### Issue Context
- Validation constrains fields like `node.state` and `edge.relation` to canonical sets, but those guarantees only apply to the validated objects.
- Mermaid rendering interpolates `node.state` and `edge.relation` directly into output lines; it relies on validated input.

### Fix Focus Areas
- bin/ultimate-agent-stack.mjs[3689-3710]

### Suggested fix
- Read the work ledger and evidence graph **once**, validate those in-memory objects, and generate the report/Mermaid from the same validated snapshot.
 - Option A: Introduce a helper like `loadValidatedWorkAndEvidence(target)` that returns `{ ledger, graph }` (or throws with collected validation errors), and use it in `commandEvidenceReport`.
 - Option B: Use `validateRepositoryContract(...)` directly in `commandEvidenceReport` to get both `value` objects and errors, then run `validateWorkEvidenceLinkage(ledger.value, graph.value)` before rendering.
- (Optional defense-in-depth) Add a small whitelist/escaping step for any Mermaid-interpolated fields that are not already sanitized, so output remains safe even if validation is bypassed or inputs are mutated mid-run.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Per-node edge cap misdocumented ✓ Resolved 🐞 Bug ≡ Correctness

Description

The manual claims each selected node is capped at four edges, but evidenceMermaid applies only one
global selected.length * 4 limit. A single node can therefore have far more than four rendered
connections, misleading operators about diagram density.

Code

docs/OPERATING_MANUAL.md[223]

+edges at four per selected node. The report counts nodes excluded by the node

Evidence

The renderer sorts all eligible edges and slices one global list at selected.length * 4; it does
not maintain per-node counters. The dense-graph test likewise verifies only the aggregate limit.

docs/OPERATING_MANUAL.md[220-225]
bin/ultimate-agent-stack.mjs[3617-3629]
test/agent-stack.test.mjs[1791-1817]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The documentation promises four rendered edges per selected node, while the implementation only caps the total edge count at four times the selected-node count.

## Issue Context
Choose the intended behavior: either document the aggregate cap precisely or enforce an individual per-node edge limit and add an asymmetric-graph test.

## Fix Focus Areas
- docs/OPERATING_MANUAL.md[222-224]
- README.md[287-292]
- docs/ARCHITECTURE.md[193-197]
- bin/ultimate-agent-stack.mjs[3617-3629]
- test/agent-stack.test.mjs[1791-1817]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Symlink check remains racy 🐞 Bug ⛨ Security

Description

The final symlink check is separated from the pathname-based write, so a concurrent actor with
workspace write access can replace a checked directory with a symlink before temporary-file creation
or rename. This can redirect a report write outside .agent-stack/reports, violating the advertised
physical-containment guarantee.

Code

bin/ultimate-agent-stack.mjs[R3731-3732]

+    mkdirSync(dirname(outputFile), { recursive: true });
+    projectFileWithoutSymlinkComponents(target, output, "evidence report");

Evidence

The helper uses existsSync and lstatSync checks but retains no trusted directory handle. After
the second check, atomicJson or atomicText creates and renames a temporary file through the same
mutable pathname, leaving a TOCTOU window despite the documentation's physical-containment claim.

bin/ultimate-agent-stack.mjs[530-578]
bin/ultimate-agent-stack.mjs[3731-3736]
docs/ARCHITECTURE.md[196-199]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Repeated pathname checks do not prevent a directory component from being replaced after validation but before the report is written. Make report creation operate relative to trusted directory handles with no-follow semantics, or otherwise use a platform-appropriate race-safe containment strategy.

## Issue Context
The attack requires a concurrent process able to mutate the report directory. A concrete interleaving replaces `.agent-stack/reports` or a descendant with a symlink after the final check and before `atomicJson` or `atomicText` creates its temporary file.

## Fix Focus Areas
- bin/ultimate-agent-stack.mjs[530-578]
- bin/ultimate-agent-stack.mjs[3713-3736]
- test/agent-stack.test.mjs[1820-1838]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

View more (1)

5. Node limit permits edge floods ✓ Resolved 🐞 Bug ➹ Performance

Description

evidenceMermaid limits nodes but emits every edge between selected nodes, allowing a valid graph
to produce as many as 50,000 Mermaid edge lines regardless of the configured node limit. Dense
graphs can therefore generate multi-megabyte reports that are impractical to render despite the
bounded-report interface.

Code

bin/ultimate-agent-stack.mjs[R3602-3606]

+    ...graph.edges
+      .filter(
+        (edge) =>
+          selectedIds.has(edge.from) && selectedIds.has(edge.to),
+      )

Evidence

Node selection is sliced by maxNodes, but all qualifying graph edges are filtered, sorted, and
emitted without another limit. Graph validation permits 50,000 edges, so that full graph-level
allowance can be concentrated among selected nodes even though the report is presented as a bounded
visualization.

bin/ultimate-agent-stack.mjs[3582-3636]
bin/ultimate-agent-stack.mjs[2669-2675]
README.md[276-290]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The Mermaid visualization applies `--max-nodes` but no proportional edge or output-size limit. A dense validated graph can emit up to the graph-wide maximum of 50,000 edge lines and produce an unusable diagram.

## Issue Context
Apply a deterministic edge cap or total output-size budget after selecting nodes. Include the omitted-edge count in the Mermaid output and returned metadata, and cover dense graphs with tests.

## Fix Focus Areas
- bin/ultimate-agent-stack.mjs[3582-3636]
- bin/ultimate-agent-stack.mjs[2669-2675]
- test/agent-stack.test.mjs[1740-1755]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

6. Test inspects commandEvidenceReport source 📜 Skill insight ▣ Testability ⭐ New

Description

The test asserts against commandEvidenceReport.toString() and regex-matches internal helper usage,
coupling the test to implementation details instead of publicly observable behavior. This makes the
test brittle to refactors that preserve behavior and violates the requirement to assert via public
outcomes rather than private structure.

Code

test/agent-stack.test.mjs[R1665-1672]

+  const reportSource = commandEvidenceReport.toString();
+  assert.match(reportSource, /\bloadValidatedWorkEvidence\b/);
+  assert.doesNotMatch(
+    reportSource,
+    /\b(?:commandWorkValidate|commandEvidenceValidate|readJson)\b/,
+    "reporting must render the same in-memory snapshot that passed validation",
+  );
+

Relevance

⭐⭐⭐ High

Team previously accepted changes to reduce brittle test coupling (loosened regex/implementation
pinning) in tests.

PR-#29
PR-#31

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 2395833 prohibits tests that assert on private/internal call sequences or internals
rather than observable outcomes. The added lines directly introspect the function source
(toString()) and assert presence/absence of internal symbols (loadValidatedWorkEvidence,
commandWorkValidate, readJson).

test/agent-stack.test.mjs[1665-1672]
Skill: build-vertical-slice

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A newly added test validates implementation details by inspecting `commandEvidenceReport.toString()` and matching internal helper names. This is brittle and violates the requirement that tests assert on publicly observable behavior.

## Issue Context
The intent appears to be ensuring the report uses the validated in-memory snapshot rather than re-reading from disk. That property should be validated via observable behavior (outputs and side effects), not by source inspection.

## Fix Focus Areas
- test/agent-stack.test.mjs[1665-1672]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

7. Symlink rejection promise overbroad ✓ Resolved 🐞 Bug ⛨ Security

Description

The trust documentation says path checks reject symlinks before protected writes, but ordinary CLI
writes accept symlink components that resolve within the project; only evidence-report output uses
the stricter helper. This overstates the filesystem guarantee provided by general CLI writes.

Code

docs/TRUST.md[46]

+programs. Path checks reject symlinks observed before a protected write; they

Evidence

Standard atomicProjectJson and atomicProjectText use projectFile, whose pathInside logic
permits symlinks resolving inside the project. The stricter projectFileWithoutSymlinkComponents
helper is used only for report output.

docs/TRUST.md[44-48]
bin/ultimate-agent-stack.mjs[556-587]
bin/ultimate-agent-stack.mjs[625-669]
bin/ultimate-agent-stack.mjs[3712-3732]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The trust documentation describes universal symlink-component rejection, while most CLI writes only reject paths whose resolved destination escapes the project.

## Issue Context
Either narrow the statement to evidence-report writes or route all protected write helpers through the strict component check if universal rejection is intended.

## Fix Focus Areas
- docs/TRUST.md[44-48]
- bin/ultimate-agent-stack.mjs[556-587]
- bin/ultimate-agent-stack.mjs[625-669]
- bin/ultimate-agent-stack.mjs[3712-3732]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

8. Omitted edge count underreports 📜 Skill insight ✧ Quality

Description

evidenceMermaid calculates omitted_edge_count only from edges whose endpoints survive the node
bound, so edges discarded because one or both nodes were omitted are not counted and the report can
incorrectly claim zero omitted edges. This contradicts the updated documentation stating that
visualization omissions are reported.

Code

bin/ultimate-agent-stack.mjs[3650]

+  const omittedEdges = eligibleEdges.length - selectedEdges.length;

Evidence

Rule 2395888 requires documentation to remain consistent with the behavior it describes.
eligibleEdges contains only edges between selected nodes, and calculating omissions as
eligibleEdges.length - selectedEdges.length at lines 3650-3653 ignores graph edges excluded by
node selection; the focused test at lines 1740-1747 confirms the mismatch by selecting one node from
a two-node, one-edge graph, rendering no edges, yet expecting omitted_edge_count to be zero
despite the documentation's promise to report omissions caused by visualization bounds.

bin/ultimate-agent-stack.mjs[3644-3653]
docs/ARCHITECTURE.md[196-199]
test/agent-stack.test.mjs[1746-1747]
bin/ultimate-agent-stack.mjs[3617-3653]
test/agent-stack.test.mjs[1740-1748]
README.md[287-292]
Skill: build-vertical-slice

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description

Fix `omitted_edge_count` so it includes graph edges discarded because one or both endpoints were omitted by `maxNodes`, as well as edges omitted by the edge cap. Ensure report metadata and Mermaid annotations no longer claim zero omitted edges when graph edges were not rendered.

## Issue Context

The updated documentation says omitted nodes and edges are reported, but the current calculation uses only `eligibleEdges`, which already excludes edges connected to omitted nodes. The existing two-node test demonstrates the defect: selecting one node renders no edges while reporting zero omitted edges; update that expectation and add coverage distinguishing edges lost through node selection from edges lost through the edge cap.

## Fix Focus Areas

- bin/ultimate-agent-stack.mjs[3617-3653]
- test/agent-stack.test.mjs[1740-1747]
- test/agent-stack.test.mjs[1779-1818]
- docs/ARCHITECTURE.md[196-199]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

_{To customize comments, go to the Qodo configuration screen, or learn more in the docs.}

Previous review results

Review updated until commit 9e0c090

Results up to commit ee6967b ⚖️ Balanced

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Report directory symlink bypass ✓ Resolved 🐞 Bug ⛨ Security

Description

The output allowlist checks the lexical path, so a project-local .agent-stack/reports symlink can
redirect report writes elsewhere inside the repository. Linking that directory to the project root
allows .agent-stack/reports/package.json to pass validation and replace the real package.json.

Code

bin/ultimate-agent-stack.mjs[R3679-3681]

+    normalizedOutput = relative(realpathSync(target), outputFile)
+      .split(sep)
+      .join("/");

Evidence

projectFile rejects only symlinks resolving outside the project and returns the unresolved
candidate, while report validation derives its allowlisted path from that lexical candidate and
checks only the final file with lstatSync. The atomic helpers create and rename files through the
destination parent, so a symlinked reports directory targeting another in-project directory is
followed despite the documented reports-only boundary.

bin/ultimate-agent-stack.mjs[602-646]
bin/ultimate-agent-stack.mjs[528-551]
bin/ultimate-agent-stack.mjs[3677-3699]
README.md[279-290]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Evidence report output validation uses a lexical path and checks only whether the final file is a symlink. A symlinked `.agent-stack/reports` ancestor can therefore redirect an allowed report write onto another file inside the repository.

## Issue Context
Resolve and validate the actual destination against a canonical `.agent-stack/reports` directory immediately before writing. Reject symlinked report-directory ancestors or otherwise guarantee that atomic temporary-file creation and rename remain physically inside that directory; add a regression test using a reports-directory symlink to the project root.

## Fix Focus Areas
- bin/ultimate-agent-stack.mjs[3677-3699]
- bin/ultimate-agent-stack.mjs[602-646]
- test/agent-stack.test.mjs[1749-1775]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Node limit permits edge floods ✓ Resolved 🐞 Bug ➹ Performance

Description

evidenceMermaid limits nodes but emits every edge between selected nodes, allowing a valid graph
to produce as many as 50,000 Mermaid edge lines regardless of the configured node limit. Dense
graphs can therefore generate multi-megabyte reports that are impractical to render despite the
bounded-report interface.

Code

bin/ultimate-agent-stack.mjs[R3602-3606]

+    ...graph.edges
+      .filter(
+        (edge) =>
+          selectedIds.has(edge.from) && selectedIds.has(edge.to),
+      )

Evidence

Node selection is sliced by maxNodes, but all qualifying graph edges are filtered, sorted, and
emitted without another limit. Graph validation permits 50,000 edges, so that full graph-level
allowance can be concentrated among selected nodes even though the report is presented as a bounded
visualization.

bin/ultimate-agent-stack.mjs[3582-3636]
bin/ultimate-agent-stack.mjs[2669-2675]
README.md[276-290]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The Mermaid visualization applies `--max-nodes` but no proportional edge or output-size limit. A dense validated graph can emit up to the graph-wide maximum of 50,000 edge lines and produce an unusable diagram.

## Issue Context
Apply a deterministic edge cap or total output-size budget after selecting nodes. Include the omitted-edge count in the Mermaid output and returned metadata, and cover dense graphs with tests.

## Fix Focus Areas
- bin/ultimate-agent-stack.mjs[3582-3636]
- bin/ultimate-agent-stack.mjs[2669-2675]
- test/agent-stack.test.mjs[1740-1755]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Results up to commit 93ffe66 ⚖️ Balanced

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (1)

1. Symlink check remains racy 🐞 Bug ⛨ Security

Description

The final symlink check is separated from the pathname-based write, so a concurrent actor with
workspace write access can replace a checked directory with a symlink before temporary-file creation
or rename. This can redirect a report write outside .agent-stack/reports, violating the advertised
physical-containment guarantee.

Code

bin/ultimate-agent-stack.mjs[R3731-3732]

+    mkdirSync(dirname(outputFile), { recursive: true });
+    projectFileWithoutSymlinkComponents(target, output, "evidence report");

Evidence

The helper uses existsSync and lstatSync checks but retains no trusted directory handle. After
the second check, atomicJson or atomicText creates and renames a temporary file through the same
mutable pathname, leaving a TOCTOU window despite the documentation's physical-containment claim.

bin/ultimate-agent-stack.mjs[530-578]
bin/ultimate-agent-stack.mjs[3731-3736]
docs/ARCHITECTURE.md[196-199]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Repeated pathname checks do not prevent a directory component from being replaced after validation but before the report is written. Make report creation operate relative to trusted directory handles with no-follow semantics, or otherwise use a platform-appropriate race-safe containment strategy.

## Issue Context
The attack requires a concurrent process able to mutate the report directory. A concrete interleaving replaces `.agent-stack/reports` or a descendant with a symlink after the final check and before `atomicJson` or `atomicText` creates its temporary file.

## Fix Focus Areas
- bin/ultimate-agent-stack.mjs[530-578]
- bin/ultimate-agent-stack.mjs[3713-3736]
- test/agent-stack.test.mjs[1820-1838]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Omitted edge count underreports 📜 Skill insight ✧ Quality

Description

evidenceMermaid calculates omitted_edge_count only from edges whose endpoints survive the node
bound, so edges discarded because one or both nodes were omitted are not counted and the report can
incorrectly claim zero omitted edges. This contradicts the updated documentation stating that
visualization omissions are reported.

Code

bin/ultimate-agent-stack.mjs[3650]

+  const omittedEdges = eligibleEdges.length - selectedEdges.length;

Evidence

Rule 2395888 requires documentation to remain consistent with the behavior it describes.
eligibleEdges contains only edges between selected nodes, and calculating omissions as
eligibleEdges.length - selectedEdges.length at lines 3650-3653 ignores graph edges excluded by
node selection; the focused test at lines 1740-1747 confirms the mismatch by selecting one node from
a two-node, one-edge graph, rendering no edges, yet expecting omitted_edge_count to be zero
despite the documentation's promise to report omissions caused by visualization bounds.

bin/ultimate-agent-stack.mjs[3644-3653]
docs/ARCHITECTURE.md[196-199]
test/agent-stack.test.mjs[1746-1747]
bin/ultimate-agent-stack.mjs[3617-3653]
test/agent-stack.test.mjs[1740-1748]
README.md[287-292]
Skill: build-vertical-slice

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description

Fix `omitted_edge_count` so it includes graph edges discarded because one or both endpoints were omitted by `maxNodes`, as well as edges omitted by the edge cap. Ensure report metadata and Mermaid annotations no longer claim zero omitted edges when graph edges were not rendered.

## Issue Context

The updated documentation says omitted nodes and edges are reported, but the current calculation uses only `eligibleEdges`, which already excludes edges connected to omitted nodes. The existing two-node test demonstrates the defect: selecting one node renders no edges while reporting zero omitted edges; update that expectation and add coverage distinguishing edges lost through node selection from edges lost through the edge cap.

## Fix Focus Areas

- bin/ultimate-agent-stack.mjs[3617-3653]
- test/agent-stack.test.mjs[1740-1747]
- test/agent-stack.test.mjs[1779-1818]
- docs/ARCHITECTURE.md[196-199]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Results up to commit 4ada866 ⚖️ Balanced

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Per-node edge cap misdocumented ✓ Resolved 🐞 Bug ≡ Correctness

Description

The manual claims each selected node is capped at four edges, but evidenceMermaid applies only one
global selected.length * 4 limit. A single node can therefore have far more than four rendered
connections, misleading operators about diagram density.

Code

docs/OPERATING_MANUAL.md[223]

+edges at four per selected node. The report counts nodes excluded by the node

Evidence

The renderer sorts all eligible edges and slices one global list at selected.length * 4; it does
not maintain per-node counters. The dense-graph test likewise verifies only the aggregate limit.

docs/OPERATING_MANUAL.md[220-225]
bin/ultimate-agent-stack.mjs[3617-3629]
test/agent-stack.test.mjs[1791-1817]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The documentation promises four rendered edges per selected node, while the implementation only caps the total edge count at four times the selected-node count.

## Issue Context
Choose the intended behavior: either document the aggregate cap precisely or enforce an individual per-node edge limit and add an asymmetric-graph test.

## Fix Focus Areas
- docs/OPERATING_MANUAL.md[222-224]
- README.md[287-292]
- docs/ARCHITECTURE.md[193-197]
- bin/ultimate-agent-stack.mjs[3617-3629]
- test/agent-stack.test.mjs[1791-1817]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Symlink rejection promise overbroad ✓ Resolved 🐞 Bug ⛨ Security

Description

The trust documentation says path checks reject symlinks before protected writes, but ordinary CLI
writes accept symlink components that resolve within the project; only evidence-report output uses
the stricter helper. This overstates the filesystem guarantee provided by general CLI writes.

Code

docs/TRUST.md[46]

+programs. Path checks reject symlinks observed before a protected write; they

Evidence

Standard atomicProjectJson and atomicProjectText use projectFile, whose pathInside logic
permits symlinks resolving inside the project. The stricter projectFileWithoutSymlinkComponents
helper is used only for report output.

docs/TRUST.md[44-48]
bin/ultimate-agent-stack.mjs[556-587]
bin/ultimate-agent-stack.mjs[625-669]
bin/ultimate-agent-stack.mjs[3712-3732]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The trust documentation describes universal symlink-component rejection, while most CLI writes only reject paths whose resolved destination escapes the project.

## Issue Context
Either narrow the statement to evidence-report writes or route all protected write helpers through the strict component check if universal rejection is intended.

## Fix Focus Areas
- docs/TRUST.md[44-48]
- bin/ultimate-agent-stack.mjs[556-587]
- bin/ultimate-agent-stack.mjs[625-669]
- bin/ultimate-agent-stack.mjs[3712-3732]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Results up to commit 77af404 ⚖️ Balanced

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Unvalidated second read ✓ Resolved 🐞 Bug ☼ Reliability

Description

commandEvidenceReport validates the work ledger/evidence graph, then re-reads both JSON files and
generates the report from that second read without validating that snapshot. If the files change
between validation and the second read, the report can be inconsistent with the validated state or
throw during report/Mermaid generation.

Code

bin/ultimate-agent-stack.mjs[R3689-3710]

+  const work = commandWorkValidate(target);
+  const evidence = commandEvidenceValidate(target);
+  if (!work.ok || !evidence.ok) {
+    throw new StackError(
+      "Evidence reporting requires valid repository work and evidence.",
+      2,
+      [...work.errors, ...evidence.errors],
+    );
+  }
+  const ledger = readJson(
+    projectFile(target, WORK_LEDGER_PATH, "work ledger"),
+    "work ledger",
+  );
+  const graph = readJson(
+    projectFile(target, EVIDENCE_GRAPH_PATH, "evidence graph"),
+    "evidence graph",
+  );
+  const report = evidenceReportData(ledger, graph);
+  const visualization =
+    format === "mermaid"
+      ? evidenceMermaid(graph, numericMaxNodes)
+      : null;

Evidence

The report command performs validation but discards the validated objects and re-reads JSON for
rendering; Mermaid output interpolates node.state and edge.relation directly, which is only safe
when those fields come from validated input. The validation schema does constrain these fields,
demonstrating the safety assumptions that the second read bypasses.

bin/ultimate-agent-stack.mjs[3689-3710]
bin/ultimate-agent-stack.mjs[3605-3642]
bin/ultimate-agent-stack.mjs[2677-2775]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`commandEvidenceReport` runs validation (`commandWorkValidate` / `commandEvidenceValidate`) but then calls `readJson(...)` again and uses those objects to build the report and Mermaid output without validating that second-read snapshot.

This creates a time-of-check/time-of-use gap: if `.agent-stack/work-items.json` or `.agent-stack/evidence-graph.json` changes between the validation step and the second read, the generated output is no longer guaranteed to reflect a validated repository state and may crash while building counts/Mermaid.

### Issue Context
- Validation constrains fields like `node.state` and `edge.relation` to canonical sets, but those guarantees only apply to the validated objects.
- Mermaid rendering interpolates `node.state` and `edge.relation` directly into output lines; it relies on validated input.

### Fix Focus Areas
- bin/ultimate-agent-stack.mjs[3689-3710]

### Suggested fix
- Read the work ledger and evidence graph **once**, validate those in-memory objects, and generate the report/Mermaid from the same validated snapshot.
 - Option A: Introduce a helper like `loadValidatedWorkAndEvidence(target)` that returns `{ ledger, graph }` (or throws with collected validation errors), and use it in `commandEvidenceReport`.
 - Option B: Use `validateRepositoryContract(...)` directly in `commandEvidenceReport` to get both `value` objects and errors, then run `validateWorkEvidenceLinkage(ledger.value, graph.value)` before rendering.
- (Optional defense-in-depth) Add a small whitelist/escaping step for any Mermaid-interpolated fields that are not already sanitized, so output remains safe even if validation is bypassed or inputs are mutated mid-run.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[samtay32]

/improve

[qodo-code-review]

Code review by qodo was updated up to the latest commit 93ffe66

[samtay32]

/improve

[qodo-code-review]

Code review by qodo was updated up to the latest commit 4ada866

[samtay32]

/improve

[qodo-code-review]

Code review by qodo was updated up to the latest commit ae7b215

[qodo-code-review]

PR Summary by Qodo

Add bounded evidence graph reports

✨ Enhancement 🧪 Tests 📝 Documentation 🕐 40+ Minutes

AI Description

• Adds provider-neutral JSON and bounded Mermaid reports from validated repository evidence.
• Restricts report files to extension-matched paths under .agent-stack/reports with symlink
 checks.
• Documents behavior and adds unit, behavioral, and packed-install coverage.

Diagram

graph TD
  CLI["Evidence CLI"] --> Validator["Contract validation"] --> Builder["Report builder"] --> Json["JSON summary"] --> Reports["Reports directory"]
  Ledger["Work ledger"] --> Validator
  Evidence["Evidence graph"] --> Validator
  Builder --> Mermaid["Mermaid renderer"] --> Reports

Loading

High-Level Assessment

The current approach is appropriate: reports are derived only after existing repository contracts validate, remain provider-neutral, and enforce explicit node, edge, sample, and write-location bounds. Provider-backed reporting was dismissed because it would weaken offline neutrality, while rendering the complete graph would create unbounded and less reviewable output.

Files changed (11) +553 / -8

Enhancement (1) +271 / -4

ultimate-agent-stack.mjsImplement bounded JSON and Mermaid evidence reporting +271/-4

Implement bounded JSON and Mermaid evidence reporting

• Adds deterministic evidence coverage aggregation, sanitized Mermaid rendering, node and edge bounds, CLI option handling, and report command exports. Restricts optional output to format-matched files under '.agent-stack/reports' and rejects observed symlink path components.

bin/ultimate-agent-stack.mjs

Tests (4) +236 / -3

scenarios.jsonAdd direct bounded evidence-report scenario +30/-0

Add direct bounded evidence-report scenario

• Adds a behavioral scenario requiring repository-contained Mermaid reporting without external provider access or unrelated skill activation.

evals/scenarios.json

packed-smoke.mjsVerify evidence reporting from packed installations +22/-0

Verify evidence reporting from packed installations

• Runs the Mermaid evidence report command against an installed package and verifies the successful response and rendered flowchart output.

scripts/packed-smoke.mjs

agent-stack.test.mjsCover deterministic, bounded, and contained evidence reports +181/-0

Cover deterministic, bounded, and contained evidence reports

• Tests deterministic summaries, sanitized Mermaid labels, node and edge limits, valid output writes, extension and traversal rejection, and symlink-component protection. It also verifies dense graph omission metadata and report contents.

test/agent-stack.test.mjs

skill-eval.test.mjsUpdate behavioral evaluation totals +3/-3

Update behavioral evaluation totals

• Adjusts expected scenario and passing-run counts to include the new evidence-report behavioral contract.

test/skill-eval.test.mjs

Documentation (5) +45 / -1

README.mdDocument evidence report CLI usage +21/-0

Document evidence report CLI usage

• Adds JSON and Mermaid command examples. Explains coverage contents, rendering bounds, omission reporting, label sanitization, and the report-write containment model.

README.md

ARCHITECTURE.mdDefine evidence reporting architecture and trust boundaries +10/-0

Define evidence reporting architecture and trust boundaries

• Documents validated repository inputs, provider neutrality, bounded JSON samples, Mermaid node and edge limits, omission semantics, and non-race-safe path confinement.

docs/ARCHITECTURE.md

OPERATING_MANUAL.mdAdd operational guidance for bounded evidence reports +7/-0

Add operational guidance for bounded evidence reports

• Explains when to use JSON or Mermaid reports, how to select a proportional node bound, and how to interpret omitted nodes and edges.

docs/OPERATING_MANUAL.md

TRUST.mdClarify report output symlink protections +4/-1

Clarify report output symlink protections

• Documents rejection of observed symlink path components and explicitly limits the guarantee against concurrent checkout mutation.

docs/TRUST.md

SKILL.mdGuide agents to use evidence reports appropriately +3/-0

Guide agents to use evidence reports appropriately

• Adds instructions for selecting JSON or bounded Mermaid reports while preserving the authority of referenced evidence artifacts.

skills/manage-project-work/SKILL.md

Other (1) +1 / -0

plugin.jsonAdvertise evidence graph reporting capability +1/-0

Advertise evidence graph reporting capability

• Adds evidence graph reports to the plugin's published capability list.

.codex-plugin/plugin.json

[qodo-code-review]

Code review by qodo was updated up to the latest commit ae7b215

[samtay32]

/improve

[qodo-code-review]

Code review by qodo was updated up to the latest commit 77af404

[samtay32]

/improve

[qodo-code-review]

Code review by qodo was updated up to the latest commit 9e0c090

[samtay32]

Qodo exact-head review is complete and every review thread is resolved. Re-evaluating the protected Qodo receipt.