Validate review claims before disposition

[samtay32]

Outcome

Makes review closure evidence-first and gives all review-specific guidance one canonical disposition vocabulary.

Root cause

The canonical policy used fixed | rebutted | deferred | decision-needed, while the executable skill restated verb-form variants. The skill also implied claim validation without explicitly requiring inspection of the cited code before changing production behavior.

Changes

• define claim validation and the four canonical dispositions in one policy reference;
• require inspection of cited code, surrounding behavior, tests, and authoritative docs before acting;
• make the execution skill and secondary guidance consume the canonical policy instead of restating it;
• align the no-coder README flow with evidence-backed rebuttal and authority decisions;
• add regression coverage that rejects the previous drifted labels.

Verification

• node --test test/maintenance.test.mjs — 13/13 passed
• skill quick_validate.py — valid
• npm run release:check — 79/79 tests, dry package, packed smoke
• markdownlint — all 49 tracked Markdown files passed
• git diff --check — passed

Risk / rollback

Documentation and agent-instruction behavior only; no runtime CLI, schema, dependency, or release-path change. Roll back the single commit if the stricter review contract causes an unexpected harness incompatibility.

Summary by CodeRabbit

• Documentation

  • Updated review workflows to require evidence-based validation before production changes.
  • Standardized review outcomes to four dispositions: fixed, rebutted, deferred, and decision-needed.
  • Clarified closure requirements, authority boundaries, verification steps, and merge-readiness decisions.

• Tests

  • Added checks ensuring review guidance consistently follows the canonical closure policy and response format.

[samtay32]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: cb8044b7-30e2-4f29-b467-c129dc5991e0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The change establishes evidence-backed validation for reviewer claims, defines four canonical review dispositions, propagates the policy through delivery and review-loop guidance, and adds maintenance tests enforcing consistent documentation.

Changes

Review closure policy

Layer / File(s)	Summary
Canonical validation and dispositions skills/close-review-loop/references/review-closure-policy.md	Defines claim validation steps and restricts responses to fixed, rebutted, deferred, or decision-needed.
Review loop procedure integration skills/close-review-loop/SKILL.md, docs/GITHUB_LOOP.md	Requires evidence-based validation, canonical response formatting, verification, and closure sequencing.
Repository guidance and validation README.md, STARTER_PROMPT.md, assets/project-template/AGENTS.md, CHANGELOG.md, test/maintenance.test.mjs	Updates delivery and project guidance, records the policy change, and tests canonical vocabulary and references.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• samtay32/ultimate-agent-stack#9: Updates the shared close-review workflow and review-receipt gating semantics.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: validating review claims before assigning a disposition.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/review-truth-dispositions

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/maintenance.test.mjs`:
- Around line 179-228: Update the guidance-source loop in the test named “review
closure validates claims and has one disposition vocabulary” so every secondary
source is checked for noncanonical disposition labels using the existing
alternate-label pattern, and for local Disposition declarations with optional
leading indentation. Preserve the canonical policy checks and the requirement
that secondary guidance references the review closure policy.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: dcc6251d-cea7-4db3-a2a4-5144ede7e9db

📥 Commits

Reviewing files that changed from the base of the PR and between e9b02c9 and e93a3e0.

📒 Files selected for processing (8)

• CHANGELOG.md
• README.md
• STARTER_PROMPT.md
• assets/project-template/AGENTS.md
• docs/GITHUB_LOOP.md
• skills/close-review-loop/SKILL.md
• skills/close-review-loop/references/review-closure-policy.md
• test/maintenance.test.mjs

[samtay32]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.