Skip to content

fix(ci): reject low-effort feature reports in issue-quality bot - #423

Merged
Wibias merged 5 commits into
lidge-jun:devfrom
Wibias:fix/issue-quality-low-effort-reports
Jul 24, 2026
Merged

fix(ci): reject low-effort feature reports in issue-quality bot#423
Wibias merged 5 commits into
lidge-jun:devfrom
Wibias:fix/issue-quality-low-effort-reports

Conversation

@Wibias

@Wibias Wibias commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Why

Issue #401 stayed open because enforce-issue-quality.yml was working, but the validator treated that report as valid.

What changed

  • Treat NA / not applicable as placeholder example text
  • Reject placeholder examples on the feature form before empty-section logic
  • Reject overly vague feature sections that lack concrete workflow detail
  • Add regression tests for issue Change voice chat to different model #401-style reports
  • Add workflow_dispatch with issue_number so maintainers can backfill already-open issues like Change voice chat to different model #401 after merge

Test plan

  • node --test .github/scripts/issue-quality.test.cjs

Summary by CodeRabbit

  • Improvements
    • Issue quality checks now treat a wider range of placeholder-like values (including common “not applicable” variants and fenced blocks with only placeholders) as empty, and more precisely differentiate placeholder content from missing required example text.
    • Feature reports with overly vague or terse goals/examples are rejected with more implementation-oriented guidance.
    • Manual enforcement supports workflow dispatch, requiring a valid numeric issue number and adding early safety checks (including default-branch-only dispatch and rejecting unsupported pull-request dispatch scenarios).
  • Tests
    • Added coverage for placeholder normalization, vagueness/too-tere detection, and workflow-dispatch input validation plus guard/operation ordering.

@github-actions github-actions Bot added the bug Something isn't working label Jul 24, 2026
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3c733370-de7d-43ec-96d3-a4924360ac30

📥 Commits

Reviewing files that changed from the base of the PR and between 158b0f3 and af28e37.

📒 Files selected for processing (2)
  • .github/scripts/issue-quality.cjs
  • .github/scripts/issue-quality.test.cjs

📝 Walkthrough

Walkthrough

Issue-quality validation recognizes expanded placeholder forms, distinguishes missing from placeholder examples, and rejects vague feature sections. The enforcement workflow supports manual issue validation with dispatch input and branch guards.

Changes

Issue Quality Enforcement

Layer / File(s) Summary
Placeholder normalization and detection
.github/scripts/issue-quality.cjs, .github/scripts/issue-quality.test.cjs
Cleanup and placeholder detection handle additional tokens, fenced placeholder-only values, HTML comments, and legacy bug-form placeholders.
Feature content validation
.github/scripts/issue-quality.cjs, .github/scripts/issue-quality.test.cjs
Feature validation separately reports placeholder examples and missing examples, and flags terse core sections lacking concrete detail.
Manual issue-quality workflow execution
.github/scripts/issue-quality.cjs, .github/workflows/enforce-issue-quality.yml, .github/scripts/issue-quality.test.cjs, tests/ci-workflows.test.ts
Manual runs accept and validate a positive issue number, enforce dispatch branch and pull-request guards, and test guard ordering before mutations.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Dispatcher
  participant EnforceIssueQuality
  participant GitHubAPI
  participant IssueQuality

  Dispatcher->>EnforceIssueQuality: workflow_dispatch with issue_number and ref
  EnforceIssueQuality->>IssueQuality: validate dispatch input and default branch
  EnforceIssueQuality->>GitHubAPI: fetch issue by issue_number
  GitHubAPI-->>EnforceIssueQuality: issue payload
  EnforceIssueQuality->>IssueQuality: reject pull-request dispatch payload
  IssueQuality-->>EnforceIssueQuality: rejection message or null
  EnforceIssueQuality->>GitHubAPI: validate and enforce issue quality
Loading

Suggested reviewers: ingwannu, lidge-j

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: tightening the issue-quality bot to reject low-effort feature reports.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]

This comment was marked as resolved.

chatgpt-codex-connector[bot]

This comment was marked as resolved.

@Wibias

Wibias commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author

Ready

coderabbitai[bot]

This comment was marked as resolved.

Wibias added 3 commits July 25, 2026 00:21
- distinguish blank example sections from explicit placeholder tokens

- scope workflow issues:write permission to the validate job

- validate workflow_dispatch issue_number as a positive safe integer

- add regression test for blank example usage
@Wibias
Wibias force-pushed the fix/issue-quality-low-effort-reports branch from 3b96465 to 70bb34b Compare July 24, 2026 22:58
Count Cyrillic/Greek/Arabic as real word tokens, reject fenced N/A-only
examples, and cover the true 6-7 word concrete-detail boundary.
coderabbitai[bot]

This comment was marked as resolved.

Count CJK glyphs separately from non-CJK tokens so stubs like
provider中 cannot bypass the feature terseness check.
@Wibias
Wibias merged commit 3fac781 into lidge-jun:dev Jul 24, 2026
12 checks passed
@Wibias
Wibias deleted the fix/issue-quality-low-effort-reports branch July 25, 2026 07:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant