Skip to content

Add --list-categories flag to validator#405

Merged
potiuk merged 2 commits into
apache:mainfrom
Nishieee:feat/list-categories-flag
May 31, 2026
Merged

Add --list-categories flag to validator#405
potiuk merged 2 commits into
apache:mainfrom
Nishieee:feat/list-categories-flag

Conversation

@Nishieee

Copy link
Copy Markdown
Contributor

Summary

  • Add --list-categories to skill-and-tool-validator so callers can discover valid --skip-categories names without grepping the source
  • Print all 11 category constants sorted alphabetically; SOFT categories marked with (advisory)
  • Add regression test asserting exact sorted output

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • prek run --all-files passes
  • For Python packages touched: uv run pytest / ruff check / mypy passes
  • For Groovy bridges touched: command-line invocation tested end-to-end
  • For skill changes: eval suite passes for the affected skill
  • For skill behaviour changes: a new or updated eval fixture is included in this PR
  • Other: uv run --directory tools/skill-and-tool-validator skill-and-tool-validate --list-categories prints sorted categories with advisory markers

RFC-AI-0004 compliance

  • HITL
  • Sandbox
  • Vendor neutrality
  • Conversational + correctable
  • Write-access discipline
  • Privacy LLM

Linked issues

Closes #377

Notes for reviewers (optional)

Screenshot 2026-05-30 at 8 09 43 PM

Discovery flag exits 0 before run_validation(). Includes all 11 named category constants (tool/capability-sync categories plus the skill-validation set).

@justinmclean
justinmclean requested a review from Copilot May 31, 2026 06:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a --list-categories CLI flag to the skill-and-tool-validator that prints every violation category (marking SOFT/advisory ones) and exits, along with a corresponding test.

Changes:

  • Introduce ALL_CATEGORIES frozenset enumerating every violation category.
  • Add --list-categories argparse flag and handler in main().
  • Add test_list_categories test covering the new flag.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
tools/skill-and-tool-validator/src/skill_and_tool_validator/init.py Defines ALL_CATEGORIES and implements the --list-categories CLI option.
tools/skill-and-tool-validator/tests/test_validator.py Adds test verifying the listed categories and advisory marking.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py Outdated
@potiuk

potiuk commented May 31, 2026

Copy link
Copy Markdown
Member

Can you also fix the prek checks ?

@potiuk
potiuk merged commit cf5c417 into apache:main May 31, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add --list-categories to print the validator's violation category names

3 participants