Summary
Current test coverage is 37% but pyproject.toml enforces a 90% threshold, causing pytest to fail. We will keep the 90% threshold and let CI fail until we improve coverage - this is intentional to drive quality improvement and block releases on test failures going forward.
Impact
pytest fails locally and in CI (expected - this is now a blocker)
Cannot merge PRs until coverage improves
Forces quality discipline and comprehensive testing
Aligns with professional software engineering practices
Current State
# pyproject.toml line 85
addopts = " -v --cov=agentready --cov-report=term-missing --cov-report=html --cov-report=xml --cov-fail-under=90"
# CLAUDE.md
** Current Coverage** : 37% (focused on core logic)
Strategy
DO NOT lower the threshold . Instead, systematically improve coverage:
Phase 1: Core Assessment Logic (Target: 50%)
Add tests for all 10 implemented assessors
Cover error paths and edge cases
Test file I/O edge cases (encoding errors, missing files)
Phase 2: Service Layer (Target: 70%)
Test scorer with various configurations
Test language detection fallbacks
Test scanner orchestration
Test LLM cache with expiry scenarios
Phase 3: Reporters & CLI (Target: 80%)
Test HTML report generation edge cases
Test Markdown report formatting
Test JSON serialization
Test CLI argument validation
Phase 4: Error Handling & Edge Cases (Target: 90%)
Test all exception handlers
Test subprocess timeout scenarios
Test LLM enrichment failure paths
Test config validation
Test report schema versioning
Areas Needing Tests
Based on code review, priority areas:
CommandFix timeout handling (Issue [P0] Command Execution Timeout Missing - DoS Vulnerability #102 ) - needs tests
LLM enricher retry logic - needs max_retries tests
File I/O error handling - encoding fallbacks, permissions
Scorer edge cases - empty findings, all excluded
Path validation - URL encoding, Unicode attacks
Subprocess utilities - timeout enforcement, output limits
Report generation - missing data, invalid Assessment
Config loading - malformed JSON, missing files
Testing Tools to Add
# Install testing utilities
uv pip install pytest-cov pytest-timeout pytest-mock coverage[toml]
# Run coverage report with branch coverage
pytest --cov=agentready --cov-branch --cov-report=term-missing --cov-report=html
# Identify untested code
coverage report --show-missing
# Find untested functions
coverage json
cat coverage.json | jq ' .files[] | select(.summary.percent_covered < 80) | .filename'
Acceptance Criteria
Quick Wins (Start Here)
Add tests for CLAUDEmdAssessor edge cases (missing file, malformed YAML)
Add tests for READMEAssessor (no README, multiple READMEs)
Add tests for TypeAnnotationsAssessor (no Python files, parse errors)
Add tests for StandardLayoutAssessor (missing dirs, empty project)
Add tests for GitignoreAssessor (no .gitignore, malformed entries)
Each assessor test file should reach ~90% coverage for that assessor.
Long-term Benefits
Confidence in refactoring : Can change code without breaking things
Faster debugging : Tests pinpoint failures immediately
Documentation : Tests show how code should be used
Professionalism : 90% coverage is industry standard for quality tools
Release confidence : CI failures block bad releases
References
Full remediation plan: .plans/code-review-remediation-plan.md
Coverage report: htmlcov/index.html (generated by pytest)
Labels : bug, P0, testing
Milestone : v1.24.0
Priority : Block all releases until 90% coverage achieved
Summary
Current test coverage is 37% but
pyproject.tomlenforces a 90% threshold, causing pytest to fail. We will keep the 90% threshold and let CI fail until we improve coverage - this is intentional to drive quality improvement and block releases on test failures going forward.Impact
Current State
Strategy
DO NOT lower the threshold. Instead, systematically improve coverage:
Phase 1: Core Assessment Logic (Target: 50%)
Phase 2: Service Layer (Target: 70%)
Phase 3: Reporters & CLI (Target: 80%)
Phase 4: Error Handling & Edge Cases (Target: 90%)
Areas Needing Tests
Based on code review, priority areas:
Testing Tools to Add
Acceptance Criteria
Quick Wins (Start Here)
CLAUDEmdAssessoredge cases (missing file, malformed YAML)READMEAssessor(no README, multiple READMEs)TypeAnnotationsAssessor(no Python files, parse errors)StandardLayoutAssessor(missing dirs, empty project)GitignoreAssessor(no .gitignore, malformed entries)Each assessor test file should reach ~90% coverage for that assessor.
Long-term Benefits
References
.plans/code-review-remediation-plan.mdhtmlcov/index.html(generated by pytest)Labels: bug, P0, testing
Milestone: v1.24.0
Priority: Block all releases until 90% coverage achieved