feat: Implement OpenAPISpecsAssessor

[jeremyeder]

feat: Implement OpenAPISpecsAssessor

Attribute Definition

Attribute ID: openapi_specs (Attribute #19 - Tier 3)

Definition: Machine-readable API documentation in OpenAPI format (formerly Swagger).

Why It Matters: OpenAPI specs define everything needed to integrate with an API: authentication, endpoints, HTTP methods, request/response schemas, error codes. AI can read specs to generate client code, tests, and integration code automatically.

Impact on Agent Behavior:

• Auto-generation of SDKs and client libraries
• Request/response validation
• API mocking for testing
• Contract compliance verification
• Interactive API exploration

Measurable Criteria:

• OpenAPI 3.0+ specification file (openapi.yaml or openapi.json)
• All endpoints documented with:
  • Description and purpose
  • HTTP method (GET, POST, PUT, DELETE, PATCH)
  • Parameters (path, query, header)
  • Request body schema
  • Response schemas (success and error cases)
  • Authentication requirements
  • Example requests/responses
• Validation: Use Swagger Editor or Spectral
• Auto-generate from code annotations OR keep manually in sync
• Hosted documentation (Swagger UI, ReDoc)

Implementation Requirements

File Location: src/agentready/assessors/documentation.py (or new file api.py)

Class Name: OpenAPISpecsAssessor

Tier: 3 (Important)

Default Weight: 0.015 (1.5% of total score)

Assessment Logic

Scoring Approach: Binary check for OpenAPI spec file, with quality scoring

Evidence to Check (score components):

1. OpenAPI file exists (60%)

   • Check for: openapi.yaml, openapi.yml, openapi.json, swagger.yaml, swagger.json
   • Validate file is parseable JSON/YAML

2. OpenAPI version (20%)

   • Check openapi: 3.0.0 or higher
   • Penalize OpenAPI 2.0 (Swagger) specs

3. Spec completeness (20%)

   • Count defined paths/endpoints
   • Check for schemas section
   • Verify response definitions
   • Look for security definitions

Scoring Logic:

if openapi_file_exists:
    file_score = 60

    # Bonus for OpenAPI 3.x
    if openapi_version >= "3.0":
        version_score = 20
    else:
        version_score = 10  # Swagger 2.0

    # Completeness check
    if has_paths and has_schemas:
        completeness_score = 20
    else:
        completeness_score = 10

    total_score = file_score + version_score + completeness_score
else:
    total_score = 0

status = "pass" if total_score >= 75 else "fail"

Code Pattern to Follow

Reference: READMEAssessor for file existence and content parsing

Pattern:

1. Check for OpenAPI spec file in common locations
2. Parse YAML/JSON to extract version and structure
3. Validate basic schema (paths, components/definitions)
4. Calculate score based on presence and quality
5. Provide remediation with examples

Example Finding Responses

Pass (Score: 100)

Finding(
    attribute=self.attribute,
    status="pass",
    score=100.0,
    measured_value="OpenAPI 3.1.0 with 15 endpoints",
    threshold="OpenAPI 3.x spec present",
    evidence=[
        "openapi.yaml found in repository root",
        "OpenAPI version: 3.1.0",
        "15 endpoints documented",
        "Schemas defined for all request/response types",
        "Security schemes configured",
    ],
    remediation=None,
    error_message=None,
)

Fail (Score: 0)

Finding(
    attribute=self.attribute,
    status="fail",
    score=0.0,
    measured_value="no OpenAPI spec",
    threshold="OpenAPI spec present",
    evidence=[
        "No OpenAPI specification found",
        "Searched: openapi.yaml, openapi.json, swagger.yaml",
    ],
    remediation=self._create_remediation(),
    error_message=None,
)

Not Applicable

Finding.not_applicable(
    self.attribute,
    reason="No API endpoints detected (not a web service)"
)

Registration

Add to src/agentready/services/scanner.py in create_all_assessors():

from ..assessors.documentation import (
    CLAUDEmdAssessor,
    READMEAssessor,
    OpenAPISpecsAssessor,  # Add this import
)

def create_all_assessors() -> List[BaseAssessor]:
    return [
        # ... existing assessors ...
        OpenAPISpecsAssessor(),  # Add this line
    ]

Testing Guidance

Test File: tests/unit/test_assessors_documentation.py

Test Cases to Add:

1. test_openapi_pass_v3: Repository with valid OpenAPI 3.x spec
2. test_openapi_partial_swagger_v2: Repository with Swagger 2.0 spec (partial score)
3. test_openapi_fail_missing: No OpenAPI spec found
4. test_openapi_fail_invalid_yaml: Spec file exists but unparseable
5. test_openapi_not_applicable: Non-API project (CLI tool, library)

Note: AgentReady is a CLI tool (not web API), should return not_applicable.

Dependencies

External Tools: None (YAML/JSON parsing only)

Python Standard Library:

• pathlib.Path for file location
• yaml (PyYAML) for parsing YAML specs
• json for parsing JSON specs

Optional Enhancement: Use openapi-spec-validator for strict validation

Remediation Steps

def _create_remediation(self) -> Remediation:
    return Remediation(
        summary="Create OpenAPI specification for API endpoints",
        steps=[
            "Create openapi.yaml in repository root",
            "Define OpenAPI version 3.x",
            "Document all API endpoints with full schemas",
            "Add request/response examples",
            "Define security schemes (API keys, OAuth, etc.)",
            "Validate spec with Swagger Editor or Spectral",
            "Generate API documentation with Swagger UI or ReDoc",
        ],
        tools=["swagger-editor", "spectral", "openapi-generator"],
        commands=[
            "# Install OpenAPI validator",
            "npm install -g @stoplight/spectral-cli",
            "",
            "# Validate spec",
            "spectral lint openapi.yaml",
            "",
            "# Generate client SDK",
            "npx @openapitools/openapi-generator-cli generate \\",
            "  -i openapi.yaml \\",
            "  -g python \\",
            "  -o client/",
        ],
        examples=[
            """# openapi.yaml - Minimal example
openapi: 3.1.0
info:
  title: My API
  version: 1.0.0
  description: API for managing users

servers:
  - url: https://api.example.com/v1

paths:
  /users/{userId}:
    get:
      summary: Get user by ID
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
      responses:
        '200':
          description: User found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/User'
        '404':
          description: User not found

components:
  schemas:
    User:
      type: object
      required:
        - id
        - email
      properties:
        id:
          type: string
          example: "user_123"
        email:
          type: string
          format: email
          example: "user@example.com"
        name:
          type: string
          example: "John Doe"
""",
        ],
        citations=[
            Citation(
                source="OpenAPI Initiative",
                title="OpenAPI Specification",
                url="https://spec.openapis.org/oas/v3.1.0",
                relevance="Official OpenAPI 3.1 specification",
            ),
            Citation(
                source="Swagger",
                title="API Documentation Best Practices",
                url="https://swagger.io/resources/articles/best-practices-in-api-documentation/",
                relevance="Guide to writing effective API docs",
            ),
        ],
    )

Implementation Notes

1. File Detection: Check for openapi.{yaml,yml,json}, swagger.{yaml,yml,json}
2. Parsing: Use PyYAML for YAML, json module for JSON
3. Version Extraction: Parse openapi field (3.x) or swagger field (2.0)
4. Completeness Check: Verify paths and components sections exist
5. API Detection: Check if repository is web service (presence of web framework)
6. Edge Cases: Return not_applicable for CLI tools, libraries, non-API projects
7. Validation: Basic structure check, not full OpenAPI compliance (optional enhancement)

[github-actions]

🎉 This issue has been resolved in version 1.22.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀