Requirements Library: Implement generic Python tool requirements

[akihikokuroda]

Implement five generic Python tool requirements.

Requirements to Implement

1. PythonCodeExtraction — code blocks are present and extractable
2. PythonSyntaxValid — code is syntactically valid Python
3. PythonExecutesWithoutError — code runs without exceptions
4. OutputSizeLimit — captured output doesn't exceed limit
5. ImportRestrictions — only whitelisted modules imported

Tests & Documentation

• Add Google-style docstrings for each requirement class
• Create test/stdlib/requirements/test_python_tools.py with positive and negative cases

Part of #1023

[akihikokuroda]

Analysis: Utilize Existing Infrastructure

Current State in python_reqs.py

The existing python_reqs.py already provides:

1. PythonCodeExtraction ✅ Already Implemented

• Function: _has_python_code_listing(ctx: Context) -> ValidationResult
• Extracts Python code blocks with intelligent scoring
• Scores blocks based on:
  • Length (bonus up to 10)
  • Function/class definitions (+5)
  • Control flow statements (+3)
  • Non-trivial content validation (penalizes import-only blocks)
• Recommendation: Wrap this as a Requirement class in python_tools.py, don't reimplement

2. PythonExecutesWithoutError ✅ Already Implemented

• Class: PythonExecutionReq(Requirement)
• Function: _python_executes_without_error(...)
• Already supports:
  • Multiple execution modes: StaticAnalysisEnvironment, UnsafeEnvironment, LLMSandboxEnvironment
  • Configurable timeout (default 5s)
  • Import allowlisting
  • Error message propagation
• Recommendation: This class already exists and is production-ready. For the factory function in Requirements Library: Create python_tool_requirements() factory function #1121, you may just reuse PythonExecutionReq directly or extend it

Revised Scope for Task #1119

The task should focus on NEW requirements:

1. PythonCodeExtraction — WRAP existing _has_python_code_listing in a Requirement class

   class PythonCodeExtraction(Requirement):
       """Validates that Python code blocks are present and extractable."""
       def __init__(self):
           super().__init__(
               description="Code blocks are present and extractable.",
               validation_fn=_has_python_code_listing,
               check_only=True,
           )

2. PythonSyntaxValid — NEW (not yet implemented)

   • Use AST parsing to validate syntax without execution
   • Should provide specific error details

3. OutputSizeLimit — NEW (not yet implemented)

   • Run code, capture stdout, check length
   • Default limit: 10,000 chars

4. ImportRestrictions — NEW (not yet implemented)

   • AST analysis for static import checking
   • Runtime import tracking if using sandbox

5. PythonExecutesWithoutError — EXTEND/WRAP existing PythonExecutionReq

   • The class already exists, may need minimal adaptation

Revised Scope for Task #1121

The python_tool_requirements() factory should:

def python_tool_requirements(
    allowed_imports: list[str] | None = None,
    output_limit_chars: int = 10000,
    timeout_seconds: int = 5,
    use_sandbox: bool = False,
) -> list[Requirement]:
    """Returns requirements for safe Python tool code generation."""
    return [
        PythonCodeExtraction(),
        PythonSyntaxValid(),
        PythonExecutesWithoutError(
            timeout=timeout_seconds,
            allow_unsafe_execution=not use_sandbox,  # Note: semantics differ
            allowed_imports=allowed_imports,
            use_sandbox=use_sandbox,
        ),
        OutputSizeLimit(limit_chars=output_limit_chars),
        ImportRestrictions(allowed_imports=allowed_imports) if allowed_imports else None,  # Optional
    ]
    # Filter out None values if any

Note: PythonExecutesWithoutError in the factory should reuse the existing PythonExecutionReq class.

Action Items

• Task Requirements Library: Implement generic Python tool requirements #1119: Focus on NEW requirements (PythonSyntaxValid, OutputSizeLimit, ImportRestrictions) and WRAP existing code extraction
• Task Requirements Library: Create python_tool_requirements() factory function #1121: Factory function should bundle existing + new requirements, reusing PythonExecutionReq directly
• Consider whether PythonExecutionReq needs to be renamed to PythonExecutesWithoutError or if we keep the existing name and wrap it

[akihikokuroda]

Requirements to Implement

The task specifies 5 generic Python tool requirements, but memory analysis reveals:

• ✅ PythonCodeExtraction — wrapper for existing _has_python_code_listing()
• ✅ PythonExecutesWithoutError — already implemented as PythonExecutionReq class
• ❌ PythonSyntaxValid — NEW (AST-based validation)
• ❌ OutputSizeLimit — NEW (captured output size enforcement)
• ❌ ImportRestrictions — NEW (import allowlist validation)

Implementation Strategy

Location: mellea/stdlib/requirements/python_tools.py

1. PythonCodeExtraction

class PythonCodeExtraction(Requirement):
    """Code blocks are present and extractable from model output."""
 
    def __init__(self):
        super().__init__(
            description="Code blocks are present and extractable.",
            validation_fn=_has_python_code_listing,  # reuse existing
            check_only=True,
        )

2. PythonSyntaxValid

import ast

class PythonSyntaxValid(Requirement):
    """Python code is syntactically valid (parses without AST errors)."""

   def __init__(self):
       super().__init__(
           description="Python code is syntactically valid.",
           validation_fn=self._validate_syntax,
           check_only=True,
       )

   def _validate_syntax(self, ctx: Context) -> ValidationResult:
       # Extract code using _has_python_code_listing
       # Try: ast.parse(code)
       # Catch SyntaxError → return False with reason
       pass

Test cases:

• Valid Python: passes
• Syntax errors: fails (unclosed parens, indentation, etc.)
• Generic code blocks: should extract first if no python marker

3. OutputSizeLimit

class OutputSizeLimit(Requirement):
    """Captured output does not exceed size limit."""
  
    def __init__(self, limit_chars: int = 10_000):
        self.limit_chars = limit_chars
        super().__init__(
           description=f"Output does not exceed {limit_chars} characters.",
            validation_fn=self._validate_output_size,
            check_only=True,
        )
 
    def _validate_output_size(self, ctx: Context) -> ValidationResult:
        # Extract code
        # Execute it (reuse _python_executes_without_error pattern)
        # Check captured stdout length
        pass

Test cases:

• Small output: passes
• Exceeds limit: fails with details
• No execution (syntax error): handle gracefully

4. ImportRestrictions

class ImportRestrictions(Requirement):
    """Only whitelisted modules are imported."""
 
    def __init__(self, allowed_imports: list[str] | None = None):
        self.allowed_imports = allowed_imports
        super().__init__(
            description=f"Only imports from {allowed_imports} are used.",
            validation_fn=self._validate_imports,
            check_only=True,
        )
 
    def _validate_imports(self, ctx: Context) -> ValidationResult:
        # AST analysis: find all Import/ImportFrom nodes
        # Check against allowlist
        # If no allowlist: accept everything
        pass

Test cases:

• Allowed imports: passes (e.g., os, sys, json)
• Forbidden imports: fails (e.g., subprocess, socket)
• No imports: passes
• Allowlist=None: passes all

5. PythonExecutesWithoutError

Already implemented as PythonExecutionReq. Task #1119 should:

• Document it in docstrings
• Ensure it appears in python_tools.py exports
• Note: do not reimplement; reuse existing

Exports from python_tools.py

__all__ = [
    "PythonCodeExtraction",
    "PythonSyntaxValid",
    "PythonExecutesWithoutError",  # alias for PythonExecutionReq
    "OutputSizeLimit",
    "ImportRestrictions",
]

Tests

• Create test/stdlib/requirements/test_python_tools.py
• Test each requirement class: positive and negative cases
• Test parameter propagation (e.g., OutputSizeLimit(limit_chars=100))
• Verify errors are informative (reason field)