Add fuzzing

[spoorcc]

See google/clusterfuzzlite#128

From ossf scorecard:

• Integrate the project with OSS-Fuzz by following the instructions here.

Risk: Medium (possible vulnerabilities in code)

This check tries to determine if the project uses fuzzing by checking:

• if the repository name is included in the OSS-Fuzz project list;
• if ClusterFuzzLite is deployed in the repository;
• if there are user-defined language-specified fuzzing functions in the repository.
  • currently only supports Go fuzzing,
  • a limited set of property-based testing libraries for Haskell including QuickCheck, Hedgehog, validity or SmallCheck,
  • a limited set of property-based testing libraries for JavaScript and TypeScript including fast-check.
  • a limited set of property-based testing libraries for Erlang, including proper and quickcheck.

Fuzzing, or fuzz testing, is the practice of feeding unexpected or random data into a program to expose bugs. Regular fuzzing is important to detect vulnerabilities that may be exploited by others, especially since attackers can also use fuzzing to find the same flaws.

Note

A project that fulfills this criterion with other tools may still receive a low score on this test. There are many ways to implement fuzzing, and it is challenging for an automated tool like Scorecard to detect them all. A low score
is therefore not a definitive indication that the project is at risk.

[spoorcc]

An example project can be seen here: https://github.com/google/oss-fuzz/tree/master/projects%2Fujson

With some documentation here https://google.github.io/oss-fuzz/getting-started/new-project-guide/python-lang/

[spoorcc]

@coderabbitai can you help planning implementation

[coderabbitai]

Coding Plan

Approach

Integrate fuzzing into dfetch using ClusterFuzzLite to detect potential vulnerabilities through continuous automated testing of manifest parsing, VCS URL handling, and input validation logic, thereby improving the project's OSSF Scorecard score.

Observations

dfetch is a Python-based VCS-agnostic dependency management tool that reads YAML manifests (dfetch.yaml) to fetch and manage external dependencies. The project uses a command-based architecture with 9 commands, extensive subprocess calls to Git, PyYAML for manifest parsing with pykwalify schema validation, and pytest/behave for testing. Key security-sensitive areas include manifest parsing (dfetch/manifest/), Git URL/path handling (dfetch/vcs/git.py), and minimal input sanitization beyond schema validation.

Assumptions

Assumption 1: Should fuzzing run real Git commands or mock them?

Options Considered:

• Mock subprocess calls to focus on parsing/validation layers without executing Git
• Allow real subprocess calls but with strict resource limits and timeouts

Chosen Option: 1

Rationale: Fuzzing will mock subprocess.run/Popen calls to focus on input parsing, URL manipulation, and validation logic without executing actual Git commands, providing fast deterministic fuzzing while still exercising security-critical code paths. The codebase makes extensive subprocess calls to Git for core functionality, but atheris best practices recommend avoiding subprocess execution during fuzzing (slow, non-deterministic, resource leaks).

Assumption 2: Should we use ClusterFuzzLite or prepare for full OSS-Fuzz submission?

Options Considered:

• Full OSS-Fuzz integration requiring submission to google/oss-fuzz repository
• ClusterFuzzLite GitHub Actions integration running fuzzing in project's own CI
• Local fuzzing only without CI integration

Chosen Option: 2

Rationale: ClusterFuzzLite provides immediate OSSF Scorecard improvement without external dependencies, aligns with existing GitHub Actions infrastructure, and maintains a path to graduate to full OSS-Fuzz if desired in the future. The ticket mentions both OSS-Fuzz and ClusterFuzzLite as acceptable solutions. Full OSS-Fuzz requires external approval and coordination, while ClusterFuzzLite can be implemented entirely within the project repository.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Fuzzing Infrastructure Setup

Establish the foundational fuzzing infrastructure by adding atheris dependency, creating the fuzzers directory structure, and implementing helper utilities for mocking subprocess calls.

Task 1: Add atheris dependency

Add atheris as a development dependency to enable Python fuzzing capabilities

• Update pyproject.toml [tool.project.optional-dependencies] section to include atheris in a new "fuzzing" extras group alongside existing dev, doc, test, build, and sbom groups
• Pin atheris to a stable version (e.g., atheris>=2.3.0) compatible with Python >=3.9 requirement

Task 2: Create fuzzers directory structure

Establish directory hierarchy for fuzzing targets and seed corpus following OSS-Fuzz conventions

• Create fuzzers/ directory in repository root for housing all fuzzing-related files
• Create fuzzers/seed_corpus/ subdirectory for storing example inputs
• Create fuzzers/dictionaries/ subdirectory for format-specific token dictionaries
• Add README.md in fuzzers/ documenting purpose, local execution instructions, and relationship to ClusterFuzzLite

Task 3: Implement subprocess mocking utilities

Create reusable helpers to mock subprocess calls during fuzzing for deterministic, fast execution

• Create fuzzers/fuzzer_utils.py with MockSubprocess class that patches subprocess.run and subprocess.Popen
• MockSubprocess should return controlled CompletedProcess objects with configurable stdout, stderr, and returncode
• Add context manager support (with statement) for clean setup/teardown in fuzz targets
• Include example Git command responses (ls-remote output, config output) as constants for realistic mocking

🤖 Prompt for AI agents

Set up the foundational fuzzing infrastructure for dfetch including atheris
dependency, directory structure, and subprocess mocking utilities.

Add atheris>=2.3.0 to pyproject.toml as a "fuzzing" extras group in
optional-dependencies

Create directory structure:
- fuzzers/ (root directory)
- fuzzers/seed_corpus/
- fuzzers/dictionaries/
- fuzzers/README.md with usage instructions

Implement fuzzers/fuzzer_utils.py:
- MockSubprocess class that patches subprocess.run and subprocess.Popen
- Return configurable CompletedProcess objects
- Implement context manager protocol
- Include constants for typical Git command responses

Phase 2: Core Fuzz Target Implementation

Implement atheris-based fuzz targets for the highest-value attack surfaces: manifest parsing, Git URL handling, and project entry validation.

Task 1: Implement manifest parsing fuzzer

Create fuzz target for YAML manifest parsing to detect parsing bugs, type confusion, and validation bypass

• Create fuzzers/fuzz_manifest.py that imports dfetch.manifest.Manifest within atheris.instrument_imports() context
• Implement TestOneInput(data) function that:
  • Attempts Manifest.from_yaml() with fuzzed data as YAML text
  • Catches expected exceptions (yaml.YAMLError, RuntimeError, pykwalify errors) to avoid false positives
  • Validates that successfully parsed manifests conform to expected structure (has version, projects list)
• Use atheris.Setup() with enable_python_coverage=True and call atheris.Fuzz()
• Handle both valid and invalid YAML structures to exercise parser edge cases

Task 2: Implement Git URL parsing fuzzer

Create fuzz target for Git URL manipulation and remote handling to detect injection and traversal vulnerabilities

• Create fuzzers/fuzz_git_url.py that imports dfetch.vcs.git within atheris.instrument_imports() context
• Use MockSubprocess from fuzzer_utils.py to prevent actual Git command execution
• Implement TestOneInput(data) function that:
  • Decodes data as UTF-8 URL string with error handling
  • Creates GitRemote instances with fuzzed URLs
  • Exercises GitRemote._ensure_abs_url() with relative URL patterns
  • Tests GitRemote methods that parse Git output (get_latest_tag, get_default_branch, list_tool_info)
• Catch expected exceptions from URL validation and Git parsing without crashing fuzzer
• Focus on URL string manipulation, regex parsing, and relative path resolution code paths

Task 3: Implement project entry validation fuzzer

Create fuzz target for ProjectEntry.from_yaml() to test field combination validation and edge cases

• Create fuzzers/fuzz_project_entry.py that imports dfetch.manifest.project within atheris.instrument_imports() context
• Implement TestOneInput(data) function that:
  • Attempts to parse data as YAML dictionary suitable for ProjectEntry.from_yaml()
  • Tests various field combinations (vcs, url, remote, branch, tag, revision, src, dst, patch)
  • Validates that successfully created entries have internally consistent state
  • Catches schema validation errors and type errors as expected behavior
• Exercise optional field handling, uniqueness constraints, and cross-field validation logic

🤖 Prompt for AI agents

Implement three atheris-based fuzz targets to test manifest parsing, Git URL
handling, and project entry validation.

Create fuzzers/fuzz_manifest.py:
- Import dfetch.manifest.Manifest within atheris.instrument_imports()
- TestOneInput(data) calls Manifest.from_yaml() with fuzzed data
- Catch expected exceptions (yaml.YAMLError, RuntimeError, pykwalify)
- Validate parsed manifest structure
- Setup with enable_python_coverage=True

Create fuzzers/fuzz_git_url.py:
- Import dfetch.vcs.git within atheris.instrument_imports()
- Use MockSubprocess to prevent Git execution
- TestOneInput(data) creates GitRemote with fuzzed URLs
- Exercise _ensure_abs_url(), get_latest_tag(), get_default_branch(),
list_tool_info()
- Catch expected exceptions from URL validation

Create fuzzers/fuzz_project_entry.py:
- Import dfetch.manifest.project within atheris.instrument_imports()
- TestOneInput(data) calls ProjectEntry.from_yaml() with fuzzed YAML dict
- Test various field combinations
- Catch schema validation and type errors

Phase 3: Seed Corpus and Dictionary Creation

Build initial seed corpus and token dictionaries to guide fuzzing toward meaningful inputs and improve coverage efficiency.

Task 1: Create manifest seed corpus

Assemble diverse example manifests covering valid, edge-case, and intentionally malformed inputs

• Add fuzzers/seed_corpus/manifest/ directory with multiple .yaml files including:
  • minimal_valid.yaml: Smallest valid manifest (version + one project)
  • complex_valid.yaml: Manifest with multiple remotes, projects, all optional fields
  • missing_version.yaml: Manifest missing required version field
  • invalid_vcs.yaml: Project with unsupported VCS type
  • path_traversal.yaml: Manifest with ../../../ patterns in src/dst
  • malformed.yaml: Partially truncated YAML for parser edge cases
• Source examples from dfetch.yaml in repository root and modify for coverage

Task 2: Create Git URL seed corpus

Build corpus of Git URLs representing various schemes, edge cases, and malicious patterns

• Add fuzzers/seed_corpus/git_url/ directory with .txt files containing:
  • https://github.com/user/repo.git (standard HTTPS)
  • git@github.com:user/repo.git (SSH)
  • ../relative/path (relative URL patterns)
  • file:///absolute/path (file protocol)
  • URLs with special characters, Unicode, very long paths, empty components
• Include patterns from exploration findings showing relative URL resolution code

Task 3: Create fuzzing dictionaries

Define token dictionaries with format-specific keywords to bias mutations toward syntactically valid inputs

• Create fuzzers/dictionaries/yaml.dict with YAML syntax tokens (keys, scalars, collections syntax)
• Create fuzzers/dictionaries/manifest.dict with dfetch-specific keywords:
  • Field names: manifest, version, remotes, projects, name, url-base, dst, src, vcs, branch, tag, revision, patch
  • VCS types: git, svn
  • Common values: origin, main, master, v0.0, HEAD
• Create fuzzers/dictionaries/git_url.dict with URL components (protocols, path segments, special characters)

🤖 Prompt for AI agents

Build seed corpus and dictionaries to improve fuzzing efficiency and guide
mutations toward meaningful inputs.

Create fuzzers/seed_corpus/manifest/ with YAML files:
- minimal_valid.yaml (smallest valid manifest)
- complex_valid.yaml (all features)
- missing_version.yaml (missing required field)
- invalid_vcs.yaml (unsupported VCS)
- path_traversal.yaml (../ patterns)
- malformed.yaml (truncated YAML)

Create fuzzers/seed_corpus/git_url/ with .txt files:
- HTTPS URLs
- SSH URLs
- Relative paths
- File protocol URLs
- Special characters, Unicode, long paths

Create fuzzers/dictionaries/:
- yaml.dict (YAML syntax tokens)
- manifest.dict (dfetch field names, VCS types, common values)
- git_url.dict (URL components and protocols)

Phase 4: ClusterFuzzLite CI Integration

Configure GitHub Actions workflow to run ClusterFuzzLite, enabling continuous fuzzing in CI and crash artifact collection.

Task 1: Create ClusterFuzzLite workflow

Add GitHub Actions workflow file to run fuzzing on pull requests and scheduled intervals

• Create .github/workflows/cfl.yml (ClusterFuzzLite workflow) following ClusterFuzzLite documentation
• Configure workflow to:
  • Trigger on pull_request and schedule (daily or weekly)
  • Use ClusterFuzzLite action (google/clusterfuzzlite/actions/*)
  • Set language: python and specify fuzz-seconds (300-600 for PR, longer for scheduled)
  • Build fuzz targets using pip install with fuzzing extras
  • Run all fuzzers in fuzzers/ directory
  • Upload crashes as artifacts and create issues for new crashes
• Reference existing action.yml GitHub Action pattern for consistency with project CI structure

Task 2: Configure ClusterFuzzLite project settings

Add ClusterFuzzLite configuration file with fuzzing parameters and sanitizer options

• Create .clusterfuzzlite/project.yaml configuration file with:
  • language: python
  • List of fuzz_targets pointing to fuzzers/fuzz_*.py files
  • Sanitizers: address (if native extensions detected) and undefined for memory safety
  • Max_total_time limits appropriate for CI environment
• Align sanitizer configuration with Python >=3.9 compatibility and atheris requirements

Task 3: Add fuzzer local execution scripts

Create helper scripts for developers to run fuzzers locally during development

• Create fuzzers/run_fuzzer.py Python script that:
  • Accepts fuzzer name as argument
  • Sets up subprocess mocking context
  • Runs specified fuzzer with atheris for configurable duration
  • Reports coverage and crash information
• Create fuzzers/run_all_fuzzers.sh Bash script to execute all fuzzers sequentially with timeout
• Document usage in fuzzers/README.md for developer onboarding

🤖 Prompt for AI agents

Integrate ClusterFuzzLite into CI/CD pipeline and provide local execution tools
for developers.

Create .github/workflows/cfl.yml:
- Trigger on pull_request and schedule (daily/weekly)
- Use google/clusterfuzzlite/actions
- Set language: python, fuzz-seconds (300-600 for PR, longer for scheduled)
- Build with pip install fuzzing extras
- Run all fuzzers in fuzzers/
- Upload crash artifacts and create issues

Create .clusterfuzzlite/project.yaml:
- language: python
- List fuzz_targets (fuzzers/fuzz_*.py)
- Configure sanitizers (address, undefined)
- Set max_total_time for CI

Create local execution scripts:
- fuzzers/run_fuzzer.py (single fuzzer with arguments)
- fuzzers/run_all_fuzzers.sh (all fuzzers with timeout)
- Update fuzzers/README.md with usage

Phase 5: Documentation and OSS-Fuzz Preparation

Document the fuzzing infrastructure, provide usage instructions, and prepare materials for potential future OSS-Fuzz submission.

Task 1: Add fuzzing documentation

Create comprehensive documentation for fuzzing infrastructure and development workflows

• Add doc/fuzzing.md (or similar location aligned with existing documentation structure) explaining:
  • Overview of fuzzing targets and their purpose
  • How to run fuzzers locally using scripts from Task 4.3
  • How ClusterFuzzLite integrates with CI/CD
  • How to interpret crash reports and triage findings
  • How to add new fuzz targets following established patterns
• Update main README.md with link to fuzzing documentation and badge for ClusterFuzzLite status

Task 2: Create OSS-Fuzz preparation guide

Document steps for future OSS-Fuzz submission if project grows and team decides to pursue it

• Create fuzzers/OSS_FUZZ_PREP.md with guidance on:
  • How current structure aligns with OSS-Fuzz requirements
  • What additional files would be needed (build.sh in oss-fuzz repo, project.yaml modifications)
  • Dependencies and build process translation to OSS-Fuzz environment
  • Differences between ClusterFuzzLite and full OSS-Fuzz integration
• Reference exploration findings about OSS-Fuzz best practices and requirements

Task 3: Update contribution guidelines

Extend existing contribution documentation to include fuzzing-related workflows

• Update CONTRIBUTING.md (or equivalent) to mention:
  • Running fuzzers before submitting PRs that touch parsing/validation code
  • Adding seed corpus entries when introducing new manifest features
  • Expectation that ClusterFuzzLite checks should pass on PRs
• Add fuzzing to checklist for security-sensitive code changes in areas identified by exploration (manifest parsing, VCS URL handling)

🤖 Prompt for AI agents

Document the fuzzing infrastructure and prepare for potential future OSS-Fuzz
submission.

Create doc/fuzzing.md:
- Overview of fuzz targets and purpose
- Local execution instructions
- ClusterFuzzLite CI/CD integration explanation
- Crash report interpretation and triage
- How to add new fuzz targets
- Update README.md with link and ClusterFuzzLite badge

Create fuzzers/OSS_FUZZ_PREP.md:
- Alignment with OSS-Fuzz requirements
- Additional files needed for full OSS-Fuzz
- Build process translation
- Differences between ClusterFuzzLite and OSS-Fuzz

Update CONTRIBUTING.md:
- Run fuzzers before PRs on parsing/validation code
- Add seed corpus for new features
- ClusterFuzzLite check expectations
- Fuzzing checklist for security-sensitive changes

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Set up the foundational fuzzing infrastructure for dfetch including atheris
dependency, directory structure, and subprocess mocking utilities.

Add atheris>=2.3.0 to pyproject.toml as a "fuzzing" extras group in
optional-dependencies

Create directory structure:
- fuzzers/ (root directory)
- fuzzers/seed_corpus/
- fuzzers/dictionaries/
- fuzzers/README.md with usage instructions

Implement fuzzers/fuzzer_utils.py:
- MockSubprocess class that patches subprocess.run and subprocess.Popen
- Return configurable CompletedProcess objects
- Implement context manager protocol
- Include constants for typical Git command responses
===============================================================================

Task: 2

Implement three atheris-based fuzz targets to test manifest parsing, Git URL
handling, and project entry validation.

Create fuzzers/fuzz_manifest.py:
- Import dfetch.manifest.Manifest within atheris.instrument_imports()
- TestOneInput(data) calls Manifest.from_yaml() with fuzzed data
- Catch expected exceptions (yaml.YAMLError, RuntimeError, pykwalify)
- Validate parsed manifest structure
- Setup with enable_python_coverage=True

Create fuzzers/fuzz_git_url.py:
- Import dfetch.vcs.git within atheris.instrument_imports()
- Use MockSubprocess to prevent Git execution
- TestOneInput(data) creates GitRemote with fuzzed URLs
- Exercise _ensure_abs_url(), get_latest_tag(), get_default_branch(),
list_tool_info()
- Catch expected exceptions from URL validation

Create fuzzers/fuzz_project_entry.py:
- Import dfetch.manifest.project within atheris.instrument_imports()
- TestOneInput(data) calls ProjectEntry.from_yaml() with fuzzed YAML dict
- Test various field combinations
- Catch schema validation and type errors
===============================================================================

Task: 3

Build seed corpus and dictionaries to improve fuzzing efficiency and guide
mutations toward meaningful inputs.

Create fuzzers/seed_corpus/manifest/ with YAML files:
- minimal_valid.yaml (smallest valid manifest)
- complex_valid.yaml (all features)
- missing_version.yaml (missing required field)
- invalid_vcs.yaml (unsupported VCS)
- path_traversal.yaml (../ patterns)
- malformed.yaml (truncated YAML)

Create fuzzers/seed_corpus/git_url/ with .txt files:
- HTTPS URLs
- SSH URLs
- Relative paths
- File protocol URLs
- Special characters, Unicode, long paths

Create fuzzers/dictionaries/:
- yaml.dict (YAML syntax tokens)
- manifest.dict (dfetch field names, VCS types, common values)
- git_url.dict (URL components and protocols)
===============================================================================

Task: 4

Integrate ClusterFuzzLite into CI/CD pipeline and provide local execution tools
for developers.

Create .github/workflows/cfl.yml:
- Trigger on pull_request and schedule (daily/weekly)
- Use google/clusterfuzzlite/actions
- Set language: python, fuzz-seconds (300-600 for PR, longer for scheduled)
- Build with pip install fuzzing extras
- Run all fuzzers in fuzzers/
- Upload crash artifacts and create issues

Create .clusterfuzzlite/project.yaml:
- language: python
- List fuzz_targets (fuzzers/fuzz_*.py)
- Configure sanitizers (address, undefined)
- Set max_total_time for CI

Create local execution scripts:
- fuzzers/run_fuzzer.py (single fuzzer with arguments)
- fuzzers/run_all_fuzzers.sh (all fuzzers with timeout)
- Update fuzzers/README.md with usage
===============================================================================

Task: 5

Document the fuzzing infrastructure and prepare for potential future OSS-Fuzz
submission.

Create doc/fuzzing.md:
- Overview of fuzz targets and purpose
- Local execution instructions
- ClusterFuzzLite CI/CD integration explanation
- Crash report interpretation and triage
- How to add new fuzz targets
- Update README.md with link and ClusterFuzzLite badge

Create fuzzers/OSS_FUZZ_PREP.md:
- Alignment with OSS-Fuzz requirements
- Additional files needed for full OSS-Fuzz
- Build process translation
- Differences between ClusterFuzzLite and OSS-Fuzz

Update CONTRIBUTING.md:
- Run fuzzers before PRs on parsing/validation code
- Add seed corpus for new features
- ClusterFuzzLite check expectations
- Fuzzing checklist for security-sensitive changes

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- @coderabbitai You can skip phase 3. Add a simple unit test case for phase 2.
- @coderabbitai For assumption 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!