Skip to content

Add GEPA optimization for MCP Tools#270

Draft
dandye wants to merge 1 commit into
mainfrom
gepa_mcp_optimizer_v0003
Draft

Add GEPA optimization for MCP Tools#270
dandye wants to merge 1 commit into
mainfrom
gepa_mcp_optimizer_v0003

Conversation

@dandye

@dandye dandye commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

Integrate GEPA Optimization and Refactor MCP Tool Routing

References

This pull request integrates GEPA (Gradient-based Engine for Prompt Ad-hoc Optimization) to automate the optimization of MCP tool docstrings and descriptions, improving model-routing accuracy across Google Threat Intelligence (GTI), SecOps SIEM, and SecOps SOAR MCP servers.

Key Changes

1. GEPA Optimization & Tool Routing

  • GEPA Integration: Added optimization packages and execution scripts (optimize_gti_mcp.py, optimize_soar_mcp.py, and optimize_secops_mcp.py) that evaluate routing accuracy and iterate on docstring formulations using Vertex AI models.
  • Evaluation Datasets: Added curated natural language queries to mcp_dataset.json for each server package to test tool resolution, parameter extraction, and baseline routing.
  • Docstring Updates: Refactored tool descriptions based on optimized outputs. Long markdown relationship tables (e.g., in netloc, files, and urls tools) were converted to concise, comma-separated lists to minimize context size and focus routing.

2. Google SecOps SOAR Improvements

  • Offline Fallback: Updated connection bindings to handle failed scope retrieval gracefully, allowing the server to continue in offline mode when API credentials are not provided.
  • Live Connection Test: Introduced test_live_soar.py to facilitate local verification of SOAR API bindings.
  • Enhanced Testing: Configured conftest.py to support automatic .env loading, and added new dynamic tests for retrieving alert events and entities.

3. Setup & Configuration

  • Environment Variables: Added a .env.example template containing required configuration variables for local server execution and GEPA optimization.
  • Git Configuration: Deleted obsolete .gitmodules and ensured local .env remains properly ignored.

Integrate Pareto-optimized tool definitions and relationship constraints under genetic-pareto framework

Update copyright headers to 2026 for touched files

optimize(soar-mcp): integrate gepa library optimization suite and refine credentials flow

test(soar-mcp): add live connection verification script

test(soar-mcp): expand integration tests with live endpoints and parameterized runs

docs, refactor: integrate GEPA optimization with Vertex AI Model Garden, clean up submodules and hardcoded paths
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant