[ci-coach] Optimize integration test matrix by isolating slow Docker build test

[github-actions]

CI Optimization: Reduce Integration Test Critical Path by 41%

Summary

This PR optimizes the integration test matrix by isolating the slow TestDockerBuild tests into a dedicated matrix group, reducing the critical path from 67.7s to ~40s (41% improvement).

Problem Analysis

The "CLI Completion & Other" catch-all group was the slowest integration test group at 67.7s, with a single test dominating execution time:

• TestDockerBuild_WithMake: 38.43s (57% of group time)
• Other Docker tests: ~0.4s combined
• Total Docker tests: 3 tests, 38.4s

This bottleneck blocked the critical path, preventing faster test groups from completing earlier.

Solution

Created a new dedicated matrix group for Docker build tests:

1. New group: "CLI Docker Build"

   • Pattern: TestDockerBuild
   • Expected duration: ~38-40s
   • Runs in parallel with other groups

2. Updated "CLI Completion & Other" skip pattern

   • Added TestDockerBuild to exclusions
   • Reduced from 67.7s to ~29s (estimated)

Changes

File: .github/workflows/ci.yml

Added new matrix entry (after line 107):

- name: "CLI Docker Build"  # Isolate slow Docker build test (~38s)
  packages: "./pkg/cli"
  pattern: "TestDockerBuild"

Updated "CLI Completion & Other" skip pattern (line 111):

skip_pattern: "...|^TestAudit|^TestInspect|TestDockerBuild"

Expected Impact

Before:

• Total matrix groups: 31
• Critical path: 67.7s (CLI Completion & Other)
• Docker tests block other test completion

After:

• Total matrix groups: 32 (+1)
• Critical path: ~40s (CLI Docker Build, runs in parallel)
• Improvement: 41% faster critical path
• Time saved: ~28 seconds per CI run

Cost-Benefit Analysis:

• Impact: High (41% speedup on integration tests)
• Effort: Low (4 line change, no logic changes)
• Risk: Low (existing tests, just rebalancing execution)
• Reversibility: High (easy to revert if issues arise)

Validation Results

✅ YAML syntax valid: Skip pattern correctly includes TestDockerBuild
✅ Matrix count: 32 groups (31 → 32, expected)
✅ CLI groups: 14 CLI test groups (13 → 14, expected)
✅ Pattern isolation: Docker tests will run in dedicated group

Note: Full validation with make lint && make build && make test-unit && make recompile cannot be performed in this workflow environment (no Go/Make available), but YAML structure has been verified.

Testing Plan

• Verify workflow syntax passes GitHub Actions validation
• Monitor first CI run after merge:
  • CLI Docker Build group should take ~38-40s
  • CLI Completion & Other should drop to ~25-30s
  • Critical path should be ~40s (down from 67.7s)
• Compare with historical baselines from previous runs
• Easy rollback: Revert this single commit if issues arise

Metrics Baseline (for comparison)

Current state (from CI run analysis):

• Success rate: 71% (71 success, 26 cancelled, 3 action_required out of 100 runs)
• Matrix groups: 31
• Longest integration group: 67.7s (CLI Completion & Other)

Expected post-merge:

• Matrix groups: 32
• Longest integration group: ~40s (CLI Docker Build)
• CLI Completion & Other: ~29s (reduced from 67.7s)
• Time saved per run: ~28s

Analysis Methodology

This optimization was identified through systematic analysis of CI run data:

1. Analyzed: 100 recent workflow runs
2. Examined: Integration test artifacts with per-test timing from successful runs
3. Identified: TestDockerBuild_WithMake taking 38.43s (57% of group time)
4. Root cause: Single slow test in large catch-all group creating bottleneck
5. Solution: Isolate slow tests into dedicated parallel group

View Detailed Test Timing Analysis

CLI Completion & Other Group (67.7s total)

Top 10 slowest tests:

1. TestDockerBuild_WithMake: 38.43s ⚠️ (isolated by this PR)
2. TestCollectWorkflowFiles_NoLockFile: 3.14s
3. TestCompileWorkflowsWithNoEmit: 3.13s
4. TestCompileWorkflows_TrialMode: 3.03s
5. TestInteractiveWorkflowBuilder_compileWorkflow_SpinnerIntegration: 1.85s
6. TestCompileWorkflows/nonexistent_specific_file: 1.62s
7. TestCompileWorkflows: 1.62s
8. TestDownloadRunArtifactsConcurrent_ConcurrencyLimit: 0.70s
9. TestCompileWithFuzzyDailyScheduleDeterministic: 0.45s
10. TestInstallPackage: 0.41s

Test grouping:

• Docker: 3 tests, 38.4s (isolated by this PR)
• Compile: 54 tests, 12.9s
• Collect: 17 tests, 3.2s
• Completion: 45 tests, 2.1s
• Interactive: 17 tests, 1.9s
• Other: ~2,300 tests, ~11s

After this PR:

• CLI Docker Build: 3 tests, ~38-40s (new dedicated group)
• CLI Completion & Other: 2,440 tests, ~29s (reduced from 67.7s)

---

References:

• Analysis workflow run: §21287478106

This optimization maintains 100% test coverage while improving CI efficiency through better workload distribution.

AI generated by CI Optimization Coach

---

Note

This was originally intended as a pull request, but the git push operation failed.

Workflow Run: View run details and download patch artifact

The patch file is available in the agent-artifacts artifact in the workflow run linked above.

To apply the patch locally:

# Download the artifact from the workflow run https://github.com/githubnext/gh-aw/actions/runs/21287478106
# (Use GitHub MCP tools if gh CLI is not available)
gh run download 21287478106 -n agent-artifacts

# The patch file will be at agent-artifacts/tmp/gh-aw/aw.patch after download
# Apply the patch
git am agent-artifacts/tmp/gh-aw/aw.patch

Show patch preview (52 of 52 lines)

From 0e700b6b2a7150a118c5d4558ac18a1c3dd0e397 Mon Sep 17 00:00:00 2001
From: CI Coach <ci-coach@github.com>
Date: Fri, 23 Jan 2026 13:29:17 +0000
Subject: [PATCH] ci: isolate slow Docker build test into dedicated matrix
 group
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Reduces integration test critical path from 67.7s to ~40s (41% improvement)

The 'CLI Completion & Other' catch-all group was the slowest at 67.7s,
with TestDockerBuild_WithMake taking 38.43s alone. This change:

- Creates dedicated 'CLI Docker Build' matrix group for Docker tests
- Updates 'CLI Completion & Other' skip pattern to exclude Docker tests
- Allows Docker tests to run in parallel, reducing critical path

Expected impact:
- Critical path: 67.7s → ~40s (41% faster)
- Time saved per CI run: ~28 seconds
- Matrix groups: 31 → 32 (minimal overhead)

Analysis based on CI run data showing 2,443 tests in the catch-all group
with Docker build test dominating execution time.
---
 .github/workflows/ci.yml | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 5a58a05..28be06e 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -105,10 +105,13 @@ jobs:
           - name: "CLI Audit & Inspect"
             packages: "./pkg/cli"
             pattern: "^TestAudit|^TestInspect"
+          - name: "CLI Docker Build"  # Isolate slow Docker build test (~38s)
+            packages: "./pkg/cli"
+            pattern: "TestDockerBuild"
           - name: "CLI Completion & Other"  # Remaining catch-all (reduced from original)
             packages: "./pkg/cli"
             pattern: ""  # Catch-all for tests not matched by other CLI patterns
-            skip_pattern: "^TestCompile[^W]|TestPoutine|TestMCPInspectPlaywright|TestMCPGateway|TestMCPAdd|TestMCPInspectGitHub|TestMCPServer|TestMCPConfig|TestLogs|TestFirewall|TestNoStopTime|TestLocalWorkflow|TestProgressFlagSignature
... (truncated)