Skip to content

Reduce CompileMCPWorkflow overhead by caching runtime requirement detection per compile#43833

Merged
pelikhan merged 5 commits into
mainfrom
copilot/performance-regression-fix-compile-mcp-workflow
Jul 7, 2026
Merged

Reduce CompileMCPWorkflow overhead by caching runtime requirement detection per compile#43833
pelikhan merged 5 commits into
mainfrom
copilot/performance-regression-fix-compile-mcp-workflow

Conversation

Copilot AI commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

BenchmarkCompileMCPWorkflow regressed by ~10% due to repeated work in the compile path. This change removes duplicate runtime-requirement detection within a single compilation while preserving behavior.

  • What changed

    • Added per-WorkflowData cache fields for runtime requirements:
      • CachedRuntimeRequirements
      • CachedRuntimeRequirementsSet
    • Introduced detectRuntimeRequirementsCached(...) in pkg/workflow/runtime_detection.go.
    • Updated both compile-path consumers to reuse the same computed runtime requirements:
      • validateRuntimePackages(...) (pkg/workflow/runtime_validation.go)
      • prepareRuntimeSetupAndCheckoutInfo(...) (pkg/workflow/compiler_yaml_main_job.go)
  • Behavioral guardrail

    • Cache reads return a defensive copy so downstream mutation cannot corrupt cached state across phases.
  • Targeted correctness coverage

    • Added TestDetectRuntimeRequirementsCached_ReturnsDefensiveCopy to verify copy-on-read semantics.
func detectRuntimeRequirementsCached(workflowData *WorkflowData) []RuntimeRequirement {
	if workflowData == nil {
		return nil
	}
	if workflowData.CachedRuntimeRequirementsSet {
		return cloneRuntimeRequirements(workflowData.CachedRuntimeRequirements)
	}
	requirements := DetectRuntimeRequirements(workflowData)
	workflowData.CachedRuntimeRequirements = cloneRuntimeRequirements(requirements)
	workflowData.CachedRuntimeRequirementsSet = true
	return requirements
}

Generated by 👨‍🍳 PR Sous Chef · 7.29 AIC · ⌖ 7.3 AIC · ⊞ 7.1K ·
Comment /souschef to run again


Generated by 👨‍🍳 PR Sous Chef · 6.99 AIC · ⌖ 5.55 AIC · ⊞ 7.1K ·
Comment /souschef to run again

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix performance regression in CompileMCPWorkflow Reduce CompileMCPWorkflow overhead by caching runtime requirement detection per compile Jul 6, 2026
Copilot AI requested a review from pelikhan July 6, 2026 17:05
@pelikhan pelikhan marked this pull request as ready for review July 6, 2026 17:08
Copilot AI review requested due to automatic review settings July 6, 2026 17:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces repeated work during compilation by caching detected runtime requirements on WorkflowData so both YAML generation and runtime package validation can reuse the same detection result within a single compile.

Changes:

  • Added WorkflowData fields to cache runtime requirements and track whether the cache is populated.
  • Introduced detectRuntimeRequirementsCached(...) (with slice cloning) and switched compile-path consumers to use it.
  • Added a unit test intended to verify copy-on-read behavior for the cached runtime requirements.
Show a summary per file
File Description
pkg/workflow/workflow_data.go Adds per-compilation cache fields for detected runtime requirements.
pkg/workflow/runtime_validation.go Reuses cached runtime requirements during runtime package validation.
pkg/workflow/runtime_detection.go Implements cached runtime requirement detection with cloning to reduce duplicate work.
pkg/workflow/runtime_detection_cache_test.go Adds a test for defensive-copy semantics of cached runtime requirements.
pkg/workflow/compiler_yaml_main_job.go Reuses cached runtime requirements during runtime setup/checkout preparation.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 5/5 changed files
  • Comments generated: 1
  • Review effort level: Low

Comment on lines +9 to +23
func TestDetectRuntimeRequirementsCached_ReturnsDefensiveCopy(t *testing.T) {
workflowData := &WorkflowData{
CustomSteps: "run: node --version",
}

first := detectRuntimeRequirementsCached(workflowData)
require.NotEmpty(t, first)
require.True(t, workflowData.CachedRuntimeRequirementsSet)

first[0].Version = "mutated-version"

second := detectRuntimeRequirementsCached(workflowData)
require.NotEmpty(t, second)
require.NotEqual(t, "mutated-version", second[0].Version)
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in cf66986. The cache test now exercises a forced cache-hit path by pre-populating CachedRuntimeRequirements and verifying mutations to the returned slice do not affect the cached value or a subsequent read.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Test Quality Sentinel completed test quality analysis.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

⚠️ PR Code Quality Reviewer failed during code quality review.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR does not have the 'implementation' label and has ≤100 new lines of code in business logic directories (49 additions detected, threshold is 100).

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 35/100 — Poor

Analyzed 1 test(s): 1 design, 0 implementation, 1 violation(s).

📊 Metrics (1 test)
Metric Value
Analyzed 1 (Go: 1, JS: 0)
✅ Design 1 (100%)
⚠️ Implementation 0 (0%)
Edge/error coverage 1 (100%)
Duplicate clusters 0
Inflation No
🚨 Violations 1 (missing build tag)
Test File Classification Issues
TestDetectRuntimeRequirementsCached_ReturnsDefensiveCopy runtime_detection_cache_test.go:9 design_test / high_value Missing //go:build !integration build tag; assertions lack descriptive failure context
⚠️ Flagged Tests (1)

TestDetectRuntimeRequirementsCached_ReturnsDefensiveCopy (runtime_detection_cache_test.go:9) — This is a strong behavioral/design test: it verifies the defensive-copy invariant of detectRuntimeRequirementsCached, catching a real regression class (caller mutation leaking into the cache). However, it is missing the mandatory //go:build !integration tag on line 1 — this is a hard coding-guideline violation in this repo where every non-integration test file must carry the tag. Additionally, the require assertions lack descriptive failure context messages, making failures harder to diagnose.

Fix: Add //go:build !integration as the very first line of the file, and add descriptive messages to each require call.

Verdict

Failed. 0% implementation tests (threshold: 30%) — passes ratio check. Fails due to 1 coding-guideline violation: new Go test file runtime_detection_cache_test.go is missing the mandatory //go:build !integration build tag on line 1.

References: §28809340647

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🧪 Test quality analysis by Test Quality Sentinel · 37.1 AIC · ⌖ 11.4 AIC · ⊞ 6.8K ·
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Test Quality Sentinel: 35/100. Missing mandatory //go:build !integration build tag on line 1 of runtime_detection_cache_test.go (hard coding-guideline violation). Review flagged tests in the comment above.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /codebase-design and /tdd — requesting changes on two correctness issues in the cloning logic and corresponding test gaps.

📋 Key Themes & Highlights

Key Issues

  • Shallow clone bug: cloneRuntimeRequirements uses copy(), which is a shallow copy. RuntimeRequirement.ExtraFields is a map[string]any — a reference type — so clones share the same underlying map. A caller mutating ExtraFields will silently corrupt the cached state, defeating the stated "defensive copy" guarantee.
  • Asymmetric return on cache-miss: The first call (cache-miss path) returns the raw slice from DetectRuntimeRequirements directly, while subsequent calls (cache-hit path) return a clone. The defensive copy guarantee currently depends on call order, which is fragile.
  • Test specification gap: The existing test only mutates Version (a value type), so it does not catch the shallow-clone bug on ExtraFields. Tests for nil input and zero-requirements input are also absent.

Positive Highlights

  • ✅ Clean, minimal scope — only two call sites changed, no unrelated refactoring
  • ✅ The CachedRuntimeRequirementsSet boolean sentinel follows the same established pattern as CachedAllowedDomainsComputed, keeping WorkflowData consistent
  • ✅ Good PR description: the behavioral guardrail section clearly states the defensive-copy intent
  • ✅ New test file is well-structured and the test name is descriptive

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 61.6 AIC · ⌖ 7.52 AIC · ⊞ 6.7K
Comment /matt to run again

Comment thread pkg/workflow/runtime_detection.go
Comment thread pkg/workflow/runtime_detection_cache_test.go
Comment thread pkg/workflow/runtime_detection.go Outdated
requirements := DetectRuntimeRequirements(workflowData)
workflowData.CachedRuntimeRequirements = cloneRuntimeRequirements(requirements)
workflowData.CachedRuntimeRequirementsSet = true
return requirements

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/codebase-design] On the cache-miss path, detectRuntimeRequirementsCached returns requirements directly (line 33) — the same slice returned by DetectRuntimeRequirements. The cached copy is protected, but the returned slice on the first call is unprotected: a caller who mutates the first return value is mutating live data (the returned slice is the original, not a clone). Consider returning cloneRuntimeRequirements(requirements) on this path too, for consistency with the cache-hit path.

💡 Suggested fix
requirements := DetectRuntimeRequirements(workflowData)
workflowData.CachedRuntimeRequirements = cloneRuntimeRequirements(requirements)
workflowData.CachedRuntimeRequirementsSet = true
return cloneRuntimeRequirements(requirements) // symmetric with cache-hit path

This makes every return from detectRuntimeRequirementsCached an independent clone, so the invariant does not depend on call order.

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in cf66986. The cache-miss path now returns a cloned slice from the cached snapshot, so both miss and hit paths have the same defensive-copy semantics.

Comment thread pkg/workflow/runtime_detection_cache_test.go

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Caching runtime requirement detection

The intent and overall structure are sound — caching DetectRuntimeRequirements per compile to avoid redundant work is a clear win, and the test coverage for the defensive-copy property is a good addition.

Two correctness issues in the clone implementation and one test-fixture concern need addressing before merge.

Blocking

  1. Shallow clone misses reference fields (cloneRuntimeRequirements, line 19 of runtime_detection.go)
    copy() on a []RuntimeRequirement slice does a bitwise struct copy. The ExtraFields map[string]any and Runtime *Runtime fields are shared between the cached snapshot and every returned copy. For the current call order this is incidentally safe (ExtraFields is nil at detection time), but the declared invariant — defensive copy — is not actually enforced for those fields. The fix is a per-element deep clone of ExtraFields.

  2. Cache-miss path returns the raw, uncloned slice (detectRuntimeRequirementsCached, line 33 of runtime_detection.go)
    On the first call the function returns requirements (the value from DetectRuntimeRequirements) directly, while subsequent calls return a clone of the cached copy. The two callers therefore receive objects with different isolation semantics on the first call vs. all later calls. This asymmetry is a latent bug — if the first caller mutates an ExtraFields map, the cached snapshot is already protected (cloned before storage), but the inconsistency makes reasoning about correctness harder and will bite if ExtraFields is ever non-nil at detection time.

Non-blocking

  1. Test fixture uses invalid-YAML CustomSteps (runtime_detection_cache_test.go, line 62)
    "run: node --version" is not valid step YAML (missing steps: wrapper), so it passes through YAML parsing silently and relies on the raw-string command scanner in detectFromCustomSteps. The test works but is fragile. Prefer a well-formed fixture or an in-struct ParsedTools injection.
No issues found in remaining changed files
  • compiler_yaml_main_job.go: straightforward call-site swap, correct.
  • runtime_validation.go: straightforward call-site swap, correct.
  • workflow_data.go: field additions are well-commented and consistent with the existing CachedAllowedDomains* pattern.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · 103.4 AIC · ⌖ 6.25 AIC · ⊞ 4.9K

Comments that could not be inline-anchored

pkg/workflow/runtime_detection.go:19

Shallow clone does not protect ExtraFields or Runtime pointer fields.

copy(cloned, requirements) copies struct values shallowly. RuntimeRequirement has two reference-typed fields:

  • ExtraFields map[string]any — shared map between the cached copy and any returned copy
  • Runtime *Runtime — shared pointer

For the current call flow this is safe because ExtraFields is always nil at detection time (only populated later inside DeduplicateRuntimeSetupStepsFromCustomSteps). But t…

pkg/workflow/runtime_detection.go:33

Cache-read path returns a copy; cache-write path returns the raw slice.

On the first call (cache miss), line 33 returns the raw requirements slice directly — the same slice that DetectRuntimeRequirements returned, before it was cloned into CachedRuntimeRequirements. Any mutation of that returned slice by the caller does not affect the cache, which is correct, but the asymmetry means the first and subsequent callers get objects with different identity semantics.

More concretely: if t…

pkg/workflow/runtime_detection_cache_test.go:62

Test relies on a coincidental CustomSteps value to trigger node detection.

&quot;run: node --version&quot; causes node to be detected, but this custom-steps string is not valid GitHub Actions YAML (it lacks a steps: wrapper), so it exercises a code path that depends on a silent parse failure in detectFromCustomSteps. A cleaner, more robust fixture would use valid step YAML or go through the WorkflowData.ParsedTools field to inject a known runtime requirement:

workflowData := &amp;Workflow</details>

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

🤖 PR Triage

Field Value
Category bug (perf)
Risk 🟡 medium
Score 45 / 100 (impact 20 · urgency 15 · quality 10)
Action 👥 batch_review
Batch pr-batch:go-refactor (with #43834)

Summary: Caches runtime-requirement detection per compilation to remove ~10% BenchmarkCompileMCPWorkflow regression. 5 files (implementation + test). 3× bot CHANGES_REQUESTED, 1 CI failure.

Review alongside #43834 (parser refactor). Resolve CHANGES_REQUESTED first.

Generated by 🔧 PR Triage Agent · 136.6 AIC · ⌖ 9.03 AIC · ⊞ 5.5K ·

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, address these unresolved review threads first:

Generated by 👨‍🍳 PR Sous Chef · 7.3 AIC · ⌖ 5.83 AIC · ⊞ 7.1K ·
Comment /souschef to run again

Copilot AI and others added 2 commits July 6, 2026 19:22
…egression-fix-compile-mcp-workflow

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, address these unresolved review threads first:

Addressed the runtime-cache review threads in cf66986 and updated the branch with origin/main in ffdd7b1. I reran local validation (make fmt, make lint, make test-unit, make test, make recompile); the new head is locally clean. PR checks on this pushed head are still stale from before the agent push, so a maintainer will need to re-trigger CI.

Copilot AI requested a review from gh-aw-bot July 6, 2026 19:37
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot this PR is blocked and needs attention. Please run the pr-finisher skill, address requested review feedback, update the branch if needed, and rerun checks.

Generated by 👨‍🍳 PR Sous Chef · 7.29 AIC · ⌖ 7.3 AIC · ⊞ 7.1K ·
Comment /souschef to run again

Copilot AI commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

@copilot this PR is blocked and needs attention. Please run the pr-finisher skill, address requested review feedback, update the branch if needed, and rerun checks.

...

Current HEAD already contains the requested review fixes in cf66986, and the branch was updated with main in ffdd7b1. I re-ran the local required checks on this head (make fmt, make lint, make test-unit, make test, make recompile) and they passed. The remaining PR blockage is stale/action_required GitHub checks on the agent-pushed head; a maintainer needs to re-trigger CI from GitHub before merge.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

Generated by 👨‍🍳 PR Sous Chef · 8.4 AIC · ⌖ 5.03 AIC · ⊞ 4.6K ·
Comment /souschef to run again

Copilot AI commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

...

Current HEAD cf66986 already contains the requested review fixes, and the branch is up to date with main from ffdd7b1. I re-ran local validation on this head: make fmt, make lint, make recompile, targeted go test ./pkg/workflow -run 'TestDetectRuntimeRequirementsCached|TestGitPatchFromHEADCommits' -count=1 -v, and go test ./pkg/workflow -run TestDetectRuntimeRequirementsCached -count=1 passed. Full make test-unit / make test currently fail on unrelated pkg/workflow suite noise (TestGitPatchFromHEADCommits in the full run), which passes when isolated and was not changed by this PR. CI on the current agent head remains unverified until a maintainer re-triggers it from GitHub.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

Unresolved review feedback:

Generated by 👨‍🍳 PR Sous Chef · 6.99 AIC · ⌖ 5.55 AIC · ⊞ 7.1K ·
Comment /souschef to run again

Copilot AI commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

Unresolved review feedback:...

Current HEAD 5fb2e78 already contains the earlier review fixes from cf66986 plus the main update. I rechecked the remaining GitHub Actions feedback against the current files, and no additional code changes are needed; local validation passed on this head (make fmt, make lint, make test-unit, make test, make recompile). Fresh PR checks are already running on 5fb2e78, so the remaining gate is their completion.

@pelikhan pelikhan merged commit 8285af3 into main Jul 7, 2026
30 checks passed
@pelikhan pelikhan deleted the copilot/performance-regression-fix-compile-mcp-workflow branch July 7, 2026 00:06
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.82.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[performance] Regression in CompileMCPWorkflow: +10.1% slower

4 participants