Copilot engine fallback model path uses --model CLI flag instead of COPILOT_MODEL env var

[samuelkahessay]

What happens

When engine.model is not explicitly configured in workflow frontmatter, the compiler generates a shell command that conditionally appends --model "$GH_AW_MODEL_AGENT_COPILOT" as a CLI flag to the Copilot CLI invocation. The Copilot CLI does not accept --model as a flag — it errors with argument '<model>' is invalid.

When engine.model is configured, the compiler correctly sets the COPILOT_MODEL environment variable, which the Copilot CLI reads natively. So the "explicit model" path works, but the "fallback from GitHub org variable" path is broken.

The test suite contains contradictory expectations: one test expects the --model flag to appear, while another asserts it should NOT appear.

What should happen

Both code paths should use COPILOT_MODEL as an environment variable. When the model comes from a GitHub org variable (GH_AW_MODEL_AGENT_COPILOT), the compiler should map it to COPILOT_MODEL in the step's env: block rather than appending it as a CLI flag.

Where in the code

All references are to main at 99b2107.

Path selection:

• copilot_engine_execution.go:78 — modelConfigured := workflowData.EngineConfig != nil && workflowData.EngineConfig.Model != "" determines which path is taken

Broken fallback path (no explicit model):

• copilot_engine_execution.go:154 — needsModelFlag := !modelConfigured
• copilot_engine_execution.go:182-183 — sandbox mode: generates ${GH_AW_MODEL_AGENT_COPILOT:+ --model "$GH_AW_MODEL_AGENT_COPILOT"}
• copilot_engine_execution.go:191-192 — non-sandbox mode: same --model flag expansion
• copilot_engine_execution.go:312-318 — sets GH_AW_MODEL_AGENT_COPILOT or GH_AW_MODEL_DETECTION_COPILOT in env block (the value source is correct, but it's consumed as a CLI flag)

Correct explicit-model path:

• copilot_engine_execution.go:309-311 — env[constants.CopilotCLIModelEnvVar] = workflowData.EngineConfig.Model — sets COPILOT_MODEL directly

Contradictory tests:

• model_env_vars_test.go:21-25 — expects --model flag in shell command (tests the broken behavior)
• model_env_vars_test.go:213-243 — asserts --model flag should NOT appear when COPILOT_MODEL env var is used

Evidence

Source-level verification (2026-03-03):

• Confirmed both code paths exist at the specified lines on current main
• Confirmed needsModelFlag is true exactly when modelConfigured is false
• The generated shell expansion ${GH_AW_MODEL_AGENT_COPILOT:+ --model "$GH_AW_MODEL_AGENT_COPILOT"} will pass --model gpt-5 (or whatever value) as a positional argument to the Copilot CLI

Copilot CLI verification (v0.0.421):

• copilot --model gpt-5 --prompt 'say hi' fails with argument 'gpt-5' is invalid
• COPILOT_MODEL=gpt-5 copilot --prompt 'say hi' runs normally
• Confirms the CLI does not accept --model as a flag; the env var is the only supported mechanism

Related issue: #17265 (closed) requested the model be provided via env var. The fix was applied for the explicit-model path (COPILOT_MODEL env var) but the fallback path was missed and still uses the broken --model flag.

Proposed fix

In copilot_engine_execution.go, when needsModelFlag is true, instead of generating ${GH_AW_MODEL_AGENT_COPILOT:+ --model "$GH_AW_MODEL_AGENT_COPILOT"} in the shell command, map the org variable to COPILOT_MODEL in the step's env: block:

// Instead of appending --model to the shell command,
// map the org variable to the native env var:
env[constants.CopilotCLIModelEnvVar] = fmt.Sprintf("${{ vars.%s || '' }}", constants.EnvVarModelAgentCopilot)

Remove the needsModelFlag / --model flag expansion from buildCopilotCommand() entirely. Update the contradictory test at model_env_vars_test.go:21-25 to expect COPILOT_MODEL in the env block instead of --model in the command string.

Impact

Frequency: Affects every workflow run where the model is configured via GitHub org variables (GH_AW_MODEL_AGENT_COPILOT / GH_AW_MODEL_DETECTION_COPILOT) rather than inline engine.model frontmatter. This is the default path for repos that don't customize per-workflow.
Cost: The --model flag produces a hard error from the Copilot CLI. If the org variable is set, the run fails immediately. If the org variable is unset, the conditional expansion is a no-op and the default model is used — so the bug is latent until someone configures the org variable.

[pelikhan]

Use COPILOT_MODEL