fix(ci): postprocess install-step swap must tolerate trailing --rootless flag by lpcox · Pull Request #5800 · github/gh-aw-firewall

lpcox · 2026-07-01T22:14:05Z

Problem

The Build Test Suite (and every other agentic workflow run through the AWF smoke harness) was failing with:

[ERROR] Fatal error: Error: The --build-local flag requires a full repository checkout. It is not supported with the standalone bundle. Use the npm package or clone the repository instead.

(Observed on run 28550453439, PR #5783.)

Root cause

gh aw v0.82+ now emits the AWF install step with a trailing flag:

- name: Install AWF binary
  run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.27.16 --rootless

scripts/ci/postprocess-smoke-workflows.ts swaps that step for a build-from-source step and rewrites --skip-pull → --build-local. But its install-step regex anchored the version directly to a newline:

… install_awf_binary\.sh"? v[0-9.]+\n

The new trailing --rootless broke that match, so the install-step replacement silently no-op'd while the --build-local rewrite still fired. Result: every non-release lock file ended up with a standalone-bundle install + --build-local — an impossible combination that awf rejects at runtime.

Fix

Widen the regex to v[0-9.]+[^\n]*\n so trailing flags (--rootless, etc.) are tolerated (in both the script and its test copy).
Add a --rootless regression test.
Recompile (gh aw compile) + re-run the postprocess script so all smoke/build-test workflows build the firewall containers from source again.

Verification

npx jest scripts/ci/postprocess-smoke-workflows.test.ts → 47 passed (incl. new case).
npm run build clean.
Consistency check: no lock file now mixes --build-local with install_awf_binary.sh. The only files still using the standalone installer are the two intentional release-mode files (build-test-network-isolation, network-isolation-test), which correctly keep --skip-pull.

The large lock-file diff is expected: the install-step swap now correctly applies to all non-release workflows (previously all silently skipped). Cron changes are normal fuzz-schedule drift from recompilation.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

gh-aw v0.82+ emits the AWF install step with a trailing flag, e.g. `install_awf_binary.sh v0.27.16 --rootless`. The postprocess regex anchored the version to an immediate newline (`v[0-9.]+\n`), so the install-step replacement silently no-op'd while the `--skip-pull -> --build-local` rewrite still fired. This left every non-release lock file in a broken state: standalone bundle install + `--build-local`, which awf rejects at runtime: The --build-local flag requires a full repository checkout. It is not supported with the standalone bundle. Widen the regex to `v[0-9.]+[^\n]*\n` so trailing flags are tolerated, add a `--rootless` regression test, and recompile + postprocess all workflows so smoke/build-test suites build the firewall containers from source locally again. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-07-01T22:16:07Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	98.64%	98.68%	📈 +0.04%
Statements	98.53%	98.56%	📈 +0.03%
Functions	99.56%	99.56%	➡️ +0.00%
Branches	94.51%	94.51%	➡️ +0.00%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/workdir-setup.ts`	93.0% → 94.8% (+1.74%)	93.0% → 94.8% (+1.74%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

This PR fixes the AWF smoke-harness postprocessing so it still replaces the compiled “Install AWF binary” step when gh aw appends trailing flags (e.g. --rootless), preventing workflows from ending up with an invalid standalone-bundle install combined with --build-local.

Changes:

Broaden installStepRegex to tolerate trailing flags after the AWF version string.
Add a regression test covering the --rootless suffix case.
Recompile + re-run the postprocess script so affected .lock.yml workflows consistently build AWF from source again (with expected cron “fuzz” drift).

Show a summary per file

File	Description
scripts/ci/postprocess-smoke-workflows.ts	Loosens AWF installer step matching so the postprocess swap still triggers with trailing flags like `--rootless`.
scripts/ci/postprocess-smoke-workflows.test.ts	Adds coverage ensuring the updated install-step regex matches `vX.Y.Z --rootless`.
.github/workflows/update-release-notes.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/test-hard-cap-ai-credits.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/test-coverage-reporter.lock.yml	Recompiled lock: swaps AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/test-coverage-improver.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/smoke-services.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/smoke-otel-tracing.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/smoke-gemini.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/smoke-copilot.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/smoke-copilot-pat.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/smoke-copilot-byok.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/smoke-copilot-byok-aoai-entra.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/smoke-copilot-byok-aoai-apikey.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/smoke-codex.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/smoke-claude.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/smoke-chroot.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/self-hosted-runner-doctor.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install (also updates metadata body hash).
.github/workflows/self-hosted-runner-doctor-updater.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift + metadata updates.
.github/workflows/security-review.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/security-guard.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/secret-digger-copilot.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/secret-digger-codex.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/secret-digger-claude.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/schema-sync.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/refactoring-scanner.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/red-team-benchmark.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/plan.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/pelis-agent-factory-advisor.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/model-api-mapping-updater.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/issue-monster.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/issue-duplication-detector.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/firewall-issue-dispatcher.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/export-audit.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/duplicate-code-detector.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/doc-maintainer.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/dependency-security-monitor.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/copilot-token-usage-analyzer.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/copilot-token-optimizer.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/contribution-check.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/config-consistency-auditor.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/cli-flag-consistency-checker.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/claude-token-usage-analyzer.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/claude-token-optimizer.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/ci-doctor.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.
.github/workflows/ci-cd-gaps-assessment.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install; cron drift from recompilation.
.github/workflows/build-test.lock.yml	Recompiled lock: swaps standalone AWF installer step for local build-from-source AWF install.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 47/47 changed files
Comments generated: 2
Review effort level: Low

gh-aw v0.82 moved AWF network/host config out of CLI flags and into the awf-config.json (--config), and added native support for a top-level `services:` frontmatter section. The compiler now emits the GitHub Actions `services:` block into the agent job AND auto-derives `--allow-host-service-ports` from the declared port mappings. The old smoke-services post-processing (inject a services block + swap a `--enable-host-access` CLI flag) silently no-op'd against the new output: the agent job gained an `if:` line that broke the injection regex, and the `--enable-host-access` flag no longer exists. Result: smoke-services ran with no service containers and no host-service-port access. - Declare redis + postgres under a native `services:` section in smoke-services.md. - Remove the entire smoke-services special-case from postprocess-smoke-workflows.ts (now handled by the compiler). - Recompile + postprocess (also drops a stale codex env_key in secret-digger-codex that a prior recompile left behind). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-07-01T22:28:02Z

✅ Copilot review passed with no inline comments.

@lpcox Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

lpcox · 2026-07-01T22:28:06Z

Follow-up: fixed a second v0.82 regression in `smoke-services`

While verifying, I found smoke-services was also silently degraded by the same v0.82 output changes — and its postprocess special-case had two independent breakages:

Services block injection — the agent job now has an if: line between needs: activation and runs-on: ubuntu-latest, which broke the agentJobNeedsRunsOnRegex, so the Redis/PostgreSQL services: block was never injected.
--enable-host-access swap — v0.82 dropped the --enable-host-access CLI flag (network/host config now lives in awf-config.json), so the --enable-host-access → --allow-host-service-ports swap had nothing to match.

Net effect: smoke-services was running with no service containers and no host-service-port access — the smoke test could never actually reach Redis/Postgres.

Clean fix (no more postprocess hack)

gh-aw v0.82 natively supports a top-level services: frontmatter section: the compiler emits the Actions services: block and auto-derives --allow-host-service-ports from the port mappings. So I:

Declared redis + postgres under a native services: section in smoke-services.md.
Deleted the entire smoke-services special-case (~100 lines) from postprocess-smoke-workflows.ts.

Verified the recompiled lock now contains the services: block and --allow-host-service-ports "${{ job.services['postgres'].ports['5432'] }},${{ job.services['redis'].ports['6379'] }}", with --build-local. Postprocess is idempotent; 63 tests pass; build clean.

The local-build install steps inject a Setup Node.js step; some workflows already emit an identical one immediately before, yielding duplicate consecutive steps. Add a backreference-guarded post-pass that collapses only byte-identical consecutive blocks, so workflows without one keep the injected step. Resolves the Copilot review nit on smoke-claude/smoke-gemini. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

lpcox · 2026-07-01T22:40:14Z

Addressed the duplicate Setup Node.js review nit in 54cbf13.

The local-build install steps inject a Setup Node.js step because many workflows (e.g. plan, ci-doctor, security-guard) have no compiler-emitted one and need it. Workflows that do already emit an identical step immediately before ended up with two consecutive identical blocks.

Fix: a backreference-guarded post-pass that collapses only byte-identical consecutive Setup Node.js blocks. It can't remove a differing or required step. Collapsed 5 duplicates across smoke-claude, smoke-gemini, red-team-benchmark, and secret-digger-claude; verified idempotent (2nd run collapses 0). Added 3 regression tests (50 pass).

Copilot AI review requested due to automatic review settings July 1, 2026 22:14

Copilot started reviewing on behalf of lpcox July 1, 2026 22:14 View session

Copilot AI reviewed Jul 1, 2026

View reviewed changes

Comment thread .github/workflows/smoke-claude.lock.yml Outdated

Comment thread .github/workflows/smoke-gemini.lock.yml Outdated

lpcox merged commit 29ece54 into main Jul 1, 2026
24 checks passed

lpcox deleted the fix/postprocess-install-step-rootless branch July 1, 2026 22:47

This was referenced Jul 1, 2026

fix: detect and expose system CA bundle for RHEL/Amazon Linux in chroot #5783

Merged

chore: upgrade gh-aw to v0.82.2 pre-release and recompile workflows #5828

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(ci): postprocess install-step swap must tolerate trailing --rootless flag#5800

fix(ci): postprocess install-step swap must tolerate trailing --rootless flag#5800
lpcox merged 3 commits into
mainfrom
fix/postprocess-install-step-rootless

lpcox commented Jul 1, 2026

Uh oh!

github-actions Bot commented Jul 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 1, 2026

Uh oh!

lpcox commented Jul 1, 2026

Uh oh!

lpcox commented Jul 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

lpcox commented Jul 1, 2026

Problem

Root cause

Fix

Verification

Uh oh!

github-actions Bot commented Jul 1, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 1, 2026

Uh oh!

lpcox commented Jul 1, 2026

Follow-up: fixed a second v0.82 regression in smoke-services

Clean fix (no more postprocess hack)

Uh oh!

lpcox commented Jul 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Follow-up: fixed a second v0.82 regression in `smoke-services`