Support running compiled workflows locally with nektos/act (setup.sh /tmp guard + copy-order bugs)

[dsyme]

Summary

I investigated running compiled gh-aw workflows locally with nektos/act (via the gh-act extension, act v0.2.89), so that the gh-aw e2e suite can be exercised without dispatching real GitHub Actions runs. The goal was one deterministic local run of test-copilot-create-issue compiled with --use-samples.

This is mostly viable, but I hit a few rough edges. Some are pure act limitations (out of gh-aw's control), but two are gh-aw bugs/hardening opportunities in actions/setup/setup.sh, and one is a design observation about the agent job doing heavy Docker work even in samples mode. Filing this so the gh-aw team can decide whether to make local/act execution a first-class supported path.

Tested against gh-aw at c3c5944139d2001faa3b8e0aedabac668b56123e (the github/gh-aw/actions/setup@<sha> pinned in a freshly compiled lockfile).

Context: how far it gets

Compiling test-copilot-create-issue with --use-samples produces a 4-job lockfile (activation, agent, conclusion, safe_outputs). In samples mode the agent job's execution step becomes Replay safe-outputs samples (deterministic) (apply_samples.cjs, no AI engine), and safe_outputs creates the real issue via the GitHub API. With the workarounds below, the activation job runs every step successfully end-to-end under act.

gh-aw issues found

1. actions/setup/setup.sh hard-fails when RUNNER_TEMP is /tmp

setup.sh aborts with ::error::RUNNER_TEMP resolves to /tmp, which conflicts with gh-aw's runtime tree (/tmp/gh-aw). Under act, RUNNER_TEMP is /tmp and the ${{ runner.temp }} expression is hardcoded to /tmp as well (overriding the RUNNER_TEMP env var alone does not change the expression, so they diverge and break other steps).

Suggestion: provide an opt-in escape hatch (e.g. honor an env var like GH_AW_ALLOW_TMP_RUNNER_TEMP=1) that downgrades this hard failure to a warning, explicitly for local/act execution. Hosted CI behavior is unchanged.

2. actions/setup/setup.sh copy-order bug wipes the destination before copying

When RUNNER_TEMP=/tmp, DESTINATION resolves under /tmp/gh-aw/actions. The script runs create_dir "${DESTINATION}" before rm -rf /tmp/gh-aw; mkdir -p /tmp/gh-aw, so the cleanup wipes the freshly-created destination and the subsequent copy fails:

cp: cannot create regular file '/tmp/gh-aw/actions/action_conclusion_otlp.cjs': No such file or directory

Reordering so the /tmp/gh-aw reset happens before create_dir "${DESTINATION}" fixes it. Even outside act, the current ordering looks fragile whenever DESTINATION lives under /tmp/gh-aw.

3. agent job runs heavy Docker setup even in --use-samples mode

In samples mode the agent never calls an engine (apply_samples.cjs only), yet the agent job still unconditionally runs Download container images (~6 ghcr.io firewall/MCP images), Start MCP Gateway (docker run), and MCP CLI mounting. For deterministic sample replay these appear unnecessary and make local (and CI) runs much heavier/slower than they need to be. Consider gating these steps when the workflow is compiled for sample replay.

act-side limitations (not gh-aw bugs, but they block local runs)

These are noted for completeness — they would need to be worked around by whatever drives act, not by gh-aw itself:

1. concurrency.queue: max — the gh-aw extension property is rejected by act's schema; it must be stripped from the lockfile before act parses it. (~97 lockfiles in the test repo carry it.)
2. Custom runner labels ubuntu-slim / ubuntu-latest need -P <label>=catthehacker/ubuntu:act-latest mappings.
3. act full-clones github/gh-aw for actions/setup@<sha>. On the real monorepo this times out; worked around with --local-repository "github/gh-aw@<sha>=<path-to-local-checkout>".
4. Cross-job artifacts (Upload/Download activation artifact) require act's artifact server (--artifact-server-path), which also sets ACTIONS_RUNTIME_TOKEN.
5. upload-artifact@v7 / download-artifact@v8 send a mime_type protobuf field that act's artifact server (current release, v0.2.89) rejects: Error decode request body: unknown field "mime_type" → Failed to CreateArtifact. This is tracked upstream in act (#6114, #6022) with unmerged fix PRs (#6115, #6039). Until one lands in an act release, cross-job artifact passing under act requires a patched act build. This is currently the main blocker to a fully green local run.

Token handling note

activation validates COPILOT_GITHUB_TOKEN by prefix only (validate_multi_secret.sh): github_pat_* passes, ghp_*/gho_* are rejected, with no live API call. In --use-samples mode the engine never runs, so a dummy github_pat_... value is sufficient for local replay. GITHUB_TOKEN still needs to be a real token for the safe_outputs job to create the issue via the API.

Ask

Would the gh-aw team consider:

• Accepting the two setup.sh fixes (the /tmp escape hatch + copy-order reorder)? Happy to open a PR.
• Gating the agent job's Docker/MCP setup when compiled for sample replay?

This would make nektos/act a practical local runner for the gh-aw e2e suite (pending the upstream act artifact-server fix).

[dsyme]

I will close this, as I think it's too much churn and change to support this, and massive downloads, but leaving it files as an issue for further comment