Skip to content

[ARC-DinD] Enable runner.topology arc-dind with rootless agent job execution #42368

Description

@lpcox

Summary

When an agentic workflow targets an ARC/DinD runner, gh-aw must:

  1. Emit runner.topology: arc-dind in the AWF config so the firewall activates all split-filesystem handling automatically (sysroot image, tool cache redirection, network isolation, path-prefix probes).
  2. Ensure every step in the agent job runs rootless — no sudo, no apt-get install, no privileged operations. On ARC, the runner container does not have root access, so anything requiring root must already be handled at image build time (by AWF's build-tools sysroot image).

Companion issue


What gh-aw needs to do

1. Detect ARC/DinD topology

When the workflow targets an ARC runner label (or the user explicitly opts in), gh-aw should know to emit the arc-dind topology config.

Detection options (not mutually exclusive):

  • Explicit: user sets runner.topology: arc-dind in their workflow config / .github/gh-aw.yml
  • Inferred: gh-aw recognizes ARC-specific runner labels (e.g., labels containing arc, dind, or custom org labels mapped to ARC runner scale sets)
  • Environment probe: at runtime, detect ACTIONS_RUNNER_CONTAINER_HOOKS or KUBERNETES_SERVICE_HOST env vars that indicate an ARC environment

2. Emit AWF config with topology selector

When ARC/DinD is detected, gh-aw emits a single key in the AWF config:

{
  "runner": {
    "topology": "arc-dind"
  }
}

AWF resolves all internal details from this signal:

  • network.isolationtrue (ARC k8s lacks NET_ADMIN)
  • container.sysrootImageghcr.io/github/gh-aw-firewall/build-tools:{version} (system base)
  • dind.preStageDirstrue
  • Split-filesystem path-prefix probes activate automatically
  • Tool cache validation (warn if RUNNER_TOOL_CACHE is under /opt, which is invisible to the DinD daemon)

An explicit value in any downstream AWF key always overrides the topology default.

3. Ensure rootless agent job execution

The core constraint: No code in the workflow action can run as root on ARC. This means:

a) No sudo or privileged commands in generated steps

Audit all generated steps in the agent job to ensure none use:

  • sudo apt-get install ...
  • sudo ... anything
  • Commands that implicitly require root (writing to /usr, /etc, /opt)

b) Tool cache redirection

On ARC, the standard RUNNER_TOOL_CACHE=/opt/hostedtoolcache is useless — /opt on the runner is invisible to the DinD daemon's filesystem. gh-aw should emit an early step that redirects the tool cache to a shared volume:

- name: Redirect tool cache for ARC/DinD
  if: runner.topology == 'arc-dind'   # pseudocode for the detection mechanism
  run: echo "RUNNER_TOOL_CACHE=/tmp/gh-aw/tool-cache" >> "$GITHUB_ENV"

This ensures that subsequent setup-* actions (setup-go, setup-node, setup-java, setup-dotnet) install to a path on the shared emptyDir volume between the runner and DinD containers.

c) setup-* actions still run on-demand

Language SDKs are NOT baked into images. They are installed on-demand by setup-* actions, which download pre-built tarballs and extract them — no root needed. The flow:

  1. setup-go runs on the runner → extracts to /tmp/gh-aw/tool-cache/go/1.22.x/x64/
  2. DinD daemon can see /tmp/gh-aw/tool-cache (shared emptyDir)
  3. AWF mounts it into the agent at /host/tmp/gh-aw/tool-cache:ro
  4. Agent wrapper discovers SDK bin/ dirs via find and adds them to PATH

d) Validate no root-requiring steps leak through

Add a lint or validation pass that flags any generated step using sudo or writing to root-owned paths when runner.topology: arc-dind is active. This prevents regressions as new workflow types are added.


Privilege hierarchy

When Root? What happens Who owns it
Image build (release pipeline) apt-get install gcc make libssl-dev ... AWF build-tools image
ARC pod startup (K8s) DinD daemon starts, shared volumes created K8s / ARC operator
Workflow steps (runner container) setup-* actions, checkout, config generation gh-aw generated workflow
Agent container (AWF) Build, test, agent execution AWF runtime

gh-aw's responsibility spans rows 3 and 4 — it generates the workflow steps and configures AWF. Neither layer has root.


End-to-end flow on ARC/DinD

┌─────────────────────────────────────────────────────────────────────┐
│ ARC Pod                                                             │
│                                                                     │
│  ┌──────────────────────┐     ┌──────────────────────────────────┐  │
│  │ Runner Container      │     │ DinD Sidecar (dockerd)           │  │
│  │                       │     │                                  │  │
│  │ 1. gh-aw detects      │     │                                  │  │
│  │    ARC/DinD topology  │     │                                  │  │
│  │                       │     │                                  │  │
│  │ 2. Redirect tool cache│     │                                  │  │
│  │    → /tmp/gh-aw/      │     │                                  │  │
│  │      tool-cache       │     │                                  │  │
│  │                       │     │                                  │  │
│  │ 3. setup-go, etc.     │     │                                  │  │
│  │    install to shared  │────▶│  (sees /tmp/gh-aw via emptyDir) │  │
│  │    tool cache         │     │                                  │  │
│  │                       │     │  4. AWF starts compose stack:    │  │
│  │ 4. AWF invoked with   │────▶│     - sysroot-stage (cp -a)     │  │
│  │    topology: arc-dind │     │     - squid proxy               │  │
│  │                       │     │     - agent container            │  │
│  │                       │     │       mounts: sysroot + cache   │  │
│  │                       │     │                                  │  │
│  │ 5. Agent runs build/  │◀───│  5. Agent executes in chroot    │  │
│  │    test, gets exit    │     │     with gcc, go, node, etc.    │  │
│  │    code               │     │                                  │  │
│  └──────────────────────┘     └──────────────────────────────────┘  │
│                                                                     │
│  Shared emptyDir volumes:                                           │
│   /home/runner/_work  (workspace)                                   │
│   /tmp/gh-aw          (tool-cache, staging, config)                 │
│   Docker socket                                                     │
└─────────────────────────────────────────────────────────────────────┘

Acceptance criteria

  • gh-aw can detect or be told that the target is an ARC/DinD runner
  • gh-aw emits { "runner": { "topology": "arc-dind" } } in the AWF config
  • Tool cache is redirected to /tmp/gh-aw/tool-cache early in the workflow when on ARC
  • No generated step in the agent job uses sudo or requires root
  • setup-* actions continue to work (install to redirected tool cache, no root needed)
  • A lint/validation check flags root-requiring operations when topology: arc-dind
  • Integration test with an ARC runner (or simulated ARC environment) passes build-test workflow
  • Documentation updated for ARC/DinD workflow configuration

Notes

  • The runner.topology key is the single stable contract between gh-aw and AWF for ARC deployments
  • AWF handles all split-FS, sysroot, and network isolation details internally once it sees this key
  • gh-aw should not need to know about sysroot images, path prefixes, or compose generation details

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions