Skip to content

feat: migrate devcontainer build from QEMU to native ARM64 runners#5506

Merged
max-sixty merged 2 commits intoPRQL:mainfrom
max-sixty:build-devcontainer-arm
Oct 19, 2025
Merged

feat: migrate devcontainer build from QEMU to native ARM64 runners#5506
max-sixty merged 2 commits intoPRQL:mainfrom
max-sixty:build-devcontainer-arm

Conversation

@max-sixty
Copy link
Copy Markdown
Member

@max-sixty max-sixty commented Oct 19, 2025

Summary

Replace QEMU emulation with GitHub's native ARM64 runners for significantly faster builds. Uses matrix strategy with push-by-digest pattern for multi-platform image creation.

Changes

  • Remove QEMU emulation: No more slow instruction translation
  • Add native ARM64 builds: Use ubuntu-24.04-arm runners (free for public repos)
  • Implement matrix strategy: AMD64 builds on x86, ARM64 builds on ARM natively
  • Push-by-digest pattern: Each platform pushes separately, then merged into multi-platform manifest
  • Add validation: Digest count check and platform verification
  • Scope GHA cache: Per-platform cache to prevent thrashing
  • Reduce timeouts: Build 4h (was 60h!), Merge 30m

Performance Results

Actual build times from CI:

  • AMD64: ~2 minutes (native x86 build)
  • ARM64: ~3 minutes (native ARM build)
  • Total: ~3-4 minutes for both platforms in parallel

These are similar to current times since the devcontainer build is relatively light. The real benefit comes with heavier builds where QEMU emulation causes 9-22x slowdowns.

Test Results

✅ Both platform builds completed successfully
✅ Build artifacts exported correctly
✅ GHA cache working per-platform
✅ All CI checks passing

Notes

  • The merge job is skipped in PRs (only runs when push: true)
  • This follows Docker's official pattern for native multi-platform builds
  • Image name fixed to lowercase (prql instead of PRQL) per Docker registry requirements

🤖 Generated with Claude Code

Replace QEMU emulation with GitHub's native ARM64 runners for 22x faster
builds. Use matrix strategy with push-by-digest pattern for multi-platform
image creation.

Key changes:
- Remove QEMU setup, add matrix build on ubuntu-24.04-arm
- Implement push-by-digest pattern with artifact-based merge
- Add digest validation and platform verification
- Scope GHA cache per platform to prevent thrashing
- Reduce timeouts (build: 4h, merge: 30m)

Expected performance: 22x faster ARM64 builds (33.5min → 1.5min based on
real-world benchmarks).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@max-sixty max-sixty changed the title Migrate devcontainer build from QEMU to native ARM64 runners feat: migrate devcontainer build from QEMU to native ARM64 runners Oct 19, 2025
Docker registry names must be lowercase. Changed from
github.repository_owner (which can be uppercase) to hardcoded
lowercase 'prql'.
@max-sixty
Copy link
Copy Markdown
Member Author

🔍 Testing Status

What's been tested:

  • ✅ Both AMD64 and ARM64 builds complete successfully
  • ✅ Build artifacts are created correctly
  • ✅ GHA cache working per-platform
  • ✅ Digest export working

What hasn't been tested yet:

  • ⏳ Merge job (creates multi-platform manifest)
  • ⏳ Digest download and validation
  • ⏳ Platform verification

Why the merge job was skipped:
The merge job only runs when push: true, which happens when:

  1. Devcontainer Dockerfile changes AND
  2. Merged to main AND
  3. Push event (not PR)

This is the correct security model - PRs shouldn't push to the registry.

To verify after merge:

  1. Merge this PR to main
  2. The devcontainer push workflow will trigger automatically
  3. Watch for the build-devcontainer / merge job
  4. Verify it:
    • Downloads both digests
    • Creates multi-platform manifest
    • Verifies both platforms are present

The merge logic is straightforward (download artifacts → create manifest → verify), but we won't know if it works until after merge.

@max-sixty max-sixty merged commit 8cd084d into PRQL:main Oct 19, 2025
46 checks passed
@max-sixty max-sixty deleted the build-devcontainer-arm branch October 19, 2025 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant