feat: use i686 layout for nanvix-unstable guests and make snapshot RWX

[danbugs]

Summary

When nanvix-unstable is enabled, selects the i686 hyperlight-common memory layout on x86_64 hosts and makes the snapshot memory region writable. Also propagates the nanvix-unstable feature from hyperlight-host to hyperlight-common so both crates share the same cfg gates.

Motivation

Nanvix runs as a 32-bit (i386) guest without CoW guest paging. Two things need to change for this to work:

1. i686 layout: The amd64 layout defines MAX_GPA/MAX_GVA for a 36-bit/48-bit address space and includes SNAPSHOT_PT_GVA_* constants for the CoW page table region. A 32-bit guest needs 32-bit address limits (0xFFFF_FFFF) and has no snapshot page tables. With nanvix-unstable, the i686 layout module is used even on x86_64 hosts.

2. Snapshot RWX: Without CoW, the guest writes directly to snapshot pages — .data, .bss, kernel page pool, and even GDT descriptors (the CPU sets the "Accessed" bit during segment loads). If the snapshot isn't writable, these writes cause EPT violations that KVM retries forever.

Changes

• hyperlight_host/Cargo.toml: nanvix-unstable = ["hyperlight-common/nanvix-unstable"] — propagates the feature
• hyperlight_common/src/layout.rs: Three-way cfg_attr selects i686 layout for native x86 OR x86_64 with nanvix-unstable; gates SNAPSHOT_PT_GVA_* exports behind all(target_arch = "x86_64", not(nanvix-unstable))
• hyperlight_common/src/arch/i686/layout.rs: Removes stale SNAPSHOT_PT_GVA_* constants and fixes MAX_GVA to 0xFFFF_FFFF; updates min_scratch_size signature to match the amd64 layout
• hyperlight_host/src/mem/shared_mem.rs: Snapshot region is READ | EXECUTE by default (CoW handles writes), READ | WRITE | EXECUTE with nanvix-unstable
• hyperlight_host/src/sandbox/snapshot.rs: Gates the snapshot page-table GVA filter behind not(nanvix-unstable)

[dblnz]

LGTM!
However, I am not that well acquainted with this part of the codebase. Maybe @syntactically can take a look

[syntactically]

I'm not clear on the need for a configurable region base for scratch. You can just set MAX_GPA/MAX_GVA based on target architecture + feature flags? (see src/hyperlight_common/src/arch/i686/layout.rs). Maybe the selection based on target_arch of which layout module to use is broken in the host for the i686 case where it's the guest arch that counts; maybe that needs to change to something like #[cfg_attr(any(target_arch = "x86", all(target_arch = "x86-64", not(feature = "init-paging")), path = "arch/i686/layout.rs")]. (Or, we could add cargo cfg aliases for guest arch, or something?). I would really prefer to avoid re-adding more configuration knobs.

I'm not clear on how the idea of making snapshot pages writable is going to work with the snapshot-first lifecycle (#1268). This seems like it will break a lot of assumptions and make a lot of code more complex/rife with opportunities for vulnerabilities. If you really are unable to run on amd64 natively, would teaching hyperlight about i686 paging structures be a slightly smaller break of the conceptual model?

[jsturtevant]

I'm not clear on how the idea of making snapshot pages writable is going to work with the snapshot-first lifecycle (#1268).

This seems like something we could flush out in that discussion? With CoW changes we've broken some downstream consumers, so it makes sense give an escape hatch (maybe behind a feature flag) while we sort through what we want to do. As far as I understand, this isn't a breaking change and doesn't introduce issues now? or does it? To be clear I think we do need to reconcile this requirement and enabling this wouldn't be a long term solution

[ludfjig]

does nanvix not use snapshotting? If we can do this without adding a config option, I think that would be good too.

[danbugs]

Addressed the config option concern. Latest push removes set_scratch_base_gpa() from SandboxConfiguration entirely. Instead, the i686 memory layout is now auto-selected at compile time:

• When init-paging feature is enabled (standard hyperlight guests), the existing amd64 layout is used unchanged.
• When init-paging feature is dsaibled (Nanvix), an i686-compatible layout is selected in hyperlight-common that places the scratch region at a lower GPA (below 4GB), compatible with 32-bit page tables.

The init-paging feature is propagated from hyperlight-host to hyperlight-common via Cargo feature unification, so no runtime configuration is needed.

Regarding the xsave commit that was on this branch — it's been dropped from the latest push since it's orthogonal to the scratch GPA changes. The writable snapshot commit remains, gated behind init-paging. Nanvix needs writable snapshots because its guest page tables and kernel state live in the snapshot EPT region and must be writable during execution.

Note that Nanvix doesn't use Hyperlight's snapshotting mechanism. Instead, it has a kernel call that, being guest-aware, can VMExit at any time to snapshot. Nanvix does this because it doesn't execute like our purpose-built guest binaries (e.g., it doesn't use guest function calls). Instead, it just calls evolve once.