improve the performance of building page tables

[simongdavies]

This PR reduces the overhead of building page tables and page table entries, especially as the size of a sandboxes memory increases.

This pull request focuses on optimizing memory management in the src/hyperlight_host/src/mem/mgr.rs file, particularly improving the efficiency of page table entry (PTE) handling and memory region lookups. Key changes include pre-allocating buffers for PTEs, reducing shared memory writes, and introducing caching for sequential memory region access patterns.

Memory Management Optimizations:

• Pre-allocation of PTE buffer: Added a pre-allocated vec to store all page table entries (pte_buffer), minimizing shared memory writes by writing the entire buffer in one operation. [1] [2]
• Efficient shared memory write: Converted the PTE buffer to bytes and wrote it to shared memory in a single operation using copy_from_slice. This approach avoids repeated conversions and individual writes, improving performance.

Sequential Access Optimization:

• Caching for memory region lookup: Updated the get_page_flags method to include a cached region index (cached_region_idx) for faster sequential access. The method first checks the cached region, then adjacent regions, and falls back to binary search for non-sequential access patterns. [1] [2]

[ludfjig]

I am curious how much this speeds things up. Do you have any numbers?

[simongdavies]

I am curious how much this speeds things up. Do you have any numbers?

I'll add some in while, this really only shows up when we start to use larger sandboxes and if we remove all the snapshot code, which needs a separate fix that will take longer

[simongdavies]

These before and after measurements are done after removing the snapshotting code (which has the largest impact on creation/function call with reset time as sandbox size increases, this will be dealt with in a separate PR. The flamegraph is from a simple sample program that creates a sandbox with a 500MB heap and then drops it.

The benchmarks show the creation of sandboxes with increasing heap size.

Flamegraph

Before

After

Benchmark

Before

After

[simongdavies]

@ludfjig ☝️

[ludfjig]

@ludfjig ☝️

Is that ran with just bench release? Could it be in debug mode? Unrelated to your PR but we should probably prevent benchmarks from ever running in debug-profile

[simongdavies]

@ludfjig ☝️

Is that ran with just bench release? Could it be in debug mode? Unrelated to your PR but we should probably prevent benchmarks from ever running in debug-profile

Good catch updated, yes we should, if only to stop me from doing it again #638

[ludfjig]

LGTM