Use mimalloc as the allocator on x86_64-pc-windows-msvc - #138764
Conversation
|
rustbot has assigned @Mark-Simulacrum. Use |
|
Local benchmarks:
We do see the memory use regressions compared to Windows' allocator too. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
just for my interest: does it work for windows-gnu too? |
|
I haven't tested that, but probably. |
|
@bors try |
Use mimalloc as the global allocator on `x86_64-pc-windows-msvc` This adds using mimalloc as the global allocator for the compiler as an option to `bootstrap.toml` and enables it for the `x86_64-pc-windows-msvc` dist builder. try-job: dist-x86_64-msvc
|
These benchmarks also seem pretty small, what's the result on a bigger project like cargo? |
|
☀️ Try build successful - checks-actions |
| #jemalloc = false | ||
|
|
||
| # Use mimalloc as the global allocator for the compiler. Rust code will prefer `mimalloc` over | ||
| # `jemalloc` if that is also enabled. |
There was a problem hiding this comment.
Could we reject setting both? That feels needlessly confusing to me and probably not hugely needed?
There was a problem hiding this comment.
We can probably just remove jemalloc if mimalloc v3 solves all regressions over it.
There was a problem hiding this comment.
Maybe name this field override-allocators in case we'd want to swap allocators in the future?
|
|
||
| #[cfg(feature = "mimalloc")] | ||
| #[global_allocator] | ||
| static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc; |
There was a problem hiding this comment.
Is this just temporary, or the long-term state? As-is this doesn't affect LLVM's malloc calls?
IOW, why is this not changign the rustc/src/main.rs malloc overrides?
There was a problem hiding this comment.
Long term. It's better than the linker hacks in main.rs.
Overriding the C allocator only in main.rs doesn't affect DLLs making it unsound without further work dealing with loadable backends and LLVM plugins.
There was a problem hiding this comment.
But isn't this still a regression for LLVM's allocations? It seems odd to measure perf for a halfway shift like this to mimalloc...
There was a problem hiding this comment.
We'd still want the linker hacks on Linux / macOS for LLVM, using global_allocator just makes Rust allocation overriding less hacky.
As for perf on this PR, LLVM uses Windows' heap allocator both before and after.
|
Nit: if we modify the bootstrap.toml config, I would go for |
|
Is there a noticeable difference with jemalloc on windows? Given, that it already exist but not enabled for windows. |
|
I haven't tried to make jemalloc work on Windows. I focused on mimalloc since it outperformed jemalloc on our Linux tests. |
|
mimalloc v3.2.7 release candidate was just released, it should worth a try. |
|
Some new benchmarks, jemalloc as baseline:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Can you show the rustc-perf benchmarks results? The collector runs on windows, at least it used to. |
It's hidden on the right of the table, but is the memory usage increase a concern? |
I don't really have a setup for that / longer benches. Release builds appear closer than last time:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Test with -Zthreads=7. Here mimalloc extends its lead. mimalloc v3 commits more memory then in the single threaded case.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Here's a test with the C allocator also overridden. It reduces the memory regression down to 11% which is reasonable for a -7.6% runtime reduction.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
x86_64-pc-windows-msvcx86_64-pc-windows-msvc
|
@bors try |
This comment has been minimized.
This comment has been minimized.
Use mimalloc as the allocator on `x86_64-pc-windows-msvc` try-job: dist-x86_64-msvc
|
💔 Test for 18f6c32 failed: CI. Failed job:
|
This comment has been minimized.
This comment has been minimized.
|
@bors try |
This comment has been minimized.
This comment has been minimized.
Use mimalloc as the allocator on `x86_64-pc-windows-msvc` try-job: dist-x86_64-msvc
|
💔 Test for ea0be1e failed: CI. Failed job:
|
This comment has been minimized.
This comment has been minimized.
|
@bors try |
This comment has been minimized.
This comment has been minimized.
Use mimalloc as the allocator on `x86_64-pc-windows-msvc` try-job: dist-x86_64-msvc
|
☔ The latest upstream changes (presumably #157205) made this pull request unmergeable. Please resolve the merge conflicts. |
…ark-Simulacrum,Kobzol Replace `jemalloc` bootstrap options with `override-allocator` I want to make [use of `mimalloc` on Windows](rust-lang#138764) so this renames `jemalloc` bootstrap options to `override-allocator` so we can reuse the same flag for this purpose.
Rollup merge of #155617 - Zoxc:allocator-feature-rename, r=Mark-Simulacrum,Kobzol Replace `jemalloc` bootstrap options with `override-allocator` I want to make [use of `mimalloc` on Windows](#138764) so this renames `jemalloc` bootstrap options to `override-allocator` so we can reuse the same flag for this purpose.
…acrum,Kobzol Replace `jemalloc` bootstrap options with `override-allocator` I want to make [use of `mimalloc` on Windows](rust-lang/rust#138764) so this renames `jemalloc` bootstrap options to `override-allocator` so we can reuse the same flag for this purpose.
View all comments
This adds using mimalloc as the global allocator for the compiler as an option to
bootstrap.tomland enables it for thex86_64-pc-windows-msvcdist builder.try-job: dist-x86_64-msvc