Update the benchmarks with Jemalloc master

[LifeIsStrange]

Jemalloc for contingent reasons has not seen a release since mi-2019.
It would be more fair and intellectually interesting to benchmarck latest mimalloc with latest jemalloc because it has received many improvements since then.
Also, do the benchmarks use the sized deallocation apis?
Not passing the size of the object to free prevent a lot of optimizations for jemalloc (but you probably already know that ;) )

I would be very interested in such results and mimalloc is for sure a nice technology.

[daanx]

Hi @LifeIsStrange, thanks for the feedback; I recently re-benchmarked and took the latest release tag at that time -- it is tricky as sometimes "master" versions are slower; generally I only pick specific commits for allocators that are in development (like mesh) but try to avoid it for mature allocators like jemalloc (which is 15 years in development now!). Nevertheless, I keep it mind for next time.

The benchmarks can be found at https://github.com/daanx/mimalloc-bench (and on Ubuntu it should be not too hard to run them yourself). None of the benchmarks use sized deallocation api's as the benchmarks target a wide variety of allocators and need to be as general as possible; I was not aware that jemalloc has specific optimizations for sized deallocation. However, as I always stress, the benchmarks should be interpreted with care and are limited in scope. This is one example of that -- there are many more things, like the limited size of the benchmarks, they don't measure latency, or commit size over long running processes etc etc. Most modern allocators all perform quite well on the benchmarks so the choice between them often comes down to other concerns (like jemalloc has great statistics support) or just by testing your specific workloads against different allocators. For mimalloc for example a lot of work went into low latency, lowering commit size over long running processes, huge page support, 100GiB+ huge workloads, etc. None of that is reflected in any of the current benchmarks as it is.

[jq-rs]

I checked the benchmarks and the larson-test seemed pretty straightforward to modify to measure also sized deallocation. The modified test can be found here, feel free @daanx to use it if you decide to extend sized deallocation to the tests. It would need some kind of generalization at least. I am also happy to send a PR. Thanks!

[mjp41]

@jq-rs you might want to apply the fix I needed for snmalloc:
daanx/mimalloc-bench@c19046b
The original benchmark was not correct C++.

Also, have you considered using the C++14 sized deallocation operator.

	operator delete(pointer, size);

This would allow many allocators to use this code path by providing the relevant overload in their .so. Rather than specialising the test for each allocator?

Unfortunately, clang++ seems to require a flag to expose this:

-fsized-deallocation

But g++ and cl.exe seemed to do it directly. Here is a mini sample of how you might rewrite the code:

#include <new>

int main() {
	auto p = operator new(64);
	operator delete(p, 64);
}

[jq-rs]

Thanks @mjp41 for the comment! C++14 handling is interesting, I think that is the proper way to do this kind of thing. This is just a prototype version for quick benchmarking which I am not planning to develop further at this point.

[jq-rs]

Hmm, I am still a bit puzzled, which API it will call with C++14?

Edit: I see, probably the application needs to define a template for it as explained in https://isocpp.org/files/papers/n3778.html#basic.stc.dynamic.deallocation

[mjp41]

So snmalloc provides them at https://github.com/microsoft/snmalloc/blob/master/src/override/new.cc#L44-L49
and mimalloc provides them at https://github.com/microsoft/mimalloc/blob/master/src/alloc-override.c#L108-L111
and jemalloc provides them at https://github.com/jemalloc/jemalloc/blob/dev/src/jemalloc_cpp.cpp#L36-L40

I don't know about any other allocators. There will be weak leakage symbols that got overridden by these definitions, when you use the relevant libraries.

[jq-rs]

Seems to work then for mimalloc too, grand. There isn't any fast path at the moment based on this thread, I guess. Maybe that is TBA?

Edit: And as a confirmation as mi_free_size shows here, it just calls mi_free and size is mostly ignored.

[devnexen]

Less known but seems rpmalloc does too https://github.com/mjansson/rpmalloc/blob/a3e46a349d393b0517f5948cfd65b90788439d7a/rpmalloc/malloc.c#L108

[jq-rs]

tcmalloc has this too https://github.com/google/tcmalloc/blob/master/tcmalloc/libc_override_gcc_and_weak.h
It was perhaps the first to have it.

Generalization should be done this way for sure for this test.

[mjp41]

I think almost every allocator could benefit from knowing the size, but most don't take it into account as you don't have code using it. @daanx I am surprised you don't have these code paths given the origins of mimalloc in Koka and Lean. snmalloc benefits from sizes in its Rust wrapper: https://github.com/microsoft/snmalloc/blob/master/src/override/rust.cc#L18-L21, and when we call the allocator in Verona.

[jq-rs]

Mimalloc Rust could use the size too, it is now ignored (perhaps due to not being speedier) https://github.com/purpleprotocol/mimalloc_rust/blob/master/src/lib.rs#L95

[LifeIsStrange]

For the record: Jemalloc reported

The performance win for a contrived microbenchmark doing an allocation
and immediately freeing it is ~10%. It may have a different impact on a
real workload.

for sized deallocation.
jemalloc/jemalloc@4cfe551
jemalloc/jemalloc@2319152
since then maybe that they leveraged more optimizations from it.

[jq-rs]

I generalized the sized deallocation test with this PR: daanx/mimalloc-bench#25