More and better benchmarks are needed

[emberian]

We have few good, solid benchmarks that stress a single part of the codegen or libraries. These are necessary to be able to be able to easily spot perfomance regressions, as well as have a good quantitative data about Rust's performance.

Benchmarks should cover both space and time characteristics, and should use the built-in bench runner where possible. Some ideas:

• Allocating an object with a destructor
• Allocating a @-ptr
• Destructuring an enum/struct/tuple/tuple struct/vector
• Pattern matching
• Numeric manipulation (should be covered fairly well by the current shootout stuff, but should be extracted)
• Data structure manipulation (vec benchmarks etc, similar to what core-map and friends do)
• String manipulation
• Method call
• Trait object method call
• Task communication (like pingpong does) with the various primitives (one-shots, streams, protocols)
• Logging

An important note: these benchmarks aren't intended for comparison to other languages (although they could be useful for that in the future), but for tracking the performance of Rust.

[emberian]

Nominating for well-covered

[emberian]

Benchmark #7543 (newtype vs original type)

[emberian]

Benchmark FFI calls

[emberian]

Benchmark task switch time

[emberian]

Benchmark all forms of allocation

[emberian]

Benchmark parallel allocation/ffi (for the hit contention would give)

[emberian]

Benchmark the impact of segmented stacks in various situations

[pnathan]

cmr:

What do you think about taking some code from here: https://github.com/pnathan/flaky-data-structures, marking a particular changeset and using that to build benchmarks on? The BST and linked list should handle these tasks, today; I'd just need to cook up a pile of data and a benchmarking driver to use against them:

Allocating a @-ptr
Destructuring an enum//tuple
Pattern matching
Method call
Trait object method call

[emberian]

@pnathan: ideally the benchmarks would be included in the rust source, I'm not sure the license is suitable (@graydon?). While it'd be fine as a datapoint, and I'd love to include it, it does "too much" in that its performance doesn't really reflect how well Rust is doing for any single feature. But, it'd be a good start. (Aside: you call it "Mozilla Rust" in your readme. it's just plain "Rust")

[pnathan]

@cmr: I don't mind relicencing the code for the Rust project if you want to include it as a datapoint. I see what you mean about the single-feature focus you want. I can cook something up this weekend for a couple of those benchmarks.

[emberian]

That'd be great. I'd love to include it and it'd certainly be useful as
part of the whole perf picture

[abhijeetgaiha]

@pnathan Are you working on individual benchmarks as well?

[graydon]

Note: rust has a unit benchmark driver in the unit test driver. These are really small things to write. I'll land a bunch of them today.

[emberian]

@graydon could you help land #7687 first? I'd like to keep the benchmarks in a separate crate from libstd etc, to make them standaloneish

[huonw]

(For anyone wondering how to use it, see the benchmark section of the unit testing wiki page.)