feat(java): add reporting suite to java bench

[slbotbm]

Which issue does this PR address?

Closes #
Relates to #

Rationale

The java bench was missing a reporting suite, which meant no metrics collection was happening.

What changed?

bench/
├── benchmarks
│   ├── actors
│   │   └── tcp
│   │       └── async
│   │           └── TcpAsyncPinnedProducerActor.java
│   │               # now collects benchmark records, tracks send progress and latency,
│   │               # returns IndividualMetrics, and logs producer benchmark lifecycle/summary data.
│   └── runners
│       └── tcp
│           └── async
│               └── TcpAsyncPinnedProducer.java
│                   # now runs actors, collects IndividualMetrics, builds GroupMetrics,
│                   # and exposes report data
├── cli
│   ├── IggyBenchCommand.java
│   │   # increased the default warmup time from 20 seconds to 60 seconds to better warm up the JVM.
│   └── PinnedProducerCommand.java
│       # builds benchmark/report metadata, constructs and writes the final report,
│       # and prints a summary.
├── common
│   └── enums
│       ├── BenchmarkKind.java
│       │   # added pretty human-readable benchmark names.
│       └── TimeSeriesKind.java
│           # defines the time-series categories used in reporting.
├── models
│   └── report
│       ├── FinalReport.java
│       │   # top-level immutable report record.
│       ├── context
│       │   ├── BenchmarkCacheMetrics.java
│       │   │   # value record for server cache hit/miss statistics.
│       │   ├── BenchmarkCacheMetricsKey.java
│       │   │   # key record for per-partition cache metrics.
│       │   ├── BenchmarkHardware.java
│       │   │   # hardware metadata record for the benchmark host.
│       │   ├── BenchmarkParams.java
│       │   │   # report metadata record for benchmark topology, sizing, command, identifiers.
│       │   └── BenchmarkServerStats.java
│       │       # server-side stats record.
│       └── metrics
│           ├── BenchmarkRecord.java
│           │   # raw per-sample benchmark record used as the input to metrics calculation.
│           ├── DistributionPercentiles.java
│           │   # percentile summary record for latency distributions.
│           ├── GroupMetrics.java
│           │   # aggregate metrics record for a group of actors plus optional latency distribution.
│           ├── GroupMetricsSummary.java
│           │   # numeric summary of aggregate throughput and latency metrics.
│           ├── HistogramBin.java
│           │   # histogram bin record for latency distributions.
│           ├── IndividualMetrics.java
│           │   # per-actor metrics record containing summary, time series,
│           │   # and non-serialized raw latency samples.
│           ├── IndividualMetricsSummary.java
│           │   # numeric summary record for one actor's benchmark results.
│           ├── LatencyDistribution.java
│           │   # latency distribution record combining histogram bins,
│           │   # log-normal parameters, and percentile markers.
│           ├── LatencyMetrics.java
│           │   # internal holder for computed latency statistics.
│           ├── LogNormalParams.java
│           │   # record for log-normal distribution parameters.
│           ├── TimePoint.java
│           │   # simple time/value point record used by report time series.
│           └── TimeSeries.java
│               # time-series record with typed series kind and an empty-series helper.
└── report
    ├── FinalReportBuilder.java
    │   # assembles the final report, serializes it as JSON,
    │   # writes it to disk, and prints generic benchmark/group summaries.
    ├── GroupMetricsCalculator.java
    │   # aggregates per-actor metrics into group throughput/latency summaries,
    │   # time series, and latency distributions.
    ├── HardwareInfoCollector.java
    │   # collects local machine hardware details such as CPU name/core count,
    │   # memory, OS name/version, and optional hostname identifier.
    ├── IndividualMetricsCalculator.java
    │   # converts raw BenchmarkRecord samples into per-actor throughput/latency summaries
    │   # and time series.
    ├── ServerStatsCollector.java
    │   # collects benchmark server stats through the admin client
    │   # and maps cache metrics into report models.
    └── StatsHelper.java
        # helper math for means, extrema, standard deviation,
        # moving averages, and percentile interpolation.

Local Execution

• Passed
• Pre-commit hooks

AI Usage

If AI tools were used, please answer:

1. Which tools? codex
2. Scope of usage? code generated
3. How did you verify the generated code works correctly? read through the code and verified multiple times.
4. Can you explain every line of the code if asked? yes

[github-actions]

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

Slash commands (own line, regular comment) move it around the queue:

• /ready - back to S-waiting-on-review after addressing feedback
• /author - flip to S-waiting-on-author while you finish changes
• /request-review @user-or-team - request a reviewer

See CONTRIBUTING.md for details.

[slbotbm]

I still need to verify some more things, so I converted this to a draft for now. Preliminary results of running the two benches are as follows:

Commands ran:
Java:

taskset -c 0-3 ./target/debug/iggy-server --fresh --with-default-root-credentials
taskset -c 4-7 ./gradlew --no-daemon :iggy-bench:run --args="pp"

Rust:

taskset -c 0-3 ./target/debug/iggy-server --fresh --with-default-root-credentials
taskset -c 4-7 ./target/debug/iggy-bench --warmup-time 60s pp tcp output -o .

Results:

Metric	Java	Rust
Total Throughput	58.80 MB/s	65.33 MB/s
Messages/s	58,782	65,308
Avg Throughput / Producer	7.35 MB/s	8.17 MB/s
p50 Latency	74.18 ms	74.07 ms
p90 Latency	243.74 ms	205.46 ms
p95 Latency	497.91 ms	307.11 ms
p99 Latency	1,661.16 ms	1,466.63 ms
p999 Latency	6,420.36 ms	6,776.00 ms
p9999 Latency	8,243.68 ms	7,800.65 ms
Avg Latency	155.02 ms	140.63 ms
Median Latency	74.18 ms	74.07 ms
Min Latency	3.13 ms	2.10 ms
Max Latency	13,927.95 ms	9,637.32 ms
Std Dev	183.00 ms	175.81 ms
Total Time	184.15 s	158.94 s

Generated reports:
Rust: rust-report.json
Java: java-report.json

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.45%. Comparing base (578f024) to head (b701d04).

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3420      +/-   ##
============================================
+ Coverage     74.28%   74.45%   +0.16%     
  Complexity      943      943              
============================================
  Files          1245     1229      -16     
  Lines        121653   121382     -271     
  Branches      97957    97931      -26     
============================================
  Hits          90370    90370              
+ Misses        28326    28055     -271     
  Partials       2957     2957              

Components	Coverage Δ
Rust Core	75.38% <ø> (ø)
Java SDK	62.30% <ø> (+3.86%)	⬆️
C# SDK	69.92% <ø> (ø)
Python SDK	81.06% <ø> (ø)
PHP SDK	83.57% <ø> (ø)
Node SDK	91.26% <ø> (ø)
Go SDK	40.20% <ø> (ø)
see 16 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[hubcio]

@slbotbm can you retest with iggy-server built with --release flag? these results with server being built in debug mode don't provide much meaning. also, preferably, don't set warmup time to 60s. do like 1s, otherwise you saturate you disk.

[slbotbm]

Results of the new run:

core/server/config.toml changes:

cpu_allocation = "0..4"

Commands ran:
Java:

./target/release/iggy-server --fresh --with-default-root-credentials
taskset -c 4-7 ./gradlew --no-daemon :iggy-bench:run --args="pp -p 4"

Java automatically warms up for 60s and outputs report.json.

Rust:

./target/release/iggy-server --fresh --with-default-root-credentials
taskset -c 4-7 ./target/debug/iggy-bench pp -p 4 tcp output -o .

Results:

Metric	Rust	Java
Total throughput	85.74 MB/s	48.45 MB/s
Messages/s	85765 messages/s	48435 messages/s
Average throughput per Producer	21.44 MB/s	12.11 MB/s
p50 latency	18.64 ms	20.77 ms
p90 latency	80.07 ms	81.87 ms
p95 latency	83.52 ms	166.28 ms
p99 latency	127.23 ms	1131.66 ms
p999 latency	5552.00 ms	7277.87 ms
p9999 latency	11998.44 ms	9005.69 ms
Average latency	48.82 ms	84.41 ms
Median latency	18.64 ms	20.77 ms
Min latency	1.88 ms	2.16 ms
Max latency	14866.58 ms	10749.03 ms
Std dev	134.91 ms	194.62 ms
Total time	58.95 s	98.62 s

Generated reports:
Rust: report.json
Java: report.json

[hubcio]

@mmodzelewski mind checking this once you have free time?