Add low-and-slow rate-ingest nightly benchmark (1–100K eps)

[Copilot]

Expands nightly benchmarks with a non-competitive, logfwd-only benchmark that measures steady-state RSS memory and CPU utilisation at six controlled ingest rates: 1, 10, 100, 1 000, 10 000, and 100 000 eps.

New: rate_bench.rs module

• Rate-limited writer thread appends JSON log lines to a temp file at the target eps (sleep/spin-loop hybrid for accuracy)
• Samples /proc/{pid}/status (RSS) and /proc/{pid}/stat (CPU ticks) every 500 ms over a 20 s measurement window after a 5 s warmup
• Outputs: Markdown table, plain-text table, structured JSON report, and github-action-benchmark customSmallerIsBetter JSON for dashboard regression tracking

CLI (--rate-bench flag)

Added to logfwd-competitive-bench. When set, bypasses the competitive benchmark flow entirely and runs the rate bench against the LOGFWD binary:

just bench-rate                  # builds + runs locally
just bench-competitive --rate-bench --markdown --json-file rate.json

Nightly workflow additions

• New step "Run rate-ingest benchmark" runs after PGO competitive step, reusing the already-built release binary
• Artifacts: rate-bench-result.json + rate-gh-bench.json (90-day retention)
• New benchmark-action dashboard step at dev/bench-rate (customSmallerIsBetter — lower RSS/CPU is better)
• Rate-bench Markdown table and rate-bench dashboard URL included in the nightly issue body

Example output:

Target (eps)	Actual (eps)	RSS Memory	CPU %
1	1.0	12345 KiB	0.1%
10	10.0	12456 KiB	0.3%
100	100.1	13002 KiB	1.2%
1000	1000.0	15300 KiB	5.7%
10000	9998.2	18800 KiB	42.1%
100000	99870.0	23400 KiB	98.6%

Original prompt

Let's expand our nightly benchmarks (non-competitive) and let's add a benchmark for a low and slow ingest where we measure our memory and cpu usage at 1 eps, 10eps, 100eps, 1000eps, 10000eps, and 100000 eps

The user has attached the following file paths as relevant context:

• .github/copilot-instructions.md
• AGENTS.md

Created from VS Code.

[github-actions]

Requesting changes due to correctness issues in rate-bench failure handling: current behavior can emit success-shaped but invalid/partial benchmark data when logfwd exits early or when some EPS levels fail.

---

What is this? | From workflow: AI: PR Review

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

If logfwd exits during warmup/measurement, this function still returns Ok with fallback averages (rss_kb=0, cpu_percent=0.0) because /proc reads are optional and empty sample vectors average to zero. That produces a success-shaped benchmark result instead of failing the run.

A concrete failure case is startup bind/config failure (for example diagnostics port conflict): writer continues appending, actual_eps remains non-zero, and CI can publish misleading data.

Please fail this EPS run when the child exits early (e.g. try_wait) and/or when no RSS/CPU samples were collected.

[github-actions]

This swallows per-rate failures and continues, so the overall rate-bench can succeed with missing EPS points.

In run_rate_bench_main, the process exits non-zero only when results.is_empty(), which means a partial result set (e.g. 5/6 rates) is still uploaded/published as if complete.

Please make incomplete runs fail (or explicitly encode failed EPS levels and fail CI when any level failed) so dashboards/issues don’t silently reflect partial data.