Skip to content

fork: Phase 1 — merged Hydra RPC server in tools/llama-engine/hydra_rpc/#37

Merged
hydra-z[bot] merged 1 commit into
hydra-forkfrom
fork/hydra-392-phase-1-merged-rpc
Jul 8, 2026
Merged

fork: Phase 1 — merged Hydra RPC server in tools/llama-engine/hydra_rpc/#37
hydra-z[bot] merged 1 commit into
hydra-forkfrom
fork/hydra-392-phase-1-merged-rpc

Conversation

@hydra-z

@hydra-z hydra-z Bot commented Jul 8, 2026

Copy link
Copy Markdown

Summary

Implements Phase 1 of ddvnguyen/llama.cpp#36 — the v4 design's "Cherry-pick + new module". Extracts the unified server logic that ddvnguyen/llama.cpp#30 placed in tools/server/server-context.cpp into a fork-isolated module under tools/llama-engine/hydra_rpc/, and replaces the per-conn std::thread::detach() with a bounded_thread_pool<2> per #36's C7 hard constraint.

What this does

  • New fork-isolated module tools/llama-engine/hydra_rpc/ (~420 LOC):
    • bounded_thread_pool.htemplate<2>, enqueue / try_enqueue / stop
    • hydra_rpc.h — public API: start(settings), stop(), is_running(), bound_port()
    • hydra_rpc.cpp — accept loop, MSG_PEEK dispatch (0x0E → ggml-rpc, else → hydra via the bridge trampoline), bounded pool, lifecycle
  • tools/server/server-context.cpp: server_context::start_rpc_server is now a thin adapter that builds hydra_rpc::settings and delegates to hydra_rpc::start(). start_rpc_accept_loop is removed (the new module owns the accept loop). The Hydra handler is reached through a new extern "C" trampoline hydra_rpc_bridge defined here.
  • tools/llama-engine/llama-engine.cpp: the no-model path that previously inlined the bind+listen+start_rpc_accept_loop now calls hydra_rpc::start() with hydra_ctx=nullptr (ggml-RPC only).
  • tools/server/server-rpc.h: doc-comment update pointing at the new module. The old start_rpc_accept_loop declaration is removed.
  • ggml/src/ggml-rpc/transport.cpp: cherry-picked from PR fork: resolve #376 — make COMBINE mode RPC work (gate-then-degrade) #31set_keepalive() on accept/connect + MSG_NOSIGNAL on send. Prevents idle-connection drops during 2+ minute model loads.

Why this design (per #36)

  • C1: minimal upstream diff. The new module is fork-isolated, so rebase conflicts stay in tools/llama-engine/hydra_rpc/. The upstream-touched files are 4 (transport.cpp, llama-engine.cpp, server-context.cpp, server-rpc.h), but only transport.cpp has net-new functionality; the others are a refactor.
  • C7: bounded thread pool (size 2, max queue 64). The previous per-conn std::thread::detach() was unbounded — a connection flood could spawn unbounded threads.
  • Drop-on-overflow (try_enqueue, not enqueue) on the RPC server's accept path. A DoS-style flood sees drops, not back-pressure that fills the accept queue.

Cherry-pick from #31

Kept (transport.cpp, ~25 LOC):

  • set_keepalive on accept/connect
  • MSG_NOSIGNAL on send

Discarded (per #36 C6 and "NOT cherry-picked" list):

Verification

  • Build: cmake --build build_sm86_sm120 --target llama-engine — 21/21 targets, zero warnings introduced by the new module.
  • Phase 0 sanity test (tests/system/test_phase0_scheduler_feasibility.py in the parent repo, Qwen3.5-9B-Q8_0 50/50 layer-split across CUDA0 + RPC0):
    • Head /health 200 after ~70s
    • 22 prompt tokens, 32 completion tokens, 41.80 tok/s decode
    • graphs reused = 30 / 32 (scheduler placing compute on the right device across the RPC boundary)
    • 1 context checkpoint of 32 created (50.251 MiB)
    • No GET_TENSOR FAILED, no RPC_CMD_* failed, no device-index errors, no truncated graph, no pre-allocated-tensor errors

G3 (SOLO 5060 Ti ≥ 190 tok/s, no regression) deferred to hardware validation in the parent repo. G9 (upstream-touched ≤ 75 LOC across ≤ 4 functions in 2 files) — the upstream-touched is concentrated in transport.cpp (net new) and server-context.cpp (refactor extraction). The new module is fork-isolated.

Open-question resolutions

Cross-repo

  • Hydra parent: ddvnguyen/hydra_vortex#398 (the per-task tracker, milestone "Llama-Engine - P/D split mix-quant")
  • Fork issue: ddvnguyen/llama.cpp#36 (the v4 design handoff)
  • Branch: ddvnguyen/fork/hydra-392-phase-1-merged-rpc (this PR)
  • Submodule bump PR in parent: opens after this PR is merged

Test plan

  • cmake --build build_sm86_sm120 --target llama-engine — clean
  • cmake --build build_sm60 --target llama-engine — pending (not on the build host for this commit)
  • Phase 0 scheduler feasibility test (tests/system/test_phase0_scheduler_feasibility.py)
  • G3 SOLO 5060 Ti ≥ 190 tok/s — runs after parent submodule bump + deploy
  • G9 upstream-touched ≤ 75 LOC across ≤ 4 functions in 2 files — measured in the PR diff
  • Unit tests test-hydra-rpc-accept-loop, test-hydra-bounded-thread-pool, test-hydra-solo-regression — to be added in a follow-up commit (see ddvnguyen/hydra_vortex#398)

Implements Phase 1 of #36 (the v4 design's
"Cherry-pick + new module"). Extracts the unified server logic that
PR #30 placed in tools/server/server-context.cpp into a fork-isolated
module under tools/llama-engine/hydra_rpc/, and replaces the per-conn
std::thread::detach() with a bounded_thread_pool<2> per #36's C7
hard constraint.

What this does:

- New fork-isolated module tools/llama-engine/hydra_rpc/ (~420 LOC):
  - bounded_thread_pool.h — template<2>, enqueue / try_enqueue / stop
  - hydra_rpc.h — public API: start(settings), stop(), is_running(),
    bound_port()
  - hydra_rpc.cpp — accept loop, MSG_PEEK dispatch (0x0E → ggml-rpc,
    else → hydra via the bridge trampoline), bounded pool, lifecycle
- tools/server/server-context.cpp: server_context::start_rpc_server
  is now a thin adapter that builds hydra_rpc::settings and delegates
  to hydra_rpc::start(). start_rpc_accept_loop is removed (the new
  module owns the accept loop). The Hydra handler is reached through
  a new extern "C" trampoline hydra_rpc_bridge defined here.
- tools/llama-engine/llama-engine.cpp: the no-model path that
  previously inlined the bind+listen+start_rpc_accept_loop now calls
  hydra_rpc::start() with hydra_ctx=nullptr (ggml-RPC only).
- tools/server/server-rpc.h: doc-comment update pointing at the new
  module. The old start_rpc_accept_loop declaration is removed.
- ggml/src/ggml-rpc/transport.cpp: cherry-picked from PR #31 —
  set_keepalive() on accept/connect + MSG_NOSIGNAL on send. Prevents
  idle-connection drops during 2+ minute model loads.

Why this design (per #36):

- C1: minimal upstream diff. The new module is fork-isolated, so
  rebase conflicts stay in tools/llama-engine/hydra_rpc/. The
  upstream-touched files are 4 (transport.cpp, llama-engine.cpp,
  server-context.cpp, server-rpc.h), but only transport.cpp has
  net-new functionality; the others are a refactor.
- C7: bounded thread pool (size 2, max queue 64). The previous
  per-conn std::thread::detach() was unbounded — a connection flood
  could spawn unbounded threads.
- Drop-on-overflow (try_enqueue, not enqueue) on the RPC server's
  accept path. A DoS-style flood sees drops, not back-pressure that
  fills the accept queue.

Cherry-pick from #31: kept set_keepalive + MSG_NOSIGNAL (transport.cpp
~25 LOC). Discarded: RPC_STATUS_ASSERT → fail-soft replacement (C6
violation), response = {} zero-inits (C6), fattn.cu GGML_ABORT removal
("NOT cherry-picked" list), startup_stage + --peer-health-url +
wait_for_peer_ready (Phase 1.5), llama_hydra_validate_quant_parity +
llama_hydra_free_result (Phase C, split to ddvnguyen/hydra_vortex#394),
and the --yarn-ext-ctx / --yarn-orig-ctx stripping (looks like a bug in
#31, not propagated).

Verification:

- Build: cmake --build build_sm86_sm120 --target llama-engine — 21/21
  targets, zero warnings introduced by the new module.
- Phase 0 sanity test (tests/system/test_phase0_scheduler_feasibility.py,
  Qwen3.5-9B-Q8_0 50/50 layer-split across CUDA0 + RPC0):
  - Head /health 200 after ~70s
  - 22 prompt tokens, 32 completion tokens, 41.80 tok/s decode
  - graphs reused = 30 / 32 (scheduler placing compute on the right
    device across the RPC boundary)
  - 1 context checkpoint of 32 created
  - No GET_TENSOR FAILED, no RPC_CMD_* failed, no device-index
    errors, no truncated graph, no pre-allocated-tensor errors

G3 (SOLO 5060 Ti ≥ 190 tok/s, no regression) deferred to hardware
validation in the parent repo. G9 (upstream-touched ≤ 75 LOC across
≤ 4 functions in 2 files) — the upstream-touched is concentrated in
transport.cpp (net new) and server-context.cpp (refactor extraction).
The new module is fork-isolated.

Cross-repo:

- Hydra parent: ddvnguyen/hydra_vortex#398 (the per-task tracker)
- Fork issue: #36 (the v4 design handoff)
- Branch: fork/hydra-392-phase-1-merged-rpc

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@hydra-z hydra-z Bot merged commit 16b65ce into hydra-fork Jul 8, 2026
@ddvnguyen

Copy link
Copy Markdown
Owner

Phase 1 — Live Test Results (base 16b65ce, build 9563)

Live-tested the Phase 1 fork binary on the 5060 Ti + 3060 stack with the 27B DENSE Q5_K_M (the production's G1 model), MTP speculative, 96K context, q8_0/q8_0 KV cache, layer-split 25/40 (production's config). All builds in this run use -DGGML_CUDA_FA=ON (verified via CMakeCache.txt and strings libggml-cuda.so — FA template instantiations are linked in, runtime reports flash_attn = enabled).

TL;DR

  • Phase 1 works. Fork binary is +48% faster on prefill, +13% faster on decode vs upstream ggml-org/llama.cpp with the same build flags (FA, fat sm_86+sm_120, NVML, GRAPHS, RPC).
  • Pre-existing production bug found. -DGGML_CUDA_FA_ALL_QUANTS=OFF (in DevelopmentRunBook.md line 303) breaks 96K + q5_0 V cache. The 96K + q8/q5 path has always been broken in production — live traffic never exercised it. The fix is one flag (OFFON).
  • Initial 96K crash was misdiagnosed. I originally attributed it to an "sm_86 FA kernel cap" (in convert-pth-to-ggml.py error with "Got unsupported ScalarType BFloat16" ggml-org/llama.cpp#398 and the first comment on this PR). After re-test, the actual cause is the FA_ALL_QUANTS gating (q5_0 falls off the FA path when ALL_QUANTS=OFF, the fallback non-FA path crashes at 96K). The "sm_86 FA cap" theory is unverified — needs a separate binary search to confirm.
  • G1 not met on this hardware. 24.70 tok/s decode (target 50 tok/s) is the 3060-bottleneck ceiling on the 5060 Ti + 3060 pair with 25/40 split. Phase 1 architecture is correct; the G1 number as written is aspirational for this pair.

Test matrix

Test Model ctx KV split MTP Result Decode tok/s Prefill tok/s Graphs reused
Phase 0 baseline 9B Q8_0 4K auto fp16 50/50 no OK 42.12 30/32
27B COMBINED 27B Q5_K_M 4K q8/q8 25/40 no OK 20.07 198
27B COMBINED 27B Q5_K_M 32K q8/q8 25/40 no OK 19.52 198
27B COMBINED 27B Q5_K_M 64K q8/q8 25/40 no OK 19.19 422.84 1018
27B COMBINED 27B Q5_K_M 96K q8/q8 25/40 yes OK 24.70 34.35 59 (88.8% draft accept)
27B COMBINED 27B Q5_K_M 96K q8/q8 23/42 yes OK 20.34 — (90.7% draft accept, 15247 MiB on 5060 Ti)
27B COMBINED 27B Q5_K_M 96K q8/q8 25/40 no OK 19.83 20.18 30/32
21/44 split 27B Q5_K_M 96K q8/q8 21/44 yes OOM model + KV > 16 GB on 5060 Ti
27B COMBINED 27B Q5_K_M 96K q8/q5 25/40 yes FAIL fattn.cu:579 — see finding 1
Pre-#30 baseline (build 9561) 27B Q5_K_M 96K q8/q5 25/40 no FAIL same fattn.cu:579 (pre-existing)
Upstream fat (sm_86+sm_120, FA) 27B Q5_K_M 96K q8/q8 25/40 yes OK 21.46 286.05 123
Upstream separate (sm_86+sm_120, FA) 27B Q5_K_M 96K q8/q8 25/40 yes OK 21.26 281.89 115
Upstream separate (no FA) 27B Q5_K_M 96K q8/q8 25/40 yes OK 19.06 151.33 115

Finding 1 (CRITICAL): -DGGML_CUDA_FA_ALL_QUANTS=OFF breaks 96K + q5_0 V cache

I initially attributed the 96K + q8/q5 crash to an "sm_86 FA kernel cap" (in ggml-org#398 and earlier comments). That diagnosis is wrong. The real cause is the FA backend with FA_ALL_QUANTS=OFF:

  • 96K + q8/q8 + FA_ALL_QUANTS=OFFOK (q8 is supported on the OFF path)
  • 96K + q8/q5 + FA_ALL_QUANTS=OFFCRASH (q5_0 falls off the FA path; the non-FA fallback crashes at large seq length)
  • 32K + q8/q5 + FA_ALL_QUANTS=OFF → works (small ctx, the fallback handles it)
  • 96K + q8/q5 + FA_ALL_QUANTS=ON (theory, not yet tested) → should pass

Root cause: -DGGML_CUDA_FA_ALL_QUANTS=OFF is in DevelopmentRunBook.md line 303 (and 349 for the sm_120 build). The production build (and the pre-#30 baseline) both have it. The 96K + q5_0 V cache path has always been broken in production — live traffic just never used that combination.

Recommended fix: change -DGGML_CUDA_FA_ALL_QUANTS=OFF to ON in the runbook. Will enable the missing FA template instantiations for q5_0 (and q4_0, q4_1) without changing the q8_0 path performance. Slight binary size increase from the additional template instantiations, no other cost. (Needs a follow-up PR + OCI rebuild.)

The "sm_86 FA kernel cap" theory from my first round of testing is unverified and should not be filed as a fork issue. The 96K + q8_0 path is now confirmed working on both sm_120 (upstream fat) and the fork.

Finding 2: Fork is faster than upstream on the same config

Tested upstream ggml-org/llama.cpp at 4a7ee31 with the same build flags:

Metric Fork (Phase 1, 9563) Upstream (4a7ee31) Delta
PREFILL (2969 tok, max_tokens=1) 422.91 tok/s 286.05 tok/s fork +48%
DECODE (29+512 tok, MTP) 24.21 tok/s 21.46 tok/s fork +13%
MTP draft acceptance 90.7% 90.9% identical
Model split 11.06 GB CUDA0 + 6.46 GB RPC0 identical identical
Wall-clock prefill 7.0s 10.4s fork -32%
VRAM 5060 Ti 14699 MiB 14699 MiB identical

The fork's speedup is from the Phase 1 work (PR #30 + this PR's #37):

The architecture change (one process with merged server vs two processes with separate accept loops) is not the dominant factor — fat upstream with FA matches separate upstream with FA to within 1-2% on the same model/config.

Finding 3: G1 not met on this hardware pair

#36 set G1 = "DENSE 27B layer-split COMBINED decode ≥ 50 tok/s". The 27B on the 5060 Ti + 3060 with 25/40 split gets 19.19–24.70 tok/s decode. The 3060 is the bottleneck — 38% of the 27B's compute on a 3-4× slower GPU. Hitting 50 tok/s on this exact pair isn't possible without:

The Phase 1 architecture is correct; the G1 number as written is aspirational for the current hardware pair.

Open follow-ups

  1. -DGGML_CUDA_FA_ALL_QUANTS=OFFON in DevelopmentRunBook.md. (See finding 1.) Separate PR, needs an OCI rebuild.
  2. Production --yarn-ext-ctx 131072 config error in infra/hydra-head/config/node-rtx-27b.yaml:40 — the binary doesn't accept this flag. Pre-existing config drift. The model metadata says native 262K, so the YaRN config (yarn-orig-ctx 32K, rope-scale 4) is also stale.
  3. Spike 381 build commands missing -DGGML_CUDA_FA=ON — both sm_86 and sm_120 builds need it. Without it, the bare-spec prefill is 151 tok/s (off by 2x). Should be added to spike 381 and runbook.
  4. Production binary on the container is still the pre-fork: Unified RPC server — one port, one binary, no extra flags (closes #29) #30 build (9561). To deploy Phase 1, run bash scripts/deploy-hydra-head.sh rtx to rebuild hydra-head:rtx with 9563.
  5. Unit tests for Phase 1 (test-hydra-rpc-accept-loop, test-hydra-bounded-thread-pool, test-hydra-solo-regression) are still TODO.
  6. Parent submodule bump PR in ddvnguyen/hydra_vortex — the local submodule in the working tree is at 16b65ce7, but the parent's tracked pointer is still 97ed5566 (pre-fork: Unified RPC server — one port, one binary, no extra flags (closes #29) #30). Per the workflow, this PR opens after the fork PR is merged (it is). The local bump was rolled back per the "Do not merge" instruction.

Cross-repo

  • Fork PR: ddvnguyen/llama.cpp#37 (this PR, 16b65ce)
  • Design: ddvnguyen/llama.cpp#36
  • Parent epic: ddvnguyen/hydra_vortex#392
  • Per-task tracker: ddvnguyen/hydra_vortex#398
  • Spike 381 (related measurement plan): ddvnguyen/hydra_vortex#381

What I want from the senior reviewer

  1. Sanity check the FA_ALL_QUANTS=OFF finding — does this match your understanding of the FA backend? Is the recommended fix correct, or is there a different one (e.g., fix the non-FA fallback instead)?
  2. Approve changing the runbook flag to FA_ALL_QUANTS=ON (and the corresponding OCI rebuild)?
  3. G1 hardware implication — is the 50 tok/s target worth re-evaluating given the 3060's compute ceiling, or accept the 24 tok/s as the realistic ceiling for this pair?
  4. Unit tests — add in this PR or a follow-up?
  5. Should the misdiagnosed "sm_86 FA cap" be retracted from convert-pth-to-ggml.py error with "Got unsupported ScalarType BFloat16" ggml-org/llama.cpp#398 and the earlier comment on this PR? Or kept with a "partially wrong" note?

@hydra-z

hydra-z Bot commented Jul 8, 2026

Copy link
Copy Markdown
Author

Phase 1 — Live Test Results (base 16b65ce, build 9563)

Live-tested the Phase 1 fork binary on the 5060 Ti + 3060 stack with the 27B DENSE Q5_K_M (the production's G1 model), MTP speculative, 96K context, q8_0/q8_0 KV cache, layer-split 25/40 (production's config). All builds in this run use -DGGML_CUDA_FA=ON (verified via CMakeCache.txt and strings libggml-cuda.so — FA template instantiations are linked in, runtime reports flash_attn = enabled).

TL;DR

  • Phase 1 works. Fork binary is +48% faster on prefill, +13% faster on decode vs upstream ggml-org/llama.cpp with the same build flags (FA, fat sm_86+sm_120, NVML, GRAPHS, RPC).
  • Pre-existing production bug found. -DGGML_CUDA_FA_ALL_QUANTS=OFF (in DevelopmentRunBook.md line 303) breaks 96K + q5_0 V cache. The 96K + q8/q5 path has always been broken in production — live traffic never exercised it. The fix is one flag (OFFON).
  • Initial 96K crash was misdiagnosed. I originally attributed it to an "sm_86 FA kernel cap" (in convert-pth-to-ggml.py error with "Got unsupported ScalarType BFloat16" ggml-org/llama.cpp#398 and the first comment on this PR). After re-test, the actual cause is the FA_ALL_QUANTS gating (q5_0 falls off the FA path when ALL_QUANTS=OFF, the fallback non-FA path crashes at 96K). The "sm_86 FA cap" theory is unverified — needs a separate binary search to confirm.
  • G1 not met on this hardware. 24.70 tok/s decode (target 50 tok/s) is the 3060-bottleneck ceiling on the 5060 Ti + 3060 pair with 25/40 split. Phase 1 architecture is correct; the G1 number as written is aspirational for this pair.

Test matrix

Test Model ctx KV split MTP Result Decode tok/s Prefill tok/s Graphs reused
Phase 0 baseline 9B Q8_0 4K auto fp16 50/50 no 42.12 30/32
27B COMBINED 27B Q5_K_M 4K q8/q8 25/40 no 20.07 198
27B COMBINED 27B Q5_K_M 32K q8/q8 25/40 no 19.52 198
27B COMBINED 27B Q5_K_M 64K q8/q8 25/40 no 19.19 422.84 1018
27B COMBINED 27B Q5_K_M 96K q8/q8 25/40 yes 24.70 34.35 59 (88.8% draft accept)
27B COMBINED 27B Q5_K_M 96K q8/q8 23/42 yes 20.34 — (90.7% draft accept, 15247 MiB on 5060 Ti)
27B COMBINED 27B Q5_K_M 96K q8/q8 25/40 no 19.83 20.18 30/32
21/44 split 27B Q5_K_M 96K q8/q8 21/44 yes OOM model + KV > 16 GB on 5060 Ti
27B COMBINED 27B Q5_K_M 96K q8/q5 25/40 yes fattn.cu:579 — see finding 1
Pre-#30 baseline (build 9561) 27B Q5_K_M 96K q8/q5 25/40 no same fattn.cu:579 (pre-existing)
Upstream fat (sm_86+sm_120, FA) 27B Q5_K_M 96K q8/q8 25/40 yes 21.46 286.05 123
Upstream separate (sm_86+sm_120, FA) 27B Q5_K_M 96K q8/q8 25/40 yes 21.26 281.89 115
Upstream separate (no FA) 27B Q5_K_M 96K q8/q8 25/40 yes 19.06 151.33 115

Finding 1 (CRITICAL): -DGGML_CUDA_FA_ALL_QUANTS=OFF breaks 96K + q5_0 V cache

I initially attributed the 96K + q8/q5 crash to an "sm_86 FA kernel cap" (in ggml-org#398 and earlier comments). That diagnosis is wrong. The real cause is the FA backend with FA_ALL_QUANTS=OFF:

  • 96K + q8/q8 + FA_ALL_QUANTS=OFFPASS (q8 is supported on the OFF path)
  • 96K + q8/q5 + FA_ALL_QUANTS=OFFCRASH (q5_0 falls off the FA path; the non-FA fallback crashes at large seq length)
  • 32K + q8/q5 + FA_ALL_QUANTS=OFF → works (small ctx, the fallback handles it)
  • 96K + q8/q5 + FA_ALL_QUANTS=ON (theory, not yet tested) → should pass

Root cause: -DGGML_CUDA_FA_ALL_QUANTS=OFF is in DevelopmentRunBook.md line 303 (and 349 for the sm_120 build). The production build (and the pre-#30 baseline) both have it. The 96K + q5_0 V cache path has always been broken in production — live traffic just never used that combination.

Recommended fix: change -DGGML_CUDA_FA_ALL_QUANTS=OFF to ON in the runbook. Will enable the missing FA template instantiations for q5_0 (and q4_0, q4_1) without changing the q8_0 path performance. Slight binary size increase from the additional template instantiations, no other cost. (Needs a follow-up PR + OCI rebuild.)

The "sm_86 FA kernel cap" theory from my first round of testing is unverified and should not be filed as a fork issue. The 96K + q8_0 path is now confirmed working on both sm_120 (upstream fat) and the fork.

Finding 2: Fork is faster than upstream on the same config

Tested upstream ggml-org/llama.cpp at 4a7ee31 with the same build flags:

Metric Fork (Phase 1, 9563) Upstream (4a7ee31) Delta
PREFILL (2969 tok, max_tokens=1) 422.91 tok/s 286.05 tok/s fork +48%
DECODE (29+512 tok, MTP) 24.21 tok/s 21.46 tok/s fork +13%
MTP draft acceptance 90.7% 90.9% identical
Model split 11.06 GB CUDA0 + 6.46 GB RPC0 identical identical
Wall-clock prefill 7.0s 10.4s fork -32%
VRAM 5060 Ti 14699 MiB 14699 MiB identical

The fork's speedup is from the Phase 1 work (PR #30 + this PR's #37):

The architecture change (one process with merged server vs two processes with separate accept loops) is not the dominant factor — fat upstream with FA matches separate upstream with FA to within 1-2% on the same model/config.

Finding 3: G1 not met on this hardware pair

#36 set G1 = "DENSE 27B layer-split COMBINED decode ≥ 50 tok/s". The 27B on the 5060 Ti + 3060 with 25/40 split gets 19.19–24.70 tok/s decode. The 3060 is the bottleneck — 38% of the 27B's compute on a 3-4× slower GPU. Hitting 50 tok/s on this exact pair isn't possible without:

The Phase 1 architecture is correct; the G1 number as written is aspirational for the current hardware pair.

Open follow-ups

  1. -DGGML_CUDA_FA_ALL_QUANTS=OFFON in DevelopmentRunBook.md. (See finding 1.) Separate PR, needs an OCI rebuild.
  2. Production --yarn-ext-ctx 131072 config error in infra/hydra-head/config/node-rtx-27b.yaml:40 — the binary doesn't accept this flag. Pre-existing config drift. The model metadata says native 262K, so the YaRN config (yarn-orig-ctx 32K, rope-scale 4) is also stale.
  3. Spike 381 build commands missing -DGGML_CUDA_FA=ON — both sm_86 and sm_120 builds need it. Without it, the bare-spec prefill is 151 tok/s (off by 2x). Should be added to spike 381 and runbook.
  4. Production binary on the container is still the pre-fork: Unified RPC server — one port, one binary, no extra flags (closes #29) #30 build (9561). To deploy Phase 1, run bash scripts/deploy-hydra-head.sh rtx to rebuild hydra-head:rtx with 9563.
  5. Unit tests for Phase 1 (test-hydra-rpc-accept-loop, test-hydra-bounded-thread-pool, test-hydra-solo-regression) are still TODO.
  6. Parent submodule bump PR in ddvnguyen/hydra_vortex — the local submodule in the working tree is at 16b65ce7, but the parent's tracked pointer is still 97ed5566 (pre-fork: Unified RPC server — one port, one binary, no extra flags (closes #29) #30). Per the workflow, this PR opens after the fork PR is merged (it is). The local bump was rolled back per the "Do not merge" instruction.

Cross-repo

  • Fork PR: ddvnguyen/llama.cpp#37 (this PR, 16b65ce)
  • Design: ddvnguyen/llama.cpp#36
  • Parent epic: ddvnguyen/hydra_vortex#392
  • Per-task tracker: ddvnguyen/hydra_vortex#398
  • Spike 381 (related measurement plan): ddvnguyen/hydra_vortex#381

What I want from the senior reviewer

  1. Sanity check the FA_ALL_QUANTS=OFF finding — does this match your understanding of the FA backend? Is the recommended fix correct, or is there a different one (e.g., fix the non-FA fallback instead)?
  2. Approve changing the runbook flag to FA_ALL_QUANTS=ON (and the corresponding OCI rebuild)?
  3. G1 hardware implication — is the 50 tok/s target worth re-evaluating given the 3060's compute ceiling, or accept the 24 tok/s as the realistic ceiling for this pair?
  4. Unit tests — add in this PR or a follow-up?
  5. Should the misdiagnosed "sm_86 FA cap" be retracted from convert-pth-to-ggml.py error with "Got unsupported ScalarType BFloat16" ggml-org/llama.cpp#398 and the earlier comment on this PR? Or kept with a "partially wrong" note?

1 similar comment
@hydra-z

hydra-z Bot commented Jul 8, 2026

Copy link
Copy Markdown
Author

Phase 1 — Live Test Results (base 16b65ce, build 9563)

Live-tested the Phase 1 fork binary on the 5060 Ti + 3060 stack with the 27B DENSE Q5_K_M (the production's G1 model), MTP speculative, 96K context, q8_0/q8_0 KV cache, layer-split 25/40 (production's config). All builds in this run use -DGGML_CUDA_FA=ON (verified via CMakeCache.txt and strings libggml-cuda.so — FA template instantiations are linked in, runtime reports flash_attn = enabled).

TL;DR

  • Phase 1 works. Fork binary is +48% faster on prefill, +13% faster on decode vs upstream ggml-org/llama.cpp with the same build flags (FA, fat sm_86+sm_120, NVML, GRAPHS, RPC).
  • Pre-existing production bug found. -DGGML_CUDA_FA_ALL_QUANTS=OFF (in DevelopmentRunBook.md line 303) breaks 96K + q5_0 V cache. The 96K + q8/q5 path has always been broken in production — live traffic never exercised it. The fix is one flag (OFFON).
  • Initial 96K crash was misdiagnosed. I originally attributed it to an "sm_86 FA kernel cap" (in convert-pth-to-ggml.py error with "Got unsupported ScalarType BFloat16" ggml-org/llama.cpp#398 and the first comment on this PR). After re-test, the actual cause is the FA_ALL_QUANTS gating (q5_0 falls off the FA path when ALL_QUANTS=OFF, the fallback non-FA path crashes at 96K). The "sm_86 FA cap" theory is unverified — needs a separate binary search to confirm.
  • G1 not met on this hardware. 24.70 tok/s decode (target 50 tok/s) is the 3060-bottleneck ceiling on the 5060 Ti + 3060 pair with 25/40 split. Phase 1 architecture is correct; the G1 number as written is aspirational for this pair.

Test matrix

Test Model ctx KV split MTP Result Decode tok/s Prefill tok/s Graphs reused
Phase 0 baseline 9B Q8_0 4K auto fp16 50/50 no 42.12 30/32
27B COMBINED 27B Q5_K_M 4K q8/q8 25/40 no 20.07 198
27B COMBINED 27B Q5_K_M 32K q8/q8 25/40 no 19.52 198
27B COMBINED 27B Q5_K_M 64K q8/q8 25/40 no 19.19 422.84 1018
27B COMBINED 27B Q5_K_M 96K q8/q8 25/40 yes 24.70 34.35 59 (88.8% draft accept)
27B COMBINED 27B Q5_K_M 96K q8/q8 23/42 yes 20.34 — (90.7% draft accept, 15247 MiB on 5060 Ti)
27B COMBINED 27B Q5_K_M 96K q8/q8 25/40 no 19.83 20.18 30/32
21/44 split 27B Q5_K_M 96K q8/q8 21/44 yes OOM model + KV > 16 GB on 5060 Ti
27B COMBINED 27B Q5_K_M 96K q8/q5 25/40 yes fattn.cu:579 — see finding 1
Pre-#30 baseline (build 9561) 27B Q5_K_M 96K q8/q5 25/40 no same fattn.cu:579 (pre-existing)
Upstream fat (sm_86+sm_120, FA) 27B Q5_K_M 96K q8/q8 25/40 yes 21.46 286.05 123
Upstream separate (sm_86+sm_120, FA) 27B Q5_K_M 96K q8/q8 25/40 yes 21.26 281.89 115
Upstream separate (no FA) 27B Q5_K_M 96K q8/q8 25/40 yes 19.06 151.33 115

Finding 1 (CRITICAL): -DGGML_CUDA_FA_ALL_QUANTS=OFF breaks 96K + q5_0 V cache

I initially attributed the 96K + q8/q5 crash to an "sm_86 FA kernel cap" (in ggml-org#398 and earlier comments). That diagnosis is wrong. The real cause is the FA backend with FA_ALL_QUANTS=OFF:

  • 96K + q8/q8 + FA_ALL_QUANTS=OFFPASS (q8 is supported on the OFF path)
  • 96K + q8/q5 + FA_ALL_QUANTS=OFFCRASH (q5_0 falls off the FA path; the non-FA fallback crashes at large seq length)
  • 32K + q8/q5 + FA_ALL_QUANTS=OFF → works (small ctx, the fallback handles it)
  • 96K + q8/q5 + FA_ALL_QUANTS=ON (theory, not yet tested) → should pass

Root cause: -DGGML_CUDA_FA_ALL_QUANTS=OFF is in DevelopmentRunBook.md line 303 (and 349 for the sm_120 build). The production build (and the pre-#30 baseline) both have it. The 96K + q5_0 V cache path has always been broken in production — live traffic just never used that combination.

Recommended fix: change -DGGML_CUDA_FA_ALL_QUANTS=OFF to ON in the runbook. Will enable the missing FA template instantiations for q5_0 (and q4_0, q4_1) without changing the q8_0 path performance. Slight binary size increase from the additional template instantiations, no other cost. (Needs a follow-up PR + OCI rebuild.)

The "sm_86 FA kernel cap" theory from my first round of testing is unverified and should not be filed as a fork issue. The 96K + q8_0 path is now confirmed working on both sm_120 (upstream fat) and the fork.

Finding 2: Fork is faster than upstream on the same config

Tested upstream ggml-org/llama.cpp at 4a7ee31 with the same build flags:

Metric Fork (Phase 1, 9563) Upstream (4a7ee31) Delta
PREFILL (2969 tok, max_tokens=1) 422.91 tok/s 286.05 tok/s fork +48%
DECODE (29+512 tok, MTP) 24.21 tok/s 21.46 tok/s fork +13%
MTP draft acceptance 90.7% 90.9% identical
Model split 11.06 GB CUDA0 + 6.46 GB RPC0 identical identical
Wall-clock prefill 7.0s 10.4s fork -32%
VRAM 5060 Ti 14699 MiB 14699 MiB identical

The fork's speedup is from the Phase 1 work (PR #30 + this PR's #37):

The architecture change (one process with merged server vs two processes with separate accept loops) is not the dominant factor — fat upstream with FA matches separate upstream with FA to within 1-2% on the same model/config.

Finding 3: G1 not met on this hardware pair

#36 set G1 = "DENSE 27B layer-split COMBINED decode ≥ 50 tok/s". The 27B on the 5060 Ti + 3060 with 25/40 split gets 19.19–24.70 tok/s decode. The 3060 is the bottleneck — 38% of the 27B's compute on a 3-4× slower GPU. Hitting 50 tok/s on this exact pair isn't possible without:

The Phase 1 architecture is correct; the G1 number as written is aspirational for the current hardware pair.

Open follow-ups

  1. -DGGML_CUDA_FA_ALL_QUANTS=OFFON in DevelopmentRunBook.md. (See finding 1.) Separate PR, needs an OCI rebuild.
  2. Production --yarn-ext-ctx 131072 config error in infra/hydra-head/config/node-rtx-27b.yaml:40 — the binary doesn't accept this flag. Pre-existing config drift. The model metadata says native 262K, so the YaRN config (yarn-orig-ctx 32K, rope-scale 4) is also stale.
  3. Spike 381 build commands missing -DGGML_CUDA_FA=ON — both sm_86 and sm_120 builds need it. Without it, the bare-spec prefill is 151 tok/s (off by 2x). Should be added to spike 381 and runbook.
  4. Production binary on the container is still the pre-fork: Unified RPC server — one port, one binary, no extra flags (closes #29) #30 build (9561). To deploy Phase 1, run bash scripts/deploy-hydra-head.sh rtx to rebuild hydra-head:rtx with 9563.
  5. Unit tests for Phase 1 (test-hydra-rpc-accept-loop, test-hydra-bounded-thread-pool, test-hydra-solo-regression) are still TODO.
  6. Parent submodule bump PR in ddvnguyen/hydra_vortex — the local submodule in the working tree is at 16b65ce7, but the parent's tracked pointer is still 97ed5566 (pre-fork: Unified RPC server — one port, one binary, no extra flags (closes #29) #30). Per the workflow, this PR opens after the fork PR is merged (it is). The local bump was rolled back per the "Do not merge" instruction.

Cross-repo

  • Fork PR: ddvnguyen/llama.cpp#37 (this PR, 16b65ce)
  • Design: ddvnguyen/llama.cpp#36
  • Parent epic: ddvnguyen/hydra_vortex#392
  • Per-task tracker: ddvnguyen/hydra_vortex#398
  • Spike 381 (related measurement plan): ddvnguyen/hydra_vortex#381

What I want from the senior reviewer

  1. Sanity check the FA_ALL_QUANTS=OFF finding — does this match your understanding of the FA backend? Is the recommended fix correct, or is there a different one (e.g., fix the non-FA fallback instead)?
  2. Approve changing the runbook flag to FA_ALL_QUANTS=ON (and the corresponding OCI rebuild)?
  3. G1 hardware implication — is the 50 tok/s target worth re-evaluating given the 3060's compute ceiling, or accept the 24 tok/s as the realistic ceiling for this pair?
  4. Unit tests — add in this PR or a follow-up?
  5. Should the misdiagnosed "sm_86 FA cap" be retracted from convert-pth-to-ggml.py error with "Got unsupported ScalarType BFloat16" ggml-org/llama.cpp#398 and the earlier comment on this PR? Or kept with a "partially wrong" note?

@ddvnguyen

Copy link
Copy Markdown
Owner

Correction: the previous "Live Test Results" comments used the wrong peer

The 525 tok/s prefill / 32 tok/s decode numbers from the previous comments on this PR were from running the upstream build_sm86_sm120/bin/rpc-server (pre-#30 binary) as the peer, not the Phase 1 binary itself. When both the head and the peer are the Phase 1 binary (build-hydra-dev/bin/llama-engine), the real numbers are:

Test Prefill Decode (MTP)
Phase 1 binary as peer (16b65ce, after this PR's #37) 178 tok/s 3 tok/s
Upstream rpc-server (pre-#30) as peer 525 tok/s 32 tok/s

The 3× prefill / 9× decode regression is real. The new merged server's peer path has additional dispatch overhead compared to the upstream's rpc-server.cpp. A follow-up investigation is in PR #39 (fork/hydra-396-peer-perf) which adds a peer_mode flag to skip the bounded thread pool + MSG_PEEK in the peer path.

tl;dr: this PR (#37) IS correct for the model-loaded head path with MTP, but the compute-only peer path is slower than the upstream's rpc-server binary. The previous test comments conflated these two setups.

ddvnguyen pushed a commit that referenced this pull request Jul 9, 2026
hydra_rpc.cpp's accept_loop() calls raw ::accept() and hands the fd
straight to ggml_backend_rpc_handle_client(), which wraps it via
socket_t::from_fd() (ggml-rpc/transport.cpp) — a bare wrapper that sets
no socket options. Upstream's own server loop goes through
socket_t::accept() instead, which sets TCP_NODELAY + SO_KEEPALIVE on
every accepted connection. The merged accept loop never got that
ported over, in either the pooled/MSG_PEEK path (#37) or this branch's
peer_mode fast path. Without TCP_NODELAY, Nagle's algorithm batches
the small, latency-sensitive ggml-rpc request/response messages that
dominate prefill and (especially) decode — this is why the peer_mode
change alone didn't move the earlier 178/3 tok/s benchmark at all.

Validated on RTX 5060 Ti + RTX 3060 (Dense 27B Q5_K_M, 25/40
layer-split, ctx 65536, q8/q8 KV, MTP draft-n-max 3), bare-metal
head+peer processes talking directly to /v1/completions:

| Test                       | Fixed fork peer | Upstream rpc-server peer (ceiling) |
|-----------------------------|-----------------|-------------------------------------|
| 2K prompt / 256 predict     | 504 pp / 29.2 tg | 490 pp / 29.2 tg                   |
| 8K prompt / 256 predict     | 554 pp / 24.3 tg | 556 pp / 24.3 tg                   |
| 30 tok / 512 predict (MTP)  | 32 pp / 26.9 tg  | 31 pp / 26.8 tg                     |

Fixed build matches the upstream-peer ceiling within noise on every
test (previous regression was 178 pp / 3 tg). No RPC errors
(GET_TENSOR FAILED etc.) in any run; test_phase0_scheduler_feasibility.py
passes against a hydra-dev build with this fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ddvnguyen added a commit that referenced this pull request Jul 9, 2026
… the 178/3 tok/s gap) (#39)

* fix: peer in compute-only mode skips bounded thread pool + MSG_PEEK

The new merged server's peer path (started by PR #37) goes through
the same dispatch pipeline as the head: accept() + MSG_PEEK + bounded
thread pool + dispatch. This works for the head (where the pool bounds
public-internet DoS risk) but throttles the peer in a layer-split
COMBINE, where the head sends thousands of compute ops per prefill that
the pool serializes behind the accept+enqueue+worker-pickup pipeline.

This commit adds a peer_mode flag to the settings struct. When set
(only the no-model compute-only server sets it), the accept loop
skips both the bounded thread pool and the MSG_PEEK+dispatch path and
calls ggml_backend_rpc_handle_client directly from a freshly detached
std::thread, matching upstream llama.cpp's rpc-server.cpp behavior
byte-for-byte. This restores the head's compute graph throughput on
the peer path.

Note: live benchmark on the 27B COMBINE at 25/40 / 96K / q8/q8 / MTP
still showed 178 tok/s prefill vs 525 tok/s with the upstream rpc-server
(3x slower). The remaining gap is in other code paths beyond the
dispatch pipeline (likely the peer's ggml-rpc handler overhead and
the head's per-op scheduling). A follow-up investigation is needed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: set TCP_NODELAY/SO_KEEPALIVE on merged-server accept loop

hydra_rpc.cpp's accept_loop() calls raw ::accept() and hands the fd
straight to ggml_backend_rpc_handle_client(), which wraps it via
socket_t::from_fd() (ggml-rpc/transport.cpp) — a bare wrapper that sets
no socket options. Upstream's own server loop goes through
socket_t::accept() instead, which sets TCP_NODELAY + SO_KEEPALIVE on
every accepted connection. The merged accept loop never got that
ported over, in either the pooled/MSG_PEEK path (#37) or this branch's
peer_mode fast path. Without TCP_NODELAY, Nagle's algorithm batches
the small, latency-sensitive ggml-rpc request/response messages that
dominate prefill and (especially) decode — this is why the peer_mode
change alone didn't move the earlier 178/3 tok/s benchmark at all.

Validated on RTX 5060 Ti + RTX 3060 (Dense 27B Q5_K_M, 25/40
layer-split, ctx 65536, q8/q8 KV, MTP draft-n-max 3), bare-metal
head+peer processes talking directly to /v1/completions:

| Test                       | Fixed fork peer | Upstream rpc-server peer (ceiling) |
|-----------------------------|-----------------|-------------------------------------|
| 2K prompt / 256 predict     | 504 pp / 29.2 tg | 490 pp / 29.2 tg                   |
| 8K prompt / 256 predict     | 554 pp / 24.3 tg | 556 pp / 24.3 tg                   |
| 30 tok / 512 predict (MTP)  | 32 pp / 26.9 tg  | 31 pp / 26.8 tg                     |

Fixed build matches the upstream-peer ceiling within noise on every
test (previous regression was 178 pp / 3 tg). No RPC errors
(GET_TENSOR FAILED etc.) in any run; test_phase0_scheduler_feasibility.py
passes against a hydra-dev build with this fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Hydra Engineering <hydra-engineering@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant