Replace LiveKit with WebSocket Opus audio relay

[tlongwell-block]

Summary

Replace the external LiveKit WebRTC SFU with a WebSocket-based Opus audio relay embedded in sprout-relay. All codec work happens in Rust — the WebView becomes a dumb mic capture + UI layer. No new services, no new ports, no new network dependencies.

34 files changed, +1464 / −1742 (net −278 lines)

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                          sprout-relay                               │
│  ┌──────────────────┐    ┌────────────────────────────────────┐     │
│  │  Nostr WS (/)    │    │  Audio WS (/huddle/{id}/audio)     │     │
│  │  kind:9 text     │    │  binary: opaque Opus frames        │     │
│  │  kind:481xx sig  │    │  text:   control JSON              │     │
│  └──────────────────┘    └──────────┬─────────────────────────┘     │
│                                     │                               │
│                          ┌──────────▼──────────┐                    │
│                          │   AudioRoomManager   │                   │
│                          │  DashMap<Uuid,Room>  │                   │
│                          │  fan-out: try_send   │                   │
│                          └──────────────────────┘                   │
└─────────────────────────────────────────────────────────────────────┘

What changed

Relay — new audio/ module

• room.rs — Room, AudioPeer, AdmissionGuard (mutex-synchronized peer admission + ended flag + index recycling), dual-channel fan-out (audio_tx + ctrl_tx), MAX_PEERS_PER_ROOM (25) defense-in-depth cap, remove_peer_and_check_ended for atomic last-peer detection
• handler.rs — WS handler with NIP-42 keypair auth, dual ctrl/data send channels with priority drain, heartbeat, lifecycle event emission (48101/48102), 8 KB text frame size limit, relay-side auto-end (archive + kind:48103 when last human disconnects)
• mod.rs — re-exports

Relay — wiring & deletions

• New route: GET /huddle/{channel_id}/audio (WebSocket upgrade)
• AppState: added audio_rooms: Arc<AudioRoomManager>, removed huddle_service and livekit_url
• Deleted: sprout-huddle crate, api/huddles.rs, api/webhooks.rs, LiveKit env var handling

Desktop Rust backend

• relay_api.rs: connect_audio_relay() performs synchronous WS handshake (connect + NIP-42 auth + wait for joined) with 5s timeout before returning — auth failures propagate, no silent degradation. Background pipeline does Opus encode/decode + rodio playback. Emits huddle-audio-disconnected Tauri event on unexpected WS drop (suppressed on intentional teardown via CancellationToken guard).
• state.rs: removed livekit_* fields, added audio_ws_cancel + audio_relay_pcm_tx
• mod.rs: simplified commands — no token fetch, no 48101/48102 emission (relay owns these). Audio relay failure is fatal (not degraded mode). Both start_huddle and join_huddle rollback state on post_connect_setup failure.
• events.rs: removed livekit_room param from build_huddle_started, deleted participant join/leave builders
• pipeline.rs: audio relay connection failure propagates to caller

Desktop frontend

• Deleted livekit.ts and livekit-client npm dependency
• HuddleContext.tsx: getUserMedia directly (48 kHz enforced), no LiveKit connection object, mic stream cleanup on failure, stable disconnectMedia callback (reads track from ref, not state) to prevent React effect cascade from self-ending huddles during startup. Listens for huddle-audio-disconnected event to auto-leave on unexpected relay WS drop.
• HuddleIndicator.tsx: extract participant pubkey from p-tag (relay-signed 48101/48102 events)
• HuddleBar.tsx: removed livekit_room, isReconnecting
• worklet.js: buffer 4800→960 samples (100ms→20ms) for Opus frame alignment, 48 kHz assumption documented

Key design decisions

• NIP-42 keypair auth — reuses existing challenge/response pattern, no tokens
• Dual per-peer channels — audio frames and control messages (joined/left) use separate queues so control is never starved by audio backpressure
• Peer index recycling — IndexPool freelist prevents the 255-join exhaustion bug
• Client-side active speakers — peer_index→pubkey map seeded from initial joined message, maintained via control messages, emitted as Tauri event every 500ms
• Synchronous handshake — connect_audio_relay() completes WS connect + auth before returning; failures are fatal
• Stable React callbacks — disconnectMedia uses a ref for the audio track instead of state, preventing the unmount-cleanup effect from re-firing mid-startup
• Auto-disconnect on relay drop — pipeline emits Tauri event on unexpected WS close; frontend auto-leaves. CancellationToken guard suppresses the event during intentional teardown (teardown_huddle cancels the token before dropping the PCM sender to prevent a race).
• Relay-side auto-end — when the last audio WS peer disconnects, the relay archives the ephemeral channel and emits kind:48103 (huddle ended). Bots never connect to the audio WS, so every peer is a human — an empty room means zero humans remain. Concurrency is handled by AdmissionGuard, a shared mutex that makes add_peer and remove_peer_and_check_ended mutually exclusive (10 codex review passes to get the synchronization right).

Testing

• All 1100+ existing workspace unit tests pass
• Crossfired across 25 review passes total:
  • Initial development: codex CLI × 6, opus model × 1 (14 bugs found and fixed)
  • Post-PR crossfire: codex o3, gpt-5.4, claude-4-opus (3 independent reviewers, 15 issues found)
  • Fix review: codex gpt-5.4 × 4 passes (7→4→6→9/10), each pass catching a real regression in the fixes themselves
  • Auto-end feature: codex gpt-5.4 × 10 passes (6→4→5→3→3→4→3→4→7→9/10), progressively closing concurrency races
• Live-tested: huddle start, audio relay connect, peer join/leave, teardown, auto-end on hard exit

Bugs found and fixed during development

1. Wire protocol mismatch (client double-prefixed peer_index)
2. Single WS send queue starving heartbeat pings → dual ctrl/data channels
3. Peer index wraps at 255 → IndexPool with recycling freelist
4. Initial joined peers not seeded into active speaker map
5. Task leak (surviving task not aborted after select)
6. Per-packet rodio Player creation → persistent player
7. text.contains("leave") hack → proper JSON parse
8. HuddleIndicator using event.pubkey for relay-signed 48101/48102 → p-tag extraction
9. Mic stream leak on failed start/join → try/catch with stream cleanup
10. Room per-peer queue shared for audio+control → split audio_tx/ctrl_tx
11. connect_audio_relay returned before handshake → synchronous connect+auth
12. Axum 0.8 route param syntax (:channel_id → {channel_id})
13. post_connect_setup failure swallowed as non-fatal → audio relay failure is now fatal
14. React effect cascade: disconnectMedia dep on localAudioTrack state caused unmount-cleanup to re-fire mid-startup → stable ref-based callback with [] deps

Post-crossfire hardening (3 reviewers × 4 fix-review passes)

15. start_huddle/join_huddle committed state before post_connect_setup with no rollback → added rollback (archive ephemeral channel for creators, reset to Idle for joiners)
16. No auto-disconnect on unexpected relay WS drop → pipeline emits huddle-audio-disconnected Tauri event; frontend listens and auto-leaves
17. Control channel capacity too small (4 slots) for state-bearing joined/left messages → increased to 32 with warn! on drop
18. Client WS handshake had no timeout for challenge/joined → 5s timeout added
19. Opus encoder errors silently swallowed (unwrap_or(0)) → logged via eprintln
20. Auth event serialization used unwrap_or_default() → hard error
21. No text frame size limit on auth/control messages → 8 KB cap added
22. Tag::parse used .expect() in production path → match/warn!/return
23. MAX_PEERS_PER_ROOM defense-in-depth cap (25) added
24. Heartbeat missed-pong counter off-by-one readability → fetch_add + 1 pattern
25. getUserMedia didn't enforce 48 kHz → sampleRate: 48000 added
26. Dead stream parameter in cleanupFailedStart → removed
27. Bytes::copy_from_slice in recv_loop → Bytes::from (zero-copy)
28. TODO(security) comment on ensure_membership strengthened (unverified parent_channel_id)
29. Disconnect event fired on intentional teardown too → is_cancelled() guard
30. leaveHuddleRef2 referenced before leaveHuddle defined (TDZ) → reuse existing ref
31. teardown_huddle dropped PCM sender before cancelling token (race) → cancel first, then drop

Auto-end feature (10 codex review passes)

32. Hard exit (Ctrl+C, crash) left huddle "active" for up to 1 hour → relay-side auto-end archives ephemeral channel + emits 48103 when last audio peer disconnects
33. ensure_membership checked is_member before archived_at → reordered so archived channels are rejected first
34. Post-get_or_create DB check was missing → added fail-closed archived_at re-check to close cross-boundary race
35. remove_peer and mark_ended were separate lock acquisitions → combined into atomic remove_peer_and_check_ended under one AdmissionGuard mutex
36. add_peer ended check was outside the lock → moved inside, held across ended-check + alloc + insert
37. Concurrent disconnects could both win auto-end → !g.ended guard ensures only first empty+!ended transition wins
38. Archive failure left room in ended state → clear_ended() rollback, TTL reaper handles cleanup

What is NOT built (per plan §7)

Video, server-side recording/mixing, TURN/NAT traversal, jitter buffer, per-participant volume, WS reconnect/resume, FEC — all deferred per spec.