feat: local Concierge voice agent on the Home screen#910
Closed
tlongwell-block wants to merge 6 commits into
Closed
feat: local Concierge voice agent on the Home screen#910tlongwell-block wants to merge 6 commits into
tlongwell-block wants to merge 6 commits into
Conversation
3096f39 to
da136f6
Compare
tlongwell-block
pushed a commit
that referenced
this pull request
Jun 9, 2026
The Home launcher was a fixed bottom-right pill that sat on top of the inbox reply composer's send arrow. Move it into the composer toolbar's extraActions slot, immediately left of the send button (mic-key placement), as a compact orb icon with the selected agent's name in a tooltip. The sidebar Concierge entry remains the persistent entry point. Reported-by: Tyler (from Max's PR #910 screenshots) Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co>
Wire the existing machinery — huddle voice (Parakeet STT + TTS + VAD), mesh-llm inference (:9337/v1 via the relay-mesh agent preset), and managed agents — into a local Concierge agent reachable from Home. - ensureConciergeSession(): find/create the Concierge managed agent on the mesh-llm preset, open the persistent DM (the memory spine), and ensure the agent is running. Derived ownership, no new persistence. - ConciergeLiveScreen: live DM transcript, voice orb that attaches a huddle to the DM (STT in, agent replies spoken via TTS), and typed input fallback. - Dispatch confirm cards: agents propose actions in a fenced dispatch JSON block; fail-closed parser + per-identity settled state; Approve posts the instruction to the target channel. The card is the safety boundary — no auto-send. - Home entry: floating Concierge launcher orb; sidebar nav + /concierge route (demo screen kept behind ?demo= for the screenshot loop). Covered by unit tests (session bootstrap, dispatch parsing, approve flow) and 4 Playwright specs against the e2e mock bridge. Signed-off-by: Eva <eva@sprout.local>
Group huddle STT posts transcripts to the ephemeral huddle channel,
which is correct for group calls but wrong for the Concierge: spoken
turns vanished from the DM transcript and never entered the memory
spine, while typed turns worked.
- start_huddle gains an opt-in transcripts_to_parent flag (default
false; group huddles byte-for-byte unchanged). A pure
resolve_transcript_channel() picks the STT destination; flag without
a parent channel is an error. HuddleState carries the flag.
- startHuddle(parent, members, { transcriptsToParent }) passes it
through; when set, the TTS subscription follows the conversation to
the parent DM (ttsChannelOverride) so agent replies are spoken.
- ConciergeLiveScreen passes transcriptsToParent: true — the DM is
now the single conversation spine for typed and spoken turns.
- New e2e: voice attach asserts start_huddle receives the DM as parent
AND transcriptsToParent: true; mock bridge records the payload.
Signed-off-by: Eva <eva@sprout.local>
Replace name-only discovery ("any agent literally named Concierge")
with an explicit per-user selection, per the agreed design: users can
make any agent their Home Concierge, named anything they like, with a
sensible default when none is selected.
- conciergeSelection.ts: versioned localStorage preference keyed
sprout-concierge.v1:<selfPubkey> -> { agentPubkey, updatedAt },
parse-validated, pubkey-keyed (rename-proof). Unit tested.
- ensureConciergeSession(selfPubkey) is selection-first: pure
resolveConciergeAgent() finds the chosen agent by pubkey; stale
selection (agent deleted) clears the preference, surfaces a toast,
and falls back; no selection provisions the default Concierge once
and records it as the selection. findConciergeAgent name-matching
is deleted; tests replaced with pubkey-resolution tests.
- Picker: "Set as Home Concierge" in the agent-row dropdown (any
agent, any backend); the Home launcher pill shows the selected
agent's name live (same-tab custom event + cross-tab storage event).
- Biome cleanup: drop both !important reduced-motion rules in favor of
properly-ordered selectors; fix string concat in dispatch test.
Signed-off-by: Eva <eva@sprout.local>
The Home launcher was a fixed bottom-right pill that sat on top of the inbox reply composer's send arrow. Move it into the composer toolbar's extraActions slot, immediately left of the send button (mic-key placement), as a compact orb icon with the selected agent's name in a tooltip. The sidebar Concierge entry remains the persistent entry point. Reported-by: Tyler (from Max's PR #910 screenshots) Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co>
Main's Rust-owned mesh coordinator (#879) deleted startRelayMeshClientForTarget; the Concierge bootstrap now calls meshPrepareRelayMeshClient like CreateAgentDialog does. Branch rebased onto main @ 4215930. Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co>
c15c965 to
788ba88
Compare
* origin/main: Fix post-compact handoff context for OpenAI providers (#931) chore(release): release version 0.3.15 (#936) fix: persona is source of truth at spawn + thread-depth conventions (#930) fix: skip avatar reconciliation for legacy agent records (#933) feat(desktop): add nest commit identity guidance with human sign-off (#929) feat: provider/model selection for personas and runtime-aware env injection (#794) fix: reconcile agent profile on startup when relay publish was missed (#921) Revamp first-run onboarding (#924) Update setup loading screen (#926) fix(dm): keep hidden DMs hidden across refetch via relay-signed visibility snapshot (NIP-DV) (#857) Maximize desktop window on launch (#925) feat: preview features (experiments settings UI) (#888) fix(updater): send no-cache header on update check to avoid stale manifest (#922) fix(desktop): refresh channel state after unarchive (#923) Add channel visibility & ephemeral TTL controls to manage sidebar (#911) ci(release): add Intel macOS (x86_64) DMG as a release target (#748) Signed-off-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co> # Conflicts: # desktop/src/features/sidebar/ui/AppSidebar.tsx
Collaborator
Author
|
Closing per direction change (Tyler, in-channel): rather than shipping Concierge as a destination, we're folding the learnings into making huddles-with-an-agent faster and easier to start. The substrate pieces here (transcripts_to_parent, DM spine, dispatch approval cards) will be re-landed incrementally on the huddle-first path as they're needed. First follow-up: voice-mode prompt improvements for sentence-at-a-time TTS streaming. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A local Concierge voice agent reachable from the Home screen — built entirely on existing machinery, no new serving stack:
:9337/v1) via the relay-mesh agent preset — config-only brain wiringHow it works
ensureConciergeSession()— find/create the Concierge managed agent on the mesh-llm preset (re-uses the exact relay-mesh path the Create-Agent flow uses), idempotently open the DM, ensure the agent is running. Ownership is derived from agent name + DM membership;start_managed_agent's existing mesh preflight makes restart-after-reboot self-heal.ConciergeLiveScreen— live DM transcript (useChannelMessagesQuery+ subscription), a voice orb that starts/ends a huddle on the DM (mic → Parakeet STT → agent → spoken reply via TTS), and typed input as fallback./conciergeroute. The static demo screen is kept behind?demo=truefor the screenshot/aesthetics loop.Out of scope (deliberately)
Sub-second pipeline work (Moonshine/Kokoro/speculative overlap), prompt-length discipline, and persona packaging — those are the latency spike, tracked separately. This PR ships the existing-machinery version; latency is whatever huddle + mesh-llm gives today.
Verification
pnpm typecheck✓,pnpm check(biome + file sizes) ✓conciergeSession.test.mjs), dispatch parsing (dispatchIntent.test.mjs), and the approve flow (approveDispatch.test.mjs)