Skip to content

Separate speech capability controls from voice settings - #947

Merged
shanselman merged 8 commits into
openclaw:mainfrom
calebeden:calebeden-separate-speech-permissions-controls
Jul 9, 2026
Merged

Separate speech capability controls from voice settings#947
shanselman merged 8 commits into
openclaw:mainfrom
calebeden:calebeden-separate-speech-permissions-controls

Conversation

@calebeden

@calebeden calebeden commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: Speech provider/model controls were split between Permissions and Voice & Audio, making Permissions feel like a duplicate configuration surface.
  • Why it matters: Capabilities should gate agent access, while Voice & Audio should own model, provider, voice, and language setup.
  • What changed: Removed speech provider setup from Permissions, added a conditional Voice & Audio link with tailored setup warnings, made Permissions the owner of STT/TTS capability toggles, and kept Voice & Audio settings interactive while showing capability notices.
  • Chat recovery: Disabled STT now routes to Permissions while missing STT model routes to Voice & Audio. Disabled TTS routes to Permissions, but missing configured TTS provider/voice setup no longer blocks chat read-aloud because the existing Windows speech fallback is preserved.
  • User impact: Users configure speech models/voices in Voice & Audio and enable or disable agent STT/TTS access in Permissions.
  • What did NOT change (scope boundary): No MCP command shape, gateway behavior, persisted credential storage, or node capability registration changes.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor
  • Docs / instructions
  • Tests / validation
  • Security hardening
  • Chore / infra

Scope (select all touched areas)

  • Tray / WinUI UX
  • Windows node capability
  • Local MCP / winnode
  • Gateway / connection / pairing
  • Setup / onboarding
  • Permissions / privacy / security
  • Tests / CI / docs

Linked Issue/PR

  • Closes #
  • Related #
  • Related to a bug or regression

Validation

  • Latest head 424b484b: dotnet test .\tests\OpenClaw.Tray.Tests\OpenClaw.Tray.Tests.csproj --no-restore --filter "SpeechInputContractTests|LocalizationValidationTests|CapabilitiesPageLocalizationCoverageTests" — 26 passed, 0 skipped
  • Latest head 424b484b: $env:OPENCLAW_REPO_ROOT=(Get-Location).Path; .\build.ps1 — all builds succeeded
  • Latest head 424b484b: dotnet test .\tests\OpenClaw.Shared.Tests\OpenClaw.Shared.Tests.csproj --no-restore — 2703 passed, 31 skipped
  • Latest head 424b484b: dotnet test .\tests\OpenClaw.Tray.Tests\OpenClaw.Tray.Tests.csproj --no-restore — 1577 passed, 0 skipped

Real behavior proof

  • Environment tested: Windows ARM64 local worktree
  • Latest PR head / commit tested: 424b484b
  • Exact steps or command run: Validation commands above; UIA check against the active OpenClaw Companion window; manual screenshot proof added by PR author.
  • Evidence after fix: Validation confirms localized Permissions/Voice & Audio contracts, warning copy, absence of duplicated provider controls, both Voice & Audio Permissions links, split chat microphone recovery, TTS disabled recovery, preserved chat TTS fallback for missing configured provider setup, and guarded speech dialogs.
  • Observed result: Permissions owns STT/TTS capability toggles; Voice & Audio owns speech model/provider/voice setup and stays configurable while disabled-capability notices are shown. UIA verified pressing the chat Voice button entered Listening… and produced transcript text when STT was enabled/configured.

Screenshots

  • Permissions: TTS setup warning to download/select a voice. image
  • Permissions: STT setup warning to download a speech model. centered-02-permissions-stt-model-warning
  • Permissions: combined warning when both STT model and TTS voice are missing. image
  • Permissions: plain Voice & Audio link when enabled capabilities are fully configured. centered-04-permissions-plain-link-configured
  • Permissions: no Voice & Audio link or warnings when both STT/TTS capabilities are disabled. centered-05-permissions-both-disabled-no-link
  • Voice & Audio: warning notices that STT/TTS capabilities must be enabled in Permissions. 07-voice-audio-tts-capability-disabled-notice

Review notes / deferred scope

Dual-model adversarial review found no high-consensus correctness/security issues that need to expand this PR further. We intentionally deferred broader speech runtime-readiness work that predates this split:

  • STT service lifecycle/transient readiness: if VoiceServiceInstance is temporarily unavailable or the model is still warming up, that should be handled by a future runtime-readiness UX pass rather than this capability/settings split.
  • Custom/legacy STT model detection: warning behavior for valid-but-unlisted speech model names is out of scope for this UI ownership change.

The TTS fallback concern is not deferred: this PR preserves the existing chat read-aloud fallback to Windows speech when the configured provider/voice setup is incomplete, while Permissions still warns that the selected provider needs setup.

Security Impact (required)

  • New permissions/capabilities? (Yes/No): No
  • Secrets/tokens handling changed? (Yes/No): No
  • New/changed network calls? (Yes/No): No
  • Command/tool execution surface changed? (Yes/No): No
  • Data access scope changed? (Yes/No): No
  • If any Yes, explain risk + mitigation: N/A

Compatibility / Migration

  • Backward compatible? (Yes/No): Yes
  • Config/env changes? (Yes/No): No
  • Migration needed? (Yes/No): No
  • If yes, exact upgrade steps: N/A

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only conversations that still need reviewer or maintainer judgment.

calebeden and others added 5 commits July 8, 2026 13:16
Replace the duplicated Permissions speech settings controls with a single conditional link to Voice & Audio, and update localization coverage for the new surface.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Localize the Permissions voice settings link, remove obsolete removed-control resources, and stop the Permissions STT capability toggle from starting a Whisper model download. The shared card now shows setup/download guidance only when an enabled speech capability is missing required Voice & Audio setup.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove the STT capability toggle from Voice & Audio, add disabled-state notices that link to Permissions, and keep speech setup controls available for preconfiguration while runtime-only test and preview actions respect the capability state.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Select the Permissions voice setup warning based on the missing speech configuration: speech model, TTS voice setup, or both. Keep the shared Voice & Audio link while making the warning text specific to the enabled capabilities that still need setup.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@clawsweeper

clawsweeper Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed July 8, 2026, 11:56 PM ET / 03:56 UTC.

Summary
The branch moves speech provider/model/voice setup from Permissions to Voice & Audio, keeps STT/TTS capability toggles in Permissions, updates chat recovery routing, localization, and tray contract tests.

Reproducibility: yes. from source inspection: current defaults set VoiceTtsEnabled=true and NodeTtsEnabled=false, current chat read-aloud can use the fallback TTS service, and the PR readiness helper returns false until NodeTtsEnabled is true.

Review metrics: 2 noteworthy metrics.

  • Diff surface: 16 files, +758/-596. The PR spans WinUI pages, localization, shared readiness logic, and tray tests, so compatibility and proof matter before merge.
  • Screenshot proof: 6 screenshots inspected. The images show settings UI states but not the changed chat read-aloud fallback or recovery flow.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦪 silver shellfish
Patch quality: 🧂 unranked krab
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Resolve or get maintainer sign-off on whether chat read-aloud should require NodeTtsEnabled.
  • [P2] Add current-head real behavior proof for the chat read-aloud fallback and disabled-TTS recovery path, with private details redacted.

Proof guidance:

  • [P1] Needs stronger real behavior proof before merge: The PR has current-head validation text and screenshots for settings states, but it still needs direct redacted proof of the changed chat read-aloud/TTS recovery behavior such as a recording, UIA transcript, live output, or logs. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P2] Merging would make automatic and manual chat read-aloud unavailable whenever NodeTtsEnabled is false, even for users whose VoiceTtsEnabled read-aloud works on current main through the fallback TTS service.
  • [P2] The posted screenshots prove several settings states, but they do not directly show the changed chat read-aloud fallback or disabled-TTS recovery behavior at the current head.

Maintainer options:

  1. Preserve read-aloud compatibility (recommended)
    Keep existing chat read-aloud working when VoiceTtsEnabled is true, while retaining Permissions as the gate for agent/node TTS access.
  2. Accept capability-gated chat TTS
    Maintainers can intentionally require NodeTtsEnabled for chat read-aloud, but the PR should document the upgrade impact and include current-head proof.
  3. Pause the chat TTS policy change
    Keep the settings ownership split and defer chat read-aloud gating to a smaller follow-up with explicit policy review.

Next step before merge

  • [P1] Human review is needed for the chat read-aloud compatibility policy and proof gate; there is no narrow automated repair until maintainers choose the intended TTS boundary.

Maintainer decision needed

  • Question: Should local chat read-aloud require the node TTS capability toggle, or should NodeTtsEnabled gate only agent/node tts.speak access while chat read-aloud continues to follow VoiceTtsEnabled?
  • Rationale: Current main and defaults allow chat read-aloud independently of the node TTS capability, so this PR changes an upgrade-visible behavior that needs maintainer intent rather than automation judgment.
  • Likely owner: shanselman — The current chat TTS defaults and fallback path trace to their recent main history, making them the best available owner for the compatibility choice.
  • Options:
    • Preserve chat read-aloud compatibility (recommended): Keep chat-local automatic/manual read-aloud controlled by VoiceTtsEnabled while leaving NodeTtsEnabled as the agent/node capability gate, then update tests and proof.
    • Accept the new TTS capability gate: Merge only with maintainer sign-off that users must enable the TTS capability before any chat read-aloud, plus current-head proof and compatibility notes.
    • Defer chat TTS gating: Land the settings ownership split without changing chat read-aloud gating and track stricter chat TTS policy separately.

Security
Cleared: The diff moves existing voice settings UI ownership and chat gating logic without adding dependencies, command execution, credential storage changes, network calls, or new node/MCP capability surface.

Review findings

  • [P1] Preserve chat read-aloud across upgrades — src/OpenClaw.Tray.WinUI/Services/SpeechSetupReadiness.cs:10
Review details

Best possible solution:

Keep the Permissions and Voice & Audio ownership split, but either preserve upgrade-compatible chat-local read-aloud or get explicit maintainer approval for the new capability gate, then add direct current-head proof.

Do we have a high-confidence way to reproduce the issue?

Yes, from source inspection: current defaults set VoiceTtsEnabled=true and NodeTtsEnabled=false, current chat read-aloud can use the fallback TTS service, and the PR readiness helper returns false until NodeTtsEnabled is true.

Is this the best way to solve the issue?

No, not yet: the UI ownership split is maintainable, but the chat read-aloud gate needs an upgrade-compatible path or explicit maintainer approval plus direct current-head proof.

Full review comments:

  • [P1] Preserve chat read-aloud across upgrades — src/OpenClaw.Tray.WinUI/Services/SpeechSetupReadiness.cs:10
    IsChatTtsPlaybackReady now makes chat read-aloud unavailable unless NodeTtsEnabled is true. Current main defaults VoiceTtsEnabled to true and NodeTtsEnabled to false, while manual/automatic chat speech can still use the local fallback TextToSpeechService; upgraded users or fresh installs can lose read-aloud until they discover the Permissions toggle. Preserve chat-local read-aloud compatibility or get explicit maintainer approval for this gate with upgrade proof.
    Confidence: 0.86

Overall correctness: patch is incorrect
Overall confidence: 0.86

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against c3371d66f72d.

Label changes

Label justifications:

  • P2: This is a normal-priority settings UX PR with a concrete compatibility blocker but no merged runtime regression yet.
  • merge-risk: 🚨 compatibility: The PR can change existing/fresh chat read-aloud behavior by requiring NodeTtsEnabled where current main only requires VoiceTtsEnabled and can fall back locally.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR has current-head validation text and screenshots for settings states, but it still needs direct redacted proof of the changed chat read-aloud/TTS recovery behavior such as a recording, UIA transcript, live output, or logs. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • proof: 📸 screenshot: Contributor real behavior proof includes screenshot evidence. The PR has current-head validation text and screenshots for settings states, but it still needs direct redacted proof of the changed chat read-aloud/TTS recovery behavior such as a recording, UIA transcript, live output, or logs.
Evidence reviewed

What I checked:

Likely related people:

  • shanselman: Current-main blame ties the chat TTS guard, settings defaults, and TTS fallback implementation to commit 4166e0f. (role: introduced current TTS/settings behavior; confidence: high; commits: 4166e0fd63f8; files: src/OpenClaw.Tray.WinUI/App.xaml.cs, src/OpenClaw.Tray.WinUI/Services/SettingsManager.cs, src/OpenClaw.Tray.WinUI/Services/TextToSpeech/TextToSpeechService.cs)
  • RBrid: Merged pull request Add Windows node text-to-speech #253 added the Windows node TTS capability, settings UI/persistence, MCP description, and validation. (role: original TTS feature contributor; confidence: medium; commits: e0c40985a718; files: src/OpenClaw.Shared/Capabilities/TtsCapability.cs, src/OpenClaw.Tray.WinUI/Services/TextToSpeech/TextToSpeechService.cs, src/OpenClaw.Tray.WinUI/Pages/VoiceSettingsPage.xaml.cs)
  • ArtLupo: Merged pull request fix: preserve full chat text for Tray TTS #900 recently preserved full chat text for Tray TTS and touched the same notification speech path. (role: recent chat TTS contributor; confidence: medium; commits: 43d40ed60acc; files: src/OpenClaw.Tray.WinUI/App.xaml.cs)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (6 earlier review cycles)
  • reviewed 2026-07-09T00:43:01.682Z sha 79bf968 :: needs real behavior proof before merge. :: [P2] Route disabled STT prompts to Permissions
  • reviewed 2026-07-09T01:42:55.394Z sha 79bf968 :: needs changes before merge. :: [P2] Route disabled STT prompts to Permissions
  • reviewed 2026-07-09T01:56:23.513Z sha 4c0d9e6 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-09T03:28:41.580Z sha 657586c :: found issues before merge. :: [P1] Preserve chat TTS fallback on upgrade
  • reviewed 2026-07-09T03:34:49.969Z sha 657586c :: found issues before merge. :: [P1] Preserve chat TTS fallback on upgrade
  • reviewed 2026-07-09T03:49:48.499Z sha 424b484 :: needs real behavior proof before merge. :: [P1] Preserve chat read-aloud across upgrades

@clawsweeper clawsweeper Bot added proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 9, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jul 9, 2026
Route chat read-aloud and speaker recovery through the same TTS readiness checks used by the Permissions voice setup warning. Keep blocked unmute visually muted before showing recovery dialogs, gate overlapping speech dialogs, and apply the same readiness predicate to automatic response speech.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 9, 2026
Keep chat TTS readiness gated on the Permissions capability only so existing Windows speech fallback can handle missing configured provider setup. Update Permissions warning copy to describe provider setup without implying all read aloud is blocked.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@clawsweeper clawsweeper Bot added status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jul 9, 2026

@shanselman shanselman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and live-tested with >90% confidence. CI is green and local build passed. Manual proof: disabled STT routes Chat voice recovery to Permissions; enabled STT with missing model routes to Voice & Audio; Voice & Audio setup controls remain available while capability gating is owned by Permissions.

@shanselman
shanselman merged commit c35a0ed into openclaw:main Jul 9, 2026
14 checks passed
@calebeden
calebeden deleted the calebeden-separate-speech-permissions-controls branch July 9, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants