Forward assistant output to realtime through handoffs - #27127
Forward assistant output to realtime through handoffs#27127guinness-oai wants to merge 8 commits into
Conversation
6211ecf to
3c85d78
Compare
be873bc to
d63806e
Compare
d63806e to
1775690
Compare
1775690 to
a678a86
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 002f9fe228
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if output_text.trim().is_empty() { | ||
| return; | ||
| } | ||
| self.conversation.send_assistant_output(output_text).await |
There was a problem hiding this comment.
When a realtime conversation is open, this forwards the full AgentMessage into the realtime model context with no truncation. Long assistant outputs can exceed context/API limits; the context guidance requires hard caps on injected items and flags >1k-token items for review. guidance
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 37d2758. Every assistant item is now capped at 1,000 approximate tokens in the shared RealtimeHandoffState::assistant_output path before either active or handoff-less sideband injection. The realtime truncator also accounts for its marker when enforcing the hard cap; assistant_output_is_capped_for_realtime_context covers the bound and head/tail preservation.
| assistant_output | ||
| .handoff_id | ||
| .unwrap_or_else(|| DEFAULT_HANDOFF_ID.to_string()), |
There was a problem hiding this comment.
If a V2 assistant output is queued and the handoff is cleared or replaced before the input task drains it (for example an abort immediately after an ItemCompleted), this still sends the captured handoff_id. The removed V2 path re-read active_handoff and dropped stale progress, so stale background output can now be appended to a cancelled or different realtime handoff.
Useful? React with 👍 / 👎.
…minal-output # Conflicts: # codex-rs/core/src/session/turn.rs
|
Closing this pull request because it has had no updates for more than 14 days. If you plan to continue working on it, feel free to reopen or open a new PR. |
Why
Realtime voice should feel like one coherent assistant even though the frontend model and Codex orchestrator are separate agents. That requires the frontend model to hear every user-facing Codex message, including preambles and finals, whether the turn began from a realtime handoff, typed input, or a worker report.
This change gives those messages one outbound contract: each completed Codex assistant message is forwarded exactly once through
conversation.handoff.append.Stack
Builds on #27116, which removes raw user-input mirroring.
What changed
conversation.handoff.appendfor both V1 and V2codexfor turns without an active handoffdeveloperitem andresponse.createpathTest plan
just test -p codex-core assistant_output_is_capped_for_realtime_contextjust test -p codex-core conversation_sends_each_assistant_message_oncejust test -p codex-core conversation_clears_handoff_after_turn_abortjust test -p codex-app-server webrtc_terminal_output_without_handoff_reaches_realtimejust test -p codex-app-server webrtc_v1_handoff_request_delegates_and_appends_resultjust test -p codex-app-server webrtc_v2_background_agent_tool_call_appends_preamble_and_final