fix(providers): normalize GitHub Copilot streamed Responses wire - #996
fix(providers): normalize GitHub Copilot streamed Responses wire#996Simon-Opopeee wants to merge 1 commit into
Conversation
GitHub Copilot's /responses stream is incompatible with Responses clients for the vscode-chat integration: function-call argument deltas are obfuscated (ciphertext + obfuscation field), reasoning items ship as GitHub-encrypted encrypted_content with no plaintext summary, and the response id plus every item id are re-encrypted per event so the terminal completed payload disagrees with the streamed events. Agentic turns on Responses-only Copilot models (gpt-5.6-luna/sol/terra, gpt-5.4, ...) stall in Codex clients: reasoning never finishes and tool calls are unreadable. Add a provider-scoped SSE rewrite wired into the passthrough payloadRewrites composition for github-copilot: drop ciphertext deltas and re-emit the plaintext arguments as one delta before done, replace encrypted reasoning items with a canonical empty plaintext summary (in output_item events and the terminal completed payload), pin response/item ids to first-seen values, and strip the obfuscation metadata from every event. Clean events pass through byte-identical.
📝 WalkthroughWalkthroughChangesGitHub Copilot SSE compatibility
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related issues
Possibly related PRs
Sequence Diagram(s)sequenceDiagram
participant GitHubCopilot
participant SsePayloadRewrite
participant ResponsesClient
GitHubCopilot->>SsePayloadRewrite: Send obfuscated SSE events
SsePayloadRewrite->>SsePayloadRewrite: Remove metadata and ciphertext
SsePayloadRewrite->>SsePayloadRewrite: Sanitize reasoning and stabilize IDs
SsePayloadRewrite->>ResponsesClient: Emit rewritten events and plaintext arguments
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f69e2be6d7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| route.providerName === "github-copilot" | ||
| ? createGithubCopilotObfuscationRewrite() | ||
| : undefined, |
There was a problem hiding this comment.
In src/server/responses/core.ts, adding the Copilot transform only to payloadRewrites leaves needsClientRewrite false when it is the sole rewrite. Consequently, macOS with streamMode: "eager-relay" and Windows when eager relay is selected enter the branch at line 1992, but rewritePayload is omitted because win32EagerRewrite is also false, so obfuscated Copilot events reach the client unchanged and tool turns still stall. Derive the rewrite requirement from payloadRewrites.length or explicitly include Copilot so every eager path either applies the transform inline or remains on the rewriting tee path.
AGENTS.md reference: src/AGENTS.md:L17-L19
Useful? React with 👍 / 👎.
| }; | ||
| // The relay writes `data: <payload>` then the block delimiter, so this | ||
| // payload becomes two consecutive valid SSE events (delta, then done). | ||
| return `${JSON.stringify(deltaEvent)}\n\ndata: ${JSON.stringify(parsed)}`; |
There was a problem hiding this comment.
When Copilot sends standard named SSE blocks such as event: response.function_call_arguments.done, embedding \n\ndata: inside the replacement payload leaves that original event: field attached to the synthetic delta, while the following done payload becomes an unnamed message event. Clients dispatching by SSE event name can therefore parse the delta as a done event and ignore the actual done event. Emit two complete blocks with matching event: fields, or make the relay rewrite framing-aware, rather than inserting an event separator inside a data payload.
AGENTS.md reference: src/AGENTS.md:L17-L19
Useful? React with 👍 / 👎.
| if (obfuscatedCalls.delete(parsed.output_index)) { | ||
| const deltaEvent = { | ||
| type: "response.function_call_arguments.delta", | ||
| item_id: parsed.item_id, |
There was a problem hiding this comment.
When Copilot re-encrypts the function-call item's ID between response.output_item.added and response.function_call_arguments.done, this synthetic delta copies the latter cipher ID even though the item lifecycle and completed snapshot have been pinned to the first ID. Consumers that correlate argument events by item_id cannot attach the plaintext arguments to the function call; for example, src/chat/outbound.ts records the added ID at line 379 and ignores a done event whose ID is absent from that map at lines 397-400. Rewrite the original deltas, synthetic delta, and done event to itemIds.get(output_index) so the entire function-call lifecycle uses one ID.
AGENTS.md reference: src/AGENTS.md:L17-L19
Useful? React with 👍 / 👎.
| export function createGithubCopilotObfuscationRewrite(): SsePayloadRewrite { | ||
| // GitHub re-encrypts `item_id` per event (delta and done carry different ids for the | ||
| // same logical call), so calls are tracked by the stable `output_index` instead. | ||
| const obfuscatedCalls = new Set<number>(); |
There was a problem hiding this comment.
If a buggy or hostile Copilot stream emits obfuscated deltas with many distinct output_index values and never sends the matching done events, obfuscatedCalls grows for the entire request with no entry cap or translator-budget charge. The relay releases its per-frame budget after every event, so a long-lived upstream can grow this set without hitting the existing 32 MiB translation limit. Charge this retained state to translatorBudget, impose a bounded index count, or fail the stream once the cap is exceeded.
Useful? React with 👍 / 👎.
| } | ||
| if (type === "response.completed" && isPayloadRecord(parsed.response)) { | ||
| if (typeof parsed.response.id === "string" && responseId !== null && parsed.response.id !== responseId) { | ||
| parsed.response.id = responseId; |
There was a problem hiding this comment.
When Copilot gives response.created and response.completed different encrypted IDs, this line changes the completed ID delivered to the client, but the inspection branch still passes the raw completed response to rememberPassthroughResponse in src/server/responses/core.ts, and rememberResponseState keys its cache by that raw terminal ID. The next request therefore supplies the rewritten first-seen ID as previous_response_id, misses the local replay cache, and forwards the tool-result turn without its prior conversation history. Store an alias under the client-visible ID or normalize the snapshot before recording it while retaining the raw output items for upstream replay.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/server/responses/core.ts`:
- Around line 1979-1981: Update the response routing logic around
needsClientRewrite to include a dedicated boolean for github-copilot traffic,
and reuse that boolean when conditionally creating
createGithubCopilotObfuscationRewrite(). Add focused regression tests covering
Copilot-only traffic on Windows relay selection and Darwin streamMode
"eager-relay", verifying the client rewrite path and rewritePayload behavior
respectively.
In `@src/server/sse-payload-rewrite.ts`:
- Around line 118-136: Normalize top-level parsed.item_id by output_index in the
argument-event handling before the early delta return, while tracking whether
the event was actually obfuscated separately so clean deltas retain their
payload. In src/server/sse-payload-rewrite.ts:118-136, use the canonical
normalized ID when constructing deltaEvent. In
tests/sse-payload-rewrite.test.ts:43-80, add a preceding
response.output_item.added event and verify all related argument delta and done
events use its first-seen item ID.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 82e0af42-c419-4150-8ce6-be9c03baf1d3
📒 Files selected for processing (3)
src/server/responses/core.tssrc/server/sse-payload-rewrite.tstests/sse-payload-rewrite.test.ts
| route.providerName === "github-copilot" | ||
| ? createGithubCopilotObfuscationRewrite() | ||
| : undefined, |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Include the Copilot rewrite in needsClientRewrite.
For Copilot-only traffic, payloadRewrites contains the new rewrite while needsClientRewrite remains false. On Windows, this skips win32EagerRewrite and sends rewrite traffic through the tee plus JS-pull path that the existing code marks unsafe. On Darwin with streamMode: "eager-relay", this selects the no-rewrite eager path and omits rewritePayload, so clients receive the incompatible Copilot events unchanged.
Add a dedicated Copilot boolean to needsClientRewrite, and reuse it when creating the rewrite. Add regression coverage for Copilot-only traffic on both affected relay selections.
Proposed fix
const repairConfig = route.provider.responsesItemIdRepair;
- const needsClientRewrite = imageGenCallAliases.size > 0 || hasResponsesItemIdRepair(repairConfig);
+ const needsGithubCopilotRewrite = route.providerName === "github-copilot";
+ const needsClientRewrite = imageGenCallAliases.size > 0
+ || hasResponsesItemIdRepair(repairConfig)
+ || needsGithubCopilotRewrite;
const payloadRewrites = [
createImageGenCallRestoreRewrite(imageGenCallAliases),
hasResponsesItemIdRepair(repairConfig)
? createResponsesItemIdPayloadRewrite(repairConfig!, translatorBudget)
: undefined,
- route.providerName === "github-copilot"
+ needsGithubCopilotRewrite
? createGithubCopilotObfuscationRewrite()
: undefined,
].filter((rewrite): rewrite is NonNullable<typeof rewrite> => rewrite !== undefined);As per path instructions, “A behavior change in src/ should come with a focused regression test near existing tests for that subsystem.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/server/responses/core.ts` around lines 1979 - 1981, Update the response
routing logic around needsClientRewrite to include a dedicated boolean for
github-copilot traffic, and reuse that boolean when conditionally creating
createGithubCopilotObfuscationRewrite(). Add focused regression tests covering
Copilot-only traffic on Windows relay selection and Darwin streamMode
"eager-relay", verifying the client rewrite path and rewritePayload behavior
respectively.
Source: Path instructions
| if (type === "response.function_call_arguments.delta" && changed) { | ||
| if (typeof parsed.output_index === "number") obfuscatedCalls.add(parsed.output_index); | ||
| // Ciphertext chunk: emit an empty delta so the client sees a valid event | ||
| // without any unusable bytes; the plaintext arrives with `.done`. | ||
| parsed.delta = ""; | ||
| return JSON.stringify(parsed); | ||
| } | ||
| if (type === "response.function_call_arguments.done" && typeof parsed.output_index === "number") { | ||
| if (obfuscatedCalls.delete(parsed.output_index)) { | ||
| const deltaEvent = { | ||
| type: "response.function_call_arguments.delta", | ||
| item_id: parsed.item_id, | ||
| output_index: parsed.output_index, | ||
| sequence_number: parsed.sequence_number, | ||
| delta: typeof parsed.arguments === "string" ? parsed.arguments : "", | ||
| }; | ||
| // The relay writes `data: <payload>` then the block delimiter, so this | ||
| // payload becomes two consecutive valid SSE events (delta, then done). | ||
| return `${JSON.stringify(deltaEvent)}\n\ndata: ${JSON.stringify(parsed)}`; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Verify whether optional generic item-ID repair currently covers GitHub Copilot.
rg -n -C 4 'responsesItemIdRepair|github-copilot' src
rg -n -C 4 'cipher-delta-id|cipher-done-id|item_id' tests/sse-payload-rewrite.test.tsRepository: lidge-jun/opencodex
Length of output: 16633
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- outline ---'
ast-grep outline src/server/sse-payload-rewrite.ts
echo '--- implementation ---'
sed -n '1,190p' src/server/sse-payload-rewrite.ts
echo '--- relevant tests ---'
sed -n '1,120p' tests/sse-payload-rewrite.test.ts
echo '--- rewrite composition and config ---'
sed -n '1960,1985p' src/server/responses/core.ts
rg -n -C 8 'function hasResponsesItemIdRepair|createResponsesItemIdPayloadRewrite|createGithubCopilotObfuscationRewrite|itemIds' src/serverRepository: lidge-jun/opencodex
Length of output: 27880
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- relay framing ---'
sed -n '180,280p' src/server/sse-payload-rewrite.ts
echo '--- remaining rewrite tests ---'
sed -n '120,260p' tests/sse-payload-rewrite.test.ts
echo '--- item-id repair state model ---'
sed -n '1,220p' src/server/responses-item-id-repair.tsRepository: lidge-jun/opencodex
Length of output: 17836
🌐 Web query:
OpenAI Responses API streaming response.function_call_arguments.delta response.function_call_arguments.done item_id output_index
💡 Result:
In the OpenAI Responses and Realtime APIs, streaming function call arguments is handled via specific server-sent events that allow developers to build arguments incrementally [1][2]. The two primary events for this process are: 1. response.function_call_arguments.delta: This event is emitted incrementally as the model generates the JSON arguments for a function [1][3]. It contains the delta string that should be appended to your local buffer for that specific function call [1][3]. Key fields include item_id (identifying the function call), output_index, and the delta string itself [1][3][4]. 2. response.function_call_arguments.done: This event is emitted once the model has finished generating all arguments for a function call [1][3]. It provides the finalized, complete argument string [1]. It also contains the item_id and output_index, allowing you to map the finished arguments back to the specific function call initiated [1][3]. Key technical details for implementation: - Buffering: You should maintain a buffer (usually a string or dictionary) for each active function call, keyed by its item_id or call_id [5][3]. - Incremental Updates: When a response.function_call_arguments.delta event arrives, append the delta field to the corresponding buffer [1][5][3]. Do not parse the JSON until the process is complete [3]. - Completion: Upon receiving response.function_call_arguments.done, you can safely parse the fully assembled JSON string to invoke your function [1][3]. - Item Mapping: Both events provide the item_id and output_index, which are critical for tracking multiple simultaneous function calls or managing state in complex responses [1][3][4]. Note that in some earlier versions of the SDK, developers reported issues where the function name field was missing in the done event [6]; ensuring you are using the latest stable version of the OpenAI SDK is recommended to avoid such issues [6].
Citations:
- 1: https://developers.openai.com/api/docs/guides/function-calling
- 2: https://developers.openai.com/api/docs/guides/migrate-to-responses
- 3: https://community.openai.com/t/responses-api-streaming-the-simple-guide-to-events/1363122
- 4: https://fossies.org/linux/openai-python/src/openai/types/realtime/response_function_call_arguments_delta_event.py
- 5: https://github.com/openai/openai-agents-python/blob/3a11cf52/examples/basic/stream_function_call_args.py
- 6: Responses API streaming response.function_call_arguments.done event missing function name openai/openai-python#2723
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
import json
events = [
{
"type": "response.output_item.added",
"output_index": 0,
"item": {"type": "function_call", "id": "first-item-id"},
},
{
"type": "response.function_call_arguments.delta",
"item_id": "cipher-delta-id",
"output_index": 0,
"sequence_number": 3,
"delta": "ciphertext-chunk-1",
"obfuscation": "abc123",
},
{
"type": "response.function_call_arguments.done",
"item_id": "cipher-done-id",
"output_index": 0,
"sequence_number": 5,
"arguments": '{"a":2,"b":2}',
},
]
# Faithful behavioral probe of the inspected implementation's relevant branches.
obfuscated = set()
item_ids = {}
out = []
for event in events:
parsed = json.loads(json.dumps(event))
changed = False
if isinstance(parsed.get("obfuscation"), str):
del parsed["obfuscation"]
changed = True
typ = parsed.get("type")
if typ == "response.function_call_arguments.delta" and changed:
obfuscated.add(parsed["output_index"])
parsed["delta"] = ""
out.append(parsed)
continue
if typ == "response.function_call_arguments.done" and isinstance(parsed.get("output_index"), (int, float)):
if parsed["output_index"] in obfuscated:
obfuscated.remove(parsed["output_index"])
out.append({
"type": "response.function_call_arguments.delta",
"item_id": parsed.get("item_id"),
"output_index": parsed["output_index"],
"sequence_number": parsed.get("sequence_number"),
"delta": parsed.get("arguments", ""),
})
if typ in ("response.output_item.added", "response.output_item.done"):
item = parsed.get("item")
index = parsed.get("output_index")
if isinstance(index, (int, float)) and isinstance(item, dict) and isinstance(item.get("id"), str):
item_ids.setdefault(index, item["id"])
out.append(parsed)
print("remembered nested item id:", item_ids[0])
print("argument event item ids:", [e.get("item_id") for e in out if "item_id" in e])
print("synthetic delta item id:", out[2]["item_id"])
assert item_ids[0] == "first-item-id"
assert [e.get("item_id") for e in out if "item_id" in e] == [
"cipher-delta-id", "cipher-done-id", "cipher-done-id"
]
assert out[2]["item_id"] != item_ids[0]
print("result: top-level function-call item_id values are not pinned")
PYRepository: lidge-jun/opencodex
Length of output: 381
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
import json
events = [
{
"type": "response.output_item.added",
"output_index": 0,
"item": {"type": "function_call", "id": "first-item-id"},
},
{
"type": "response.function_call_arguments.delta",
"item_id": "cipher-delta-id",
"output_index": 0,
"sequence_number": 3,
"delta": "ciphertext-chunk-1",
"obfuscation": "abc123",
},
{
"type": "response.function_call_arguments.done",
"item_id": "cipher-done-id",
"output_index": 0,
"sequence_number": 5,
"arguments": '{"a":2,"b":2}',
},
]
obfuscated = set()
item_ids = {}
out = []
for event in events:
parsed = json.loads(json.dumps(event))
changed = False
if isinstance(parsed.get("obfuscation"), str):
del parsed["obfuscation"]
changed = True
typ = parsed.get("type")
if typ == "response.function_call_arguments.delta" and changed:
obfuscated.add(parsed["output_index"])
parsed["delta"] = ""
out.append(parsed)
continue
if typ == "response.function_call_arguments.done":
if parsed["output_index"] in obfuscated:
obfuscated.remove(parsed["output_index"])
out.append({
"type": "response.function_call_arguments.delta",
"item_id": parsed.get("item_id"),
"output_index": parsed["output_index"],
"sequence_number": parsed.get("sequence_number"),
"delta": parsed.get("arguments", ""),
})
if typ in ("response.output_item.added", "response.output_item.done"):
item = parsed.get("item")
index = parsed.get("output_index")
if isinstance(index, (int, float)) and isinstance(item, dict) and isinstance(item.get("id"), str):
item_ids.setdefault(index, item["id"])
out.append(parsed)
print("remembered nested item id:", item_ids[0])
print("argument event item ids:", [e.get("item_id") for e in out if "item_id" in e])
print("synthetic delta item id:", out[2]["item_id"])
assert item_ids[0] == "first-item-id"
assert [e.get("item_id") for e in out if "item_id" in e] == [
"cipher-delta-id", "cipher-done-id", "cipher-done-id"
]
assert out[2]["item_id"] != item_ids[0]
print("result: top-level function-call item_id values are not pinned")
PYRepository: lidge-jun/opencodex
Length of output: 381
Normalize top-level item_id values by output_index.
itemIds currently normalizes only nested item.id. Argument delta and done events retain different top-level parsed.item_id values, and the synthetic plaintext delta copies the done-event ID. This breaks function-call correlation.
src/server/sse-payload-rewrite.ts:118-136: Normalizeparsed.item_idbefore the early delta return and use the canonical ID indeltaEvent. TrackhadObfuscationseparately so ID normalization does not cause clean deltas to be replaced with empty strings.tests/sse-payload-rewrite.test.ts:43-80: Add a precedingresponse.output_item.addedevent and assert that all argument delta and done events use its first-seen item ID.
📍 Affects 2 files
src/server/sse-payload-rewrite.ts#L118-L136(this comment)tests/sse-payload-rewrite.test.ts#L43-L80
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/server/sse-payload-rewrite.ts` around lines 118 - 136, Normalize
top-level parsed.item_id by output_index in the argument-event handling before
the early delta return, while tracking whether the event was actually obfuscated
separately so clean deltas retain their payload. In
src/server/sse-payload-rewrite.ts:118-136, use the canonical normalized ID when
constructing deltaEvent. In tests/sse-payload-rewrite.test.ts:43-80, add a
preceding response.output_item.added event and verify all related argument delta
and done events use its first-seen item ID.
Summary
GitHub Copilot's
/responseswire is not compatible with Responses clients for thevscode-chatintegration, so agentic turns on Responses-only Copilot models (gpt-5.6-luna,gpt-5.6-sol,gpt-5.6-terra,gpt-5.4,gpt-5.4-mini,gpt-5.5,gpt-5.3-codex) stall in Codex clients. Fixes #995.Root causes in the upstream stream:
delta+obfuscationfield); the.doneevent carries the plaintextarguments.encrypted_contentwith no plaintext summary; clients cannot decrypt them and stay stuck on "thinking".completedpayload disagrees with the streamed events (call_idis the only stable field).Fix: a provider-scoped SSE rewrite (
createGithubCopilotObfuscationRewrite) wired into the passthroughpayloadRewritescomposition whenroute.providerName === "github-copilot"(applies to all Copilot models on both relay paths):argumentsare re-emitted as one delta immediately before.done, keeping the stream protocol-conformant.encrypted_contentare replaced with a canonical empty plaintext summary, inoutput_itemevents and the terminalcompletedpayload.created,output_item, andcompletedevents.obfuscationmetadata is stripped from every event; clean events pass through byte-identical.Verification
bun test tests/sse-payload-rewrite.test.ts— 8 pass (4 new: delta un-obfuscation, reasoning neutralization, id pinning, clean passthrough).bun x tsc --noEmit— clean.github-copilot/gpt-5.6-lunastreamed tool-call turn now delivers plaintext arguments, plaintext reasoning summaries, and consistent ids (response.idand item ids identical acrosscreated/output_item/completed).Checklist
Summary by CodeRabbit