Skip to content

fix(llm): surface server error bodies instead of silent NOTFOUND - #68

Merged
vxfemboy merged 1 commit into
mainfrom
fix/surface-llm-server-errors
Jul 26, 2026
Merged

fix(llm): surface server error bodies instead of silent NOTFOUND#68
vxfemboy merged 1 commit into
mainfrom
fix/surface-llm-server-errors

Conversation

@vxfemboy

Copy link
Copy Markdown
Member

Problem

When an OpenAI-compatible server returns an error body at HTTP 200 — e.g. mlx_lm.server given an [ask].model it doesn't have loaded returns {"error": "404 … Repository Not Found for kuro"} while trying to fetch it from HuggingFace — kibble silently swallowed it:

  • chat_turn read v["choices"][0]["message"] (null) → content = None.
  • chat_turn_stream's non-SSE error body matched no data: line → empty.

Both looked like an empty completion, which ask classifies as NOTFOUND → the user sees "Searched the corpus — nothing covers this" with zero indication the LLM call actually failed. (Cost me a long debugging session to trace a mis-set model name.)

Fix

  • Add server_error() — pulls the message from an {"error": …} body (string or {message} object).
  • chat_turn: return a clean Err("LLM server error: …") when the body is an error.
  • chat_turn_stream: after the loop, if nothing streamed and the leftover buffer parses as an error body, Err the same way.

Now a bad model / auth / 404 surfaces the real message instead of a phantom "not found in corpus."

Tests

  • chat_turn_surfaces_server_error + chat_turn_stream_surfaces_server_error (buffered + streamed error bodies → Err), verified RED before the fix.
  • Live-verified: kibble ask with a bad [ask].model now prints the HF 404 instead of "nothing covers this."
  • Full suite 381 + integration, cargo clippy --all-targets -- -D warnings clean.

🤖 Generated with Claude Code

Some servers (e.g. mlx_lm.server given a model id it must fetch) return HTTP
200 with an {"error": ...} body. chat_turn read v["choices"][0] (null) ->
content None, and chat_turn_stream's non-SSE error body matched no data: line
-> empty. Both looked like an empty completion, which ask misread as NOTFOUND
("nothing covers this") — hiding the real cause (bad [ask].model, 404, etc).

Add server_error() and return a clean Err from both paths. New tests cover the
buffered and streamed error bodies. Derive Debug on ToolCall (test needs it).
@vxfemboy
vxfemboy merged commit d3e8281 into main Jul 26, 2026
1 check passed
@vxfemboy
vxfemboy deleted the fix/surface-llm-server-errors branch July 26, 2026 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant