fix(llm): surface server error bodies instead of silent NOTFOUND - #68
Merged
Conversation
Some servers (e.g. mlx_lm.server given a model id it must fetch) return HTTP
200 with an {"error": ...} body. chat_turn read v["choices"][0] (null) ->
content None, and chat_turn_stream's non-SSE error body matched no data: line
-> empty. Both looked like an empty completion, which ask misread as NOTFOUND
("nothing covers this") — hiding the real cause (bad [ask].model, 404, etc).
Add server_error() and return a clean Err from both paths. New tests cover the
buffered and streamed error bodies. Derive Debug on ToolCall (test needs it).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When an OpenAI-compatible server returns an error body at HTTP 200 — e.g.
mlx_lm.servergiven an[ask].modelit doesn't have loaded returns{"error": "404 … Repository Not Found for kuro"}while trying to fetch it from HuggingFace — kibble silently swallowed it:chat_turnreadv["choices"][0]["message"](null) →content = None.chat_turn_stream's non-SSE error body matched nodata:line → empty.Both looked like an empty completion, which
askclassifies asNOTFOUND→ the user sees "Searched the corpus — nothing covers this" with zero indication the LLM call actually failed. (Cost me a long debugging session to trace a mis-set model name.)Fix
server_error()— pulls the message from an{"error": …}body (string or{message}object).chat_turn: return a cleanErr("LLM server error: …")when the body is an error.chat_turn_stream: after the loop, if nothing streamed and the leftover buffer parses as an error body,Errthe same way.Now a bad model / auth / 404 surfaces the real message instead of a phantom "not found in corpus."
Tests
chat_turn_surfaces_server_error+chat_turn_stream_surfaces_server_error(buffered + streamed error bodies →Err), verified RED before the fix.kibble askwith a bad[ask].modelnow prints the HF 404 instead of "nothing covers this."cargo clippy --all-targets -- -D warningsclean.🤖 Generated with Claude Code