Fix Codex named tool_choice serialization in reflect#734
Merged
nicoloboschi merged 1 commit intoMar 30, 2026
Merged
Conversation
nicoloboschi
left a comment
Collaborator
There was a problem hiding this comment.
Thansk, can you use pytest for the test? Just follow the codebase standard
|
Can confirm this bug. I'm using |
nicoloboschi
added a commit
that referenced
this pull request
Mar 30, 2026
Follow-up to #734: replace unittest.TestCase + manual sys.path manipulation with idiomatic pytest + @pytest.mark.asyncio, matching the rest of the test suite.
2 tasks
nicoloboschi
added a commit
that referenced
this pull request
Mar 30, 2026
Follow-up to #734: replace unittest.TestCase + manual sys.path manipulation with idiomatic pytest + @pytest.mark.asyncio, matching the rest of the test suite.
nicoloboschi
added a commit
that referenced
this pull request
Mar 30, 2026
* Convert codex tool_choice test to pytest style Follow-up to #734: replace unittest.TestCase + manual sys.path manipulation with idiomatic pytest + @pytest.mark.asyncio, matching the rest of the test suite. * Fix test_hierarchical_fields_categorization for new configurable fields Update expected count from 20 to 21 and add assertions for fields added by recent PRs: retain_default_strategy, retain_strategies, max_observations_per_scope, reflect_source_facts_max_tokens, llm_gemini_safety_settings, mcp_enabled_tools. * Add LlamaIndex doc to v0.4 versioned docs and sidebars The LlamaIndex integration doc was added to docs/ (next version) in #672 but not to versioned_docs/version-0.4/, causing a broken link on the /integrations page which resolves to the latest version. * Regenerate docs skill references Run generate-docs-skill.sh to pick up new integration pages (codex, llamaindex) and updated configuration docs. * Add Codex integration doc to v0.4 versioned docs and sidebar Same issue as LlamaIndex: doc was added to docs/ (next) but not versioned_docs/version-0.4/, causing broken link on /integrations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This fixes a Codex provider compatibility issue in reflect/tool-calling paths.
Reflect can force a specific tool on early iterations using the legacy named tool choice shape:
{"type":"function","function":{"name":"recall"}}The current Codex Responses API expects the function name at the top level instead:
{"type":"function","name":"recall"}Without that normalization, Codex rejects the request and reflect can end up with empty
based_on.memorieseven when recallable evidence exists.What changed
tool_choicedicts inCodexLLM.call_with_tools()tool_choice.functionWhy this belongs in the provider
This follows the same design direction as the existing provider-specific tool-choice compatibility handling for LM Studio / OpenAI-compatible providers:
Validation
Targeted regression:
python -m pytest -o addopts="" --noconftest hindsight-api-slim/tests/test_codex_tool_choice.py -qResult:
I also verified the original failure mode before the patch via a live Codex-backed daemon, where reflect requests were rejected with:
After this patch, the same integration recovered real reflect evidence instead of failing in the forced-tool iterations.
Non-goals
This PR does not claim to fix every possible
reflectempty-evidence case.It specifically fixes the Codex provider request-shape incompatibility for forced named tool choice.