[codex] Add token budget context feature - #27438
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a19ec71e8d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| let turn_metadata_header = turn_context | ||
| .turn_metadata_state | ||
| .current_header_value_for_model_request(&window_id); | ||
| let tokens_before_sampling = sess.get_total_token_usage().await; |
There was a problem hiding this comment.
Because this snapshot is taken after run_hooks_and_record_inputs has already appended the current user turn and any pending input, a large input can move the thread from below a budget threshold to above it before this comparison starts. For example, going from 24% used to 26% used via the user's message makes tokens_before_sampling already above 25%, so maybe_record_token_budget_remaining_context never emits the 25% reminder on this or any later turn unless another threshold is crossed.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Ok for the initial version.
bolinfest
left a comment
There was a problem hiding this comment.
Small suggestions: feel free to take or leave.
| assert_eq!(requests.len(), 2); | ||
|
|
||
| let expected = vec![format!( | ||
| "<token_budget>\nCurrent context window 0.\nYou have {EFFECTIVE_CONTEXT_WINDOW} tokens left in this context window.\n</token_budget>" |
There was a problem hiding this comment.
prefer raw string to make this more readable?
Why
The model should be able to see bounded context-window budget metadata when the
token_budgetfeature is enabled. The full-window message is only injected with full context, while normal turns get a smaller follow-up only when reported usage first crosses a budget threshold.What changed
TokenBudgetfeature flag.<token_budget>developer fragments for full context-window metadata and current-window remaining tokens.Verification
just test -p codex-core token_budgetgit diff --check