Skip to content

core: strip image detail from Responses Lite requests - #27246

Merged
fjord-oai merged 1 commit into
mainfrom
fjord/responses_strict_mode_v2---4ixxk-hc0zkmy-
Jun 11, 2026
Merged

core: strip image detail from Responses Lite requests#27246
fjord-oai merged 1 commit into
mainfrom
fjord/responses_strict_mode_v2---4ixxk-hc0zkmy-

Conversation

@fjord-oai

@fjord-oai fjord-oai commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Strip image detail fields from every Responses Lite request.
  • Apply stripping to message images and function/custom tool-output images.
  • Transform only the formatted request copy without mutating stored history.
  • Preserve image URLs byte-for-byte, including HTTP(S) URLs, without downloading, validating, or resizing them.
  • Preserve all image detail fields for non-Responses-Lite models.

Motivation

Responses Lite does not support image detail tags, so Codex must omit them whenever model_info.use_responses_lite is enabled. This transport requirement is independent of the resize_all_images feature.

Stored history retains the original detail values. This keeps request-specific formatting isolated from conversation state and preserves the information for local image preparation and non-Responses-Lite requests.

git stack

@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 9, 2026 21:49
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from c1cd2b4 to ce2208b Compare June 9, 2026 21:49
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 9, 2026 21:49
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 9, 2026 22:03
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from ce2208b to 6956206 Compare June 9, 2026 22:03
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 9, 2026 22:03
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 9, 2026 23:50
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from 6956206 to 4ab6a84 Compare June 9, 2026 23:50
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 9, 2026 23:50
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 10, 2026 02:42
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from 4ab6a84 to 04183f0 Compare June 10, 2026 02:42
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 10, 2026 02:42
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 10, 2026 03:45
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from 04183f0 to da0224f Compare June 10, 2026 03:45
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 10, 2026 03:45
@fjord-oai
fjord-oai marked this pull request as ready for review June 10, 2026 04:13
@fjord-oai
fjord-oai requested a review from a team as a code owner June 10, 2026 04:13
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 10, 2026 19:37
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from da0224f to 7ede0c2 Compare June 10, 2026 19:37
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 10, 2026 19:37
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 10, 2026 19:54
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from 7ede0c2 to 377055f Compare June 10, 2026 19:54
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 10, 2026 19:54
fjord-oai added a commit that referenced this pull request Jun 10, 2026
## Summary

Add shared image-processing primitives needed for centralized image
preparation in a follow-up PR.

- Add `load_data_url_for_prompt` for decoding and preparing base64 image
data URLs.
- Add configurable maximum-dimension and 32px patch-budget resizing.
- Enforce a 1 GiB sanity limit on both encoded and decoded data-URL
representations.
- Preserve original PNG, JPEG, and WebP bytes when resizing is
unnecessary.
- Preserve the existing GIF-to-PNG behavior.
- Move image utility tests into the existing sidecar test module.

## Behavior

This PR is intended to be runtime behavior-preserving.

Existing production callers continue using
`PromptImageMode::ResizeToFit` and `PromptImageMode::Original` with
their existing semantics. The new data-URL entrypoint and configurable
resize mode have no production callers in this PR; they are used by the
next PR in the stack.

This PR does not change user-input handling, `view_image`, history
insertion, request construction, HTTP image URL forwarding, or
app-server behavior.


#### [git stack](https://github.com/magus/git-stack-cli)
- 👉 `1` #27245
- ⏳ `2` #27247
- ⏳ `3` #27246
- ⏳ `4` #27266
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 10, 2026 22:49
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from 377055f to 1b83959 Compare June 10, 2026 22:49
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 10, 2026 22:49
Comment thread codex-rs/core/tests/suite/responses_lite.rs Outdated
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 11, 2026 00:37
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from 1b83959 to 43167c8 Compare June 11, 2026 00:37
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 11, 2026 00:37
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 11, 2026 01:16
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from 43167c8 to 6ae7b9d Compare June 11, 2026 01:16
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 11, 2026 01:16
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 11, 2026 01:26
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from 6ae7b9d to 39fd6a4 Compare June 11, 2026 01:26
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy June 11, 2026 01:26
fjord-oai added a commit that referenced this pull request Jun 11, 2026
## Summary

Adds complete client-side image preparation behind the default-off
`resize_all_images` feature flag.

When enabled, local image producers defer decoding and resizing. Images
are prepared centrally before insertion into conversation history,
covering user input, `view_image`, and structured tool-output images.

## Behavior

- Processes base64 `data:` images in messages and function/custom tool
outputs.
- Leaves non-data URLs, including HTTP(S) URLs, unchanged.
- Applies image-detail budgets:
  - `high` and omitted: 2048px maximum dimension and 2.5K 32px patches.
  - `original`: 6000px maximum dimension and 10K 32px patches.
  - `auto`: uses the same 2048px / 2.5K-patch budget as high.
  - `low`: unsupported and replaced with an actionable placeholder.
- Preserves original image bytes when no resize or format conversion is
needed.
- Enforces the shared 1 GiB encoded and decoded data-URL sanity limits.
- Replaces only an image that fails preparation, preserving sibling
content and tool-output metadata.
- Uses bounded placeholders distinguishing generic processing failures,
oversized images, and unsupported `low` detail.
- Prepares resumed and forked history before installing it as live
history without modifying persisted rollouts.

## Flag-Off Behavior

When `resize_all_images` is disabled:

- Existing local user-input and `view_image` processing remains
unchanged.
- Existing decoding and error behavior remains unchanged.
- Arbitrary tool-output images are not processed.
- HTTP(S) image URLs continue to be forwarded unchanged.


#### [git stack](https://github.com/magus/git-stack-cli)
- ✅ `1` #27245
- 👉 `2` #27247
- ⏳ `3` #27246
- ⏳ `4` #27266
Base automatically changed from fjord/responses_strict_mode_v2---4ixxk-7dlcgvwy to main June 11, 2026 02:21
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from 39fd6a4 to 3a6d4bd Compare June 11, 2026 02:21

@rka-oai rka-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (just one test nit)

Comment thread codex-rs/core/src/client_common.rs Outdated
Comment thread codex-rs/core/src/session/turn_context.rs Outdated
git-stack-id: fjord/responses_strict_mode_v2---4ixxk-hc0zkmy-
git-stack-title: core: strip image detail from Responses Lite requests
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch from 3a6d4bd to 5eb40cb Compare June 11, 2026 03:29
@fjord-oai
fjord-oai merged commit 4435ff2 into main Jun 11, 2026
47 of 62 checks passed
@fjord-oai
fjord-oai deleted the fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- branch June 11, 2026 03:43
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 11, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants