Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs-site/src/content/docs/guides/codex-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,11 @@ The proxy listens on port `10100` by default and serves `POST /v1/responses`,
Codex's built-in `image_gen` tool does not go through `/v1/responses` — the codex-rs extension
POSTs `{base_url}/images/generations` (or `/images/edits` when reference images are attached)
directly, with the same ChatGPT bearer auth it uses for chat. Because the injected `base_url`
points at opencodex, the proxy relays those calls to the OpenAI upstream:
points at opencodex, the proxy relays those calls to the OpenAI upstream.

This is separate from the [Image Bridge](/guides/image-bridge/), which only activates when a
**Responses** turn lists the hosted `image_generation` tool while a non-OpenAI model is selected.
Standalone `/images/generations` calls never enter that bridge.

- **One mode-aware forward candidate:** Pool selects an eligible main/added account; Direct uses the
caller OAuth bearer. The configured mode applies consistently to the image request.
Expand Down
88 changes: 88 additions & 0 deletions docs-site/src/content/docs/guides/image-bridge.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
title: Image Bridge
description: Route image_generation hosted-tool calls to xAI Grok Imagine when using a non-OpenAI provider.
---

## Overview

When you route Codex through a non-OpenAI model (Claude, Gemini, Grok, etc.), the
`image_generation` **hosted tool** normally doesn't work — it requires OpenAI's server-side
execution environment. The Image Bridge detects these calls and transparently reroutes them to
xAI Grok Imagine, so the model you're actually chatting with can still generate images.

## Prerequisites

- **Enable the bridge** by setting `images.bridgeEnabled: true` in your config (it is off by
default to avoid unexpected xAI charges — see [Configuration](#configuration) below).
- An `xai` provider entry with credentials. The bridge pins fulfillment to the registry xAI
endpoint (`https://api.x.ai/v1`); any configured `baseUrl` override is ignored for image calls.
A minimal key-mode entry is enough:

```json
{
"providers": {
"xai": { "adapter": "openai-chat", "apiKey": "xai-…" }
}
}
```

- Authentication via `authMode: "oauth"` (`ocx login xai` — uses a stored, auto-refreshed
bearer token) or `authMode: "key"` (a configured API key). See
[Providers](/guides/providers/) for the shared auth modes.
- A non-OpenAI model selected as your active provider. (When the active provider is OpenAI,
the native hosted tool is used directly and the bridge is bypassed.)

## Configuration

Image Bridge options live under `images` in `~/.opencodex/config.json`. Bridging is
**opt-in** — you must set `bridgeEnabled: true` to enable paid xAI Grok Imagine generation:

```json
{
"images": {
"bridgeEnabled": true,
"bridgeModel": "grok-imagine-image-quality",
"maxRounds": 3,
"timeoutMs": 60000
}
}
```

| Option | Default | Description |
| --- | --- | --- |
| `bridgeEnabled` | `false` | Master switch. Set `true` to enable bridging. Off by default to avoid unexpected xAI charges. |
| `bridgeModel` | `grok-imagine-image-quality` | The xAI image model id to send prompts to. |
| `maxRounds` | `3` | Maximum image-generation loop iterations per turn. Floored to an integer and clamped to `[0, 10]`; non-finite values fall back to `3`. |
| `timeoutMs` | `60000` | Per-call xAI deadline in milliseconds. Finite positive values are floored and passed to the xAI request. |

## How It Works

The Image Bridge activates only on **Responses** turns that include the hosted
`image_generation` tool in the `/v1/responses` tools array while a **non-OpenAI**
model is selected. It does **not** intercept Codex's built-in `image_gen` tool,
which POSTs directly to `/v1/images/generations` (or `/images/edits`) — that path
is covered separately in [Codex Integration](/guides/codex-integration/#built-in-image-generation-image_gen).

1. When a Responses request lists `image_generation` in `tools`, OpenCodex detects it
during request preprocessing.
2. The hosted tool is replaced with a **synthetic function tool** that the routed model can call
normally — the model sees a callable tool rather than an opaque hosted tool it can't execute.
3. When the model invokes that tool, OpenCodex intercepts the call and sends the prompt to xAI's
image generation API.
4. Generated images are saved to `~/.opencodex/artifacts/` and the **local file path** is returned
to the model as the tool result.
5. The model continues the conversation with knowledge of the generated image and its location.

From the model's perspective nothing changed — it called a tool and got a result. From the user's
perspective, image generation works with any routed provider instead of silently failing.

## Limitations

- **Only xAI Grok Imagine is supported.** DALL-E and other image providers may be added later.
- **Web search takes priority** on adapters that support the web-search sidecar loop. If both web
search and image generation are requested in the same turn, web-search runs and image
generation is skipped. Cursor/`runTurn` adapters cannot use that sidecar today, so the image
bridge may still run for those dual-tool turns.
- **xAI costs apply.** Image generation via xAI requires an active xAI subscription or API credits.
- **Streaming only.** The bridge works by intercepting the SSE response stream; requests with
`stream: false` are rejected with a 400 error.
Loading
Loading