Custom OpenAI-compatible providers: image file attachments do not reach vision-capable models correctly

[GravityPoet]

Summary

When using a custom OpenAI-compatible provider, image attachments sent through OpenCode sessions are not reaching the model as usable vision input.

In my case:

• provider: longent
• model: gpt-5.4(xhigh)
• transport: OpenCode desktop / session API

The exact same provider/model can successfully answer a vision request when called directly via the provider's OpenAI-compatible /chat/completions API using image_url.

But when I send an image through OpenCode as a file part (mime: image/png), the assistant responds with:

I can't read remote-test.png here because this model doesn't support image input.

So this looks like an OpenCode adapter/translation issue in the custom provider + file attachment path, not a model capability issue.

Expected behavior

If a custom OpenAI-compatible provider/model supports image input, sending an image attachment through OpenCode should be translated into a valid multimodal request and the model should be able to read the image.

Actual behavior

OpenCode session messages with file image parts end up behaving as if the model does not support image input, even though the same model works with direct API vision calls.

Repro

1. Configure a custom OpenAI-compatible provider

Example config shape:

{
  "provider": {
    "longent": {
      "name": "Longent",
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "https://longent.tech/v1",
        "apiKey": "REDACTED"
      },
      "models": {
        "gpt-5.4(xhigh)": {
          "name": "GPT-5.4 XHigh",
          "options": {
            "reasoningEffort": "xhigh"
          }
        }
      }
    }
  }
}

2. Direct provider API works

A direct request to the same provider/model with image_url succeeds.

Example payload:

{
  "model": "gpt-5.4(xhigh)",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "What color is this image? Reply with one word."},
        {"type": "image_url", "image_url": {"url": "https://dummyimage.com/10x10/ff0000/ffffff.png"}}
      ]
    }
  ],
  "max_tokens": 30
}

Response is correct (red / Red).

3. OpenCode session API fails for image file attachment

Create a session, then send a message with parts:

{
  "model": {"providerID": "longent", "modelID": "gpt-5.4(xhigh)"},
  "agent": "build",
  "parts": [
    {"type": "text", "text": "What color is this image? Reply with one word."},
    {
      "type": "file",
      "mime": "image/png",
      "filename": "remote-test.png",
      "url": "https://dummyimage.com/10x10/ff0000/ffffff.png"
    }
  ]
}

Returned assistant text:

I can't read `remote-test.png` here because this model doesn't support image input.

Additional observations

• I also reproduced the issue from the OpenCode desktop app by attaching a local PNG file.
• In the recorded session message, the user input is stored as a file part with mime: image/png.
• The session itself is definitely using the custom provider/model (providerID=longent, modelID=gpt-5.4(xhigh)).
• This strongly suggests the problem is in how OpenCode maps file image parts for custom OpenAI-compatible providers.

Environment

• OpenCode desktop
• OpenCode session version observed: 1.3.13
• macOS

Hypothesis

OpenCode may be handling image attachments correctly for some built-in providers, but for @ai-sdk/openai-compatible custom providers it looks like file image parts are not being converted into the multimodal format the upstream model expects.

Possibly the provider path needs to translate image file parts into something equivalent to image_url / image content parts for OpenAI-compatible chat requests.

[github-actions]

This issue might be a duplicate of existing issues. Please check:

• [FEATURE]: Add supportsAttachments/vision capability for custom providers #8875: Same root cause — custom providers using @ai-sdk/openai-compatible have image attachments silently stripped because vision capability (SupportsAttachments) is hardcoded for built-in providers only, not inherited by custom providers
• [ModelScope] Custom Provider with Qwen Model Cannot Recognize Images - "Image Input Not Supported" #15040: Same symptom — custom OpenAI-compatible provider (ModelScope/Qwen) returns "Image Input Not Supported" even though the model officially supports vision

If your issue is distinct (e.g. specifically about the file part → image_url translation path rather than capability gating), please clarify the difference.

[GravityPoet]

Thanks — I checked the suggested duplicates and added one more data point from my side.

What I verified locally:

1. The failing session was definitely using the custom provider/model:

• providerID: longent
• modelID: gpt-5.4(xhigh)

2. The exact same provider/model works for vision when called directly through the provider's OpenAI-compatible API with image_url.

• I tested against https://longent.tech/v1/chat/completions
• same model
• image URL input
• response correctly identified the image color

3. The failure happens specifically when OpenCode sends the image as a file part in a session message.

• message part type: file
• mime: image/png
• then the assistant returns: "this model doesn't support image input"

So from my testing, this is at least one of:

• custom providers are not marked as vision-capable in the OpenCode pipeline
• and/or file(image/*) parts are not being translated correctly for OpenAI-compatible providers

If #8875 already covers the capability-gating root cause, then this issue may be the same bug manifesting through the session file attachment path.

If helpful, I can also provide a minimal repro using:

• direct provider API success with image_url
• OpenCode session API failure with parts: [text, file]

[XIYBHK]

Adding root-cause analysis and a workaround after digging through the opencode source.

Root cause: the modalities field is required but undocumented

Capability detection for image input lives in packages/opencode/src/provider/provider.ts, roughly:

capabilities: {
  input: {
    image: model.modalities?.input?.includes("image") ?? false,
    // ...
  }
}

If modalities is not defined on a custom model entry in opencode.json, capabilities.input.image defaults to false, and image parts are stripped client-side before the request is built. The model then receives text only and often replies literally "I don't support image input", which looks like a backend capability issue but is actually opencode's own gating check.

This explains why the same backend works with a direct image_url curl but fails through opencode — the file → image_url mapping logic is never reached, because the attachment is already gone by then.

Minimal repro

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "my-kimi": {
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "https://your-openai-compatible-gateway/v1",
        "apiKey": "sk-..."
      },
      "models": {
        "kimi-k2.5": {
          "name": "Kimi K2.5"
          // no `modalities` → image gets stripped client-side
        }
      }
    }
  }
}

Attach an image, send → assistant responds "I don't support image input."

Workaround: declare modalities explicitly

"kimi-k2.5": {
  "name": "Kimi K2.5",
  "modalities": {
    "input": ["text", "image"],
    "output": ["text"]
  }
}

Restart opencode (the capability cache is built at startup, not hot-reloaded). Images now flow through correctly, verified end-to-end with Kimi K2.5 and K2.6 multimodal responses.

Suggestions

1. Document modalities under the "Custom provider" section at opencode.ai/docs/providers. It's currently referenced nowhere in user-facing docs, even though it's the only switch that enables vision for custom providers.
2. Tighten the opencode.json JSON schema so provider.*.models.* describes modalities, reasoning, toolCall, temperature, etc. explicitly. Right now the inner model-entry shape is unconstrained, so IDE completion and schema validation give users zero hint that modalities exists.
3. Optional UX: when an image attachment is stripped because the model lacks modalities.input.image, surface a warning. That would turn a confusing symptom ("model says it doesn't support images") into an actionable one ("opencode stripped your image because modalities.input doesn't include 'image'").

Happy to open a docs PR for the first suggestion above.