Skip to content

feat(codec): add OCI Generative AI provider codec in Rust core - #552

Closed
fede-kamel wants to merge 1 commit into
NVIDIA:mainfrom
fede-kamel:feat/oci-genai-rust-codec
Closed

feat(codec): add OCI Generative AI provider codec in Rust core#552
fede-kamel wants to merge 1 commit into
NVIDIA:mainfrom
fede-kamel:feat/oci-genai-rust-codec

Conversation

@fede-kamel

Copy link
Copy Markdown

Overview

Adds Oracle Cloud Infrastructure (OCI) Generative AI as a built-in provider codec in the Rust core, with exposure through the Python and Node bindings — the Rust rework requested in #549 review.

  • I confirm this contribution is my own work, or I have the right to submit it under this project's license.
  • I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

What: OCIGenAIChatCodec (LlmCodec + LlmResponseCodec) and OCIGenAIStreamingCodec in crates/core/src/codec/oci_genai.rs, covering both OCI chat wire formats — GENERIC (Meta Llama, Google Gemini, xAI Grok, OpenAI, and imported open-weights models such as NVIDIA Nemotron on dedicated AI clusters) and COHERE — with typed ApiSpecificRequest::OCIGenAI / ApiSpecificResponse::OCIGenAI variants, registration in the provider-surface resolver (strongest-signal detection on the chatRequest/servingMode envelope, placed first; oci/oci.genai hints honored), PyOCIGenAIChatCodec (pyo3) and OCIGenAIChatCodec (napi) bindings, and full support in the guardrails, PII-redaction overlay, and adaptive components that match exhaustively on provider surfaces.

Why: OCI GenAI traffic is currently opaque to Relay middleware and observability. With a core codec, request intercepts (PII redaction, guardrails, policy), streaming, cost accounting, and LLMEnd annotations work on OCI payloads in every language binding, exactly as they do for the OpenAI and Anthropic built-ins.

How: Follows the established provider pattern (anthropic.rs as the closest template): unit-struct codec, PROVIDER_SURFACE descriptor, in-module tests, multi-convention key handling (SDK camelCase, CLI kebab-case, snake_case). encode() uses merge-not-replace baseline-compare-and-patch per the LlmCodec contract: unchanged messages pass through from the raw payload verbatim, so encode(decode(original), original) == original at the JSON level and unmodeled provider fields survive edits (fully for GENERIC; COHERE holds identity for unedited requests, while message edits rebuild the modeled message/chatHistory/preambleOverride fields). Model edits are rejected with InvalidArgument; tool/tool-choice edits encode via Function mapping with ProviderNative passthrough.

Testing:

  • 32 new codec unit tests (fixtures ported from real OCI wire traffic): decode/encode round-trips for both formats, identity invariants with unmodeled fields at envelope/request/message level, redaction-style edits preserving untouched messages verbatim, tool-call conversion, param patch-only-changed, response decoding across all three key conventions, request/response detection incl. non-shadowing against the other three surfaces, and streaming.
  • 7 new cross-provider parity cases; 2 PII-redaction overlay tests (GENERIC choices + COHERE text); resolver, guardrails-component, and Python test_builtin_codecs.py coverage for the new surface.
  • cargo build --workspace clean; cargo test --workspace fully green (core lib 982+ tests, integration suite passing); cargo clippy --workspace --all-targets zero warnings; cargo fmt clean; missing_docs satisfied.
  • End-to-end validation: the Rust codec (via the Python binding) decoded, identity-encoded, and round-tripped a request whose encoded payload was posted verbatim to the signed OCI REST chat endpoint and answered by a dedicated AI cluster serving an imported NVIDIA Nemotron 3 model; the response codec normalized the reply including token usage.

Breaking changes: None intended. from_provider_surface in the adaptive ACG request-surface mapping now returns Option (OCI has no ACG applier) — internal API, call sites updated.

Where should the reviewer start?

crates/core/src/codec/oci_genai.rs — detection rules and the baseline-compare-and-patch encode() are the key decisions — then the identity-invariant cases in crates/core/tests/unit/codec/oci_genai_tests.rs and the new cases in parity_tests.rs.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

Add a fourth built-in provider surface for the Oracle Cloud
Infrastructure (OCI) Generative AI chat API so request intercepts
(PII redaction, guardrails, policy) and LLM observability operate on
OCI chat traffic the same way they do for OpenAI Chat, OpenAI
Responses, and Anthropic Messages.

What:
- OCIGenAIChatCodec (LlmCodec + LlmResponseCodec) in
  crates/core/src/codec/oci_genai.rs. Requests are accepted as a full
  ChatDetails envelope ({compartmentId, servingMode, chatRequest}) or
  as a bare chatRequest, in both chatRequest.apiFormat variants:
  GENERIC (UPPERCASE roles, typed TEXT content-part lists, flat
  toolCalls {id, type: FUNCTION, name, arguments}, toolCallId on tool
  messages) and COHERE (preambleOverride, chatHistory USER/CHATBOT/
  SYSTEM turns, current message string). GENERIC maxTokens/
  temperature/topP/stop and COHERE stopSequences normalize into
  GenerationParams; unmodeled params (topK, seed, penalties, ...)
  stay in the raw payload untouched.
- encode() uses baseline-compare-and-patch semantics per the LlmCodec
  contract: the original request is re-decoded as the baseline and
  only fields an intercept actually changed are rewritten, so
  unchanged messages pass through from the raw payload verbatim,
  per-message provider fields survive edits, and
  encode(decode(original), original) == original at the JSON level.
- Response decode covers ChatResult ({modelId, chatResponse}) and
  bare chat responses for both formats, tolerates camelCase,
  kebab-case, and snake_case key conventions (SDK vs CLI shapes),
  parses GENERIC string-encoded tool-call arguments, maps
  promptTokens/completionTokens/totalTokens into Usage, and maps
  finish reasons (stop/COMPLETE -> complete, length/MAX_TOKENS ->
  length, tool_calls -> tool_use, else unknown).
- OCIGenAIStreamingCodec assembles OCI SSE choice deltas (GENERIC)
  and text fragments (COHERE) back into a non-streaming ChatResult
  shape that decode_response can consume, matching the streaming
  strategy of the existing provider codecs.
- ProviderSurface::OCIGenAI registered first in
  BUILTIN_PROVIDER_SURFACES: the ChatDetails envelope and apiFormat
  markers are the strongest request signal and never appear in the
  other surfaces' shapes. Typed ApiSpecificRequest::OCIGenAI
  (compartment_id, serving_mode, api_format) and
  ApiSpecificResponse::OCIGenAI (api_format, model_version) variants.
- Bindings: OCIGenAIChatCodec pyclass (with the fast-path builtin
  response-codec downcast) plus stubs/re-exports, and the matching
  napi class for Node.
- Plugin integration for the new surface: pii-redaction response
  overlay for GENERIC choices/COHERE text plus flat toolCalls, local
  NeMo Guardrails codec selection and stream-chunk text extraction,
  and 'oci_genai' added to the codec config enums, schemas, and
  validation messages. Adaptive ACG surface resolution returns None
  for OCI (no hint applier exists yet) instead of misclassifying.

Testing:
- crates/core/tests/unit/codec/oci_genai_tests.rs: 32 cases covering
  GENERIC/COHERE decode, envelope and bare payloads, identity
  round-trips with unmodeled fields at envelope/request/message
  level, redaction edits preserving untouched messages verbatim,
  tool-call conversion, param patch-only-changed, response decode for
  both formats plus kebab-case CLI shapes and non-object payloads,
  request/response surface detection (including non-shadowing of the
  other three surfaces), and streaming assembly for text, tool-call
  argument accumulation, and COHERE fragments.
- Cross-provider parity cases in parity_tests.rs and OCI overlay
  coverage in the pii-redaction tests.
- cargo build/test/clippy/fmt clean across the workspace.

Signed-off-by: Federico Kamelhar <federico.kamelhar@oracle.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 24, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: b5ed41f6-3f7e-4551-a27b-527155f9f1f3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size:XL PR is extra large Feature a new feature lang:js PR changes/introduces Javascript/Typescript code lang:python PR changes/introduces Python code lang:rust PR changes/introduces Rust code labels Jul 24, 2026
@fede-kamel

Copy link
Copy Markdown
Author

Splitting this into a stacked series of smaller PRs (≤~1k lines each) in foundation order for reviewability: (1) types + response codec, (2) request codec with merge-not-replace encode, (3) resolver registration + guardrails/PII/adaptive surface support, (4) streaming codec + Python/Node bindings. Same commit content, same test coverage, same end-to-end validation; series tracked in #548. First PR of the series follows shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature a new feature lang:js PR changes/introduces Javascript/Typescript code lang:python PR changes/introduces Python code lang:rust PR changes/introduces Rust code size:XL PR is extra large

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement]: Support Oracle Cloud Infrastructure (OCI) Generative AI as a provider

1 participant