feat(codec): add OCI Generative AI provider codec in Rust core - #552
feat(codec): add OCI Generative AI provider codec in Rust core#552fede-kamel wants to merge 1 commit into
Conversation
Add a fourth built-in provider surface for the Oracle Cloud
Infrastructure (OCI) Generative AI chat API so request intercepts
(PII redaction, guardrails, policy) and LLM observability operate on
OCI chat traffic the same way they do for OpenAI Chat, OpenAI
Responses, and Anthropic Messages.
What:
- OCIGenAIChatCodec (LlmCodec + LlmResponseCodec) in
crates/core/src/codec/oci_genai.rs. Requests are accepted as a full
ChatDetails envelope ({compartmentId, servingMode, chatRequest}) or
as a bare chatRequest, in both chatRequest.apiFormat variants:
GENERIC (UPPERCASE roles, typed TEXT content-part lists, flat
toolCalls {id, type: FUNCTION, name, arguments}, toolCallId on tool
messages) and COHERE (preambleOverride, chatHistory USER/CHATBOT/
SYSTEM turns, current message string). GENERIC maxTokens/
temperature/topP/stop and COHERE stopSequences normalize into
GenerationParams; unmodeled params (topK, seed, penalties, ...)
stay in the raw payload untouched.
- encode() uses baseline-compare-and-patch semantics per the LlmCodec
contract: the original request is re-decoded as the baseline and
only fields an intercept actually changed are rewritten, so
unchanged messages pass through from the raw payload verbatim,
per-message provider fields survive edits, and
encode(decode(original), original) == original at the JSON level.
- Response decode covers ChatResult ({modelId, chatResponse}) and
bare chat responses for both formats, tolerates camelCase,
kebab-case, and snake_case key conventions (SDK vs CLI shapes),
parses GENERIC string-encoded tool-call arguments, maps
promptTokens/completionTokens/totalTokens into Usage, and maps
finish reasons (stop/COMPLETE -> complete, length/MAX_TOKENS ->
length, tool_calls -> tool_use, else unknown).
- OCIGenAIStreamingCodec assembles OCI SSE choice deltas (GENERIC)
and text fragments (COHERE) back into a non-streaming ChatResult
shape that decode_response can consume, matching the streaming
strategy of the existing provider codecs.
- ProviderSurface::OCIGenAI registered first in
BUILTIN_PROVIDER_SURFACES: the ChatDetails envelope and apiFormat
markers are the strongest request signal and never appear in the
other surfaces' shapes. Typed ApiSpecificRequest::OCIGenAI
(compartment_id, serving_mode, api_format) and
ApiSpecificResponse::OCIGenAI (api_format, model_version) variants.
- Bindings: OCIGenAIChatCodec pyclass (with the fast-path builtin
response-codec downcast) plus stubs/re-exports, and the matching
napi class for Node.
- Plugin integration for the new surface: pii-redaction response
overlay for GENERIC choices/COHERE text plus flat toolCalls, local
NeMo Guardrails codec selection and stream-chunk text extraction,
and 'oci_genai' added to the codec config enums, schemas, and
validation messages. Adaptive ACG surface resolution returns None
for OCI (no hint applier exists yet) instead of misclassifying.
Testing:
- crates/core/tests/unit/codec/oci_genai_tests.rs: 32 cases covering
GENERIC/COHERE decode, envelope and bare payloads, identity
round-trips with unmodeled fields at envelope/request/message
level, redaction edits preserving untouched messages verbatim,
tool-call conversion, param patch-only-changed, response decode for
both formats plus kebab-case CLI shapes and non-object payloads,
request/response surface detection (including non-shadowing of the
other three surfaces), and streaming assembly for text, tool-call
argument accumulation, and COHERE fragments.
- Cross-provider parity cases in parity_tests.rs and OCI overlay
coverage in the pii-redaction tests.
- cargo build/test/clippy/fmt clean across the workspace.
Signed-off-by: Federico Kamelhar <federico.kamelhar@oracle.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
Splitting this into a stacked series of smaller PRs (≤~1k lines each) in foundation order for reviewability: (1) types + response codec, (2) request codec with merge-not-replace encode, (3) resolver registration + guardrails/PII/adaptive surface support, (4) streaming codec + Python/Node bindings. Same commit content, same test coverage, same end-to-end validation; series tracked in #548. First PR of the series follows shortly. |
Overview
Adds Oracle Cloud Infrastructure (OCI) Generative AI as a built-in provider codec in the Rust core, with exposure through the Python and Node bindings — the Rust rework requested in #549 review.
Details
What:
OCIGenAIChatCodec(LlmCodec+LlmResponseCodec) andOCIGenAIStreamingCodecincrates/core/src/codec/oci_genai.rs, covering both OCI chat wire formats —GENERIC(Meta Llama, Google Gemini, xAI Grok, OpenAI, and imported open-weights models such as NVIDIA Nemotron on dedicated AI clusters) andCOHERE— with typedApiSpecificRequest::OCIGenAI/ApiSpecificResponse::OCIGenAIvariants, registration in the provider-surface resolver (strongest-signal detection on thechatRequest/servingModeenvelope, placed first;oci/oci.genaihints honored),PyOCIGenAIChatCodec(pyo3) andOCIGenAIChatCodec(napi) bindings, and full support in the guardrails, PII-redaction overlay, and adaptive components that match exhaustively on provider surfaces.Why: OCI GenAI traffic is currently opaque to Relay middleware and observability. With a core codec, request intercepts (PII redaction, guardrails, policy), streaming, cost accounting, and
LLMEndannotations work on OCI payloads in every language binding, exactly as they do for the OpenAI and Anthropic built-ins.How: Follows the established provider pattern (
anthropic.rsas the closest template): unit-struct codec,PROVIDER_SURFACEdescriptor, in-module tests, multi-convention key handling (SDK camelCase, CLI kebab-case, snake_case).encode()uses merge-not-replace baseline-compare-and-patch per theLlmCodeccontract: unchanged messages pass through from the raw payload verbatim, soencode(decode(original), original) == originalat the JSON level and unmodeled provider fields survive edits (fully forGENERIC;COHEREholds identity for unedited requests, while message edits rebuild the modeledmessage/chatHistory/preambleOverridefields). Model edits are rejected withInvalidArgument; tool/tool-choice edits encode via Function mapping with ProviderNative passthrough.Testing:
test_builtin_codecs.pycoverage for the new surface.cargo build --workspaceclean;cargo test --workspacefully green (core lib 982+ tests, integration suite passing);cargo clippy --workspace --all-targetszero warnings;cargo fmtclean;missing_docssatisfied.Breaking changes: None intended.
from_provider_surfacein the adaptive ACG request-surface mapping now returnsOption(OCI has no ACG applier) — internal API, call sites updated.Where should the reviewer start?
crates/core/src/codec/oci_genai.rs— detection rules and the baseline-compare-and-patchencode()are the key decisions — then the identity-invariant cases incrates/core/tests/unit/codec/oci_genai_tests.rsand the new cases inparity_tests.rs.Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)