Skip to content

fix: Improved LangChain serialization - #165

Merged
rapids-bot[bot] merged 20 commits into
NVIDIA:mainfrom
dagardner-nv:david-03-fix-LC-serialization
May 28, 2026
Merged

fix: Improved LangChain serialization#165
rapids-bot[bot] merged 20 commits into
NVIDIA:mainfrom
dagardner-nv:david-03-fix-LC-serialization

Conversation

@dagardner-nv

@dagardner-nv dagardner-nv commented May 27, 2026

Copy link
Copy Markdown
Contributor

Overview

Fix LangChain serialization, by adding a LangChain specific codec

  • I confirm this contribution is my own work, or I have the right to submit it under this project's license.
  • I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

  • This fixes the ability to use LLM intercepts
  • First pass at documenting the immutability of LLMRequest and the immutability of AnnotatedLLMRequest (I need to go back and updagte this for the other language bindings)

Where should the reviewer start?

python/nemo_relay/integrations/langchain/_serialization.py

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • Closes #

Summary by CodeRabbit

  • Documentation

    • Clarified that LLM request objects are immutable; examples now show returning new request instances instead of mutating originals.
  • Integrations

    • LangChain integration reworked to use a dedicated codec for reliable request/response translation, role normalization, tool-call handling, and preservation of extra fields.
  • Tests

    • Expanded LangChain and middleware tests, including codec round-trip and interceptor behavior; added an end-to-end agent integration test.

Review Change Stack

Signed-off-by: David Gardner <dagardner@nvidia.com>
…utes are present

Signed-off-by: David Gardner <dagardner@nvidia.com>
…ngChainModelRequestCodec

Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
@dagardner-nv dagardner-nv self-assigned this May 27, 2026
@github-actions github-actions Bot added size:M PR is medium Bug issue describes bug; PR fixes bug lang:python PR changes/introduces Python code labels May 27, 2026
@coderabbitai

coderabbitai Bot commented May 27, 2026

Copy link
Copy Markdown

Walkthrough

This PR introduces a LangChainCodec for LangChain↔NeMo Relay translation, updates middleware to use the codec and accept immutable LLMRequest objects, revises Python examples to return new LLMRequest instances when modifying headers/annotated fields, and expands tests to cover codec round-trips and interceptor behavior.

Changes

LangChain Integration with Request Immutability

Layer / File(s) Summary
Request Immutability Contract and Examples
docs/build-plugins/code-examples.mdx, docs/build-plugins/register-behavior.mdx, docs/integrate-into-frameworks/provider-codecs.mdx, python/nemo_relay/__init__.py, python/nemo_relay/intercepts.py
Clarifies that LLM request intercepts receive immutable request objects and must return new instances; updates Python examples to copy headers and return new nemo_relay.LLMRequest objects instead of mutating the original.
LangChainCodec Implementation
python/nemo_relay/integrations/langchain/_serialization.py
Adds LangChainCodec class with message/role normalization and tool-call conversions; decode extracts modeled fields into AnnotatedLLMRequest (preserving extra fields); encode rehydrates edited annotated messages back to LangChain payloads; removes codec inference scaffolding.
Payload Request Handling
python/nemo_relay/integrations/langchain/_serialization.py
Updates payload_to_model_request to accept LLMRequest and apply overrides from llm_request.content: message edits, copied model_settings, merged headers into model_settings['extra_headers'], and tool_choice override.
Middleware Integration
python/nemo_relay/integrations/langchain/middleware.py
Updates NemoRelayMiddleware to use LangChainCodec() as the explicit model codec; tightens _call wrapper parameters to typed nemo_relay.LLMRequest and passes requests directly to payload_to_model_request.
Test Coverage
python/tests/integrations/langchain_tests/test_middleware.py, python/tests/integrations/deepagents_tests/test_deepagents_integration.py
Removes codec-inference unit tests; adds LangChainCodec round-trip tests, parameterized interceptor tests (sync/async) validating annotated message mutations propagate, adds agent integration test, and a whitespace-only test adjustment.
sequenceDiagram
  participant LangChainApp
  participant NemoRelayMiddleware
  participant LangChainCodec
  participant Interceptor
  participant ModelHandler
  LangChainApp->>NemoRelayMiddleware: wrap_model_call(model)
  NemoRelayMiddleware->>NemoRelayMiddleware: _prepare_model_call(codec=LangChainCodec())
  LangChainApp->>NemoRelayMiddleware: _call(llm_request)
  NemoRelayMiddleware->>LangChainCodec: decode(model_request)
  LangChainCodec->>NemoRelayMiddleware: AnnotatedLLMRequest
  NemoRelayMiddleware->>Interceptor: apply_llm_request_intercepts(annotated)
  Interceptor->>NemoRelayMiddleware: return modified/new LLMRequest
  NemoRelayMiddleware->>ModelHandler: handler(updated_payload)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 41.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title follows Conventional Commits format with 'fix' type and provides a concise summary, staying under 72 characters.
Description check ✅ Passed The description includes all required template sections: Overview with confirmations, Details explaining the changes and documentation scope, reviewer guidance pointing to the key file, and Related Issues section.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Signed-off-by: David Gardner <dagardner@nvidia.com>
@github-actions

github-actions Bot commented May 27, 2026

Copy link
Copy Markdown

Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
…-LC-serialization

Signed-off-by: David Gardner <dagardner@nvidia.com>
@dagardner-nv
dagardner-nv marked this pull request as ready for review May 28, 2026 00:42
@dagardner-nv
dagardner-nv requested a review from a team as a code owner May 28, 2026 00:42

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/nemo_relay/integrations/langchain/_serialization.py`:
- Around line 240-249: The code aliases the incoming model_settings dict causing
mutations to the original ModelRequest when headers are merged; in
model_request_to_payload() make a copy of model_settings before assigning to
overrides (e.g., overrides["model_settings"] = model_settings.copy() or deepcopy
for nested structures) and also create extra_headers as a copy (extra_headers =
overrides["model_settings"].get("extra_headers", {}).copy()) before updating
with llm_request.headers so you never modify llm_request or its model_settings
in-place.
- Around line 107-141: _langchain_message_to_annotated flattens and discards
LangChain structured/multimodal content blocks (and
_annotated_message_to_langchain only reconstructs simple fields), causing loss
of content_blocks/typed metadata; change both functions to preserve and
round-trip BaseMessage.content_blocks (and any non-text block fields) by
encoding the entire message.data or content_blocks into the annotated dict
(e.g., include a "data" or "content_blocks" entry when present) and when
rebuilding in _annotated_message_to_langchain use that field to populate
BaseMessage.data/content_blocks instead of only
role/content/name/tool_calls/tool_call_id so that structured blocks (type,
image_url, etc.) survive the serialization round-trip.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 2cb62b6d-d00c-4b49-8b98-aaf4c6cf66e1

📥 Commits

Reviewing files that changed from the base of the PR and between 7547fab and 3d59365.

📒 Files selected for processing (9)
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
  • docs/integrate-into-frameworks/provider-codecs.mdx
  • python/nemo_relay/__init__.py
  • python/nemo_relay/integrations/langchain/_serialization.py
  • python/nemo_relay/integrations/langchain/middleware.py
  • python/nemo_relay/intercepts.py
  • python/tests/integrations/deepagents_tests/test_deepagents_integration.py
  • python/tests/integrations/langchain_tests/test_middleware.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (22)
{docs/**,README.md,CONTRIBUTING.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

{docs/**,README.md,CONTRIBUTING.md}: For docs-only changes, run targeted checks only if commands, package names, or examples changed. Use just docs for docs-site builds and just docs-linkcheck when links changed
Run docs site build with just docs

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
{docs/**,README.md,CONTRIBUTING.md,**/*.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Run docs link validation with just docs-linkcheck when links change

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
{docs/**,README.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Verify README and docs entry points still match current package names and paths for large or public-facing changes

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
{docs/**,examples/**,README.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Verify examples still run with documented commands for large or public-facing changes

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
{docs/**,README.md,**/Cargo.toml,**/package.json,**/*.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Ensure renamed public surfaces are reflected consistently in manifests and docs for large or public-facing changes

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
**/*.{md,mdx,py,sh,yaml,yml,toml,json}

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

Keep package names, repo references, and build commands current

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • python/nemo_relay/__init__.py
  • docs/build-plugins/register-behavior.mdx
  • python/nemo_relay/intercepts.py
  • python/tests/integrations/deepagents_tests/test_deepagents_integration.py
  • python/nemo_relay/integrations/langchain/middleware.py
  • python/tests/integrations/langchain_tests/test_middleware.py
  • python/nemo_relay/integrations/langchain/_serialization.py
**/*.mdx

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

In MDX files, top-of-file comments must use JSX comment delimiters: {/* to open and */} to close. Do not use HTML comments for MDX SPDX headers.

MDX top-of-file SPDX comments must use {/* ... */} delimiters instead of HTML comment delimiters (Must-Fix)

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
**/*.{html,md,mdx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Include SPDX license header in HTML and Markdown files using HTML comment syntax

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
docs/**/*.{md,mdx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Update embedded documentation snippets, patch docs, and binding-support notes if examples or supported bindings changed

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
docs/**

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Run just docs or ./scripts/build-docs.sh html to regenerate ignored Fern API reference pages before validation for documentation site changes

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.

Files:

  • docs/integrate-into-frameworks/provider-codecs.mdx
  • docs/build-plugins/code-examples.mdx
  • docs/build-plugins/register-behavior.mdx
{crates/python/src/py_api/**/*.rs,python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi}

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

Update Python native binding in crates/python/src/py_api/mod.rs with Python wrapper docstring in python/nemo_relay/<module>.py and type stubs in python/nemo_relay/*.pyi modules

Files:

  • python/nemo_relay/__init__.py
  • python/nemo_relay/intercepts.py
  • python/nemo_relay/integrations/langchain/middleware.py
  • python/nemo_relay/integrations/langchain/_serialization.py
python/nemo_relay/**/*.py

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

Use snake_case naming convention for Python identifiers (e.g., nemo_relay.tools.call)

Format changed Python wrapper and test files with uv run ruff format python

Files:

  • python/nemo_relay/__init__.py
  • python/nemo_relay/intercepts.py
  • python/nemo_relay/integrations/langchain/middleware.py
  • python/nemo_relay/integrations/langchain/_serialization.py
{pyproject.toml,**/*.py}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Maintain consistency between Python package names in pyproject.toml and import paths used throughout the codebase

Files:

  • python/nemo_relay/__init__.py
  • python/nemo_relay/intercepts.py
  • python/tests/integrations/deepagents_tests/test_deepagents_integration.py
  • python/nemo_relay/integrations/langchain/middleware.py
  • python/tests/integrations/langchain_tests/test_middleware.py
  • python/nemo_relay/integrations/langchain/_serialization.py
**/*.{py,txt,toml,cfg,yaml,yml}

📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)

Update Python package names and top-level module imports during coordinated rename operations

Files:

  • python/nemo_relay/__init__.py
  • python/nemo_relay/intercepts.py
  • python/tests/integrations/deepagents_tests/test_deepagents_integration.py
  • python/nemo_relay/integrations/langchain/middleware.py
  • python/tests/integrations/langchain_tests/test_middleware.py
  • python/nemo_relay/integrations/langchain/_serialization.py
**/*.py

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

**/*.py: Run Python formatting with uv run ruff format python
Run Python testing with uv run pytest -k "<pattern>"

**/*.py: Keep SPDX headers on Python source files. The project is Apache-2.0.
Use snake_case for Python binding naming conventions.
Run just test-python for Python binding or wrapper changes.

**/*.py: Use Ruff with rule sets E, F, W, I for Python linting
Use Ruff formatter with line length 120 and double quotes for Python code formatting
Run ty for Python type checking
Use Python snake_case naming convention for Python identifiers
Include SPDX license header in all Python source files using hash comment syntax
Validate Python code with uv run pre-commit run --all-files to enforce Ruff linting and formatting, and ty type checking

Files:

  • python/nemo_relay/__init__.py
  • python/nemo_relay/intercepts.py
  • python/tests/integrations/deepagents_tests/test_deepagents_integration.py
  • python/nemo_relay/integrations/langchain/middleware.py
  • python/tests/integrations/langchain_tests/test_middleware.py
  • python/nemo_relay/integrations/langchain/_serialization.py
python/nemo_relay/**/*

⚙️ CodeRabbit configuration file

python/nemo_relay/**/*: Review Python wrapper changes for typed API consistency, contextvars-based scope isolation, async behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.

Files:

  • python/nemo_relay/__init__.py
  • python/nemo_relay/intercepts.py
  • python/nemo_relay/integrations/langchain/middleware.py
  • python/nemo_relay/integrations/langchain/_serialization.py
**/test_*.{py,py}

📄 CodeRabbit inference engine (.agents/skills/add-integration/SKILL.md)

Relevant integration tests or smoke coverage must exist for the integration path

Files:

  • python/tests/integrations/deepagents_tests/test_deepagents_integration.py
  • python/tests/integrations/langchain_tests/test_middleware.py
{crates/adaptive/**/*.rs,**/*test*.{rs,py,go,ts,js},**/*adaptive*test*.{rs,py,go,ts,js},docs/plugins/adaptive/**}

📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)

Maintain documented and tested validation and report behavior for adaptive surfaces

Files:

  • python/tests/integrations/deepagents_tests/test_deepagents_integration.py
  • python/tests/integrations/langchain_tests/test_middleware.py
python/**/*test*.py

📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)

python/**/*test*.py: Do not add @pytest.mark.asyncio to any test in Python test files
Do not add a -> None return type annotation to test functions
When mocking a class, use unittest.mock.MagicMock or unittest.mock.AsyncMock with the spec constructor argument when necessary, rather than defining a new class
Prefix mocked class names with mock, not fake
Prefer pytest fixtures over helper methods in Python tests
Prefer pytest.mark.parametrize over creating individual tests for different input types

Files:

  • python/tests/integrations/deepagents_tests/test_deepagents_integration.py
  • python/tests/integrations/langchain_tests/test_middleware.py
python/**/{conftest.py,*test*.py}

📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)

When creating a fixture follow the pattern: @pytest.fixture(name="<fixture_name>"[, scope="<scope>"]) def <fixture_name>_fixture() -> <return_type>: and only specify the scope argument when the value is something other than "function"

Files:

  • python/tests/integrations/deepagents_tests/test_deepagents_integration.py
  • python/tests/integrations/langchain_tests/test_middleware.py
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}

⚙️ CodeRabbit configuration file

{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.

Files:

  • python/tests/integrations/deepagents_tests/test_deepagents_integration.py
  • python/tests/integrations/langchain_tests/test_middleware.py
🪛 Ruff (0.15.14)
python/nemo_relay/integrations/langchain/middleware.py

[warning] 86-86: Dynamically typed expressions (typing.Any) are disallowed in _call

(ANN401)


[warning] 109-109: Dynamically typed expressions (typing.Any) are disallowed in _call

(ANN401)

python/tests/integrations/langchain_tests/test_middleware.py

[warning] 249-249: Boolean-typed positional argument in function definition

(FBT001)


[warning] 259-259: Missing return type annotation for private function change_request

(ANN202)


[warning] 259-259: Dynamically typed expressions (typing.Any) are disallowed in annotated

(ANN401)


[warning] 274-274: Boolean positional value in function call

(FBT003)

python/nemo_relay/integrations/langchain/_serialization.py

[warning] 69-69: Dynamically typed expressions (typing.Any) are disallowed in tool_calls

(ANN401)


[warning] 127-127: Prefer TypeError exception for invalid type

(TRY004)


[warning] 127-127: Avoid specifying long messages outside the exception class

(TRY003)


[warning] 158-158: Avoid specifying long messages outside the exception class

(TRY003)

🔇 Additional comments (7)
python/tests/integrations/deepagents_tests/test_deepagents_integration.py (1)

237-237: LGTM!

docs/build-plugins/code-examples.mdx (1)

16-17: LGTM!

Also applies to: 35-37

docs/build-plugins/register-behavior.mdx (1)

125-127: LGTM!

docs/integrate-into-frameworks/provider-codecs.mdx (1)

92-94: LGTM!

python/nemo_relay/__init__.py (1)

41-43: LGTM!

python/nemo_relay/intercepts.py (1)

14-16: LGTM!

Also applies to: 182-184

python/nemo_relay/integrations/langchain/_serialization.py (1)

182-185: ⚡ Quick win

Guard annotated.extra before merging it into the payload.

payload.update(annotated.extra) will throw TypeError if annotated.extra is None (e.g., dict.update(None) raises TypeError: 'NoneType' object is not iterable), so add a guard before updating.

Suggested fix
     def encode(self, annotated: AnnotatedLLMRequest, original: LLMRequest) -> LLMRequest:
         """Encode annotated request edits back into a LangChain-shaped payload."""
         payload = dict(original.content)
-        payload.update(annotated.extra)
+        if annotated.extra:
+            payload.update(annotated.extra)
         payload["messages"] = messages_to_dict(
             [self._annotated_message_to_langchain(message) for message in annotated.messages]
         )

Comment thread python/nemo_relay/integrations/langchain/_serialization.py
Comment thread python/nemo_relay/integrations/langchain/_serialization.py
willkill07
willkill07 previously approved these changes May 28, 2026

@willkill07 willkill07 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coderabbit feedback seems sensible to address.

Otherwise, I checked out the branch and tested locally!

Conditionally approving. Due to low risk, I still think this can target 0.3

Signed-off-by: David Gardner <dagardner@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
python/nemo_relay/integrations/langchain/_serialization.py (1)

242-254: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Copy extra_headers before mutating it.

dict(model_settings) is only a shallow copy. If Line 244 returns an existing dict, Line 253 still updates the same nested extra_headers object held by original.model_settings, so interceptor header injection is still mutating the original request.

Suggested fix
     if isinstance(model_settings, dict):
         # Using dict() to ensure we have a copy
         model_settings_copy = dict(model_settings)
-        extra_headers = model_settings_copy.get("extra_headers")
-        if not isinstance(extra_headers, dict):
-            extra_headers = {}
+        extra_headers_value = model_settings_copy.get("extra_headers")
+        extra_headers = dict(extra_headers_value) if isinstance(extra_headers_value, dict) else {}
+        model_settings_copy["extra_headers"] = extra_headers
         overrides["model_settings"] = model_settings_copy
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/nemo_relay/integrations/langchain/_serialization.py` around lines 242
- 254, The code currently shallow-copies model_settings into model_settings_copy
but then mutates the nested extra_headers in place; to avoid mutating the
original request, make a defensive copy of extra_headers when it's a dict (e.g.,
replace extra_headers = extra_headers with extra_headers = dict(extra_headers))
before calling extra_headers.update(llm_request.headers), and then assign
overrides["model_settings"]["extra_headers"] = extra_headers; update logic in
the block that sets model_settings_copy, extra_headers, and the final merge with
llm_request.headers to ensure only the copied objects are mutated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@python/nemo_relay/integrations/langchain/_serialization.py`:
- Around line 242-254: The code currently shallow-copies model_settings into
model_settings_copy but then mutates the nested extra_headers in place; to avoid
mutating the original request, make a defensive copy of extra_headers when it's
a dict (e.g., replace extra_headers = extra_headers with extra_headers =
dict(extra_headers)) before calling extra_headers.update(llm_request.headers),
and then assign overrides["model_settings"]["extra_headers"] = extra_headers;
update logic in the block that sets model_settings_copy, extra_headers, and the
final merge with llm_request.headers to ensure only the copied objects are
mutated.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 0e338ef5-3294-4df1-bcdb-ec22ae54299d

📥 Commits

Reviewing files that changed from the base of the PR and between 3d59365 and 0d5b04a.

📒 Files selected for processing (1)
  • python/nemo_relay/integrations/langchain/_serialization.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (7)
{crates/python/src/py_api/**/*.rs,python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi}

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

Update Python native binding in crates/python/src/py_api/mod.rs with Python wrapper docstring in python/nemo_relay/<module>.py and type stubs in python/nemo_relay/*.pyi modules

Files:

  • python/nemo_relay/integrations/langchain/_serialization.py
python/nemo_relay/**/*.py

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

Use snake_case naming convention for Python identifiers (e.g., nemo_relay.tools.call)

Format changed Python wrapper and test files with uv run ruff format python

Files:

  • python/nemo_relay/integrations/langchain/_serialization.py
{pyproject.toml,**/*.py}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Maintain consistency between Python package names in pyproject.toml and import paths used throughout the codebase

Files:

  • python/nemo_relay/integrations/langchain/_serialization.py
**/*.{py,txt,toml,cfg,yaml,yml}

📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)

Update Python package names and top-level module imports during coordinated rename operations

Files:

  • python/nemo_relay/integrations/langchain/_serialization.py
**/*.py

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

**/*.py: Run Python formatting with uv run ruff format python
Run Python testing with uv run pytest -k "<pattern>"

**/*.py: Keep SPDX headers on Python source files. The project is Apache-2.0.
Use snake_case for Python binding naming conventions.
Run just test-python for Python binding or wrapper changes.

**/*.py: Use Ruff with rule sets E, F, W, I for Python linting
Use Ruff formatter with line length 120 and double quotes for Python code formatting
Run ty for Python type checking
Use Python snake_case naming convention for Python identifiers
Include SPDX license header in all Python source files using hash comment syntax
Validate Python code with uv run pre-commit run --all-files to enforce Ruff linting and formatting, and ty type checking

Files:

  • python/nemo_relay/integrations/langchain/_serialization.py
**/*.{md,mdx,py,sh,yaml,yml,toml,json}

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

Keep package names, repo references, and build commands current

Files:

  • python/nemo_relay/integrations/langchain/_serialization.py
python/nemo_relay/**/*

⚙️ CodeRabbit configuration file

python/nemo_relay/**/*: Review Python wrapper changes for typed API consistency, contextvars-based scope isolation, async behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.

Files:

  • python/nemo_relay/integrations/langchain/_serialization.py

@dagardner-nv

Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot
rapids-bot Bot merged commit a3c552b into NVIDIA:main May 28, 2026
34 checks passed
@dagardner-nv
dagardner-nv deleted the david-03-fix-LC-serialization branch May 29, 2026 14:56
@dagardner-nv
dagardner-nv restored the david-03-fix-LC-serialization branch June 3, 2026 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug issue describes bug; PR fixes bug lang:python PR changes/introduces Python code size:M PR is medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants