[None][fix] propagate chat prompt token ids (#14420)

[reasonsolo]

Summary by CodeRabbit

• Bug Fixes
  • Context-only chat completions now correctly include prompt token IDs in responses, ensuring complete token information is available for these request types
  • Streaming responses improved to properly emit token ID information independently, allowing token tracking even when text content deltas are not immediately available

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[reasonsolo]

/bot run --disable-fail-fast

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 05da203a-4550-4f66-b958-6e43c4482d5f

📥 Commits

Reviewing files that changed from the base of the PR and between 209f371 and b41cfae.

📒 Files selected for processing (2)

• tensorrt_llm/serve/openai_server.py
• tensorrt_llm/serve/postprocess_handlers.py

---

📝 Walkthrough

Walkthrough

The PR adds two targeted improvements to disaggregated token handling in the TensorRT-LLM serving layer. It ensures context-only responses include prompt token IDs and extends streaming logic to emit token-bearing updates independently of text content.

Changes

Disaggregated Inference Token Handling

Layer / File(s)	Summary
Context-only prompt token IDs tensorrt_llm/serve/openai_server.py	For context-only disaggregated requests, chat_response.prompt_token_ids is populated from promise.prompt_token_ids before disaggregated-params validation, ensuring token IDs are included in the response.
Streaming token delta emission tensorrt_llm/serve/postprocess_handlers.py	The streaming post-processor detects token-id deltas (has_token_delta) and extends the delta-chunk emission condition to send output when tokens are present, even without detokenized text/reasoning content.

---

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description is blank; it only contains the template with all sections empty (Description, Test Coverage) and the checklist unchecked.	Fill in the Description section explaining the issue and solution, and the Test Coverage section listing relevant tests that validate the changes.
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: propagating chat prompt token IDs in the OpenAI server implementation.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[tensorrt-cicd]

PR_Github #51571 [ run ] triggered by Bot. Commit: b41cfae Link to invocation

[tensorrt-cicd]

PR_Github #51571 [ run ] completed with state SUCCESS. Commit: b41cfae
/LLM/main/L0_MergeRequest_PR pipeline #40962 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[reasonsolo]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #51789 [ run ] triggered by Bot. Commit: b41cfae Link to invocation

[tensorrt-cicd]

PR_Github #51789 [ run ] completed with state SUCCESS. Commit: b41cfae
/LLM/main/L0_MergeRequest_PR pipeline #41152 completed with status: 'SUCCESS'

CI Report

Link to invocation