[None][refactor] Organize SMG gRPC adapter by protocol

[QiJune]

Dev Engineer Review

• The refactor moves SMG gRPC code under tensorrt_llm.grpc.smg.
• serve.py now delegates gRPC startup to launch_smg_server.
• Review should verify backend selection, signal handling, graceful shutdown, and API compatibility.
• No configuration files or test-list files were changed.

QA Engineer Review

• Modified tests/unittest/grpc/smg/test_smg.py.
• Updated imports and package paths only.
• No test functions were added, modified, or removed.
• The test suite includes sampling, request conversion, validation, service, and multimodal end-to-end coverage.
• No test-db/ or qa/ coverage entry was identified.
• Verdict: needs follow-up because CBTS coverage cannot be verified.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[QiJune]

/bot run

[coderabbitai]

Walkthrough

Changes

SMG gRPC integration

Layer / File(s)	Summary
SMG adapter bindings and imports tensorrt_llm/grpc/..., tensorrt_llm/grpc/smg/..., tests/unittest/grpc/smg/test_smg.py	The SMG package adds protobuf bindings and updates adapter and test imports to use the new package paths.
SMG server lifecycle tensorrt_llm/grpc/smg/server.py	launch_smg_server constructs the selected backend, starts the gRPC server, enables reflection when available, and performs graceful shutdown.
Serve command wiring tensorrt_llm/commands/serve.py	The gRPC command invokes launch_smg_server with the existing host, port, LLM arguments, and served model name.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Suggested reviewers: juney-nvidia

Sequence Diagram(s)

sequenceDiagram
  participant ServeCommand
  participant launch_smg_server
  participant RequestManager
  participant SMGServicer
  participant grpcServer
  participant LLMEngine
  ServeCommand->>launch_smg_server: Pass server configuration
  launch_smg_server->>LLMEngine: Construct selected backend
  launch_smg_server->>RequestManager: Create request manager
  launch_smg_server->>SMGServicer: Register servicer
  launch_smg_server->>grpcServer: Bind and start server
  grpcServer-->>launch_smg_server: Signal shutdown
  launch_smg_server->>grpcServer: Stop gracefully
  launch_smg_server->>LLMEngine: Shut down engine

Loading

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description contains only the template; the issue, solution, test coverage, and checklist review are not completed.	Add a concise issue and solution summary, list relevant tests, and complete the PR checklist.
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly describes the protocol-based organization refactor and follows the repository title format.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

tensorrt_llm/grpc/smg/server.py (1)

117-120: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Dead except KeyboardInterrupt branch.

loop.add_signal_handler(signal.SIGINT, signal_handler) at Line 115 replaces the default SIGINT handling for this event loop. Once registered, SIGINT no longer raises KeyboardInterrupt inside the coroutine; it invokes signal_handler, which sets stop_event. The except KeyboardInterrupt at Line 119 therefore does not trigger during normal operation after Line 115 runs.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/grpc/smg/server.py` around lines 117 - 120, Remove the
unreachable KeyboardInterrupt handler around stop_event.wait in the server
startup flow. Keep loop.add_signal_handler(signal.SIGINT, signal_handler) and
the existing stop_event-based shutdown behavior unchanged.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/grpc/smg/server.py`:
- Around line 62-129: Widen the cleanup scope in the server lifecycle so the
existing shutdown logic covering llm.shutdown() also handles failures from
server.add_insecure_port(), server.start(), and subsequent startup or serving
operations after LLM construction. Ensure cleanup runs exactly once for any
post-construction exception while preserving the current graceful server.stop()
behavior.

---

Nitpick comments:
In `@tensorrt_llm/grpc/smg/server.py`:
- Around line 117-120: Remove the unreachable KeyboardInterrupt handler around
stop_event.wait in the server startup flow. Keep
loop.add_signal_handler(signal.SIGINT, signal_handler) and the existing
stop_event-based shutdown behavior unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 822a751f-5f58-42a0-9edd-1f2f2d73f128

📥 Commits

Reviewing files that changed from the base of the PR and between bf1ddb7 and fd9c25e.

📒 Files selected for processing (8)

• tensorrt_llm/commands/serve.py
• tensorrt_llm/grpc/__init__.py
• tensorrt_llm/grpc/smg/__init__.py
• tensorrt_llm/grpc/smg/bindings.py
• tensorrt_llm/grpc/smg/request_manager.py
• tensorrt_llm/grpc/smg/server.py
• tensorrt_llm/grpc/smg/servicer.py
• tests/unittest/grpc/smg/test_smg.py

[tensorrt-cicd]

PR_Github #63353 [ run ] triggered by Bot. Commit: fd9c25e Link to invocation

[BowenFu]

Reviewed the move itself: launch_smg_server is semantically identical to the old launch_grpc_server — message limits, keepalive values, signal handling, stop(grace=5.0) and the shutdown ordering are all unchanged — and nothing left in the tree references the old module paths. Holding only until blossom reports; I'd rather not be the first approver on a package relocation with no pipeline result yet.

Three things worth settling before merge:

1. tests/unittest/grpc/smg/test_smg.py isn't referenced in any test-db/ or jenkins/ list under either its old or new path, so the adapter's entire suite is dead in CI. The reorg is a good moment to register it — otherwise the reshuffle is unverified by anything.

2. tensorrt_llm/grpc/__init__.py drops GrpcRequestManager, TrtllmServiceServicer, trtllm_service_pb2 and the create_*_from_proto helpers with no shim, and the old module paths go away too. Nothing in-repo uses them and the package is only about five weeks old and undocumented, so I'm not blocking on it — but a one-release alias would be cheap if any router-side code pins the old paths.

3. Carried over from the old code but newly written into server.py: the finally only wraps await stop_event.wait(), so a failure in add_insecure_port() or server.start() (port already bound, for instance) leaves the loaded LLM engine unshut. Cheap to fix while the file is new.

The *_AVAILABLE flags being deleted is a straight improvement — smg-grpc-proto is a hard requirement in requirements.txt, so they were always True, and the old None fallbacks only turned a missing dependency into an AttributeError later.

[tensorrt-cicd]

PR_Github #63353 [ run ] completed with state FAILURE. Commit: fd9c25e
/LLM/main/L0_MergeRequest_PR pipeline #51341 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[QiJune]

/bot run

[tensorrt-cicd]

PR_Github #63365 [ run ] triggered by Bot. Commit: fd9c25e Link to invocation

[tensorrt-cicd]

PR_Github #63365 [ run ] completed with state FAILURE. Commit: fd9c25e
/LLM/main/L0_MergeRequest_PR pipeline #51350 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[BowenFu]

On the two red runs — neither looks like your refactor.

Build 51350 (#63365) ran 27211 passing tests, 0 test-level failures; the only FAILED entries are Stage Failed on RTX5080-PyTorch-1 and RTXPro6000D-PyTorch-1, with no test attributed to either. On top of that, every pipeline on this base is red on Check Test List: waives.txt:106-107 on main (24d0a8e1c, merged 03:15Z) waive cache_transceiver_precheck[ctx_0]/[gen_0], which are synthetic JUnit case names from jenkins/scripts/perf/disaggregated/slurm_ct_precheck_gate.sh, not pytest IDs in any l0/qa list — so the validator rejects them. #16432, #17121 and #17175 are all failing the same way right now.

That is also why I wouldn't read the earlier build 51341 as signal either — it only got 81 tests in before dying.

Nothing new from me on the diff itself; my earlier notes stand (the tensorrt_llm/grpc/__init__.py export removal with no shim, and tests/unittest/grpc/smg/test_smg.py still being in no test-db or jenkins list under either path). Once the base is unblocked and a clean pipeline lands, this is otherwise ready from my side.