feat: Support Nemotron-nano-v3 Omni AutoModel Path #2362
Conversation
Signed-off-by: Yuekai Zhang <yuekaiz@cw-dfw-cs-001-vscode-02.cm.cluster> Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Add .claude/settings.local.json, .codex, and .humanize/ to .gitignore as these are local tool configuration/cache files that should not be tracked. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
- Add MMPRTinyDataset class with HF download fallback and local cache support
- Add format_mmpr_tiny_dataset for OpenAI-API message conversion
- Port verl_geo3k reward function from old Megatron-Bridge implementation
- Register mmpr-tiny in DATASET_REGISTRY and vlm_hf_data_processor
- Register verl_geo3k reward in VLMVerifyWorker
- Add mathruler dependency to pyproject.toml for answer grading
- Create debug-friendly YAML config with step_400 checkpoint
- Create launch script with uv sync pre-step
- Create CoT prompt file with \boxed{} instruction
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
- Remove self.preprocessor from MMPRTinyDataset to prevent double formatting (vlm_hf_data_processor already handles format dispatch) - Add pylatexenc to pyproject.toml as transitive dependency of mathruler (mathruler does not declare it in its own metadata) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
- Fix prompt file brace escaping: \boxed{} -> \boxed{{}} so
str.format() has exactly one replacement placeholder (fixes AC-2/AC-5)
- Add explicit split parameter validation to MMPRTinyDataset with
ValueError for unsupported splits (fixes AC-1 negative test)
- Regenerate uv.lock with mathruler and pylatexenc entries (fixes AC-6)
- Add unit tests for dataset formatting, split validation, prompt
file format compatibility, and verl_geo3k_reward (14 tests, all pass)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Add 3 processor smoke tests using a stub NemotronNanoVLV2Processor:
- test_processor_produces_valid_datum_spec: verifies DatumSpec fields
- test_prompted_text_contains_boxed_literal: verifies \boxed{} survives
- test_placeholder_conversion_for_nemotron_processor: verifies <image>
placeholder and question text in vllm_content
Uses a tiny 1x1 PNG fixture for image resolution. All 17 MMPR tests
pass (11 dataset + 6 reward).
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
- Stub now captures the exact text arg passed to __call__ - Assert exact equality: vllm_content == "<image>\n" + prompted_question - Assert exactly one <image> token in output (no duplicates) - Negative assertion: raw dataset string "<image>\nQuestion" not in output - Assert captured __call__ text matches expected tokenizer input - All tests run with -p no:testmon --override-ini='addopts=' Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Clean up partial state (stale images_dir, parquet, temp dir) before re-downloading when the ready marker is absent. Prevents shutil.move from nesting images/images when images_dir already exists from an interrupted prior attempt. Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
MMPR-Tiny has 4,192 rows with multiple images (up to 11). The previous code truncated to images[0], losing visual context for multi-image questions. - _load_mmpr_tiny_from_cache: keep all image paths instead of [imgs[0]] - format_mmpr_tiny_dataset: split question on <image> and <image_N> placeholders, interleave image content items with text segments - Add tests for multi-image and numbered-placeholder formatting Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: root <zhangyuekai@foxmail.com> Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: root <zhangyuekai@foxmail.com> Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
…nting Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
…sound_projection to resume training Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
|
/ok to test 4177252 |
❌ Submodule Fast-Forward Check FailedCheck based on commit: 4177252 (PR #2362 from ❌ Submodules that need attention:Automodel: ❌ Commits have DIVERGED from a common ancestor Please ensure all submodule commits are fast-forwards of the main branch before merging. |
|
@jinglinglingling Hi Linglin, could you please review the PR again? Thanks! |
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
❌ Submodule Fast-Forward Check FailedCheck based on commit: 2a49359 (PR #2362 from ❌ Submodules that need attention:Automodel: ❌ Commits have DIVERGED from a common ancestor Please ensure all submodule commits are fast-forwards of the main branch before merging. |
Resolve the unresolved merge conflict between the nemotron (1970h) and main (2000h) branches by setting the nightly compute budget to 2070 hours. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com>
|
/ok to test 86c1c1f |
❌ Submodule Fast-Forward Check FailedCheck based on commit: 86c1c1f (PR #2362 from ❌ Submodules that need attention:Automodel: ❌ Commits have DIVERGED from a common ancestor Please ensure all submodule commits are fast-forwards of the main branch before merging. |
yuki-97
left a comment
There was a problem hiding this comment.
thanks @yuekaizhang , overall LGTM and left some minor comments.
Rename the reward function and its "verl_geo3k" config registration name to "geo3k", and note in the docstring that it has the same behavior as verl's geo3k reward function. Updates the function definition, the VLM environment dispatch/import, the unit test (file + call sites), the exemplar/recipe YAMLs, and the docs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: root <zhangyuekai@foxmail.com>
❌ Submodule Fast-Forward Check FailedCheck based on commit: 15cc685 (PR #2362 from ❌ Submodules that need attention:Automodel: ❌ Commits have DIVERGED from a common ancestor Please ensure all submodule commits are fast-forwards of the main branch before merging. |
|
/ok to test 15cc685 |
|
/ok to test c4f6247 |
❌ Submodule Fast-Forward Check FailedCheck based on commit: c4f6247 (PR #2362 from ❌ Submodules that need attention:Automodel: ❌ Commits have DIVERGED from a common ancestor Please ensure all submodule commits are fast-forwards of the main branch before merging. |
|
/ok to test 04c62f0 |
❌ Submodule Fast-Forward Check FailedCheck based on commit: 04c62f0 (PR #2362 from ❌ Submodules that need attention:Automodel: ❌ Commits have DIVERGED from a common ancestor Please ensure all submodule commits are fast-forwards of the main branch before merging. |
…y recipes Move and rename the Nemotron-Omni VLM GRPO configs into examples/configs/recipes/vlm/ following the VLM recipe naming convention, and run both as functional nightly tests: - vlm_grpo-nemotron-omni-30ba3b-clevr-1n8g-automodel-ep8.v1 (CLEVR-CoGenT, 1 node) - vlm_grpo-nemotron-omni-30ba3b-mmpr-4n8g-automodel-ep8.v1 (MMPR-Tiny, 4 nodes) Both configs are minimized snapshots that inherit from vlm_grpo_3B.yaml; the MMPR config carries cluster.num_nodes=4. The previous thin wrapper recipe (same mmpr name) is replaced by the full config under that name, and the nightly now points at these recipes directly via matching driver scripts. Nightly GPU-hours total stays at 2041h, under the 2070h budget. Update the Nemotron-Omni guide to the new config paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: root <zhangyuekai@foxmail.com>
❌ Submodule Fast-Forward Check FailedCheck based on commit: bc85d5f (PR #2362 from ❌ Submodules that need attention:Automodel: ❌ Commits have DIVERGED from a common ancestor Please ensure all submodule commits are fast-forwards of the main branch before merging. |
|
/ok to test bc85d5f |
terrykong
left a comment
There was a problem hiding this comment.
PR #2362 — feat: Support Nemotron-nano-v3 Omni AutoModel Path
Nice work — this adds Nemotron-Nano-Omni 30B-A3B AutoModel recipes (CLEVR-CoGenT + MMPR-Tiny), the MMPR-Tiny loader, a geo3k reward, docs, and nightly tests. Reviewed by a team of agents (RL/config, Automodel SME, tests, bug-finder) plus an adversarial pass.
No blocking bugs found. Every bug candidate was verified and refuted: the geo3k brace parser is correct (validated with nested/multi-last/no-close/empty cases), the MMPR download-cache lock is safe, the placeholder <image> token count matches len(images), and AutoConfig.from_pretrained in ModelFlag.matches() runs only at worker init (not a hot path). Convergence curves vs. Megatron-Bridge are present in the comments, and the nightly GPU-hour bump (2000→2070) is justified (actual total ≈2041h).
The inline comments are all suggestions/nits — the most substantive is migrating off the deprecated enable_deepep knob (Automodel upstream did the same for its own Nemotron-Omni recipes in this bump). Existing unit tests all pass locally (geo3k 6/6, mmpr_tiny 10/10).
Generated by Claude Code
- Migrate off deprecated enable_deepep knob to dispatcher=deepep + experts=torch_mm in both CLEVR and MMPR recipes (avoids grouped_gemm dep) - Remove leftover NRL_DEBUG_PICKLE debug env var from MMPR recipe - Fix docs max_total_sequence_length 10000 -> 8192 to match MMPR recipe - Update mmpr_tiny.py copyright year to 2026 - Add geo3k tests for last-wins multi-boxed and nested/unbalanced braces - Declare experts/dispatcher and freeze_config in Automodel TypedDicts Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: root <zhangyuekai@foxmail.com>
# Conflicts: # tests/unit/test_recipes_and_test_suites.py
❌ Submodule Fast-Forward Check FailedCheck based on commit: a3ca738 (PR #2362 from ❌ Submodules that need attention:Automodel: ❌ Commits have DIVERGED from a common ancestor Please ensure all submodule commits are fast-forwards of the main branch before merging. |
|
/ok to test a3ca738 |
|
Doc cross-check for Two non-blocking doc nits from the POR run:
Neither is a blocker for the PR. |
POR Test Summary — Nemotron-3-Nano-Omni-30B VLM GRPO (PR #2362)PR under test: NVIDIA-NeMo/RL #2362 — Nemotron-Nano-v3 Omni AutoModel VLM GRPO support Model & Backend Under Test
What the Tests DoBoth drivers are a 1:1 mirror of the PR's own oracle scripts
Function Tests1. CLEVR — function tier (
|
| Field | Value |
|---|---|
| Task | CLEVR-CoGenT |
| Scale | 1 node × 8 GPU |
| Execution | Interactive |
| Timeout | 7200 s |
| Recipe | vlm_grpo-nemotron-omni-30ba3b-clevr-1n8g-automodel-ep8.v1.yaml |
Result: REAL GREEN — run 5532636, 10 steps, reward 0.83 (>0.5), val acc 0.713
2. MMPR-tiny — e2e tier (testcases/e2e/)
test_vlm_grpo_nemotron_omni_30ba3b_mmpr_4n8g_automodel_ep8.sh
| Field | Value |
|---|---|
| Task | MMPR-tiny (geo3k / MMPR reward, vlm_environment path) |
| Scale | 4 nodes × 8 GPU = 32 GPU |
| Execution | sbatch via ray.sub (batched-job model) |
| Recipe | vlm_grpo-nemotron-omni-30ba3b-mmpr-4n8g-automodel-ep8.v1.yaml |
Result: REAL GREEN (verified ×2) — run 5533255, SLURM COMPLETED 0:0,
runner failures=0, reward 0.799 (>0.5), step 10/10, val acc 0.664
Coverage
Covers both VLM GRPO recipes that PR #2362 introduces, across two scales and two execution models:
- Single-node interactive (CLEVR) + multi-node batched
ray.sub(MMPR) — exercises the batched-job submission path the PR adds. - New mmpr-tiny dataset + geo3k / MMPR reward function +
vlm_environmentpath. - AutoModel VLM with frozen vision/audio towers, MoE EP=8 / TP=8 triton backend.
Tags: codecoverage, por_v06, current_por, rc_p0
Doc Check — docs/guides/nemotron-3-nano-omni.md
The recipes run green exactly as the PR ships them — no functional/run-breaking doc issue.
5 minor cosmetic/consistency doc nits were noted during the doc-vs-recipe re-check; no PR
comment or patch filed (per decision to leave doc fix out of scope).
Artifacts
| Test | Run ID | Cluster |
|---|---|---|
| CLEVR | 5532636 |
EOS |
| MMPR | 5533255 |
EOS, nodes eos[0548,0558-0559,0563] |
Remote logs base: /lustre/fsw/coreai_dlalgo_ci/qiaochuz/nemo_llm_nightly/test_suite/rl/logs/
Container: nemo-rl-nightly-20260629.sqsh
This PR follows the nano-v3-omni mbridge training branch to add AutoModel backend support for Nemotron-Nano-Omni.