From a1209d161a2bec296171f2ea6569c90b33e7cd68 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Tue, 7 Jul 2026 12:46:33 +0000 Subject: [PATCH] fix(vllm): pin L4T arm64 backend to vllm==0.24.0 for GB10 stability The nvidia-l4t-cuda-13-arm64 vLLM backend left `vllm` unpinned, so the prebuilt image drifted onto whatever aarch64 wheel was latest at build time (0.23.x). On GB10 / DGX Spark (Grace Blackwell, unified memory), 0.23 crashes deterministically during cold model loads with an empty "Engine core initialization failed" set and pins GPU memory until a host reboot. vLLM 0.24.0 carries vllm-project/vllm#45179 ("release cached device memory under pressure on UMA GPUs during weight loading"), which the reporter verified fixes the crash on GB10. Pin the L4T requirements to 0.24.0 to match the already-pinned cublas13 build (requirements-cublas13-after.txt) and keep the image deterministic. Editing this file also re-triggers the single-arch L4T image build via the path filter, republishing the gallery image with 0.24.0 (the single-arch matrix builds again after #10703). Closes #10722 Signed-off-by: Ettore Di Giacinto Assisted-by: Claude:claude-opus-4-8 [Claude Code] --- backend/python/vllm/requirements-l4t13-after.txt | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/backend/python/vllm/requirements-l4t13-after.txt b/backend/python/vllm/requirements-l4t13-after.txt index c959c6ae03bb..31456efab599 100644 --- a/backend/python/vllm/requirements-l4t13-after.txt +++ b/backend/python/vllm/requirements-l4t13-after.txt @@ -1,4 +1,12 @@ # vLLM 0.20+ ships an aarch64 manylinux wheel on PyPI whose Requires-Dist pins # torch==2.11.0 / torchvision==0.26.0 / torchaudio==2.11.0, locking an ABI- # consistent set with the cu130 torch wheel installed above. -vllm +# +# Pinned to match the cublas13 build (requirements-cublas13-after.txt) and to +# stay deterministic on GB10 / DGX Spark: 0.24.0 carries vllm-project/vllm#45179 +# ("release cached device memory under pressure on UMA GPUs during weight +# loading"), without which cold model loads on the Grace Blackwell unified- +# memory architecture crash deterministically with an empty "Engine core init +# failed" set (mudler/LocalAI#10722). Leaving this unpinned let the L4T image +# drift onto whatever wheel was latest at build time. +vllm==0.24.0