Misc. bug: GLM 4.7 crash

[virtuald]

Name and Version

$ git log -1
commit 01d8eaa28d57bfc6d06e30072085ed0ef12e06c5 (HEAD -> master, origin/master, origin/HEAD)
Author: Anav Prasad <anavp@nvidia.com>
Date:   Sat Feb 14 05:07:00 2026 -0800

    mtmd : Add Nemotron Nano 12B v2 VL support (#19547)
    
    * nemotron nano v2 vlm support added
    
    * simplified code; addressed reviews
    
    * pre-downsample position embeddings during GGUF conversion for fixed input size

Compiled with:

cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON

Using configuration suggested at #19564

Thread 1 "llama-server" received signal SIGSEGV, Segmentation fault.
0x0000555555bc61eb in ggml_backend_cuda_context::stream (stream=0, device=1688275464, this=0x555564a109f0)
    at /disk3/dustin/llama.cpp/ggml/src/ggml-cuda/../ggml-cuda/common.cuh:1388
1388	        if (streams[device][stream] == nullptr) {
(gdb) bt
#0  0x0000555555bc61eb in ggml_backend_cuda_context::stream (stream=0, device=1688275464, this=0x555564a109f0)
    at /disk3/dustin/llama.cpp/ggml/src/ggml-cuda/../ggml-cuda/common.cuh:1388
#1  ggml_backend_cuda_context::stream (this=0x555564a109f0) at /disk3/dustin/llama.cpp/ggml/src/ggml-cuda/../ggml-cuda/common.cuh:1395
#2  ggml_backend_cuda_synchronize (backend=0x555564a11320) at /disk3/dustin/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:2857
#3  0x00005555563c9ac5 in ggml_backend_synchronize (backend=<optimized out>) at /disk3/dustin/llama.cpp/ggml/src/ggml-backend.cpp:334
#4  ggml_backend_sched_synchronize (sched=0x5555649790d0) at /disk3/dustin/llama.cpp/ggml/src/ggml-backend.cpp:1811
#5  0x00005555559d66f9 in llama_context::synchronize (this=this@entry=0x5555642ecff0) at /disk3/dustin/llama.cpp/src/llama-context.cpp:546
#6  0x00005555559d8b84 in llama_get_sampled_probs_ith (ctx=0x5555642ecff0, i=65) at /disk3/dustin/llama.cpp/src/llama-context.cpp:3167
#7  0x00005555558987c5 in common_sampler::set_logits (this=this@entry=0x55558f3bbbe0, ctx=ctx@entry=0x5555642ecff0, idx=idx@entry=65)
    at /disk3/dustin/llama.cpp/common/sampling.cpp:123
#8  0x000055555589604e in common_sampler_sample (gsmpl=0x55558f3bbbe0, ctx=0x5555642ecff0, idx=idx@entry=65, grammar_first=grammar_first@entry=false)
    at /disk3/dustin/llama.cpp/common/sampling.cpp:480
#9  0x0000555555715f31 in server_context_impl::update_slots (this=0x55555dd999b0) at /usr/include/c++/13/bits/unique_ptr.h:199
#10 0x000055555575b650 in std::function<void ()>::operator()() const (this=0x55555dd99af0) at /usr/include/c++/13/bits/std_function.h:591
#11 server_queue::start_loop (this=0x55555dd999c8, idle_sleep_ms=-1000) at /disk3/dustin/llama.cpp/tools/server/server-queue.cpp:163
#12 0x00005555556ea0bd in server_context::start_loop (this=<optimized out>) at /disk3/dustin/llama.cpp/tools/server/server-context.cpp:2867
#13 0x000055555561fa8f in main (argc=<optimized out>, argv=<optimized out>) at /disk3/dustin/llama.cpp/tools/server/server.cpp:305
(gdb) info locals
__func__ = <optimized out>
err_ = <optimized out>
(gdb) print streams
$1 = {{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {
    0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x555564979c50, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
    0x0, 0x0}}
(gdb) 

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

No response

Command line

#!/bin/bash

export CUDA_SCALE_LAUNCH_QUEUES=4x
export CUDA_VISIBLE_DEVICES=1,2,3,4,5,6,7

llama.cpp/build/bin/llama-server \
    --model /path/to/gguf/GLM-4.7-Flash-Q8_0.gguf --host 0.0.0.0 --port 8013 \
    --spec-type ngram-mod --spec-ngram-size-n 24 --draft-min 48 --draft-max 64 \
    -c 2838528 -np 14 -b 8192 -ub 256 \
    --top-p 1.0 --top-k 0 --temp 0.2 -bs \
    -cram 32768 \
    --chat-template-kwargs '{"enable_thinking": false}'

Problem description & steps to reproduce

Happens consistently if I connect three pi.dev agents via openai-completions and tell them to do something.

First Bad Commit

I don't think I have the time to bisect this right now.

Relevant log output

No response

[virtuald]

According to @ggerganov using -bs with speculative decoding will cause this crash. I removed the argument and it no longer crashes.