Name and Version
|D:\a\llama.cpp\llama.cpp\ggml\src\ggml-cuda\ggml-cuda.cu:98: CUDA error
ggml_cuda_compute_forward: MUL_MAT_ID failed
PS G:\llmam\llama-b8611-bin-win-cuda-12.4-x64> .\llama-cli.exe --version
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 16310 MiB):
Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 16310 MiB
load_backend: loaded CUDA backend from G:\llmam\llama-b8611-bin-win-cuda-12.4-x64\ggml-cuda.dll
load_backend: loaded RPC backend from G:\llmam\llama-b8611-bin-win-cuda-12.4-x64\ggml-rpc.dll
load_backend: loaded CPU backend from G:\llmam\llama-b8611-bin-win-cuda-12.4-x64\ggml-cpu-alderlake.dll
version: 8611 (d43375f)
built with Clang 19.1.5 for Windows x86_64
Operating systems
Windows
Which llama.cpp modules do you know to be affected?
llama-cli, llama-server
Command line
.\llama-cli.exe -m G:\llmam\exported_models\GLM-4.7-Flash-Q3_K_S.gguf -c 81920 -ngl 48 --cache-type-k q4_0 --cache-type-v q4_0 --flash-attn on --top-k 40 --verbose
Problem description & steps to reproduce
parser generation prompt: <|assistant|>
res add_waiting_: add task 0 to waiting list. current waiting = 0 (before add)
que post: new task, id = 0, front = 0
|ggml_backend_cuda_graph_compute: CUDA graph warmup reset
ggml_cuda_compute_forward: MUL_MAT_ID failed
D:\a\llama.cpp\llama.cpp\ggml\src\ggml-cuda\ggml-cuda.cu:98: CUDA error
CUDA error: invalid argument
current device: 0, in function ggml_cuda_compute_forward at D:\a\llama.cpp\llama.cpp\ggml\src\ggml-cuda\ggml-cuda.cu:2825
First Bad Commit
No response
Relevant log output
Logs
Name and Version
|D:\a\llama.cpp\llama.cpp\ggml\src\ggml-cuda\ggml-cuda.cu:98: CUDA error
ggml_cuda_compute_forward: MUL_MAT_ID failed
PS G:\llmam\llama-b8611-bin-win-cuda-12.4-x64> .\llama-cli.exe --version
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 16310 MiB):
Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 16310 MiB
load_backend: loaded CUDA backend from G:\llmam\llama-b8611-bin-win-cuda-12.4-x64\ggml-cuda.dll
load_backend: loaded RPC backend from G:\llmam\llama-b8611-bin-win-cuda-12.4-x64\ggml-rpc.dll
load_backend: loaded CPU backend from G:\llmam\llama-b8611-bin-win-cuda-12.4-x64\ggml-cpu-alderlake.dll
version: 8611 (d43375f)
built with Clang 19.1.5 for Windows x86_64
Operating systems
Windows
Which llama.cpp modules do you know to be affected?
llama-cli, llama-server
Command line
Problem description & steps to reproduce
parser generation prompt: <|assistant|>
res add_waiting_: add task 0 to waiting list. current waiting = 0 (before add)
que post: new task, id = 0, front = 0
|ggml_backend_cuda_graph_compute: CUDA graph warmup reset
ggml_cuda_compute_forward: MUL_MAT_ID failed
D:\a\llama.cpp\llama.cpp\ggml\src\ggml-cuda\ggml-cuda.cu:98: CUDA error
CUDA error: invalid argument
current device: 0, in function ggml_cuda_compute_forward at D:\a\llama.cpp\llama.cpp\ggml\src\ggml-cuda\ggml-cuda.cu:2825
First Bad Commit
No response
Relevant log output
Logs