Eval bug: llama-server crashes with heap corruption (SIGTRAP/EXC_BREAKPOINT) serving Qwen3-Embedding-0.6B with --embedding on macOS

[CVanF5]

Environment

• llama.cpp version: b9140 (libllama.0.0.9140.dylib)
• OS: macOS 26.4.1 (25E253)
• Hardware: Apple Silicon ARM64 (Mac15,7 / M3 Pro)
• Install: Homebrew

Model

Qwen/Qwen3-Embedding-0.6B GGUF, served with --embedding

Steps to Reproduce

llama-server --embedding --hf-repo Qwen/Qwen3-Embedding-0.6B-GGUF --hf-file <file>.gguf
# Send any embedding request to /v1/embeddings

What Happens

llama-server crashes immediately on the first embedding request with heap corruption detected by libsystem_malloc:

BUG IN CLIENT OF LIBMALLOC: memory corruption of free block
EXC_BREAKPOINT / SIGTRAP

Crash Callstack (faulting thread)

server_context_impl::update_slots()
llama_decode
llama_context::decode(llama_batch const&)
llama_kv_cache::init_batch(llama_batch_allocr&, unsigned int, bool)
llama_kv_cache::prepare(std::vector<llama_ubatch> const&)
std::vector<llama_kv_cache::slot_info>::push_back(...)
std::vector<llama_kv_cache::slot_info>::__emplace_back_slow_path(...)
std::allocator<llama_kv_cache::slot_info>::construct(...)
std::vector<std::vector<unsigned int>>::__init_with_size(...)
std::vector<unsigned int>::__init_with_size(...)
operator new(unsigned long)
_xzm_xzone_malloc_from_freelist_chunk  ← crash

Analysis

The crash occurs in llama_kv_cache::prepare() when copying slot_info objects (which contain vector<vector<unsigned int>> for position sequences) into the cache. The heap corruption is detected at the operator new call inside the copy constructor, indicating a prior out-of-bounds write to a block that was already freed.

This may be related to the M-RoPE position buffer size mismatch described in #20093 (pos buffer allocated as n_tokens instead of n_tokens * n_pos_per_embd), which would cause an OOB write corrupting adjacent heap memory.

Note: this is distinct from #20085 (all-zero embeddings with --reranking). This is a hard crash on macOS with no --reranking flag involved. The libggml-metal.so backend is active (Metal GPU) at crash time.

[spikezz]

Additional reproduction: 8B variant on b9305 — same family of bug, different surface

I'm hitting what looks like the same root cause with Qwen3-Embedding-8B (Q8_0 GGUF) on a different Mac, and wanted to add data points since this issue currently only documents the 0.6B variant.

Environment

• llama.cpp b9305 (commit 63248fc), tested both as a self-compiled build and via the official prebuilt assets (llama-b9305-bin-macos-arm64.tar.gz and the KleidiAI variant)
• macOS 15.7.4 (build 24G517), Apple M4 mini, 32 GB unified memory
• Model: Qwen3-Embedding-8B Q8_0 GGUF (ollama blob sha256:d20ddc71e8a5..., 8.0 GB)

Launch

llama-server \
  --model <path-to-Qwen3-Embedding-8B-q8_0.gguf> \
  --embedding \
  --ctx-size 2048 \
  --parallel 1 \
  --batch-size 512 --ubatch-size 128 \
  --n-gpu-layers 99 \
  --host 127.0.0.1 --port 11435 \
  --alias qwen3-emb-8b --no-webui

Symptom

POSTing single-input requests to /v1/embeddings with different texts, one at a time. First two prompts succeed; the third crashes the server (silent SIGKILL — no stderr, no assertion text, just connection reset and process gone).

Final log lines before death:

slot launch_slot_: id  0 | task 5 | processing task, is_child = 0
slot update_slots: id  0 | task 5 | n_past = 1, slot.prompt.tokens.size() = 141, seq_id = 0, pos_min = 0, n_swa = 0
slot update_slots: id  0 | task 5 | forcing full prompt re-processing due to lack of cache data
  (likely due to SWA or hybrid/recurrent memory, see #13194 (comment))

So the trigger is the same path the 0.6B report points at (kv-cache prepare / m-rope position handling) — the crash just shows up later for 8B because the first two prompts happen to succeed, while 0.6B reportedly dies on the first one.

Variants tested (all crash)

Build	Result
b9305 self-compiled (cmake -DGGML_METAL=ON)	crash after prompt #3
b9305 prebuilt macos-arm64	crash after prompt #3
b9305 prebuilt macos-arm64 KleidiAI	crash after prompt #2 (earlier!)

Flags I tried that don't help

Flag	Result
--swa-full	crash pushed to prompt #4 (one more)
--cache-ram 0 (disable prompt cache entirely)	still crashes
--kv-unified	still crashes

Isolations that do NOT crash

• -ngl 0 (CPU only) — 8 unique prompts process cleanly. So the bug is in the Metal path, not in the model or the generic embedding code.
• ollama (which on macOS uses its own --ollama-engine Go runner, not llama.cpp) serves the exact same blob with no crash. Useful as a "model is fine" control.

Repro script

Minimal stdlib-only repro that sends the 8 chunks one by one and reports which one kills the server:

import json, urllib.request, time

CHUNKS = [...]  # 8 short Chinese text snippets, 43–205 tokens each

for i, c in enumerate(CHUNKS):
    t0 = time.perf_counter()
    payload = json.dumps({"input": [c], "model": "qwen3-emb-8b"}).encode()
    req = urllib.request.Request(
        "http://127.0.0.1:11435/v1/embeddings",
        data=payload, headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=60) as r:
        json.loads(r.read())
    print(f"  #{i+1} ok ({(time.perf_counter()-t0)*1000:.0f} ms)")

Happy to attach the full log, a binary repro fixture, or test additional flags / a patch if it would help.

[jackwh]

Wish I'd seen this issue before spending all day debugging it! All part of the fun though, right?!

Here's what I figured out about it anyway. Hope it helps 🤞

Environment

• llama.cpp: Homebrew llama-server --version reports b9430 (d48a56eff)
• macOS: 26.5 (25F71)
• Hardware: Apple Silicon, M5 Max, 128 GB unified memory
• Install: Homebrew
• Backend: Metal active (libggml-metal.so loaded)

Model

Qwen/Qwen3-Embedding-8B-GGUF, Q4_K_M

Command

llama-server \
  -m Qwen3-Embedding-8B-Q4_K_M.gguf \
  --alias qwen3-embedding-8b \
  --embedding \
  --pooling last \
  --embd-normalize -1 \
  -c 32768 \
  -b 512 \
  -ub 512 \
  -np 1 \
  --no-cont-batching \
  --no-kv-unified \
  --threads-http 4 \
  --cache-ram 0 \
  --ctx-checkpoints 0 \
  --no-cache-prompt \
  --no-cache-idle-slots \
  --no-ui \
  --host 127.0.0.1 \
  --port 8080

Reproduction

Sending sequential single-input requests to /v1/embeddings, no concurrency. Inputs were random alphanumeric strings with increasing length.

This crashes well below context length. In one run it crashed around a few hundred tokens. In other configurations it got further, but still failed well below 32k.

Crash report

macOS crash report shows:

EXC_BREAKPOINT / SIGTRAP
Trace/BPT trap: 5
BUG IN CLIENT OF LIBMALLOC: memory corruption of free block

Faulting stack includes:

_xzm_xzone_malloc_freelist_outlined
operator new(unsigned long)
llama_context::decode(llama_batch const&)
llama_decode
server_context_impl::update_slots()
server_queue::start_loop(long long)
llama_server(int, char**)

This looks very similar to the existing heap corruption report.

Related null-vector symptom

With other server settings, especially larger ubatch / multiple slots, I also reproduced the related failure where /v1/embeddings returns a 4096-length embedding whose values are all JSON null. Restarting the server recovers it. That seems related to #14812.

Flags tried

These did not make the crash go away reliably:

• --cache-ram 0
• --ctx-checkpoints 0
• --no-cache-prompt
• --no-cache-idle-slots
• --no-kv-unified
• -np 1
• -b 512 -ub 512
• --embd-normalize -1
• --threads-http 4

So this does not appear to be prompt cache alone, request concurrency, embedding normalization alone, or simply exceeding context length.