Skip to content

VULKAN CRASH FIX Gate DFlash graph argmax/top-k to CUDA backend#88

Open
scriptnapps wants to merge 1 commit into
Anbeeld:mainfrom
scriptnapps:fix-dflash-vulkan
Open

VULKAN CRASH FIX Gate DFlash graph argmax/top-k to CUDA backend#88
scriptnapps wants to merge 1 commit into
Anbeeld:mainfrom
scriptnapps:fix-dflash-vulkan

Conversation

@scriptnapps

Copy link
Copy Markdown

FIX crashing on Vulkan windows build

DFlash builds ggml_argmax_ext / ggml_topk_ext into the draft graph, but only the CUDA backend implements their extended semantics; on other backends (Vulkan, Metal, CPU, ...) the stock argmax kernel leaves most of the compact ids/probs tensor uninitialized, producing garbage drafts. This PR adds a dflash_graph_argmax cparams flag, set only when the output device backend is CUDA, and falls back to full-logits CPU scanning (and full-logits verification, with a one-time warning) everywhere else. No behavior change on CUDA.

Additional information

Observed on AMD RX 9070 XT with the Vulkan backend, where DFlash drafting was broken out of the box. The check is name-based (strcmp(reg_name, "CUDA")); could be replaced with a proper backend capability query if other backends gain support.

Requirements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant