llama : make all KQ masks f16 if FA is used, remove zero attention bias, remove raw_k repeats in DeepSeek V4

[fairydreaming]

Overview

This PR removes the following skeletons from DeepSeek V4 closet:

• it makes KQ masks f16 if FA is used (only the lightning indexer mask is kept f32 since it's added to scores),
• it removes KQ mask type casts entirely,
• it removes dsv4_repeat_streams() that repeats raw_k cache for each stream, as both raw_k and compressed caches are always non-unified it's no longer needed,
• it removes zero kq_b bias tensors that were causing OOM errors like this:

0.06.078.756 E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 437894.04 MiB on device 0: cudaMalloc failed: out of memory
0.06.078.764 E ggml_gallocr_reserve_n_impl: failed to allocate CUDA0 buffer of size 459165182720
0.06.078.766 E graph_reserve: failed to allocate compute buffers

Needs #25351, #25344, #25367, #25434

Additional information

I still don't understand why constructs like:

const bool use_fattn = cparams.flash_attn && (!cparams.kv_unified || n_stream == 1);

were needed at all and why the bias tensors were only created if not

(!cparams.kv_unified || !cparams.flash_attn || kq_mask->ne[3] == 1)

so that means they were created if:

cparams.kv_unified && cparams.flash_attn && kq_mask->ne[3] != 1

My head hurts when I think about what was the purpose. @am17an let me know if that breaks anything.

Tested the code with all combinations of FA and kv_unified, seems to work just fine. Performance is slightly improved.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO

[fairydreaming]

Would be nice if someone prepared f16 GGML_SET_ROWS for Metal, otherwise there will be performance regression on Macs after this PR. Unfortunately I have no access to the hardware and no experience with Metal at all.

[ggerganov]

Would be nice if someone prepared f16 GGML_SET_ROWS for Metal, otherwise there will be performance regression on Macs after this PR. Unfortunately I have no access to the hardware and no experience with Metal at all.

Taking a look

[ggerganov]

@fairydreaming Feel free to cherry-pick this commit wherever it makes sense: d4a193e

[fairydreaming]

@fairydreaming Feel free to cherry-pick this commit wherever it makes sense: d4a193e

@ggerganov Thanks, backend ops tests for this are in #25344 (hopefully Vulkan tests will pass now and it will be ready to merge), so when #25344 is merged I'm going to create PR from your branch. Not adding it here, that would be against contributing guidelines. 😱