DeepseekV4: clear cache only for seq rather than full

[am17an]

Overview

Currently the dsv4 cache has amneisa (#25171, #25171), this is because on clear we clear the entire cache. Instead we should clear for the particular seq_id.

Additional information

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES, paired with codex

[drrros]

This had fixed full prompt recomputations on every agentic turn for me, now it only computes last turn

[ggerganov]

Instead of introducing clear_compressed and clear_compressed_stream, can't we call the seq_rm of all the caches?

[ggerganov]

Moreover, do we even have to clear anything in the compressed caches? Wouldn't the masks produced by the kv_raw mask away any stale data?

[ggerganov]

I guess we don't use masks for the compressed caches and instead rely on the position:

llama.cpp/src/llama-kv-cache-dsv4.cpp

Lines 434 to 438 in 0badc06

	const int64_t n_visible = (int64_t) (pos + 1)/ratio;
	plan.n_visible[i] = (int32_t) n_visible;
	plan.n_kv = std::max(plan.n_kv, n_visible);

So similar comment as in #25325 (comment). This should eventually be reworked to follow the llama_kv_cache pattern of managing a host-side KV cells information, which is decoupled from the token positions and the contents of the buffers. This way, clearing here would not be necessary (hence why I was confused).

Maybe add a reference to this comment inside the code.

[am17an]

Yes we rely on n_visible. Let me add a TODO

[ggerganov]

Is it correct to clear here?

[am17an]

No, all of these need the correct sequence wise remove

[am17an]

@TacoTakumi I assume you have read the contributing guidelines. Copy pasting LLM outputs is not allowed

[TacoTakumi]

@TacoTakumi I assume you have read the contributing guidelines. Copy pasting LLM outputs is not allowed

My apologies, I was tired and in a rush. I knew it was a bad look.

[am17an]

@ggml-org/maintainers can I get another approval?

[drrros]

This had fixed full prompt recomputations on every agentic turn for me, now it only computes last turn

@am17an actually it's just better now, but full reprocessing still happens even after this PR merge (it now reprocess every 5-7 turns instead of every one (like before)), also tested with #25588 applied - same behavior