ggml : add alloc_buffer_n to buffer type interface#23671
Draft
ggerganov wants to merge 2 commits into
Draft
Conversation
michaelw9999
added a commit
to michaelw9999/advanced-gguf-quantizer
that referenced
this pull request
May 26, 2026
Port ggml-org/llama.cpp#23671 into advanced-gguf-quantizer as one local integration commit. Includes upstream commits 0743dce657dc9901ee56734dbce571cf0a6abb8d (add alloc_buffer_n to buffer type interface) and 3f9bcab883c84c6336e100ce5d7408bb4bf47ac9 (fix cur_buf_size after buffer flush).
Add alloc_buffer_n method to ggml_backend_buffer_type_i interface, with a public API ggml_backend_buft_alloc_buffer_n. - Default implementation in ggml-backend.cpp handles multi-buffer splitting and tensor allocation via ggml_tallocr - Meta buffer type provides custom implementation that creates per-device sub-contexts and delegates to simple buffer types - ggml_backend_alloc_ctx_tensors_from_buft now collects tensors into a list and delegates to the new API - Remove temporary ggml_backend_meta_alloc_ctx_tensors_from_buft - Add NULL alloc_buffer_n to all existing buffer type interfaces (cpu, metal, openvino, hexagon, webgpu, zdnn, virtgpu, repack) Assisted-by: llama.cpp:local pi
3f9bcab to
7c7be0f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
cont #19378
The
ggml_backend_meta_alloc_ctx_tensors_from_buftwas a temporary workaround. This patch avoids the function by extending the buffer type interface with an API that allocates a backend buffer from a list of tensors. This decouples theggml-allocfrom meta backend specifics and allows more flexible buffer allocations in the future.Additional information
Detailed review is still pending - atm this is a one-shot with Qwen3.6, so I have yet to check that the implementation is good. For now mainly interested in running the full CI over this.
Requirements