Motivation
With the recent introduction of eval-callback example, we now having more tools for debugging when working with llama.cpp. However, one of the tool that I feel missing is the ability to dump everything inside a gguf file into a human-readable (and interactive) interface.
Inspired from huggingface.js where users can visualize the KV and list of tensors on huggingface.com, I would like to implement the same thing in llama.cpp. I find this helpful in these situations:
- Debugging
convert.py script when adding a new architecture
- Debugging tokenizers
- Debugging changes related to gguf (model splits for example)
- Debugging tensors (i.e. display N first elements of a tensor, just like
eval-callback)
- Debugging control vectors
- ... (maybe other usages in the future)
The reason why I can't use huggingface.js is because it's based on browser, which make it tricky when reading a huge local file. It also don't have access to quantized types (same for gguf-py).
Possible Implementation
Ideally, I want the implementation to be a binary named gguf-viewer that when run, will open a web page in localhost:8080. User can then go to the web page to explore the gguf file. It will have these sections:
- Complete list of KV
- Tokenizer-related info (for example: list all tokens, lookup one token)
- List of all tensors
Motivation
With the recent introduction of
eval-callbackexample, we now having more tools for debugging when working with llama.cpp. However, one of the tool that I feel missing is the ability to dump everything inside a gguf file into a human-readable (and interactive) interface.Inspired from
huggingface.jswhere users can visualize the KV and list of tensors on huggingface.com, I would like to implement the same thing in llama.cpp. I find this helpful in these situations:convert.pyscript when adding a new architectureeval-callback)The reason why I can't use
huggingface.jsis because it's based on browser, which make it tricky when reading a huge local file. It also don't have access to quantized types (same forgguf-py).Possible Implementation
Ideally, I want the implementation to be a binary named
gguf-viewerthat when run, will open a web page inlocalhost:8080. User can then go to the web page to explore the gguf file. It will have these sections: