Update IPFS links to quantized alpaca with new tokenizer format

[antimatter15]

No description provided.

[Green-Sky]

its actually v1 now. but the magic changed to ggmf :)
edit: not sure how best to proceed.

[gjmulder]

@antimatter15 I included sha256 checksums for the alpaca 7B, 13B and 30B models in pull request #338, but I suspect I have the old models.

Do we have a standardized naming convention for the alpaca model subdirs and model names that is consistent with the llama model dir and file names?

[gjmulder]

its actually v1 now. but the magic changed to ggmf :) edit: not sure how best to proceed.

How does one determine the version of a model? I ran file on the model data files but it didn't return any magic numbers.

[Green-Sky]

the first couple of bytes are the magic (in little endian, so reversed) + a version byte
https://github.com/ggerganov/llama.cpp/blob/2e664f1ff413995506c9a54f3a8d5b8c64e37a91/convert-pth-to-ggml.py#L70-L71
(the comment there is out of date, its ggmf now)

before it was just the ggml magic bytes and no version id.

(just open any hex editor and check the first 4+1 bytes)

the file command depends on a public data set of known magic ids. ggml @ggerganov 's personal library is likely now known to them.

[ggerganov]

Thanks! Would be useful to add links and instructions for the bigger Alpaca models as well

[gjmulder]

FYI @antimatter15 I'd like to provide sha256 sums for #238 for the alpaca models, if we're going to support them.

llama.cpp/models$ cat chk_versions.sh 
#!/bin/sh

for B in *B/ggml-model*bin*; do
	xxd $B | head -1 | awk -v model=$B '{printf("Model: %30s, magic: 0x%8s, version: 0x%4s\n", model, $3$2, $4)}'
done
llama.cpp/models$ ./chk_versions.sh | sort -nk 2
Model:  alpaca-7B/ggml-model-q4_0.bin, magic: 0x67676c6d, version: 0x007d
Model: alpaca-13B/ggml-model-q4_0.bin, magic: 0x67676c6d, version: 0x007d
Model: alpaca-30B/ggml-model-q4_0.bin, magic: 0x67676c6d, version: 0x007d
Model:          7B/ggml-model-f16.bin, magic: 0x6767666d, version: 0x0100
Model:         7B/ggml-model-q4_0.bin, magic: 0x6767666d, version: 0x0100
Model:         13B/ggml-model-f16.bin, magic: 0x6767666d, version: 0x0100
Model:        13B/ggml-model-q4_0.bin, magic: 0x6767666d, version: 0x0100
Model:       13B/ggml-model-f16.bin.1, magic: 0x6767666d, version: 0x0100
Model:      13B/ggml-model-q4_0.bin.1, magic: 0x6767666d, version: 0x0100
Model:         30B/ggml-model-f16.bin, magic: 0x6767666d, version: 0x0100
Model:        30B/ggml-model-q4_0.bin, magic: 0x6767666d, version: 0x0100
Model:       30B/ggml-model-f16.bin.1, magic: 0x6767666d, version: 0x0100
Model:       30B/ggml-model-f16.bin.2, magic: 0x6767666d, version: 0x0100
Model:       30B/ggml-model-f16.bin.3, magic: 0x6767666d, version: 0x0100
Model:      30B/ggml-model-q4_0.bin.1, magic: 0x6767666d, version: 0x0100
Model:      30B/ggml-model-q4_0.bin.2, magic: 0x6767666d, version: 0x0100
Model:      30B/ggml-model-q4_0.bin.3, magic: 0x6767666d, version: 0x0100
Model:         65B/ggml-model-f16.bin, magic: 0x6767666d, version: 0x0100
Model:        65B/ggml-model-q4_0.bin, magic: 0x6767666d, version: 0x0100
Model:       65B/ggml-model-f16.bin.1, magic: 0x6767666d, version: 0x0100
Model:       65B/ggml-model-f16.bin.2, magic: 0x6767666d, version: 0x0100
Model:       65B/ggml-model-f16.bin.3, magic: 0x6767666d, version: 0x0100
Model:       65B/ggml-model-f16.bin.4, magic: 0x6767666d, version: 0x0100
Model:       65B/ggml-model-f16.bin.5, magic: 0x6767666d, version: 0x0100
Model:       65B/ggml-model-f16.bin.6, magic: 0x6767666d, version: 0x0100
Model:       65B/ggml-model-f16.bin.7, magic: 0x6767666d, version: 0x0100
Model:      65B/ggml-model-q4_0.bin.1, magic: 0x6767666d, version: 0x0100
Model:      65B/ggml-model-q4_0.bin.2, magic: 0x6767666d, version: 0x0100
Model:      65B/ggml-model-q4_0.bin.3, magic: 0x6767666d, version: 0x0100
Model:      65B/ggml-model-q4_0.bin.4, magic: 0x6767666d, version: 0x0100
Model:      65B/ggml-model-q4_0.bin.5, magic: 0x6767666d, version: 0x0100
Model:      65B/ggml-model-q4_0.bin.6, magic: 0x6767666d, version: 0x0100
Model:      65B/ggml-model-q4_0.bin.7, magic: 0x6767666d, version: 0x0100

[ggerganov]

Ok, so it looks like these alpaca models are not with the latest tokenizer from #252 .
This makes them incompatible with latest master branch.
Probably can be converted to compatible format with this script: #324 (comment)