 amirai21andGitHub
|
477a66b035
|
convert : correctly handle LLaMA tokenizer for Jamba (#16470)
* fix: convert_hf_to_gguf - change Jamba non-sentencepiece mode (tokenizer.json) vocab construction
* fix: convert_hf_to_gguf - jamba non-sentencepiece tokenizer to use _set_vocab_llama_hf func
* fix: convert_hf_to_gguf - removed get_vocab_base_pre from jamba
|
2025-10-11 10:33:41 +02:00 |
|