llama_cpp

2,298 Commits 1 Branch 0 Tags

3 Commits

Author	SHA1	Message	Date
compiladeandGitHub	adcb12a9ba	llama : fix non-quantization of expert gating tensors (#5754 ) This reverts a single line from #5475	2024-02-28 10:52:56 +02:00
compiladeandGitHub	f7625019c5	server : fix crash when system prompt is bigger than batch size (#5714 ) The system prompt is now decoded in batches. * server : fix off-by-one n_past when start of prompt matches whole cache The tokens right after the matching part would otherwise skip a pos value.	2024-02-25 20:43:50 +02:00
compiladeandGitHub	d6bd4d46dd	llama : support StableLM 2 1.6B (#5052 ) * llama : support StableLM 2 1.6B * convert : fix Qwen's set_vocab wrongly naming all special tokens [PAD{id}] * convert : refactor Qwen's set_vocab to use it for StableLM 2 too * nix : add tiktoken to llama-python-extra * convert : use presence of tokenizer.json to determine StableLM tokenizer loader It's a less arbitrary heuristic than the vocab size.	2024-01-22 13:21:52 +02:00

Author

SHA1

Message

Date

compiladeandGitHub

adcb12a9ba

llama : fix non-quantization of expert gating tensors (#5754 )

This reverts a single line from #5475

2024-02-28 10:52:56 +02:00

compiladeandGitHub

f7625019c5

server : fix crash when system prompt is bigger than batch size (#5714 )

The system prompt is now decoded in batches.

* server : fix off-by-one n_past when start of prompt matches whole cache

The tokens right after the matching part would otherwise skip a pos value.

2024-02-25 20:43:50 +02:00

compiladeandGitHub

d6bd4d46dd

llama : support StableLM 2 1.6B (#5052 )

* llama : support StableLM 2 1.6B

* convert : fix Qwen's set_vocab wrongly naming all special tokens [PAD{id}]

* convert : refactor Qwen's set_vocab to use it for StableLM 2 too

* nix : add tiktoken to llama-python-extra

* convert : use presence of tokenizer.json to determine StableLM tokenizer loader

It's a less arbitrary heuristic than the vocab size.

2024-01-22 13:21:52 +02:00