nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 29 Packages Projects Releases Wiki Activity
4,677 Commits 1 Branch 0 Tags
19d3c8293b1f61acbe2dab1d49a17950fd788a4a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Georgi GerganovandGitHub aaa5505307 server : minor log updates (#11760) 
ggml-ci
 2025-02-08 18:08:43 +02:00
Georgi Gerganov bdcf8b6a56 cont : fix mmap flag print (#11699) 2025-02-08 16:49:38 +02:00
Georgi GerganovandGitHub ed926d8833 llama : fix defrag logic (#11707) 
* llama : fix defrag logic

ggml-ci

* cont : better logic

ggml-ci

* cont : clamp fragmentation to 0.0

ggml-ci
 2025-02-07 16:05:34 +02:00
Georgi Gerganov 8a59053f63 sync : ggml 2025-02-06 21:23:03 +02:00
Georgi GerganovandGitHub 9dd7a0390f llama : add log about loading model tensors (#11699) 2025-02-06 13:41:37 +02:00
Georgi GerganovandGitHub d774ab3acc metal : adjust support conditions for norm operators (#11671) 
cont #11659

ggml-ci
 2025-02-05 10:57:42 +02:00
Georgi Gerganov 387a1598ca authors : update 2025-02-04 13:04:10 +02:00
Georgi Gerganov 7c9e0ca520 sync : ggml 2025-02-04 12:59:21 +02:00
Georgi Gerganov b34aedd558 ci : do not stale-close roadmap issues 2025-02-04 09:31:01 +02:00
Georgi Gerganov 8ec05832fa sync : ggml 2025-02-03 14:57:08 +02:00
Georgi Gerganov 815857791d sync : ggml 2025-01-29 11:25:29 +02:00
Georgi GerganovandGitHub 178a7eb952 metal : use residency sets (#11427) 
* metal : use residency sets

ggml-ci

* metal : restore commandBufferWithUnretainedReferences calls [no ci]

* metal : release descriptors

ggml-ci

* metal : check env GGML_METAL_NO_RESIDENCY

ggml-ci

* metal : fix build + clean-up

ggml-ci
 2025-01-26 20:06:16 +02:00
Georgi GerganovandGitHub 2cc9b8c32c readme : update hot topics 2025-01-26 14:30:15 +02:00
Georgi GerganovandGitHub 00c24acb2a ci : fix line breaks on windows builds (#11409) 
* ci : fix line breaks on windows builds

* cont : another try

* ci : fix powershell line breaks
 2025-01-25 13:36:48 +02:00
Georgi GerganovandGitHub 9755129c27 release : pack /lib in the packages (#11392) 
* release : pack /lib and /include in the packages

* cmake : put libs in /bin

* TMP : push artifacts

* Revert "TMP : push artifacts"

This reverts commit 4decf2c4dfc5cdf5d96ea44c03c8f9801ab41262.

* ci : fix HIP cmake compiler options to be on first line

* ci : restore the original HIP commands

* ci : change ubuntu build from latest to 20.04

* ci : try to fix macos build rpaths

* ci : remove obsolete MacOS build

* TMP : push artifacts

* ci : change back to ubuntu latest

* ci : macos set build rpath to "@loader_path"

* ci : fix typo

* ci : change ubuntu package to 22.04

* Revert "TMP : push artifacts"

This reverts commit 537b09e70ffc604c414ee78acf3acb4c940ec597.
 2025-01-24 18:41:30 +02:00
Georgi GerganovandGitHub 16d3df7ab0 readme : add plugin links (#11355) 2025-01-22 19:44:26 +02:00
Georgi GerganovandGitHub 2139667ec4 metal : fix out-of-bounds write (#11314) 
ggml-ci
 2025-01-21 08:48:13 +02:00
Georgi GerganovandGitHub 80d0d6b4b7 common : add -hfd option for the draft model (#11318) 
* common : add -hfd option for the draft model

* cont : fix env var

* cont : more fixes
 2025-01-20 22:29:43 +02:00
Georgi GerganovandGitHub 9f7add1cde examples : fix add_special conditions (#11311) 2025-01-20 16:36:08 +02:00
Georgi Gerganov ef6dada60c cont : fix whitespaces (#11305) 2025-01-20 09:29:32 +02:00
Georgi GerganovandGitHub 92bc493917 tests : increase timeout when sanitizers are enabled (#11300) 
* tests : increase timeout when sanitizers are enabled

* tests : add DEFAULT_HTTP_TIMEOUT
 2025-01-19 20:22:30 +02:00
Georgi GerganovandGitHub b9daaffe02 simple-chat : fix BOS being added to each message (#11278) 2025-01-19 18:12:09 +02:00
Georgi GerganovGitHubJohannes Gäßler
 		4dd34ff831 cmake : add sanitizer flags for llama.cpp (#11279) 
* cmake : add sanitizer flags for llama.cpp

ggml-ci

* tests : fix compile warnings

ggml-ci

* cmake : move sanitizer flags to llama_add_compile_flags

ggml-ci

* cmake : move llama.cpp compile flags to top level lists

ggml-ci

* cmake : apply only sanitizer flags at top level

ggml-ci

* tests : fix gguf context use in same_tensor_data

* gguf-test: tensor data comparison

* dummy : trigger ggml-ci

* unicode : silence gcc warnings

ggml-ci

* ci : use sanitizer builds only in Debug mode

ggml-ci

* cmake : add status messages [no ci]

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2025-01-18 16:18:15 +02:00
Georgi GerganovandGitHub f26c874179 scripts : restore hf.sh (#11288) 
ggml-ci
 2025-01-18 13:18:32 +02:00
Georgi GerganovandGitHub a133566d34 vocab : fix double-eos check (#11273) 
ggml-ci
 2025-01-17 09:28:00 +02:00
Georgi GerganovandGitHub f11cfdfd7f ci : use -no-cnv in gguf-split tests (#11254) 
* ci : use -no-cnv in gguf-split tests

ggml-ci

* ci : use -no-cnv in requantize tests

ggml-ci

* scripts : fix [no ci]
 2025-01-15 18:28:35 +02:00
Georgi GerganovandGitHub bbf3e55e35 vocab : add dummy tokens for "no_vocab" type (#11231) 
* vocab : add dummy tokens for "no_vocab" type

ggml-ci

* vocab : minor [no ci]
 2025-01-14 11:54:58 +01:00
Georgi Gerganov 44d1e796d0 sync : ggml 2025-01-14 10:39:42 +02:00
Georgi Gerganov a4f3f5d8e6 scripts : sync gguf (cont) 2025-01-14 09:40:52 +02:00
Georgi Gerganov 48e1ae0e61 scripts : sync gguf 2025-01-14 09:36:58 +02:00
Georgi Gerganov d00a80e89d scripts : sync opencl 2025-01-14 09:19:58 +02:00
Georgi Gerganov a29f0870d4 contrib : add naming guidelines (cont) (#11177) 2025-01-13 15:59:26 +02:00
Georgi GerganovandGitHub ca001f6656 contrib : add naming guidelines (cont) (#11177) 2025-01-13 15:08:44 +02:00
Georgi GerganovandGitHub 7426a26b24 contrib : add naming guidelines (#11177) 
* contrib : add naming guidelines

* contrib : expand naming guidelines [no ci]

* contrib : cont [no ci]

* contrib : add `_t` suffix guideline [no ci]

* contrib : cont [no ci]

* minor [no ci]

* contrib : move coding guidelines to correct section [no ci]

* contrib : minor reword coding guidelines [no ci]

* contrib : add TODO for preprocessor directives [no ci]

* contrib : expand [no ci]

* minor [no ci]

* contrib : clarify `_context` suffix usage [no ci]

* contrib : filename guidelines [no ci]

* contrib : fix notes [no ci]
 2025-01-13 14:46:36 +02:00
Georgi GerganovandGitHub 08f10f69c3 llama : remove notion of CLS token (#11064) 
ggml-ci
 2025-01-12 12:15:53 +02:00
Georgi GerganovGitHubDiego Devesa
 		afa8a9ec9b llama : add llama_vocab, functions -> methods, naming (#11110) 
* llama : functions -> methods (#11110)

* llama : add struct llama_vocab to the API (#11156)

ggml-ci

* hparams : move vocab params to llama_vocab (#11159)

ggml-ci

* vocab : more pimpl (#11165)

ggml-ci

* vocab : minor tokenization optimizations (#11160)

ggml-ci

Co-authored-by: Diego Devesa <slarengh@gmail.com>

* lora : update API names (#11167)

ggml-ci

* llama : update API names to use correct prefix (#11174)

* llama : update API names to use correct prefix

ggml-ci

* cont

ggml-ci

* cont

ggml-ci

* minor [no ci]

* vocab : llama_vocab_add_[be]os -> llama_vocab_get_add_[be]os (#11174)

ggml-ci

* vocab : llama_vocab_n_vocab -> llama_vocab_n_tokens (#11174)

ggml-ci

---------

Co-authored-by: Diego Devesa <slarengh@gmail.com>
 2025-01-12 11:32:42 +02:00
Georgi Gerganov be0e950c91 media : remove old img [no ci] 2025-01-09 11:15:15 +02:00
Georgi GerganovandGitHub c07d437bbd llama : avoid hardcoded QK_K (#11061) 
ggml-ci
 2025-01-08 16:19:36 +02:00
Georgi Gerganov 99a3755a3c sync : ggml 2025-01-08 13:40:30 +02:00
Georgi GerganovandGitHub a3c1232c3f arg : option to exclude arguments from specific examples (#11136) 
* arg : option to exclude arguments from specific examples

ggml-ci

* readme : remove old args [no ci]
 2025-01-08 12:55:36 +02:00
Georgi GerganovandGitHub 0d52a69e4b ci : fix cmake option (#11125) 2025-01-08 11:29:34 +02:00
Georgi GerganovandGitHub ecebbd292d llama : remove unused headers (#11109) 
ggml-ci
 2025-01-06 17:52:35 +02:00
Georgi GerganovandGitHub e6e7c75d94 server : fix extra BOS in infill endpoint (#11106) 
* server : fix extra BOS in infill endpoing

ggml-ci

* server : update infill tests
 2025-01-06 15:36:08 +02:00
Georgi GerganovandGitHub 47182dd03f llama : update llama_model API names (#11063) 
* llama : deprecate llama_free_model, add llama_model_free

ggml-ci

* llama : change `llama_load_model_from_file` -> `llama_model_load_from_file`

ggml-ci
 2025-01-06 10:55:18 +02:00
Georgi GerganovandGitHub 3e6e7a6bc2 tokenize : escape the prompt (#11058) 
* tokenize : escape the prompt

* tokenize : update help
 2025-01-06 10:54:25 +02:00
Georgi GerganovandGitHub ae2f606bb5 mmap : fix fileno macro clash (#11076) 
* mmap : fix fileno macro clash

ggml-ci

* cont

ggml-ci
 2025-01-06 10:52:38 +02:00
Georgi GerganovandGitHub 727368c60f llama : use LLAMA_TOKEN_NULL (#11062) 
ggml-ci
 2025-01-06 10:52:15 +02:00
Georgi GerganovandGitHub 5047dd3546 llama : use _impl suffix instead of _internal (#11060) 
ggml-ci
 2025-01-06 10:52:01 +02:00
Georgi Gerganov 78c6785175 sync : ggml 2025-01-04 16:09:53 +02:00
Georgi Gerganov 5e3b08d606 ggml : do not install metal source when embed library (ggml/1054) 2025-01-04 16:09:53 +02:00
Georgi Gerganov e7da954ecc metal : avoid uint (#11019) 2025-01-03 11:26:14 +02:00
Georgi GerganovandGitHub f66f582927 llama : refactor src/llama.cpp (#10902) 
* llama : scatter llama.cpp into multiple modules (wip)

* llama : control-vector -> adapter

* llama : arch

* llama : mmap

ggml-ci

* ci : remove BUILD_SHARED_LIBS=OFF

ggml-ci

* llama : arch (cont)

ggml-ci

* llama : chat

ggml-ci

* llama : model

ggml-ci

* llama : hparams

ggml-ci

* llama : adapter

ggml-ci

* examples : fix

ggml-ci

* rebase

ggml-ci

* minor

* llama : kv cache

ggml-ci

* llama : impl

ggml-ci

* llama : batch

ggml-ci

* cont

ggml-ci

* llama : context

ggml-ci

* minor

* llama : context (cont)

ggml-ci

* llama : model loader

ggml-ci

* common : update lora

ggml-ci

* llama : quant

ggml-ci

* llama : quant (cont)

ggml-ci

* minor [no ci]
 2025-01-03 10:18:53 +02:00
Georgi GerganovandGitHub 30caac3a68 llama : the WPM vocabs use the CLS token as BOS (#10930) 
* llama : the WPM vocabs use the CLS token as BOS

ggml-ci

* llama : add comment
 2024-12-24 09:44:20 +02:00
Georgi GerganovandGitHub 5cd85b5e00 convert : add BertForMaskedLM (#10919) 2024-12-21 10:10:18 +02:00
Georgi GerganovandGitHub d408bb9268 clip : disable GPU support (#10896) 
ggml-ci
 2024-12-19 18:47:15 +02:00
Georgi GerganovandGitHub 5cab3e4aaa llama : minor grammar refactor (#10897) 
ggml-ci
 2024-12-19 17:42:13 +02:00
Georgi GerganovandGitHub 36319dec5d tts : small QoL for easy model fetch (#10903) 2024-12-19 17:35:15 +02:00
Georgi GerganovandGitHub 0bf2d10c55 tts : add OuteTTS support (#10784) 
* server : add "tokens" output

ggml-ci

* server : output embeddings for all tokens when pooling = none

ggml-ci

* server : be explicit about the pooling type in the tests

ggml-ci

* server : do not normalize embeddings when there is no pooling

ggml-ci

* llama : add OuteTTS support (wip)

* wip

* extract features

* first conv

* group norm

* resnet conv

* resnet

* attn

* pos net

* layer norm

* convnext

* head

* hann window

* fix n_embd + remove llama.cpp hacks

* compute hann window

* fft

* spectrum processing

* clean-up

* tts : receive input text and generate codes

* clip : fix new conv name

* tts : minor fix

* tts : add header + minor fixes

ggml-ci

* tts : add matchematical constant

ggml-ci

* tts : fix sampling + cut initial noise

* tts : fixes

* tts : update default samplers

ggml-ci

* tts : text pre-processing

* tts : outetts-voc -> wavtokenizer-dec

* tts : remove hardcoded constants

ggml-ci

* tts : fix tensor shapes

* llama : refactor wavtokenizer tensors

ggml-ci

* cont

ggml-ci

* cont [no ci]

* llama : update WavTokenizer to non-causal attn

* llama : handle no-vocab detokenization

* tts : add Python example for OuteTTS (wip)

* tts : extend python example to generate spectrogram

ggml-ci

* server : fix rebase artifacts

* tts : enable "return_tokens" in Python example

ggml-ci

* tts : minor fixes

* common : support HF download for vocoder
 2024-12-18 19:27:21 +02:00
Georgi GerganovGitHubXuan Son Nguyen
 		152610eda9 server : output embeddings for all tokens when pooling = none (#10861) 
* server : add "tokens" output

ggml-ci

* server : output embeddings for all tokens when pooling = none

ggml-ci

* server : update readme [no ci]

* server : fix spacing [no ci]

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* server : be explicit about the pooling type in the tests

ggml-ci

* server : update /embeddings and /v1/embeddings endpoints

ggml-ci

* server : do not normalize embeddings when there is no pooling

ggml-ci

* server : update readme

ggml-ci

* server : fixes

* tests : update server tests

ggml-ci

* server : update readme [no ci]

* server : remove rebase artifact

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-12-18 13:01:41 +02:00
Georgi GerganovGitHubXuan Son Nguyen
 		0e70ba686e server : add "tokens" output (#10853) 
* server : add "tokens" output

ggml-ci

* server : update readme

ggml-ci

* server : return tokens ids only if requested

ggml-ci

* tests : improve "tokens" type check

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* server : remove "tokens" from the OAI endpoint

ggml-ci

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-12-18 11:05:29 +02:00
Georgi Gerganov 5437d4aaf5 sync : ggml 2024-12-17 18:36:02 +02:00
Georgi Gerganov 78f766768d cmake : fix "amd64" processor string (whisper/2638) 2024-12-17 18:35:49 +02:00
Georgi GerganovandGitHub 0006f5a74a ggml : update ggml_backend_cpu_device_supports_op (#10867) 
* ggml : fix cpy op for IQ-quants to use reference impl

ggml-ci

* ggml : disable tests involving i-matrix quantization

* ggml : update ggml_backend_cpu_device_supports_op

ggml-ci
 2024-12-17 18:35:42 +02:00
Georgi GerganovandGitHub 08ea539df2 unicode : improve naming style (#10838) 
* unicode : improve naming style

ggml-ci

* cont [no ci]
 2024-12-16 12:31:45 +02:00
Georgi GerganovGitHubDiego Devesa
 		644fd71b44 sampling : refactor + optimize penalties sampler (#10803) 
* sampling : refactor + optimize penalties sampler

ggml-ci

* common : apply ignore_eos as logit bias

ggml-ci

* batched : remove penalties sampler

* params : allow penalty_last_n == -1 to be equal to context size

ggml-ci

* common : by default, move the penalties at the end of the sampling chain

ggml-ci

* common : ignore all EOG tokens

Co-authored-by: Diego Devesa <slarengh@gmail.com>

* common : move back the penalties at the front of the sampling chain

ggml-ci

* readme : restore hint about --ignore-eos flag [no ci]

* llama : minor

ggml-ci

* webui : update

---------

Co-authored-by: Diego Devesa <slarengh@gmail.com>
 2024-12-16 12:31:14 +02:00
Georgi GerganovandGitHub 87cf323cef scripts : change build path to "build-bench" for compare-commits.sh (#10836) 2024-12-15 18:44:47 +02:00
Georgi Gerganov b5ae1ddff9 gguf-py : bump to v0.13.0 2024-12-15 13:16:42 +02:00
Georgi GerganovandGitHub 973f328b1e Merge pull request #10788 from ggerganov/gg/gguf-py-0.11.0 2024-12-11 23:14:46 +02:00
Georgi Gerganov fb18934a97 gguf-py : bump version to 0.11.0 2024-12-11 23:13:31 +02:00
Georgi GerganovandGitHub d9c3ba2b77 ggml : disable iq4_nl interleave size 8 (#10709) 
ggml-ci
 2024-12-07 18:38:15 +02:00
Georgi GerganovGitHubXuan Son NguyenXuan Son Nguyen
 		ce4a7b8493 server : various fixes (#10704) 
* server : various fixes

ggml-ci

* server : show curent seed in slot_params

ggml-ci

* fix /slots endpoint

* Update examples/server/server.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* server : reflect endpoint response changes in the readme

ggml-ci

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-12-07 18:02:05 +02:00
Georgi GerganovandGitHub c2a16c0bdb server : fix free of spec context and batch (#10651) 
ggml-ci
 2024-12-07 11:52:44 +02:00
Georgi Gerganov c5ede3849f convert : add custom attention mapping 2024-12-06 21:33:49 +02:00
Georgi Gerganov 0cd182ebcc sync : ggml 2024-12-05 13:27:42 +02:00
Georgi GerganovandGitHub 1da7b76569 server : fix speculative decoding with context shift (#10641) 
* server : fix speculative decoding with context shift

ggml-ci

* server : take into account speculative limits

ggml-ci

* server : add tests
 2024-12-04 22:38:20 +02:00
Georgi Gerganov 1cd3df46bd scripts : remove amx sync 
ggml-ci
 2024-12-03 20:04:49 +02:00
Georgi Gerganov c505471857 sync : ggml 2024-12-03 20:04:49 +02:00
Georgi GerganovandGitHub 0115df2f65 metal : small-batch mat-mul kernels (#10581) 
* metal : small-batch mat-mul kernels

ggml-ci

* metal : add rest of types

ggml-ci

* metal : final adjustments

ggml-ci

* metal : add comments

ggml-ci
 2024-12-03 11:52:33 +02:00
Georgi Gerganov 515d4e5372 github : minify link [no ci] (revert) 
this doesn't work as expected
 2024-12-03 11:21:43 +02:00
Georgi Gerganov 844e2e1fee github : minify link [no ci] 2024-12-03 11:20:35 +02:00
Georgi GerganovandGitHub 70b98fadbc server : fix default draft model parameters (#10586) 
* server : force F16 KV cache for the draft model

ggml-ci

* server : fix draft params

ggml-ci

* server : various params fixes

ggml-ci
 2024-12-03 11:20:00 +02:00
Georgi GerganovGitHubDiego Devesa
 		8648c52101 make : deprecate (#10514) 
* make : deprecate

ggml-ci

* ci : disable Makefile builds

ggml-ci

* docs : remove make references [no ci]

* ci : disable swift build

ggml-ci

* docs : remove obsolete make references, scripts, examples

ggml-ci

* basic fix for compare-commits.sh

* update build.md

* more build.md updates

* more build.md updates

* more build.md updates

* Update Makefile

Co-authored-by: Diego Devesa <slarengh@gmail.com>

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-12-02 21:22:53 +02:00
Georgi GerganovandGitHub 4cb003dd8d contrib : refresh (#10593) 
* contrib : refresh

* contrib : expand [no ci]

* contrib : expand test-backend-ops instructions

* contrib : add CODEOWNERS

* prs : update template to not have checkbox [no ci]
 2024-12-02 08:53:27 +02:00
Georgi GerganovandGitHub 5e1ed95583 grammars : add English-only grammar (#10612) 2024-12-01 21:37:54 +02:00
Georgi GerganovandGitHub 6acce39710 readme : update the usage section with examples (#10596) 
* readme : update the usage section with examples

* readme : more examples
 2024-12-01 11:25:17 +02:00
Georgi GerganovandGitHub 3e0ba0e604 readme : remove old badge 2024-11-30 10:09:21 +02:00
Georgi GerganovandGitHub abadba05be readme : refresh (#10587) 
* readme : refresh

* readme : move section [no ci]

* readme : clarify [no ci]

* readme : fixes [no ci]

* readme : more fixes [no ci]

* readme : simplify [no ci]

* readme : clarify GGUF
 2024-11-30 09:47:07 +02:00
Georgi GerganovandGitHub f0678c5ff4 ggml : fix I8MM Q4_1 scaling factor conversion (#10562) 
ggml-ci
 2024-11-29 16:25:39 +02:00
Georgi Gerganov dc22344088 ggml : remove redundant copyright notice + update authors 2024-11-28 20:46:40 +02:00
Georgi Gerganov 4c0a95b107 llama : add missing model types 2024-11-28 20:45:07 +02:00
Georgi GerganovandGitHub 76b27d29c2 ggml : fix row condition for i8mm kernels (#10561) 
ggml-ci
 2024-11-28 14:56:37 +02:00
Georgi GerganovandGitHub eea986f215 cmake : fix ARM feature detection (#10543) 
ggml-ci
 2024-11-28 14:56:23 +02:00
Georgi Gerganov 9e2301f4a4 metal : fix group_norm support condition (#0) 2024-11-27 11:22:14 +02:00
Georgi Gerganov fee824a1a1 sync : ggml 2024-11-27 11:10:42 +02:00
Georgi GerganovandGitHub ab96610b1e cmake : enable warnings in llama (#10474) 
* cmake : enable warnings in llama

ggml-ci

* cmake : add llama_get_flags and respect LLAMA_FATAL_WARNINGS

* cmake : get_flags -> ggml_get_flags

* speculative-simple : fix warnings

* cmake : reuse ggml_get_flags

ggml-ci

* speculative-simple : fix compile warning

ggml-ci
 2024-11-26 14:18:08 +02:00
Georgi GerganovandGitHub 84e1c33cde server : fix parallel speculative decoding (#10513) 
ggml-ci
 2024-11-26 13:36:40 +02:00
Georgi GerganovandGitHub 811872a59d speculative : simplify the implementation (#10504) 
ggml-ci
 2024-11-26 12:29:38 +02:00
Georgi Gerganov 9fd8c2687f server : add more information about error (#10455) 2024-11-25 22:28:59 +02:00
Georgi GerganovandGitHub 47f931c8f9 server : enable cache_prompt by default (#10501) 
ggml-ci
 2024-11-25 21:50:07 +02:00
Georgi GerganovandGitHub 106964e3d2 metal : enable mat-vec kernels for bs <= 4 (#10491) 2024-11-25 21:49:31 +02:00