llama_cpp

Files

Yes You Can Have Your Own 50e0ad08fb server: save and clear idle slots on new task (--clear-idle) (#20993 )

* server: clear idle slots KV from VRAM (LLAMA_KV_KEEP_ONLY_ACTIVE)

* server: move idle slot KV clearing to slot release

The save "cost" is now paid by the finishing request.

* server: add --kv-clear-idle flag, enable by default

* server: skip clearing last idle slot, clear on launch

* server: test --no-kv-clear-idle flag

* server: simplify on-release clearing loop

* server: remove on-release KV clearing, keep launch-only

* cont : clean-up

* tests: update log strings after --clear-idle rename

* tests: use debug tags instead of log message matching

* test: fix Windows CI by dropping temp log file unlink

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

2026-04-03 19:02:27 +02:00

test_basic.py

server : support multiple model aliases via comma-separated --alias (#19926 )

2026-02-27 07:05:23 +01:00

test_chat_completion.py

server: Add cached_tokens info to oaicompat responses (#19361 )

2026-03-19 19:09:33 +01:00

test_compat_anthropic.py

server: Add cached_tokens info to oaicompat responses (#19361 )

2026-03-19 19:09:33 +01:00

test_compat_oai_responses.py

server: /v1/responses (partial) (#18486 )