907e08c110
server : add llama2 chat template ( #5425 )
...
* server: add mistral chat template
* server: fix typo
* server: rename template mistral to llama2
* server: format_llama2: remove BOS
* server: validate "--chat-template" argument
* server: clean up using_chatml variable
Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com >
---------
Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com >
2024-02-11 12:16:22 +02:00
Xuan Son Nguyen and GitHub
6b91b1e0a9
docker : add build for SYCL, Vulkan + update readme ( #5228 )
...
* add vulkan dockerfile
* intel dockerfile: compile sycl by default
* fix vulkan dockerfile
* add docs for vulkan
* docs: sycl build in docker
* docs: remove trailing spaces
* docs: sycl: add docker section
* docs: clarify install vulkan SDK outside docker
* sycl: use intel/oneapi-basekit docker image
* docs: correct TOC
* docs: correct docker image for Intel oneMKL
2024-02-02 09:56:31 +02:00
Xuan Son Nguyen and GitHub
48c857aa10
server : refactored the task processing logic ( #5065 )
...
* server: add llama_server_queue struct
* server: add llama_server_response_event
* server: add comments
* server: move all mutexes away from server.cpp
* server: correct multitask response
* server: only add back deferred tasks when one slot is available
* server: fix a race condition cause by "request_completion"
2024-01-26 14:42:20 +02:00
2bed4aa3f3
devops : add intel oneapi dockerfile ( #5068 )
...
Co-authored-by: Xuan Son Nguyen <xuanson.nguyen@snowpack.eu >
2024-01-23 09:11:39 +02:00
821f0a271e
server : defer tasks when "slot unavailable" ( #5018 )
...
* server: defer task when no slot is available
* remove unnecessary log
---------
Co-authored-by: Xuan Son Nguyen <xuanson.nguyen@snowpack.eu >
2024-01-18 22:33:05 +02:00
