nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 27 Packages Projects Releases Wiki Activity
2,253 Commits 1 Branch 0 Tags
4c4cb30736582cacb1a164a9d4bc8e17b1014be7
Commit Graph
10 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan Son NguyenandGitHub 373ee3fbba Add Gemma chat template (#5665) 
* add gemma chat template

* gemma: only apply system_prompt on non-model message
 2024-02-22 19:10:21 +01:00
Xuan Son NguyenandGitHub a46f50747b server : fallback to chatml, add AlphaMonarch chat template (#5628) 
* server: fallback to chatml

* add new chat template

* server: add AlphaMonarch to test chat template

* server: only check model template if there is no custom tmpl

* remove TODO
 2024-02-22 10:33:24 +02:00
Xuan Son NguyenandGitHub 7c8bcc11dc Add docs for llama_chat_apply_template (#5645) 
* add docs for llama_chat_apply_template

* fix typo
 2024-02-22 00:31:00 +01:00
Xuan Son NguyenGitHubGeorgi Gerganov
 		9c405c9f9a Server: use llama_chat_apply_template (#5593) 
* server: use llama_chat_apply_template

* server: remove trailing space

* server: fix format_chat

* server: fix help message

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* server: fix formatted_chat

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-02-20 15:58:27 +01:00
Xuan Son NguyenandGitHub 11b12de39b llama : add llama_chat_apply_template() (#5538) 
* llama: add llama_chat_apply_template

* test-chat-template: remove dedundant vector

* chat_template: do not use std::string for buffer

* add clarification for llama_chat_apply_template

* llama_chat_apply_template: add zephyr template

* llama_chat_apply_template: correct docs

* llama_chat_apply_template: use term "chat" everywhere

* llama_chat_apply_template: change variable name to "tmpl"
 2024-02-19 10:23:37 +02:00
Xuan Son NguyenGitHubJared Van Bortel
 		907e08c110 server : add llama2 chat template (#5425) 
* server: add mistral chat template

* server: fix typo

* server: rename template mistral to llama2

* server: format_llama2: remove BOS

* server: validate "--chat-template" argument

* server: clean up using_chatml variable

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>

---------

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>
 2024-02-11 12:16:22 +02:00
Xuan Son NguyenandGitHub 6b91b1e0a9 docker : add build for SYCL, Vulkan + update readme (#5228) 
* add vulkan dockerfile

* intel dockerfile: compile sycl by default

* fix vulkan dockerfile

* add docs for vulkan

* docs: sycl build in docker

* docs: remove trailing spaces

* docs: sycl: add docker section

* docs: clarify install vulkan SDK outside docker

* sycl: use intel/oneapi-basekit docker image

* docs: correct TOC

* docs: correct docker image for Intel oneMKL
 2024-02-02 09:56:31 +02:00
Xuan Son NguyenandGitHub 48c857aa10 server : refactored the task processing logic (#5065) 
* server: add llama_server_queue struct

* server: add llama_server_response_event

* server: add comments

* server: move all mutexes away from server.cpp

* server: correct multitask response

* server: only add back deferred tasks when one slot is available

* server: fix a race condition cause by "request_completion"
 2024-01-26 14:42:20 +02:00
Xuan Son NguyenGitHubXuan Son Nguyen
 		2bed4aa3f3 devops : add intel oneapi dockerfile (#5068) 
Co-authored-by: Xuan Son Nguyen <xuanson.nguyen@snowpack.eu>
 2024-01-23 09:11:39 +02:00
Xuan Son NguyenGitHubXuan Son Nguyen
 		821f0a271e server : defer tasks when "slot unavailable" (#5018) 
* server: defer task when no slot is available

* remove unnecessary log

---------

Co-authored-by: Xuan Son Nguyen <xuanson.nguyen@snowpack.eu>
 2024-01-18 22:33:05 +02:00