nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity
2,825 Commits 1 Branch 0 Tags
07cd41d0965829463eff73eda3348aedbfd3a444
Commit Graph
26 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan Son NguyenandGitHub 1fd9c1741d clean up json_value & server_log (#7142) 2024-05-08 13:24:14 +02:00
Xuan Son NguyenandGitHub 842500144e gguf-split: add --no-tensor-first-split (#7072) 2024-05-04 18:56:22 +02:00
Xuan Son NguyenGitHubslaren
 		7bb36ccf91 gguf : enforce that tensor names are unique (#6905) 
* not allow adding duplicated tensor name

* no duplicated tensor while reading gguf

* typo

* throw exception inside llama_model_loader

Co-authored-by: slaren <slarengh@gmail.com>

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-04-28 17:36:18 +02:00
Xuan Son NguyenandGitHub f7fc5f6c6f split: allow --split-max-size option (#6343) 
* split by max size

* clean up arg parse

* split: ok

* add dry run option

* error on 0 tensors

* be positive

* remove next_metadata_size
 2024-03-29 22:34:44 +01:00
Xuan Son NguyenandGitHub ad3a0505e3 Server: clean up OAI params parsing function (#6284) 
* server: clean up oai parsing function

* fix response_format

* fix empty response_format

* minor fixes

* add TODO for logprobs

* update docs
 2024-03-25 09:42:17 +01:00
Xuan Son NguyenandGitHub 91f8ad167d Server: version bump for httplib and json (#6169) 
* server: version bump for httplib and json

* fix build

* bring back content_length
 2024-03-20 13:30:36 +01:00
Xuan Son NguyenandGitHub dfbfdd60f9 readme : add wllama as a wasm binding (#6100) 2024-03-16 17:42:08 +02:00
Xuan Son NguyenandGitHub aab606a11f llama : add Orion chat template (#6066) 2024-03-15 10:44:57 +02:00
Xuan Son NguyenandGitHub 99b71c068f Server: Use multi-task for embeddings endpoint (#6001) 
* use multitask for embd endpoint

* specify types

* remove redundant {"n_predict", 0}
 2024-03-13 11:39:11 +01:00
Xuan Son NguyenandGitHub caa106d4e0 Server: format error to json (#5961) 
* server: format error to json

* server: do not crash on grammar error

* fix api key test case

* revert limit max n_predict

* small fix

* correct coding style

* update completion.js

* launch_slot_with_task

* update docs

* update_slots

* update webui

* update readme
 2024-03-11 10:56:41 +01:00
Xuan Son NguyenandGitHub 950ba1ab84 Server: reorganize some http logic (#5939) 
* refactor static file handler

* use set_pre_routing_handler for validate_api_key

* merge embedding handlers

* correct http verb for endpoints

* fix embedding response

* fix test case CORS Options

* fix code style
 2024-03-09 11:27:53 +01:00
Xuan Son NguyenandGitHub 4ffcdce2ff add alias for chat template (#5858) 2024-03-04 12:22:08 +01:00
Xuan Son NguyenandGitHub 6c32d8c7ad llama : refactor internal quantization functions (#5830) 2024-03-02 16:19:09 +02:00
Xuan Son NguyenandGitHub 052051d8ae Server: normalize naming (#5779) 
* server: normalize naming

* fix spacing
 2024-02-29 21:42:11 +01:00
Xuan Son NguyenGitHubJared Van Bortel
 		a693bea1e6 server : hit Ctrl+C twice to exit (#5734) 
* server: twice ctrl+C to exit

* std::atomic_flag

* sigint: message

* sigint: stderr

* Update examples/server/server.cpp

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>

---------

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>
 2024-02-28 10:55:37 +02:00
Xuan Son NguyenandGitHub b11a93df41 fix server hangs on empty prompt (#5733) 2024-02-26 23:15:48 +01:00
Xuan Son NguyenandGitHub 373ee3fbba Add Gemma chat template (#5665) 
* add gemma chat template

* gemma: only apply system_prompt on non-model message
 2024-02-22 19:10:21 +01:00
Xuan Son NguyenandGitHub a46f50747b server : fallback to chatml, add AlphaMonarch chat template (#5628) 
* server: fallback to chatml

* add new chat template

* server: add AlphaMonarch to test chat template

* server: only check model template if there is no custom tmpl

* remove TODO
 2024-02-22 10:33:24 +02:00
Xuan Son NguyenandGitHub 7c8bcc11dc Add docs for llama_chat_apply_template (#5645) 
* add docs for llama_chat_apply_template

* fix typo
 2024-02-22 00:31:00 +01:00
Xuan Son NguyenGitHubGeorgi Gerganov
 		9c405c9f9a Server: use llama_chat_apply_template (#5593) 
* server: use llama_chat_apply_template

* server: remove trailing space

* server: fix format_chat

* server: fix help message

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* server: fix formatted_chat

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-02-20 15:58:27 +01:00
Xuan Son NguyenandGitHub 11b12de39b llama : add llama_chat_apply_template() (#5538) 
* llama: add llama_chat_apply_template

* test-chat-template: remove dedundant vector

* chat_template: do not use std::string for buffer

* add clarification for llama_chat_apply_template

* llama_chat_apply_template: add zephyr template

* llama_chat_apply_template: correct docs

* llama_chat_apply_template: use term "chat" everywhere

* llama_chat_apply_template: change variable name to "tmpl"
 2024-02-19 10:23:37 +02:00
Xuan Son NguyenGitHubJared Van Bortel
 		907e08c110 server : add llama2 chat template (#5425) 
* server: add mistral chat template

* server: fix typo

* server: rename template mistral to llama2

* server: format_llama2: remove BOS

* server: validate "--chat-template" argument

* server: clean up using_chatml variable

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>

---------

Co-authored-by: Jared Van Bortel <cebtenzzre@gmail.com>
 2024-02-11 12:16:22 +02:00
Xuan Son NguyenandGitHub 6b91b1e0a9 docker : add build for SYCL, Vulkan + update readme (#5228) 
* add vulkan dockerfile

* intel dockerfile: compile sycl by default

* fix vulkan dockerfile

* add docs for vulkan

* docs: sycl build in docker

* docs: remove trailing spaces

* docs: sycl: add docker section

* docs: clarify install vulkan SDK outside docker

* sycl: use intel/oneapi-basekit docker image

* docs: correct TOC

* docs: correct docker image for Intel oneMKL
 2024-02-02 09:56:31 +02:00
Xuan Son NguyenandGitHub 48c857aa10 server : refactored the task processing logic (#5065) 
* server: add llama_server_queue struct

* server: add llama_server_response_event

* server: add comments

* server: move all mutexes away from server.cpp

* server: correct multitask response

* server: only add back deferred tasks when one slot is available

* server: fix a race condition cause by "request_completion"
 2024-01-26 14:42:20 +02:00
Xuan Son NguyenGitHubXuan Son Nguyen
 		2bed4aa3f3 devops : add intel oneapi dockerfile (#5068) 
Co-authored-by: Xuan Son Nguyen <xuanson.nguyen@snowpack.eu>
 2024-01-23 09:11:39 +02:00
Xuan Son NguyenGitHubXuan Son Nguyen
 		821f0a271e server : defer tasks when "slot unavailable" (#5018) 
* server: defer task when no slot is available

* remove unnecessary log

---------

Co-authored-by: Xuan Son Nguyen <xuanson.nguyen@snowpack.eu>
 2024-01-18 22:33:05 +02:00