nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 26 Packages Projects Releases Wiki Activity
3,432 Commits 1 Branch 0 Tags
45f2c19cc57286eead7b232ce8028273a817aa4d
Commit Graph
9 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Minsoo CheongandGitHub 7dda1b727e ci: exempt master branch workflows from getting cancelled (#6486) 
* ci: exempt master branch workflows from getting cancelled

* apply to bench.yml
 2024-04-04 18:30:53 +02:00
Minsoo CheongGitHubXuan Son Nguyen
 		deb7240100 embedding : adjust n_ubatch value (#6296) 
* embedding: assign `n_ubatch` value, print error on `n_batch` overflow

* Update examples/embedding/embedding.cpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* use %ld instead of %lld

* Revert "use %ld instead of %lld"

This reverts commit ea753ede90a86a0699f65878cc8e2020ff5eabb8.

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-03-26 11:11:46 +02:00
Minsoo CheongGitHubGeorgi Gerganov
 		64e7b47c69 examples : add "retrieval" (#6193) 
* add `retrieval` example

* add README

* minor fixes

* cast filepos on print

* remove use of variable sized array

* store similarities in separate vector

* print error on insufficient batch size

* fix error message printing

* assign n_batch value to n_ubatch

* fix param definitions

* define retrieval-only parameters in retrieval.cpp

* fix `--context-file` option to be provided multiple times for multiple files

* use vector for `query_emb`

* add usage description in README

* fix merge conflict

* fix usage printing

* remove seed setting

* fix lint

* increase file read buffer size

* retrieval : minor

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-03-25 09:38:22 +02:00
Minsoo CheongandGitHub 586e7bc561 sampling : deduplicated code for probability distribution access (#6240) 
* sampling: remove duplicated code for probability distribution access

* free original_logits

* fix original_logits allocation

* fixes based on review @cebtenzzre

* change function name to `llama_sampling_prepare`
 2024-03-24 10:54:07 +02:00
Minsoo CheongandGitHub ee804f6223 ci: apply concurrency limit for github workflows (#6243) 2024-03-22 19:15:06 +02:00
Minsoo CheongGitHubGeorgi Gerganov
 		332bdfd798 server : maintain chat completion id for streaming responses (#5988) 
* server: maintain chat completion id for streaming responses

* Update examples/server/utils.hpp

* Update examples/server/utils.hpp

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-03-11 10:09:32 +02:00
Minsoo CheongandGitHub 55a2a900ff server : add /v1/completions endpoint (#5914) 
* add-`/v1/completions`-endpoint

* add legacy comment to `/completion` endpoint
 2024-03-07 12:42:39 +02:00
Minsoo CheongandGitHub 6a87ac3a52 fix editorconfig check break (#5879) 2024-03-05 11:42:23 +05:30
Minsoo CheongandGitHub 6d341ab6c5 speculative : implement stochastic speculative sampling (#5625) 
* (WIP) Implement stochastic speculative decoding

* sample from residual distribution on draft accept failure

* fix #5657: force greedy sampling with probs when temp is 0

* remove p_accept parameter

* fix style

* remove unused variables

* add srand() in speculative.cpp

* replace use of rand() with mt19937 sampling

* fixes based on review (@JohannesGaessler)

* fix r random generation

* randomly select next sequence to verify + fix bug in memory freeing

* fix bug in active_seqs sync

* fix uniform int distribution initialization

* remove warnings from comparison between int and size_t

* check grammar in `llama_sample_probability_distribution_impl`

* remove malloc code by utilizing vectors

* add PR link to README
 2024-03-04 20:24:00 +02:00