4 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
3 people	332bdfd798	server : maintain chat completion id for streaming responses (#5988) ... * server: maintain chat completion id for streaming responses * Update examples/server/utils.hpp * Update examples/server/utils.hpp --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2024-03-11 10:09:32 +02:00
Minsoo CheongandGitHub	55a2a900ff	server : add /v1/completions endpoint (#5914) ... * add-`/v1/completions`-endpoint * add legacy comment to `/completion` endpoint	2024-03-07 12:42:39 +02:00
Minsoo CheongandGitHub	6a87ac3a52	fix editorconfig check break (#5879)	2024-03-05 11:42:23 +05:30
Minsoo CheongandGitHub	6d341ab6c5	speculative : implement stochastic speculative sampling (#5625) ... * (WIP) Implement stochastic speculative decoding * sample from residual distribution on draft accept failure * fix #5657: force greedy sampling with probs when temp is 0 * remove p_accept parameter * fix style * remove unused variables * add srand() in speculative.cpp * replace use of rand() with mt19937 sampling * fixes based on review (@JohannesGaessler) * fix r random generation * randomly select next sequence to verify + fix bug in memory freeing * fix bug in active_seqs sync * fix uniform int distribution initialization * remove warnings from comparison between int and size_t * check grammar in `llama_sample_probability_distribution_impl` * remove malloc code by utilizing vectors * add PR link to README	2024-03-04 20:24:00 +02:00