nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 29 Packages Projects Releases Wiki Activity
2,079 Commits 1 Branch 0 Tags
2c516611f1d0f1e5e9754f8ea1cf97cb1b17bf2c
Commit Graph
2 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Leng YueGitHubGeorgi Gerganov
 		35f73049af speculative : add heuristic algorithm (#3006) 
* Add heuristic algo for speculative

* Constrain minimum n_draft to 2

* speculative : improve heuristic impl

* speculative : be more rewarding upon guessing max drafted tokens

* speculative : fix typos

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-09-14 19:14:44 +03:00
Leng YueandGitHub 5b8530d88c make : add speculative example (#3003) 2023-09-04 13:39:57 +03:00