nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 30 Packages Projects Releases Wiki Activity
5,925 Commits 1 Branch 0 Tags
760b4484e3c192a2649a6ffd0d90086c5558f849
Commit Graph
3 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Benson WongandGitHub 5d01670266 server : include speculative decoding stats when timings_per_token is enabled (#12603) 
* Include speculative decoding stats when timings_per_token is true

New fields added to the `timings` object:

  - draft_n           : number of draft tokens generated
  - draft_accepted_n  : number of draft tokens accepted
  - draft_accept_ratio: ratio of accepted/generated

* Remove redundant draft_accept_ratio var

* add draft acceptance rate to server console output
 2025-03-28 10:05:44 +02:00
Benson WongandGitHub a45433ba20 readme : add llama-swap to infrastructure section (#11032) 
* list llama-swap under tools in README

* readme: add llama-swap to Infrastructure
 2025-01-02 09:14:54 +02:00
Benson WongGitHubDiego Devesa
 		da6aac91f1 Add docs for creating a static build (#10268) (#10630) 
* Add notes for a static build

* Update docs/build.md

---------

Co-authored-by: Diego Devesa <slarengh@gmail.com>
 2024-12-04 01:40:36 +01:00