nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 26 Packages Projects Releases Wiki Activity
1,652 Commits 1 Branch 0 Tags
919c40660fd27157b391b5832d2a577d5afef4cb
Commit Graph
1 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Oleksii MaryshchenkoGitHubOleksii Maryshchenko
 		d6069051de cuda : use CUDA memory pool with async memory allocation/deallocation when available (#3903) 
* Using cuda memory pools for async alloc/dealloc.

* If cuda device doesnt support memory pool than use old implementation.

* Removed redundant cublasSetStream

---------

Co-authored-by: Oleksii Maryshchenko <omaryshchenko@dtis.com>
 2023-11-02 19:10:39 +02:00