nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 27 Packages Projects Releases Wiki Activity
2,039 Commits 1 Branch 0 Tags
d71ac90985854b0905e1abba778e407e17f9f887
Commit Graph
1 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Oleksii MaryshchenkoGitHubOleksii Maryshchenko
 		d6069051de cuda : use CUDA memory pool with async memory allocation/deallocation when available (#3903) 
* Using cuda memory pools for async alloc/dealloc.

* If cuda device doesnt support memory pool than use old implementation.

* Removed redundant cublasSetStream

---------

Co-authored-by: Oleksii Maryshchenko <omaryshchenko@dtis.com>
 2023-11-02 19:10:39 +02:00