nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity

CUDA: remove -sm row, refactor cuBLAS (#24216)

Browse Source 
* CUDA: remove -sm row, refactor cuBLAS

* fix CDNA + BF16 logic

* fix bad return

* fix src0 strides, contiguous requirements

* fix GGML_CUDA_FORCE_CUBLAS

* fix casts to BF16
This commit is contained in:
Johannes Gäßler
2026-07-06 20:04:53 +02:00
committed by GitHub
parent 9abce7473a
commit 74976e1aef
6 changed files with 276 additions and 1167 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/include/ggml-cuda.h
-3
View File
@@ -30,9 +30,6 @@ GGML_BACKEND_API ggml_backend_buffer_type_t ggml_backend_cuda_buffer_type(int de
// conduct allreduce operation between devices
GGML_BACKEND_API bool ggml_backend_cuda_allreduce_tensor(ggml_backend_t * backends, struct ggml_tensor ** tensors, size_t n_backends);
// split tensor buffer that splits matrices by rows across multiple devices
GGML_BACKEND_API ggml_backend_buffer_type_t ggml_backend_cuda_split_buffer_type(int main_device, const float * tensor_split);
// pinned host buffer for use with the CPU backend for faster copies between CPU and GPU
GGML_BACKEND_API ggml_backend_buffer_type_t ggml_backend_cuda_host_buffer_type(void);