nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 34 Packages Projects Releases Wiki Activity
4,792 Commits 1 Branch 0 Tags
c43a3e7996e585e2addde1e44057a4f3cdbadef8
Commit Graph
5 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Rémy OandGitHub 438a83926a vulkan: add specific MMV kernels for IQ2 and IQ3 quants + optimizations (#11595) 
* vulkan: implement specialized MMV kernels for IQ2 quantizations

* vulkan: add MMV kernels for IQ3 quants

* vulkan: Increase MMV batch size and unroll IQ LUT setup

* vulkan: fix init_iq_shmem for WG sizes larger than tables

* vulkan: common batch size for all I-quants
 2025-02-28 09:42:52 +01:00
Rémy OandGitHub 61d4f39dfe vulkan: implement more backpropagation operators (#11914) 
* vulkan: implement GGML_OP_ROPE_BACK

* vulkan: implement GGML_OP_RMS_NORM_BACK

* vulkan: implement GGML_OP_SILU_BACK

* vulkan: implement GGML_OP_SOFTMAX_BACK
 2025-02-25 12:04:45 +01:00
Rémy OandGitHub 2eea03d86a vulkan: implement several ops relevant for ggml_opt (#11769) 
* vulkan: support memset_tensor

* vulkan: support GGML_OP_SUM

* vulkan: implement GGML_OP_ARGMAX

* vulkan: implement GGML_OP_SUB

* vulkan: implement GGML_OP_COUNT_EQUAL

* vulkan: implement GGML_OP_OPT_STEP_ADAMW

* vulkan: fix check_results RWKV_WKV6 crash and memory leaks

* vulkan: implement GGML_OP_REPEAT_BACK

* tests: remove invalid test-backend-ops REPEAT_BACK tests

* vulkan: fix COUNT_EQUAL memset using a fillBuffer command
 2025-02-17 07:55:57 +01:00
Rémy OandGitHub fc1b0d0936 vulkan: initial support for IQ1_S and IQ1_M quantizations (#11528) 
* vulkan: initial support for IQ1_S and IQ1_M quantizations

* vulkan: define MMV kernels for IQ1 quantizations

* devops: increase timeout of Vulkan tests again

* vulkan: simplify ifdef for init_iq_shmem
 2025-02-15 09:01:40 +01:00
Rémy OandGitHub 8a7e3bf17a vulkan: initial support for IQ4_XS quantization (#11501) 2025-02-06 07:09:59 +01:00