nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 28 Packages Projects Releases Wiki Activity
5,744 Commits 1 Branch 0 Tags
72c6bc3f3d0cf3bf160dddf1b803fed52bcbb0a3
Commit Graph
7 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Rémy OandGitHub 68d0027f3d ggml-cpu: faster AVX2 variant for IQ1_M (#12216) 2025-03-07 13:54:22 +02:00
Rémy OandGitHub 07d1572347 ggml-cpu: Faster IQ1 mul_mat_vec on AVX2 using BMI2 instructions (#12154) 
* ggml-cpu: Faster IQ1 mul_mat_vec on AVX2 using BMI2 instructions

* cmake: Add GGML_BMI2 build option

* ggml: enable BMI2 on relevant CPU variants

* ggml-cpu: include BMI2 in backend score

* ggml-cpu: register BMI2 in ggml_backend_cpu_get_features

* ggml-cpu: add __BMI2__ define when using MSVC
 2025-03-06 02:26:10 +01:00
Rémy OandGitHub 438a83926a vulkan: add specific MMV kernels for IQ2 and IQ3 quants + optimizations (#11595) 
* vulkan: implement specialized MMV kernels for IQ2 quantizations

* vulkan: add MMV kernels for IQ3 quants

* vulkan: Increase MMV batch size and unroll IQ LUT setup

* vulkan: fix init_iq_shmem for WG sizes larger than tables

* vulkan: common batch size for all I-quants
 2025-02-28 09:42:52 +01:00
Rémy OandGitHub 61d4f39dfe vulkan: implement more backpropagation operators (#11914) 
* vulkan: implement GGML_OP_ROPE_BACK

* vulkan: implement GGML_OP_RMS_NORM_BACK

* vulkan: implement GGML_OP_SILU_BACK

* vulkan: implement GGML_OP_SOFTMAX_BACK
 2025-02-25 12:04:45 +01:00
Rémy OandGitHub 2eea03d86a vulkan: implement several ops relevant for ggml_opt (#11769) 
* vulkan: support memset_tensor

* vulkan: support GGML_OP_SUM

* vulkan: implement GGML_OP_ARGMAX

* vulkan: implement GGML_OP_SUB

* vulkan: implement GGML_OP_COUNT_EQUAL

* vulkan: implement GGML_OP_OPT_STEP_ADAMW

* vulkan: fix check_results RWKV_WKV6 crash and memory leaks

* vulkan: implement GGML_OP_REPEAT_BACK

* tests: remove invalid test-backend-ops REPEAT_BACK tests

* vulkan: fix COUNT_EQUAL memset using a fillBuffer command
 2025-02-17 07:55:57 +01:00
Rémy OandGitHub fc1b0d0936 vulkan: initial support for IQ1_S and IQ1_M quantizations (#11528) 
* vulkan: initial support for IQ1_S and IQ1_M quantizations

* vulkan: define MMV kernels for IQ1 quantizations

* devops: increase timeout of Vulkan tests again

* vulkan: simplify ifdef for init_iq_shmem
 2025-02-15 09:01:40 +01:00
Rémy OandGitHub 8a7e3bf17a vulkan: initial support for IQ4_XS quantization (#11501) 2025-02-06 07:09:59 +01:00