nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 32 Packages Projects Releases Wiki Activity
Files
f84270ea10b2413b4a56b91cfa40ddec27f7300f
llama_cpp/ggml/src
T
History
Adrien GallouëtandGitHub f84270ea10 ggml : use 64 bytes aligned tile buffers (#21058) 
| Model                            | Test   |   t/s OLD |   t/s NEW |   Speedup |
|:---------------------------------|:-------|----------:|----------:|----------:|
| qwen35 0.8B BF16                 | pp512  |    584.59 |    595.41 |      1.02 |
| qwen35 0.8B BF16                 | tg128  |     52.23 |     52.82 |      1.01 |
| qwen35 0.8B IQ2_M - 2.7 bpw      | pp512  |    260.64 |    261.70 |      1.00 |
| qwen35 0.8B IQ2_M - 2.7 bpw      | tg128  |     81.17 |     80.89 |      1.00 |
| qwen35 0.8B IQ2_XXS - 2.0625 bpw | pp512  |    302.36 |    302.56 |      1.00 |
| qwen35 0.8B IQ2_XXS - 2.0625 bpw | tg128  |     84.93 |     85.12 |      1.00 |
| qwen35 0.8B IQ3_XXS - 3.0625 bpw | pp512  |    263.22 |    260.01 |      0.99 |
| qwen35 0.8B IQ3_XXS - 3.0625 bpw | tg128  |     80.29 |     78.94 |      0.98 |
| qwen35 0.8B IQ4_NL - 4.5 bpw     | pp512  |    728.65 |    742.09 |      1.02 |
| qwen35 0.8B IQ4_NL - 4.5 bpw     | tg128  |     82.39 |     84.46 |      1.03 |
| qwen35 0.8B IQ4_XS - 4.25 bpw    | pp512  |    681.33 |    677.06 |      0.99 |
| qwen35 0.8B IQ4_XS - 4.25 bpw    | tg128  |     80.18 |     79.28 |      0.99 |
| qwen35 0.8B Q2_K_M               | pp512  |    413.28 |    415.94 |      1.01 |
| qwen35 0.8B Q2_K_M               | tg128  |     81.90 |     82.78 |      1.01 |
| qwen35 0.8B Q3_K_M               | pp512  |    493.17 |    495.08 |      1.00 |
| qwen35 0.8B Q3_K_M               | tg128  |     82.75 |     83.23 |      1.01 |
| qwen35 0.8B Q3_K_S               | pp512  |    429.35 |    427.64 |      1.00 |
| qwen35 0.8B Q3_K_S               | tg128  |     86.69 |     87.02 |      1.00 |
| qwen35 0.8B Q4_0                 | pp512  |    783.46 |    782.32 |      1.00 |
| qwen35 0.8B Q4_0                 | tg128  |     88.23 |     87.90 |      1.00 |
| qwen35 0.8B Q4_1                 | pp512  |    741.71 |    729.76 |      0.98 |
| qwen35 0.8B Q4_1                 | tg128  |     85.44 |     86.01 |      1.01 |
| qwen35 0.8B Q4_K_M               | pp512  |    676.24 |    681.31 |      1.01 |
| qwen35 0.8B Q4_K_M               | tg128  |     76.59 |     77.06 |      1.01 |
| qwen35 0.8B Q4_K_S               | pp512  |    683.12 |    688.81 |      1.01 |
| qwen35 0.8B Q4_K_S               | tg128  |     80.50 |     81.19 |      1.01 |
| qwen35 0.8B Q5_K_M               | pp512  |    635.33 |    642.11 |      1.01 |
| qwen35 0.8B Q5_K_M               | tg128  |     72.07 |     72.49 |      1.01 |
| qwen35 0.8B Q5_K_S               | pp512  |    660.95 |    658.18 |      1.00 |
| qwen35 0.8B Q5_K_S               | tg128  |     72.19 |     72.95 |      1.01 |
| qwen35 0.8B Q6_K                 | pp512  |    647.97 |    638.84 |      0.99 |
| qwen35 0.8B Q6_K                 | tg128  |     72.83 |     72.49 |      1.00 |
| qwen35 0.8B Q8_0                 | pp512  |    805.01 |    785.49 |      0.98 |
| qwen35 0.8B Q8_0                 | tg128  |     70.10 |     70.13 |      1.00 |

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-04-27 09:30:55 +03:00
..
ggml-blas
ggml: backend-agnostic tensor parallelism (experimental) (#19378)
2026-04-09 16:42:19 +02:00
ggml-cann
ggml: backend-agnostic tensor parallelism (experimental) (#19378)
2026-04-09 16:42:19 +02:00
ggml-cpu
ggml : use 64 bytes aligned tile buffers (#21058)
2026-04-27 09:30:55 +03:00
ggml-cuda
CUDA: better coalesce data-access for contiguous concat (#22330)
2026-04-26 09:21:45 +02:00
ggml-hexagon
hexagon: guard HMX clock request for v75+ platforms (#22377)
2026-04-25 17:58:26 -07:00
ggml-hip
ggml: backend-agnostic tensor parallelism (experimental) (#19378)
2026-04-09 16:42:19 +02:00
ggml-metal
metal : optimize Metal Tensor API usage for GGML_OP_MUL_MAT (#20962)
2026-04-25 15:14:28 +03:00
ggml-musa
ggml-cuda: native bf16 flash attention for vec kernel (#20525)
2026-03-22 11:05:51 +01:00
ggml-opencl
opencl: add iq4_nl support (#22272)
2026-04-25 21:21:58 -07:00
ggml-openvino
openvino: driver setup, CI split, thread safety, and NPU optimizations (#21944)
2026-04-21 18:58:34 +03:00
ggml-rpc
rpc : refactor the RPC transport (#21998)
2026-04-19 10:21:53 +03:00
ggml-sycl
[SYCL] Optimize Q4_0 mul_mat for Arc770, add scripts (#22291)
2026-04-25 09:20:14 +03:00
ggml-virtgpu
ggml: backend-agnostic tensor parallelism (experimental) (#19378)
2026-04-09 16:42:19 +02:00
ggml-vulkan
vulkan: Support F16 OP_FILL (#22177)
2026-04-21 11:01:56 +02:00
ggml-webgpu
add performance-portable tuning for register-tile and subgroup matmul (#22241)
2026-04-26 09:26:28 -07:00
ggml-zdnn
ggml: backend-agnostic tensor parallelism (experimental) (#19378)
2026-04-09 16:42:19 +02:00
ggml-zendnn
ggml: backend-agnostic tensor parallelism (experimental) (#19378)
2026-04-09 16:42:19 +02:00
CMakeLists.txt
ggml-base: use MATH_LIBRARY variable instead of hardcoded 'm' (#22239)
2026-04-23 08:22:08 +03:00
ggml-alloc.c
ggml : remove ggml-ext.h (#21869)
2026-04-14 17:32:58 +03:00
ggml-backend-dl.cpp
hexagon: enable offloading to Hexagon on Windows on Snapdragon (#19150)
2026-01-29 12:33:21 -08:00
ggml-backend-dl.h
hexagon: enable offloading to Hexagon on Windows on Snapdragon (#19150)
2026-01-29 12:33:21 -08:00
ggml-backend-impl.h
ggml: backend-agnostic tensor parallelism (experimental) (#19378)
2026-04-09 16:42:19 +02:00
ggml-backend-meta.cpp
Fix recurrent state serialization for partial reads and writes (#22362)
2026-04-26 13:34:40 +02:00
ggml-backend-reg.cpp
ggml : add OpenVINO backend (#15307)
2026-03-14 07:56:55 +02:00
ggml-backend.cpp
ggml: add graph_reused (#21764)
2026-04-16 17:21:28 +08:00
ggml-common.h
ggml: add Q1_0 1-bit quantization support (CPU) (#21273)
2026-04-06 20:55:21 +02:00
ggml-impl.h
ggml: add graph_reused (#21764)
2026-04-16 17:21:28 +08:00
ggml-opt.cpp
fix: free ctx_copy in ggml_opt_free to plug per-training-session leak (#21592)
2026-04-08 17:40:15 +02:00
ggml-quants.c
ggml: add Q1_0 1-bit quantization support (CPU) (#21273)
2026-04-06 20:55:21 +02:00
ggml-quants.h
ggml: add Q1_0 1-bit quantization support (CPU) (#21273)
2026-04-06 20:55:21 +02:00
ggml-threading.cpp
ggml : build backends as libraries (#10256)
2024-11-14 18:04:35 +01:00
ggml-threading.h
remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS (#10797)
2024-12-12 19:02:49 +01:00
ggml.c
ggml : minor coding style (#22308)
2026-04-24 11:02:00 +03:00
ggml.cpp
ggml : Print backtrace on uncaught C++ exceptions (ggml/1232)
2025-06-01 13:43:57 +03:00
gguf.cpp
llama: fix llama-model-saver (#20503)
2026-03-25 12:53:16 +02:00