nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 17 Packages Projects Releases Wiki Activity

hexagon: HMX quantized matmul rework (#23368)

Browse Source 
* hmx-mm: update debug logging in hmx-mm

* hmx-mm: update dequant logic to use HVX_vector_x2/4

* hmx-mm: remove non-pipelined version of the quantize matmul

It seems that we don't reall need non-pipelined version

* hmx-mm: use activation depth mode and update naming

Co-authored-by: Kim-Chyan Gan <kgan@qti.qualcomm.com>

* hex-mm: minor hmx matmul naming updates

* hmx-mm: remove unused vars

* snapdragon: scripts bump default ubatch-size to 1K

* hexagon: combine HMX and power and clock settings into a single set_power call

* hmx-mm: remove leftover of the scale repl helper

* hexagon: fix editconf error

---------

Co-authored-by: Kim-Chyan Gan <kgan@qti.qualcomm.com>
This commit is contained in:
Max Krasnyansky
2026-05-20 07:39:01 -07:00
committed by GitHub
co-authored by Kim-Chyan Gan
parent e947228222
commit c9872a2575
12 changed files with 212 additions and 523 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/snapdragon/windows/run-bench.ps1
+1 -1
View File
@@ -45,4 +45,4 @@ $env:ADSP_LIBRARY_PATH="$basedir\lib"
& "$basedir\bin\llama-bench.exe" `
--mmap 0 -m $basedir\..\..\gguf\$model `
--poll 1000 -t 6 --cpu-mask 0xfc --cpu-strict 1 `
--batch-size 128 -ngl 99 --device $device $cli_opts
--ubatch-size 1024 -ngl 99 --device $device $cli_opts