nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 30 Packages Projects Releases Wiki Activity

hexagon: HMX quantized matmul rework (#23368)

Browse Source 
* hmx-mm: update debug logging in hmx-mm

* hmx-mm: update dequant logic to use HVX_vector_x2/4

* hmx-mm: remove non-pipelined version of the quantize matmul

It seems that we don't reall need non-pipelined version

* hmx-mm: use activation depth mode and update naming

Co-authored-by: Kim-Chyan Gan <kgan@qti.qualcomm.com>

* hex-mm: minor hmx matmul naming updates

* hmx-mm: remove unused vars

* snapdragon: scripts bump default ubatch-size to 1K

* hexagon: combine HMX and power and clock settings into a single set_power call

* hmx-mm: remove leftover of the scale repl helper

* hexagon: fix editconf error

---------

Co-authored-by: Kim-Chyan Gan <kgan@qti.qualcomm.com>
This commit is contained in:
Max Krasnyansky
2026-05-20 07:39:01 -07:00
committed by GitHub
co-authored by Kim-Chyan Gan
parent e947228222
commit c9872a2575
12 changed files with 212 additions and 523 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/snapdragon/windows/run-mtmd.ps1
+1 -1
View File
@@ -64,5 +64,5 @@ $env:ADSP_LIBRARY_PATH="$basedir\lib"
--mmproj $basedir\..\..\gguf\$mmproj `
--image $basedir\..\..\gguf\$image `
--poll 1000 -t 6 --cpu-mask 0xfc --cpu-strict 1 `
--ctx-size 8192 --ubatch-size 256 -fa on `
--ctx-size 8192 --ubatch-size 1024 -fa on `
-ngl 99 --device $device -v $cli_opts