nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 19 Packages Projects Releases Wiki Activity
6,851 Commits 1 Branch 0 Tags
2b9bd9bf4e759c05db629ec1c391dc8aeaa71887
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Diego DevesaandGitHub a2e0088d92 Revert "ggml : Leverage the existing GGML_F32_VEC helpers to vectorize ggml_v…" (#16723) 
This reverts commit 19a5a3edfd.
 2025-10-22 20:20:55 +02:00
Diego DevesaandGitHub b617cfd289 ggml-alloc : fix leak when reusing a tensor with a larger size (#16679) 2025-10-20 14:53:50 +02:00
Diego DevesaandGitHub 97870e6497 cuda : avoid initializing unused devices (#16510) 2025-10-11 13:02:26 +02:00
Diego DevesaandGitHub dc381aa9a6 docker : enable rocWMMA in ROCm images, add gfx1151 (#15997) 2025-09-15 23:38:52 +02:00
Diego DevesaandGitHub 10d197409b releases : switch to rocWMMA develop branch, add gfx1151 (#15992) 
* releases : switch to rocWMMA develop branch, add gfx1151

* remove unused variable ROCM_VERSION
 2025-09-15 23:38:42 +02:00
Diego DevesaandGitHub 9ecb884346 releases : update ROCM, add gfx1200, gfx1201, gfx1151 (#15972) 
* releases : update ROCM, add gfx1200, gfx1201, gfx1151

* releases : set target to 13.3 for macos-x64

* add hipblaslt.dll to release

* add hipblaslt/library to release
 2025-09-14 02:21:59 -07:00
Diego DevesaandGitHub 50f4281a6f llama : allow using iGPUs with --device (#15951) 
* llama : allow using iGPUs with --device

* mtmd : allow iGPU

* rpc-server : allow iGPU
 2025-09-13 16:49:49 +02:00
Diego DevesaandGitHub 360d6533db ggml-backend : add GGML_BACKEND_DEVICE_TYPE_IGPU device type (#15797) 
* ggml-backend : add GGML_BACKEND_DEVICE_TYPE_IGPU device type

ggml-backend : add device id to device props

llama : only use iGPU devices if there are no GPU devices

llama : do not use multiple devices from different backends with the same device id
 2025-09-11 22:47:38 +02:00
Diego DevesaandGitHub 274966226f llama : fix fattn reserve call n_seqs parameter (#15699) 
ggml-ci
 2025-08-31 18:47:05 +03:00
Diego DevesaandGitHub 9777032dcc llama : separate compute buffer reserve from fattn check (#15696) 
Exposes ggml_backend_sched_split_graph() to allow splitting the graph without allocating compute buffers and uses it to split the graph for the automatic Flash Attention check.
 2025-08-31 15:49:03 +02:00
Diego DevesaandGitHub dd892555b0 Update build.md to remove MSVC arm64 notes (#15684) 
Removed information about MSVC compiler limitations for arm64 builds.
 2025-08-30 23:51:28 +08:00
Diego DevesaandGitHub bcbddcd54f tests : fix test-opt with GGML_BACKEND_DL (#15599) 2025-08-26 22:14:38 +02:00
Diego DevesaandGitHub 54a241f505 sched : fix possible use of wrong ids tensor when offloading moe prompt processing (#15488) 2025-08-21 23:09:32 +02:00
Diego DevesaandGitHub 5682a3745f sched : copy only the used experts when offloading prompt processing (#15346) 2025-08-21 01:35:28 +02:00
Diego DevesaandGitHub 7aeee88cfe ci : move ccache action to ggml-org fork (#15328) 2025-08-15 12:27:02 +02:00
Diego DevesaandGitHub f75b830647 chat : include kwargs in template example (#15309) 2025-08-14 10:28:29 -07:00
Diego DevesaandGitHub 0d8831543c ggml : fix fallback to CPU for ununsupported ops (#15118) 2025-08-06 14:37:35 +02:00
Diego DevesaandGitHub ec428b02c3 llama : add --n-cpu-moe option (#15077) 
* llama : add --n-cpu-moe option

Keeps the MoE weights of the first N layers in the CPU
 2025-08-05 01:05:36 +02:00
Diego DevesaandGitHub a06ed5feae llama : add simple option to enable CPU for MoE weights (--cpu-moe) (#14992) 2025-07-31 20:15:41 +02:00
Diego DevesaandGitHub d6818d06a6 llama : allow other bufts when overriding to CPU, add --no-repack option (#14990) 2025-07-31 18:11:34 +02:00
Diego DevesaandGitHub c12bbde372 sched : fix multiple evaluations of the same graph with pipeline parallelism (#14855) 
ggml-ci
 2025-07-25 11:07:26 +03:00
Diego DevesaandGitHub eb3fa2913e test-backend-ops : disable llama test (#14461) 2025-06-30 12:43:15 +02:00
Diego DevesaandGitHub e28c1b93fd cuda : synchronize graph capture and cublas handle destruction (#14288) 
Workarounds an issue that may cause CUDA graph capture to fail when a cuBLAS handle is destroyed in a different thread
 2025-06-20 13:57:36 +02:00
Diego DevesaandGitHub 8f71d0f3e8 ggml-cpu : remove unnecesary arm feature detection (#14281) 
Support for Arm runtime feature detection has now been added to GGML_CPU_ALL_VARIANTS. This removes the old and not very functional code.
 2025-06-19 21:24:14 +02:00
Diego DevesaGitHubGeorgi Gerganov
 		6adc3c3ebc llama : add thread safety test (#14035) 
* llama : add thread safety test

* llamafile : remove global state

* llama : better LLAMA_SPLIT_MODE_NONE logic

when main_gpu < 0 GPU devices are not used

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-06-16 08:11:43 -07:00
Diego DevesaGitHubGeorgi Gerganov
 		7f4fbe5183 llama : allow building all tests on windows when not using shared libs (#13980) 
* llama : allow building all tests on windows when not using shared libraries

* add static windows build to ci

* tests : enable debug logs for test-chat

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-06-09 20:03:09 +02:00
Diego DevesaandGitHub 8f47e25f56 cuda : fix device sync on buffer clear (#14033) 2025-06-09 16:36:26 +02:00
Diego DevesaandGitHub 247e5c6e44 cuda : fix buffer type check with integrated GPUs (#14069) 2025-06-08 11:39:56 -07:00
Diego DevesaandGitHub 3a077146a4 llama : allow using mmap without PrefetchVirtualMemory, apply GGML_WIN_VER to llama.cpp sources (#14013) 2025-06-05 11:57:42 +02:00
Diego DevesaandGitHub 2589ad3704 ci : remove cuda 11.7 releases, switch runner to windows 2022 (#13997) 2025-06-04 15:37:40 +02:00
Diego DevesaandGitHub 482548716f releases : use dl backend for linux release, remove arm64 linux release (#13996) 2025-06-04 13:15:54 +02:00
Diego DevesaandGitHub b47ab7b8e9 sched : avoid changing cur_copy when a graph is already allocated (#13922) 2025-05-30 18:56:19 +02:00
Diego DevesaandGitHub df0c0c7d02 cuda : prevent using split buffers with 3d/4d matrices (#13919) 2025-05-30 16:37:18 +02:00
Diego DevesaandGitHub 952f3953c1 ggml : allow CUDA graphs when using pipeline parallelism (#13814) 2025-05-27 13:05:18 +02:00
Diego DevesaandGitHub a2d02d5793 releases : bundle llvm omp library in windows release (#13763) 2025-05-25 00:55:16 +02:00
Diego DevesaandGitHub 17fc817b58 releases : enable openmp in windows cpu backend build (#13756) 2025-05-24 22:27:03 +02:00
Diego DevesaandGitHub 2bd1b30f69 ggml-cpu : set openmp wait time if not set (#13758) 2025-05-24 22:26:47 +02:00
Diego DevesaandGitHub b775345d78 ci : enable winget package updates (#13734) 2025-05-23 23:14:00 +03:00
Diego DevesaandGitHub a70a8a69c2 ci : add winget package updater (#13732) 2025-05-23 22:09:38 +02:00
Diego DevesaandGitHub 3079e9ac8e release : fix windows hip release (#13707) 
* release : fix windows hip release

* make single hip release with multiple targets
 2025-05-23 00:21:37 +02:00
Diego DevesaandGitHub d643bb2c79 releases : build CPU backend separately (windows) (#13642) 2025-05-21 22:09:57 +02:00
Diego DevesaandGeorgi Gerganov 8b5e19aea6 ggml : fix apple OS check in ggml_print_backtrace (ggml/1229) 2025-05-19 13:29:56 +03:00
Diego DevesaandGitHub 415e40a357 releases : use arm version of curl for arm releases (#13592) 2025-05-16 19:36:51 +02:00
Diego DevesaandGitHub 5364ae4ba5 llama : print hint when loading a model when no backends are loaded (#13589) 2025-05-16 16:38:07 +02:00
Diego DevesaandGitHub c6a2c9e741 gguf : use ggml log system (#13571) 
* gguf : use ggml log system

* llama : remove unnecessary new lines in exception messages
 2025-05-15 19:13:11 +02:00
Diego DevesaandGitHub 6c8b91500e llama-bench : fix -ot with dl backends (#13563) 2025-05-15 15:46:55 +02:00
Diego DevesaandGitHub b7d2672082 llama : fix quantize with dl backends (#13539) 2025-05-14 16:12:36 +02:00
Diego DevesaandGitHub cf0a43bb64 llama-bench : add defrag-thold, check for invalid ranges (#13487) 2025-05-13 00:31:37 +02:00
Diego DevesaandGitHub 22cdab343b llama-bench : accept ranges for integer parameters (#13410) 2025-05-12 13:08:22 +02:00
Diego DevesaandGitHub 27ebfcacba llama : do not crash if there is no CPU backend (#13395) 
* llama : do not crash if there is no CPU backend

* add checks to examples
 2025-05-09 13:02:07 +02:00
Diego DevesaandGitHub 15e03282bb ci : limit write permission to only the release step + fixes (#13392) 
* ci : limit write permission to only the release step

* fix win cuda file name

* fix license file copy on multi-config generators
 2025-05-08 23:45:22 +02:00
Diego DevesaandGitHub 70a6991edf ci : move release workflow to a separate file (#13362) 2025-05-08 13:15:28 +02:00
Diego DevesaandGitHub f061021206 llama : print size and type of overridden tensors (#13364) 2025-05-08 13:15:15 +02:00
Diego DevesaandGitHub 814f795e06 docker : disable arm64 and intel images (#13356) 2025-05-07 16:36:33 +02:00
Diego DevesaandGitHub f4ed10b69c cmake : remove arm64 msvc presets (#13342) 2025-05-06 20:15:31 +02:00
Diego DevesaandGitHub 9fdfcdaedd rpc : use backend registry, support dl backends (#13304) 2025-05-04 21:25:43 +02:00
Diego DevesaandGitHub 86bd60d3fe llava/mtmd : fixes to fully support dl backends (#13303) 2025-05-04 17:05:20 +02:00
Diego DevesaandGitHub 9f2da5871f llama : build windows releases with dl backends (#13220) 2025-05-04 14:20:49 +02:00
Diego DevesaGitHubXuan Son Nguyen
 		1d36b3670b llama : move end-user examples to tools directory (#13249) 
* llama : move end-user examples to tools directory

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2025-05-02 20:27:13 +02:00
Diego DevesaandGitHub d7a14c42a1 build : fix build info on windows (#13239) 
* build : fix build info on windows

* fix cuda host compiler msg
 2025-05-01 21:48:08 +02:00
Diego DevesaandGeorgi Gerganov 4254bb4951 ggml : fix ggml_gallocr_ptr type (ggml/1205) 2025-05-01 09:58:44 +03:00
Diego DevesaandGitHub 295354ea68 llama : fix K-shift with quantized K and BLAS backend (#13113) 2025-04-25 19:40:11 +02:00
Diego DevesaandGitHub 1d735c0b4f ggml : add SSE 4.2 and x64 base variant for CPUs without AVX (#12871) 
* ggml : add SSE 4.2 variant for CPUs without AVX

* ggml : add x64 base ABI variant
 2025-04-21 18:13:51 +02:00
Diego DevesaandGeorgi Gerganov fe92821ea9 ggml : add bilinear upscale support (ggml/1185) 2025-04-11 00:17:47 +03:00
Diego DevesaandGeorgi Gerganov 459895c326 ggml : add more generic custom op, remove deprecated custom ops (ggml/1183) 
* ggml : add more generic ggml_custom op

* ggml : remove deprecated custom ops
 2025-04-11 00:17:47 +03:00
Diego DevesaandGitHub e0e912f49b llama : add option to override model tensor buffers (#11397) 
* llama : add option to override tensor buffers

* ggml : fix possible underflow in ggml_nbytes
 2025-04-02 14:52:01 +02:00
Diego DevesaandGeorgi Gerganov 6d4c23b81b whisper : support GGML_BACKEND_DL (whisper/2843) 
* whisper : support GGML_BACKEND_DL

* fix DTW crash

* whisper.objc : fix build - add ggml-cpp.h

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-03-03 18:18:11 +02:00
Diego DevesaandGitHub d5c63cd7f9 test-backend-ops : add option -p to filter by op params (#12155) 2025-03-03 14:00:46 +01:00
Diego DevesaandGitHub cc473cac7c ggml-backend : keep paths in native string type when possible (#12144) 2025-03-02 22:11:00 +01:00
Diego DevesaandGitHub 94b87f87b5 cuda : add ampere to the list of default architectures (#11870) 2025-02-14 15:33:52 +01:00
Diego DevesaandGitHub a394039db0 ggml-cpu : add chunking support to mul_mat_id (#11666) 
* ggml-cpu : add chunking support to mul_mat_id

* allocate chunk counter in wdata
parallelize src1 quantization by column to allows parallelization even when there is only one row

* disable for arm

* cleanup

* better way to disable for arm

* fix uninitialized counter when using 1 thread only

* revert test-backend-ops changes
 2025-02-13 01:02:38 +01:00
Diego DevesaandGitHub 6e264a905b docker : add GGML_CPU_ARM_ARCH arg to select ARM architecture to build for (#11419) 2025-01-25 17:22:41 +01:00
Diego DevesaandGitHub 20a758155b docker : fix CPU ARM build (#11403) 
* docker : fix CPU ARM build

* add CURL to other builds
 2025-01-25 15:22:29 +01:00
Diego DevesaandGitHub 6152129d05 main : update README documentation for batch size (#11353) 
* main : update README documentation for batch size

* fix formatting

* minor
 2025-01-22 19:22:20 +01:00
Diego DevesaandGitHub 12c2bdf2de server : fix draft context not being released (#11354) 2025-01-22 17:44:40 +01:00
Diego DevesaandGitHub 017cc5f446 ggml-backend : only offload from host buffers (fix) (#11124) 2025-01-07 16:11:57 +01:00
Diego DevesaandGitHub a3d50bc022 ggml-backend : only offload from host buffers (#11120) 2025-01-07 12:38:05 +01:00
Diego DevesaandGitHub 60cfa728e2 ggml : use wstring for backend search paths (#10960) 
ggml-ci
 2024-12-24 04:05:27 +01:00
Diego DevesaandGitHub 3327bb0f8d ggml : fix arm enabled features check (#10961) 2024-12-24 04:05:17 +01:00
Diego DevesaandGitHub 32d6ee6385 ggml : fix const usage in SSE path (#10962) 2024-12-23 20:25:52 +01:00
Diego DevesaandGitHub 21ae3b9be8 ggml : add test for SVE and disable when it fails (#10906) 2024-12-20 13:31:28 +01:00
Diego DevesaGitHubAdrien Gallouët
 		9177484f58 ggml : fix arm build (#10890) 
* ggml: GGML_NATIVE uses -mcpu=native on ARM

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* ggml: Show detected features with GGML_NATIVE

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* remove msvc support, add GGML_CPU_ARM_ARCH option

* disable llamafile in android example

* march -> mcpu, skip adding feature macros

ggml-ci

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Co-authored-by: Adrien Gallouët <angt@huggingface.co>
 2024-12-18 23:21:42 +01:00
Diego DevesaandGitHub 4da69d1abd Revert "llama : add Falcon3 support (#10864)" (#10876) 
This reverts commit 382bc7f2e8.
 2024-12-18 01:36:46 +01:00
Diego DevesaandGitHub cb13ef85a4 remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS (#10797) 
other windows build fixes
 2024-12-12 19:02:49 +01:00
Diego DevesaGitHubGeorgi Gerganov
 		43ed389a3f llama : use cmake for swift build (#10525) 
* llama : use cmake for swift build

* swift : <> -> ""

* ci : remove make

* ci : disable ios build

* Revert "swift : <> -> """

This reverts commit d39ffd9556482b77d4ea5b118b453fc1c097a31d.

* ci : try fix ios build

* ci : cont

* ci : cont

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-12-08 13:14:54 +02:00
Diego DevesaandGitHub 59f4db1088 ggml : add predefined list of CPU backend variants to build (#10626) 
* ggml : add predefined list of CPU backend variants to build

* update CPU dockerfiles
 2024-12-04 14:45:40 +01:00
Diego DevesaandGitHub 2803540814 ggml-cpu : fix HWCAP2_I8MM value (#10646) 2024-12-04 14:40:44 +01:00
Diego DevesaandGitHub 3420909dff ggml : automatic selection of best CPU backend (#10606) 
* ggml : automatic selection of best CPU backend

* amx : minor opt

* add GGML_AVX_VNNI to enable avx-vnni, fix checks
 2024-12-01 16:12:41 +01:00
Diego DevesaGitHubGeorgi Gerganov
 		7cc2d2c889 ggml : move AMX to the CPU backend (#10570) 
* ggml : move AMX to the CPU backend

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-11-29 21:54:58 +01:00
Diego DevesaandGitHub a3a3048e7a cleanup UI link list (#10577) 
* cleanup UI link list

* sort list alphabetically

* add missing licenses
 2024-11-29 17:45:08 +01:00
Diego DevesaandGitHub e90688edd0 ci : fix tag name in cuda and hip releases (#10566) 2024-11-28 15:58:54 +01:00
Diego DevesaandGitHub 46c69e0e75 ci : faster CUDA toolkit installation method and use ccache (#10537) 
* ci : faster CUDA toolkit installation method and use ccache

* remove fetch-depth

* only pack CUDA runtime on master
 2024-11-27 11:03:25 +01:00
Diego DevesaandGitHub c9b00a70b0 ci : fix cuda releases (#10532) 2024-11-26 22:12:10 +01:00
Diego DevesaandGitHub 5a349f2809 ci : remove nix workflows (#10526) 2024-11-26 21:13:54 +01:00
Diego DevesaandGitHub 30ec398321 llama : disable warnings for 3rd party sha1 dependency (#10527) 2024-11-26 21:01:47 +01:00
Diego DevesaandGitHub 7db3846a94 ci : publish the docker images created during scheduled runs (#10515) 2024-11-26 13:05:20 +01:00
Diego DevesaandGitHub c6807b3f28 ci : add ubuntu cuda build, build with one arch on windows (#10456) 2024-11-26 13:05:07 +01:00
Diego DevesaandGitHub 50d5cecbda ci : build docker images only once daily (#10503) 2024-11-25 22:05:39 +01:00
Diego DevesaandGitHub 10bce0450f llama : accept a list of devices to use to offload a model (#10497) 
* llama : accept a list of devices to use to offload a model

* accept `--dev none` to completely disable offloading

* fix dev list with dl backends

* rename env parameter to LLAMA_ARG_DEVICE for consistency
 2024-11-25 19:30:06 +01:00
Diego DevesaGitHubGeorgi Gerganov
 		5931c1f233 ggml : add support for dynamic loading of backends (#10469) 
* ggml : add support for dynamic loading of backends

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-11-25 15:13:39 +01:00