nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 27 Packages Projects Releases Wiki Activity
5,330 Commits 1 Branch 0 Tags
17512a94d636c4b6c1332370acb3e5af3ca70918
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Diego DevesaandGitHub 27ebfcacba llama : do not crash if there is no CPU backend (#13395) 
* llama : do not crash if there is no CPU backend

* add checks to examples
 2025-05-09 13:02:07 +02:00
Diego DevesaandGitHub 15e03282bb ci : limit write permission to only the release step + fixes (#13392) 
* ci : limit write permission to only the release step

* fix win cuda file name

* fix license file copy on multi-config generators
 2025-05-08 23:45:22 +02:00
Diego DevesaandGitHub 70a6991edf ci : move release workflow to a separate file (#13362) 2025-05-08 13:15:28 +02:00
Diego DevesaandGitHub f061021206 llama : print size and type of overridden tensors (#13364) 2025-05-08 13:15:15 +02:00
Diego DevesaandGitHub 814f795e06 docker : disable arm64 and intel images (#13356) 2025-05-07 16:36:33 +02:00
Diego DevesaandGitHub f4ed10b69c cmake : remove arm64 msvc presets (#13342) 2025-05-06 20:15:31 +02:00
Diego DevesaandGitHub 9fdfcdaedd rpc : use backend registry, support dl backends (#13304) 2025-05-04 21:25:43 +02:00
Diego DevesaandGitHub 86bd60d3fe llava/mtmd : fixes to fully support dl backends (#13303) 2025-05-04 17:05:20 +02:00
Diego DevesaandGitHub 9f2da5871f llama : build windows releases with dl backends (#13220) 2025-05-04 14:20:49 +02:00
Diego DevesaGitHubXuan Son Nguyen
 		1d36b3670b llama : move end-user examples to tools directory (#13249) 
* llama : move end-user examples to tools directory

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2025-05-02 20:27:13 +02:00
Diego DevesaandGitHub d7a14c42a1 build : fix build info on windows (#13239) 
* build : fix build info on windows

* fix cuda host compiler msg
 2025-05-01 21:48:08 +02:00
Diego DevesaandGeorgi Gerganov 4254bb4951 ggml : fix ggml_gallocr_ptr type (ggml/1205) 2025-05-01 09:58:44 +03:00
Diego DevesaandGitHub 295354ea68 llama : fix K-shift with quantized K and BLAS backend (#13113) 2025-04-25 19:40:11 +02:00
Diego DevesaandGitHub 1d735c0b4f ggml : add SSE 4.2 and x64 base variant for CPUs without AVX (#12871) 
* ggml : add SSE 4.2 variant for CPUs without AVX

* ggml : add x64 base ABI variant
 2025-04-21 18:13:51 +02:00
Diego DevesaandGeorgi Gerganov fe92821ea9 ggml : add bilinear upscale support (ggml/1185) 2025-04-11 00:17:47 +03:00
Diego DevesaandGeorgi Gerganov 459895c326 ggml : add more generic custom op, remove deprecated custom ops (ggml/1183) 
* ggml : add more generic ggml_custom op

* ggml : remove deprecated custom ops
 2025-04-11 00:17:47 +03:00
Diego DevesaandGitHub e0e912f49b llama : add option to override model tensor buffers (#11397) 
* llama : add option to override tensor buffers

* ggml : fix possible underflow in ggml_nbytes
 2025-04-02 14:52:01 +02:00
Diego DevesaandGeorgi Gerganov 6d4c23b81b whisper : support GGML_BACKEND_DL (whisper/2843) 
* whisper : support GGML_BACKEND_DL

* fix DTW crash

* whisper.objc : fix build - add ggml-cpp.h

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-03-03 18:18:11 +02:00
Diego DevesaandGitHub d5c63cd7f9 test-backend-ops : add option -p to filter by op params (#12155) 2025-03-03 14:00:46 +01:00
Diego DevesaandGitHub cc473cac7c ggml-backend : keep paths in native string type when possible (#12144) 2025-03-02 22:11:00 +01:00
Diego DevesaandGitHub 94b87f87b5 cuda : add ampere to the list of default architectures (#11870) 2025-02-14 15:33:52 +01:00
Diego DevesaandGitHub a394039db0 ggml-cpu : add chunking support to mul_mat_id (#11666) 
* ggml-cpu : add chunking support to mul_mat_id

* allocate chunk counter in wdata
parallelize src1 quantization by column to allows parallelization even when there is only one row

* disable for arm

* cleanup

* better way to disable for arm

* fix uninitialized counter when using 1 thread only

* revert test-backend-ops changes
 2025-02-13 01:02:38 +01:00
Diego DevesaandGitHub 6e264a905b docker : add GGML_CPU_ARM_ARCH arg to select ARM architecture to build for (#11419) 2025-01-25 17:22:41 +01:00
Diego DevesaandGitHub 20a758155b docker : fix CPU ARM build (#11403) 
* docker : fix CPU ARM build

* add CURL to other builds
 2025-01-25 15:22:29 +01:00
Diego DevesaandGitHub 6152129d05 main : update README documentation for batch size (#11353) 
* main : update README documentation for batch size

* fix formatting

* minor
 2025-01-22 19:22:20 +01:00
Diego DevesaandGitHub 12c2bdf2de server : fix draft context not being released (#11354) 2025-01-22 17:44:40 +01:00
Diego DevesaandGitHub 017cc5f446 ggml-backend : only offload from host buffers (fix) (#11124) 2025-01-07 16:11:57 +01:00
Diego DevesaandGitHub a3d50bc022 ggml-backend : only offload from host buffers (#11120) 2025-01-07 12:38:05 +01:00
Diego DevesaandGitHub 60cfa728e2 ggml : use wstring for backend search paths (#10960) 
ggml-ci
 2024-12-24 04:05:27 +01:00
Diego DevesaandGitHub 3327bb0f8d ggml : fix arm enabled features check (#10961) 2024-12-24 04:05:17 +01:00
Diego DevesaandGitHub 32d6ee6385 ggml : fix const usage in SSE path (#10962) 2024-12-23 20:25:52 +01:00
Diego DevesaandGitHub 21ae3b9be8 ggml : add test for SVE and disable when it fails (#10906) 2024-12-20 13:31:28 +01:00
Diego DevesaGitHubAdrien Gallouët
 		9177484f58 ggml : fix arm build (#10890) 
* ggml: GGML_NATIVE uses -mcpu=native on ARM

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* ggml: Show detected features with GGML_NATIVE

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* remove msvc support, add GGML_CPU_ARM_ARCH option

* disable llamafile in android example

* march -> mcpu, skip adding feature macros

ggml-ci

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Co-authored-by: Adrien Gallouët <angt@huggingface.co>
 2024-12-18 23:21:42 +01:00
Diego DevesaandGitHub 4da69d1abd Revert "llama : add Falcon3 support (#10864)" (#10876) 
This reverts commit 382bc7f2e8.
 2024-12-18 01:36:46 +01:00
Diego DevesaandGitHub cb13ef85a4 remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS (#10797) 
other windows build fixes
 2024-12-12 19:02:49 +01:00
Diego DevesaGitHubGeorgi Gerganov
 		43ed389a3f llama : use cmake for swift build (#10525) 
* llama : use cmake for swift build

* swift : <> -> ""

* ci : remove make

* ci : disable ios build

* Revert "swift : <> -> """

This reverts commit d39ffd9556482b77d4ea5b118b453fc1c097a31d.

* ci : try fix ios build

* ci : cont

* ci : cont

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-12-08 13:14:54 +02:00
Diego DevesaandGitHub 59f4db1088 ggml : add predefined list of CPU backend variants to build (#10626) 
* ggml : add predefined list of CPU backend variants to build

* update CPU dockerfiles
 2024-12-04 14:45:40 +01:00
Diego DevesaandGitHub 2803540814 ggml-cpu : fix HWCAP2_I8MM value (#10646) 2024-12-04 14:40:44 +01:00
Diego DevesaandGitHub 3420909dff ggml : automatic selection of best CPU backend (#10606) 
* ggml : automatic selection of best CPU backend

* amx : minor opt

* add GGML_AVX_VNNI to enable avx-vnni, fix checks
 2024-12-01 16:12:41 +01:00
Diego DevesaGitHubGeorgi Gerganov
 		7cc2d2c889 ggml : move AMX to the CPU backend (#10570) 
* ggml : move AMX to the CPU backend

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-11-29 21:54:58 +01:00
Diego DevesaandGitHub a3a3048e7a cleanup UI link list (#10577) 
* cleanup UI link list

* sort list alphabetically

* add missing licenses
 2024-11-29 17:45:08 +01:00
Diego DevesaandGitHub e90688edd0 ci : fix tag name in cuda and hip releases (#10566) 2024-11-28 15:58:54 +01:00
Diego DevesaandGitHub 46c69e0e75 ci : faster CUDA toolkit installation method and use ccache (#10537) 
* ci : faster CUDA toolkit installation method and use ccache

* remove fetch-depth

* only pack CUDA runtime on master
 2024-11-27 11:03:25 +01:00
Diego DevesaandGitHub c9b00a70b0 ci : fix cuda releases (#10532) 2024-11-26 22:12:10 +01:00
Diego DevesaandGitHub 5a349f2809 ci : remove nix workflows (#10526) 2024-11-26 21:13:54 +01:00
Diego DevesaandGitHub 30ec398321 llama : disable warnings for 3rd party sha1 dependency (#10527) 2024-11-26 21:01:47 +01:00
Diego DevesaandGitHub 7db3846a94 ci : publish the docker images created during scheduled runs (#10515) 2024-11-26 13:05:20 +01:00
Diego DevesaandGitHub c6807b3f28 ci : add ubuntu cuda build, build with one arch on windows (#10456) 2024-11-26 13:05:07 +01:00
Diego DevesaandGitHub 50d5cecbda ci : build docker images only once daily (#10503) 2024-11-25 22:05:39 +01:00
Diego DevesaandGitHub 10bce0450f llama : accept a list of devices to use to offload a model (#10497) 
* llama : accept a list of devices to use to offload a model

* accept `--dev none` to completely disable offloading

* fix dev list with dl backends

* rename env parameter to LLAMA_ARG_DEVICE for consistency
 2024-11-25 19:30:06 +01:00
Diego DevesaGitHubGeorgi Gerganov
 		5931c1f233 ggml : add support for dynamic loading of backends (#10469) 
* ggml : add support for dynamic loading of backends

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-11-25 15:13:39 +01:00
Diego DevesaandGitHub dc39012cba llama : fix op mul check with command-r-plus (#10476) 2024-11-24 16:10:26 +01:00
Diego DevesaandGitHub 55ed008b2d ggml : do not use ARM features not included in the build (#10457) 2024-11-23 14:41:12 +01:00
Diego DevesaGitHubJohannes Gäßler
 		a5e47592b6 cuda : optimize argmax (#10441) 
* cuda : optimize argmax

* remove unused parameter

ggml-ci

* fixup : use full warps

ggml-ci

* Apply suggestions from code review

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* fix ub

* ggml : check ne00 <= INT32_MAX in argmax and argsort

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2024-11-21 18:18:50 +01:00
slarenandGeorgi Gerganov 59b9172822 ggml/sched : do not skip views in pre-assignments 2024-11-21 09:22:05 +02:00
Diego DevesaandGitHub fab5d30ff6 llama : add .clang-format file (#10415) 2024-11-20 12:57:53 +01:00
Diego DevesaandGitHub 3ee6382d48 cuda : fix CUDA_FLAGS not being applied (#10403) 2024-11-19 14:29:38 +01:00
Diego DevesaandGitHub d3481e6316 cuda : only use native when supported by cmake (#10389) 2024-11-18 18:43:40 +01:00
Diego DevesaandGitHub be5caccef9 llama : only use default buffer types for the KV cache (#10358) 2024-11-17 12:25:45 +01:00
Diego DevesaandGitHub eda7e1d4f5 ggml : fix possible buffer use after free in sched reserve (#9930) 2024-11-17 08:31:17 +02:00
slarenandGeorgi Gerganov 883d206fbd ggml : fix some build issues 2024-11-15 21:45:32 +02:00
Diego DevesaGitHubGeorgi GerganovR0CKSTAR
 		ae8de6d50a ggml : build backends as libraries (#10256) 
* ggml : build backends as libraries

---------

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: R0CKSTAR <xiaodong.ye@mthreads.com>
 2024-11-14 18:04:35 +01:00
Diego DevesaandGitHub 97404c4a03 ggml : add ggml-cpu.h to the public headers (#10204) 2024-11-07 18:16:08 +01:00
Diego DevesaandGitHub 94d8cb8be1 metal : fix from ptr buffer name (#10189) 2024-11-06 12:10:07 +01:00
Diego DevesaandGitHub a9e8a9a030 ggml : fix arch check in bf16_to_fp32 (#10164) 2024-11-04 23:17:01 +01:00
Diego DevesaandGitHub d5a409e57f ggml : fix gelu tables initialization (#10172) 2024-11-04 20:06:58 +01:00
Diego DevesaandGitHub 401558b7ba ggml : fix q4xx mat mul, increase ggml_aligned_malloc alignment (#10167) 2024-11-04 17:34:08 +01:00
Diego DevesaandGitHub ea02c753eb cuda : clear error after changing peer access (#10153) 2024-11-04 13:10:23 +01:00
Diego DevesaandGitHub 9f40989351 ggml : move CPU backend to a separate file (#10144) 2024-11-03 19:34:08 +01:00
Diego DevesaandGitHub b634f8a26f simple-chat : only add bos on first prompt (#10129) 2024-11-02 13:08:53 +01:00
Diego DevesaGitHubXuan Son Nguyen
 		a6744e43e8 llama : add simple-chat example (#10124) 
* llama : add simple-chat example

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-11-01 23:50:59 +01:00
Diego DevesaandGitHub e991e3127f llama : use smart pointers for ggml resources (#10117) 2024-11-01 23:48:26 +01:00
Diego DevesaandGitHub 85679d37f3 llama : improve output buffer type selection (#10098) 2024-11-01 00:49:53 +01:00
Diego DevesaandGitHub 1e9f94994e quantize : fix --keep-split (#10114) 2024-11-01 00:45:34 +01:00
Diego DevesaandGitHub c02e5ab2a6 llama : fix buffer checks for mamba and rwk (#10111) 
* llama : fix buffer checks for mamba and rwk

* llama : fix missing worst case flag during reserve

* cuda : fix supports_op for norm

* disable sched SET_CAUSE
 2024-10-31 22:54:23 +01:00
Diego DevesaandGitHub dea5e86051 ggml : check tensor name lengths in gguf files (#10100) 2024-10-31 11:40:59 +01:00
Diego DevesaandGitHub b9e02e8184 ggml : fix memory leaks when loading invalid gguf files (#10094) 
* ggml : fix gguf string leak when reading kv pairs fails

* ggml : avoid crashing with GGML_ABORT when the KV has an invalid type

* ggml : avoid crashing on failed memory allocations when loading a gguf file
 2024-10-30 14:51:21 +01:00
Diego DevesaandGitHub c5b0f4b5d9 llama : refactor model loader with backend registry (#10026) 2024-10-30 02:01:23 +01:00
Diego DevesaandGitHub f010b77a37 vulkan : add backend registry / device interfaces (#9721) 
* vulkan : add backend registry / device interfaces

* llama : print devices used on model load
 2024-10-17 02:46:58 +02:00
Diego DevesaandGitHub 96776405a1 ggml : move more prints to the ggml log system (#9839) 
* ggml : move more prints to the ggml log system

* show BLAS OpenMP warnings in all builds using debug print
 2024-10-11 15:34:45 +02:00
Diego DevesaGitHubGeorgi Gerganov
 		7eee341bee common : use common_ prefix for common library functions (#9805) 
* common : use common_ prefix for common library functions

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-10-10 22:57:42 +02:00
Diego DevesaandGitHub 0e9f760eb1 rpc : add backend registry / device interfaces (#9812) 
* rpc : add backend registry / device interfaces

* llama : add llama_supports_rpc API

* ggml_backend_rpc_start_rpc_server -> ggml_backend_rpc_start_server
 2024-10-10 20:14:55 +02:00
Diego DevesaandGitHub c7499c557c examples : do not use common library in simple example (#9803) 
* examples : do not use common library in simple example

* add command line parser, simplify code
 2024-10-10 19:50:49 +02:00
Diego DevesaandGitHub c81f3bbb05 cmake : do not build common library by default when standalone (#9804) 2024-10-09 18:49:52 +02:00
Diego DevesaandGitHub dca1d4b58a ggml : fix BLAS with unsupported types (#9775) 
* ggml : do not use BLAS with types without to_float

* ggml : return pointer from ggml_internal_get_type_traits to avoid unnecessary copies

* ggml : rename ggml_internal_get_type_traits -> ggml_get_type_traits

it's not really internal if everybody uses it
 2024-10-08 14:21:43 +02:00
Diego DevesaandGitHub 6374743747 ggml : add backend registry / device interfaces to BLAS backend (#9752) 
* ggml : add backend registry / device interfaces to BLAS backend

* fix mmap usage when using host buffers
 2024-10-07 21:55:08 +02:00
Diego DevesaandGeorgi Gerganov ff565769f2 ggml : fixes after sync (ggml/983) 
ggml : remove test-backend-buffer

ggml : fix CUDA build warnings
 2024-10-04 18:50:04 +03:00
Diego DevesaandGitHub a7ad553513 ggml-backend : add device description to CPU backend (#9720) 2024-10-03 17:39:18 +02:00
Diego DevesaGitHubJohannes Gäßler
 		c83ad6d01e ggml-backend : add device and backend reg interfaces (#9707) 
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2024-10-03 01:49:47 +02:00
slarenandGitHub 1b2f992cd2 test-backend-ops : use flops for some performance tests (#9657) 
* test-backend-ops : use flops for some performance tests

- parallelize tensor quantization

- use a different set of cases for performance and correctness tests

- run each test for at least one second
 2024-09-28 14:32:46 +02:00
slarenandGitHub d09770cae7 ggml-alloc : fix list of allocated tensors with GGML_ALLOCATOR_DEBUG (#9573) 2024-09-21 14:24:23 +02:00
slarenandGitHub 63351143b2 quantize : improve type name parsing (#9570) 
quantize : do not ignore invalid types in arg parsing

quantize : ignore case of type and ftype arguments
 2024-09-20 20:55:36 +02:00
slarenGitHubMax Krasnyansky
 		64c6af3195 ggml : fix n_threads_cur initialization with one thread (#9538) 
* ggml : fix n_threads_cur initialization with one thread

* Update ggml/src/ggml.c

---------

Co-authored-by: Max Krasnyansky <quic_maxk@quicinc.com>
 2024-09-18 10:13:08 -07:00
slarenandGitHub 23e0d70bac ggml : move common CPU backend impl to new header (#9509) 2024-09-16 16:22:07 +02:00
slarenandGitHub e6deac31f7 gguf-split : add basic checks (#9499) 
* gguf-split : do not overwrite existing files when merging

* gguf-split : error when too many arguments are passed
 2024-09-15 19:02:27 +02:00
slarenandGitHub 1b28061400 llama : skip token bounds check when evaluating embeddings (#9437) 2024-09-11 17:52:13 +02:00
slarenandGitHub 49006c67b4 llama : move random seed generation to the samplers (#9398) 
* llama_sampler_penalties : clamp penalty_last_n to zero
 2024-09-10 18:04:25 +02:00
slarenandGitHub fb3f249815 make : do not run llama-gen-docs when building (#9399) 2024-09-10 09:23:33 +03:00
slarenandGitHub 5fb5e24811 llama : minor sampling refactor (2) (#9386) 2024-09-09 17:10:46 +02:00
slarenandGitHub a249843d89 common : restore --n-gpu-layers (#9371) 2024-09-08 16:44:42 +02:00