nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity
7,508 Commits 1 Branch 0 Tags
6ce863c8039b2dbfef2a22408f347842d438f03d
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Sigbjørn SkjæretandGitHub 74e05131e9 ci : remove non-windows zip artifacts (#18201) 
* remove non-windows zip artifacts

* add cuda dll links
 2025-12-19 22:29:46 +01:00
Sigbjørn SkjæretandGitHub f74747d886 ci : only save ccache on master (#18207) 2025-12-19 22:29:37 +01:00
Sigbjørn SkjæretandGitHub ec7b9329ae gguf-py : use copy-on-write mode for localtensor (#18162) 2025-12-18 13:45:38 +01:00
Sigbjørn SkjæretandGitHub 9cff4cc554 convert : sort and use file parts from model index if present (#18043) 
* keep file part order from model index

* treat index as authoritative

* sort index parts
 2025-12-18 07:54:54 +01:00
Sigbjørn SkjæretandGitHub 5166aaf868 convert : force patch_merger tensors to f16/f32 (#18124) 2025-12-17 22:15:53 +01:00
Sigbjørn SkjæretandGitHub 6853bee680 ci : clean up webui jobs (#18116) 
* clean up webui jobs

* refined step control

* forgot dependencies

* apparently always() is needed
 2025-12-17 10:45:40 +01:00
Sigbjørn SkjæretandGitHub d6742125c3 ci : separate webui from server (#18072) 
* separate webui from server

* add public to path
 2025-12-16 08:17:26 +01:00
Sigbjørn SkjæretandGitHub d6a1e18c65 convert : move rope_parameters to TextModel class (#18061) 
* make sure to search text_config for rope parameters

* move rope_parameters to TextModel class
 2025-12-15 22:03:16 +01:00
Sigbjørn SkjæretandGitHub 5c8a717128 convert : refactor rope scaling handling (#18013) 
* refactor rope scaling handling

* ws--

* missed a couple

* use find_hparam
 2025-12-14 16:04:37 +01:00
Sigbjørn SkjæretandGitHub 8e4d678528 common : skip model validation when --completion-bash is requested (#17975) 2025-12-13 08:40:50 +01:00
Sigbjørn SkjæretandGitHub 2bc94e7928 add llama-completion to completion-bash executables (#17976) 2025-12-13 08:35:50 +01:00
Sigbjørn SkjæretandGitHub b7f5f46e03 docker : include legacy llama-completion binary (#17964) 2025-12-12 19:39:23 +01:00
Sigbjørn SkjæretandGitHub dcb7d17758 cann : fix ops broken by circular padding guard (#17825) 2025-12-12 15:49:27 +01:00
Sigbjørn SkjæretandGitHub 45e350e3d3 ci: fix riscv64-native build (#17916) 2025-12-10 23:24:31 +01:00
Sigbjørn SkjæretandGitHub 4df6e859e9 cuda : add missing support check for xielu (#17895) 2025-12-10 16:16:20 +01:00
Sigbjørn SkjæretandGitHub 63391852b0 docs : update cpu and cuda ops (#17890) 
* update cuda ops

* update CPU as well
 2025-12-09 23:31:29 +01:00
Sigbjørn SkjæretandGitHub 86a3f0fad8 ggml : allow fill node alloc inplace (#17870) 2025-12-09 12:23:47 +01:00
Sigbjørn SkjæretandGitHub 42b12b5608 model : nit, DeepSeek V1 MoE is 16B and GigaChat is 20B (#12652) 
* nit, DeepSeek V1 MoE is 16B

* base type on n_ff_exp instead
 2025-12-09 12:15:06 +01:00
Sigbjørn SkjæretandGitHub c8554b66e0 graph : use fill instead of scale_bias in grouped expert selection (#17867) 
* use fill instead of scale_bias in grouped expert selection

* do not explicitly use _inplace
 2025-12-08 21:29:59 +01:00
Sigbjørn SkjæretandGitHub 0a540f9abd ci : add windows-cuda 13.1 release (#17839) 2025-12-07 14:02:04 +01:00
Sigbjørn SkjæretandGitHub 22577583a3 common : change --color to accept on/off/auto, default to auto (#17827) 2025-12-07 03:43:50 +01:00
Sigbjørn SkjæretandGitHub 03d9a77b85 ci : transform release binary root dir in tar to llama-bXXXX (#17773) 
* transform release binary root dir in tar to llama-bXXXX

* bsdtar supports -s instead of --transform
 2025-12-05 01:50:19 +01:00
Sigbjørn SkjæretandGitHub 7dba049b07 ci : disable ggml-ci-x64-amd-* (#17753) 2025-12-04 11:25:08 +01:00
Sigbjørn SkjæretandGitHub b3e3060f4e ci : move release details to the top visible by default (#17719) 2025-12-03 09:22:46 +01:00
Sigbjørn SkjæretandGitHub b61de2b2df convert : allow quantizing lora again (#17453) 2025-11-24 15:50:55 +01:00
Sigbjørn SkjæretandGitHub 96ac5a2329 cuda : support non-contiguous i32 to i32 copy (#17326) 
* support non-contiguous i32 to i32 copy

* add tests

* rename cpy_flt to cpy_scalar and reindent params
 2025-11-23 11:13:34 +01:00
Sigbjørn SkjæretandGitHub 07b0e7a5ac convert : use self.block_count everywhere instead of reading hparams (#17359) 2025-11-19 11:52:38 +01:00
Sigbjørn SkjæretandGitHub 662192e1dc convert : remove unnecessary chat template patching (#17289) 2025-11-15 20:58:59 +01:00
Sigbjørn SkjæretandGitHub 9a8860cf5d convert : use all parts in safetensors index (#17286) 2025-11-15 14:12:39 +01:00
Sigbjørn SkjæretandGitHub 9d3ef4809f convert : set expert gating func in base class (#17279) 2025-11-15 14:06:24 +01:00
Sigbjørn SkjæretandGitHub 7bef684118 models : move build_inp_out_ids outside loop (#17151) 
* move build_inp_out_ids outside loop

* realign
 2025-11-10 22:55:30 +01:00
Sigbjørn SkjæretandGitHub 9008027aa3 hparams : add n_embd_inp() to support extended embed (#16928) 
* add n_embd_full to support extended embed

* don't change output

* rename to n_embd_inp

* restore n_embd where applicable
 2025-11-07 19:27:58 +01:00
Sigbjørn SkjæretandGitHub d945834366 ci : apply model label to models (#16994) 2025-11-04 12:29:39 +01:00
Sigbjørn SkjæretandGitHub b164259bba chore : fix models indent after refactor (#16992) 2025-11-04 12:29:15 +01:00
Sigbjørn SkjæretandGitHub dd52868050 ci : disable failing riscv cross build (#16952) 2025-11-02 23:11:21 +01:00
Sigbjørn SkjæretandGitHub 961660b8c3 common : allow --system-prompt-file for diffusion-cli (#16903) 2025-11-01 11:01:42 +01:00
Sigbjørn SkjæretandGitHub 74fef4129f codeowners : update after refactor (#16905) 2025-11-01 09:55:25 +02:00
Sigbjørn SkjæretandGitHub 6eb208d17e ci : enable free-disk-space on cuda docker build (#16877) 2025-10-31 00:34:27 +01:00
Sigbjørn SkjæretandGitHub 229bf68628 cuda : fix argsort with 64k+ rows (#16849) 2025-10-30 08:56:28 +01:00
Sigbjørn SkjæretandGitHub 144a4ce824 vendor : sync minja (#16500) 
* sync minja.hpp

Adds Call/EndCall support, used in MiniCPM3 and MiniCPM4-MCP.

* remove spurious semicolon

* sync from ochafik/minja
 2025-10-29 14:09:50 +01:00
Sigbjørn SkjæretandGitHub bd562fe4f7 cuda : use fast copy when src and dst are of different type and contiguous (#16789) 
* use fast copy when src and dst are contiguous and same shape

* use int64_t ne and ignore shape
 2025-10-26 21:31:41 +01:00
Sigbjørn SkjæretandGitHub 73a48c9790 convert : enable expert group selection for all models with it (#16691) 2025-10-26 17:21:23 +01:00
Sigbjørn SkjæretandGitHub f696428ce8 graph : add clamping to ffn_moe_weights_sum to avoid div-by-zero (#16655) 
* add missing norm topk bias

* use clamping instead, update number and add comment
 2025-10-26 17:20:32 +01:00
Sigbjørn SkjæretandGitHub 7cce4f8158 model : set res->t_embd in SmallThinker models (#16782) 2025-10-26 16:08:52 +01:00
Sigbjørn SkjæretandGitHub 84bf3c6778 model : add BailingMoeV2 support (#16063) 
* add BailingMoeV2 support

* update llm types

* undo

* undo

* update llm types

* add model collection link

* update

* almost working

* correct group selection and rename n_group_exp

* avoid large top_k and use argmax instead for now

if we had something like argmax2 that would be equivalent, but this works fine until then

* poke

* skip group selection when there are no tokens

* fix 1T conversion

* hopefully fixed expert group selection

third time's the charm?

* make expert group selection generally available

The new LLaDA2Moe model uses this method too, make it generally available regardless of architecture.

* allow n_expert_groups to be 1 (Kimi K2)

* address review suggestions
 2025-10-20 21:38:20 +02:00
Sigbjørn SkjæretandGitHub cec5edbcae ci : avoid manual updates of docs/ops.md (#16663) 2025-10-19 14:03:25 +02:00
Sigbjørn SkjæretandGitHub 3a002afafa ci : refactor sdk caching to minimize storage (#16414) 
* refactor sdk caching to minimize storage

* use correct action

* add myself as owner to /.github/actions/ [no ci]
 2025-10-06 17:40:21 +02:00
Sigbjørn SkjæretandGitHub 946f71ed9a llama : fix shapes for bert/mpt q/k norm (#16409) 2025-10-03 14:40:25 +02:00
Sigbjørn SkjæretGitHubJeff Bolz
 		d64c8104f0 test-barrier : do not use more threads than physically available (#16389) 
* do not use more threads than physically available

* ensure n_threads > 0

Co-authored-by: Jeff Bolz <jbolz@nvidia.com>

---------

Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
 2025-10-02 20:10:12 +02:00
Sigbjørn SkjæretandGitHub 72ee736c44 ci : fix ubuntu-latest-cmake-rpc (disable ccache) (#16388) 2025-10-02 13:51:36 +02:00
Sigbjørn SkjæretandGitHub 1104ca1a1c ci : use registry cache for docker builds (#16366) 2025-10-01 14:09:52 +02:00
Sigbjørn SkjæretandGitHub b2ba81dbe0 ci : fix ccache key for ubuntu-cpu-cmake (#16355) 
* fix ccache key for ubuntu-cpu-cmake

* set it for release as well [no ci]
 2025-09-30 21:41:42 +02:00
Sigbjørn SkjæretandGitHub 2df5bcf357 ci : disable ccache for android (#16348) 2025-09-30 15:38:01 +02:00
Sigbjørn SkjæretandGitHub adc76347d7 ggml : check cuda and metal argsort limits and add test (#16323) 
* check cuda argsort limits and add test

* add metal check
 2025-09-29 11:09:00 +02:00
Sigbjørn SkjæretandGitHub b887d2f341 ggml : fix GGML_F32_VEC_FMA argument order in ggml_vec_mad1_f32 (#16307) 
* fix GGML_F32_VEC_FMA argument order in ggml_vec_mad1_f32

* add test that fails on simd
 2025-09-28 23:15:03 +02:00
Sigbjørn SkjæretandGitHub 835b2b915c model : add GroveMoE support (#15510) 
* add GroveMoE support

* remove constexpr that fails on certain compilers

* revert crude scalar div implementation, use cast

* build_attn_inp_kv_unified -> build_attn_inp_kv

* fix build_attn

* re-apply ffn_exps regex changes
 2025-09-25 19:50:28 +02:00
Sigbjørn SkjæretandGitHub c0c59c1157 codeowners : match all requirements files (#16214) 2025-09-24 08:53:20 +02:00
Sigbjørn SkjæretandGitHub f6b4af3d04 ggml : fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl (#15928) 
* fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl

* change initialization to true
 2025-09-23 10:25:20 +02:00
Sigbjørn SkjæretGitHubGeorgi GerganovJeff Bolz
 		3ecb2f671a ggml : implement set_rows with i32 index (#16159) 
* implement set_rows with i32 index

* template fix

* test quantized path

warnings--

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* forgotten name change

* deduplicate cuda/sycl and test-fix

* indent++

* vulkan: support set_rows with i32 index type (#16162)

* disable i32 index for webgpu for now

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
 2025-09-22 19:13:00 +02:00
Sigbjørn SkjæretandGitHub b2d980fce0 codeowners : claim responsibility for ci, models, gguf-py and convert (#16124) 
* claim responsibility for ci, gguf-py and convert

* add myself to various src/llama- files
 2025-09-22 10:59:05 +03:00
Sigbjørn SkjæretandGitHub ad6bd9083b cuda : add missing F32<->I32 entries in ggml_cuda_cpy_fn (#16060) 2025-09-18 13:28:22 +02:00
Sigbjørn SkjæretandGitHub b8e09f08b9 model : add grok-2 support (#15539) 
* add grok-2 support

* type fix

* type fix

* type fix

* "fix" vocab for invalid sequences

* fix expert tensor mapping and spaces in vocab

* add chat template

* fix norm tensor mapping

* rename layer_out_norm to ffn_post_norm

* ensure ffn_post_norm is mapped

* fix experts merging

* remove erroneous FFN_GATE entry

* concatenate split tensors and add more metadata

* process all expert layers and try cat instead of hstack

* add support for community BPE vocab

* fix expert feed forward length and ffn_down concat

* commit this too

* add ffn_up/gate/down, unsure if sequence is right

* add ffn_gate/down/up to tensor names

* correct residual moe (still not working)

* mess--

* fix embedding scale being applied twice

* add built in chat template

* change beta fast for grok if default value

* remove spm vocab in favor of community bpe vocab

* change attention temp length metadata type to integer

* update attention temp length metadata

* remove comment

* replace M_SQRT2 with std::sqrt(2)

* add yarn metadata, move defaults to hparams
 2025-09-14 23:00:59 +02:00
Sigbjørn SkjæretandGitHub 6c019cb04e server : only attempt to enable thinking if using jinja (#15967) 2025-09-14 21:17:04 +02:00
Sigbjørn SkjæretGitHubGeorgi Gerganov
 		6ab397e12b graph : support non-contiguous Q in build_attn_mha (#15908) 
* support non-contiguous Q in build_attn_mha

* Update src/llama-graph.cpp

ggml-ci

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-09-10 19:08:59 +02:00
Sigbjørn SkjæretandGitHub 5ef22d281d CUDA: non-contiguous src0 not supported for PAD (#15869) 2025-09-08 12:55:44 +03:00
Sigbjørn SkjæretandGitHub 4281c7b315 ci : exempt correct research label (#15825) 2025-09-06 01:21:15 +02:00
Sigbjørn SkjæretandGitHub 7d3c9f2b21 ci : explicitly set fa off or on (#15692) 2025-08-31 15:30:20 +02:00
Sigbjørn SkjæretandGitHub 84ab83cc0b model : jina-embeddings-v3 support (#13693) 
* initial jina-embeddings-v3 support

* initial jina-embeddings-v3 support

* initial jina-embeddings-v3 support

* fix vocab parsing with only tokenizer.json

* set mask token lstrip attribute

* additional unk_token_id fallback just in case [no ci]

* revert vocab_size() change [no ci]

* merge tensor loading into general bert

* rope

* add lora embedding and loading (non-functional)

* export separate lora ggufs instead

* add adapter metadata api

* use std::string

* convert_hf_to_lora compatibility

* fix assert

* apply suggestions from review

* apply suggestion from review
 2025-08-28 15:49:50 +02:00
Sigbjørn SkjæretandGitHub 39842a7f73 gguf-py : remove erroneous FFN_GATE entry (#15583) 2025-08-26 09:08:08 +02:00
Sigbjørn SkjæretandGitHub 0fd90db585 metal : remove contiguous assertion for src0 in IM2COL (#15577) 
* remove contiguous assertion for src0 in IM2COL

* add contiguous check in supports_op
 2025-08-26 09:51:43 +03:00
Sigbjørn SkjæretandGitHub baa9255a45 llama : merge conts and reshapes and remove unnecessary cont (#15380) 
* remove unnecessary conts and merge reshapes

* restore necessary conts

* merge more conts and reshapes

* merge even more conts and reshapes
 2025-08-18 19:30:17 +02:00
Sigbjørn SkjæretandGitHub 4d196981d4 convert : force patch_embd weights to F16 or F32 to avoid broken GGUFs (#15367) 
* force patch_embd weights to f32

* use MmprojModel base tensor_force_quant instead
 2025-08-17 14:47:42 +02:00
Sigbjørn SkjæretandGitHub b143fbc87a ci : fix hang in windows-hip build/release (#15365) 
* fix hang in windows-latest-cmake-hip

* apply fix to release as well
 2025-08-17 13:30:23 +02:00
Sigbjørn SkjæretandGitHub d3248d9b65 ci : fix ios-xcode-build (#15324) 
* fix ios-xcode-build

* use xcode-select with fixed version

* switch to macos-15 to get xcode 16.4
 2025-08-15 14:02:39 +02:00
Sigbjørn SkjæretandGitHub 4ebd0c125b cuda : fix GGML_CUDA_GRAPHS=OFF (#15300) 
* fix USE_CUDA_GRAPH=OFF

ggml-ci

* check capture status

* completely disable capturing check instead
 2025-08-14 13:22:07 +03:00
Sigbjørn SkjæretandGitHub b3e16665e1 server : enable -td and -tbd parameters (#15172) 2025-08-13 15:43:00 +02:00
Sigbjørn SkjæretandGitHub 07aa869a91 ci : add more python requirements to copilot-setup-steps (#15289) 
* ci : add flake8 and pyright to copilot-setup-steps.yml

* add tools/server/tests/requirements.txt
 2025-08-13 11:30:45 +02:00
Sigbjørn SkjæretandGitHub bc5182272c ci : add copilot-setup-steps.yml (#15214) 2025-08-13 09:07:13 +02:00
Sigbjørn SkjæretandGitHub 50e81bdf5d convert : fix merge conflicts (#15229) 2025-08-11 11:15:44 +02:00
Sigbjørn SkjæretandGitHub 65c797c4fa chat : fix yandex chat template (#15116) 2025-08-06 13:26:49 +02:00
Sigbjørn SkjæretandGitHub f324a3b715 chat : only remove double bos/eos if added (#15086) 
* only remove double bos/eos if added

* fix tests
 2025-08-05 20:43:36 +02:00
Sigbjørn SkjæretandGitHub e5bebe5251 gguf-py : add --chat-template-file to gguf_new_metadata (#15075) 2025-08-04 21:01:48 +02:00
Sigbjørn SkjæretandGitHub 2721257e3e quantize : fix confusing error message if ftype is invalid (#15071) 2025-08-04 18:11:02 +02:00
Sigbjørn SkjæretandGitHub 2bf3fbf0b5 ci : check that pre-tokenizer hashes are up-to-date (#15032) 
* torch is not required for convert_hf_to_gguf_update

* add --check-missing parameter

* check that pre-tokenizer hashes are up-to-date
 2025-08-02 14:39:01 +02:00
Sigbjørn SkjæretandGitHub 138b288b59 cuda : add softcap fusion (#14907) 2025-07-29 14:22:03 +02:00
Sigbjørn SkjæretandGitHub 221c0e0c58 ci : correct label refactor->refactoring (#14832) 2025-07-23 14:27:54 +02:00
Sigbjørn SkjæretandGitHub e28c0b80c2 cuda : implement bf16 cpy ops and enable bf16 cont (#14763) 
* implement bf16 cpy ops and enable bf16 cont

* deduplicate copy functions

* deduplicate checks
 2025-07-22 12:33:10 +02:00
Sigbjørn SkjæretandGitHub 38d3af1b73 opencl: fix im2col when KW!=KH (#14803) 2025-07-21 13:55:10 -07:00
Sigbjørn SkjæretandGitHub 1ba45d4982 ci : disable failing vulkan crossbuilds (#14723) 2025-07-16 20:52:08 -03:00
Sigbjørn SkjæretandGitHub 19e5943d9e convert : make hf token optional (#14717) 
* make hf token optional

* fail if we can't get necessary tokenizer config
 2025-07-16 23:17:43 +02:00
Sigbjørn SkjæretandGitHub 4b91d6f71f convert : only check for tokenizer folder if we need it (#14704) 2025-07-16 08:52:04 +02:00
Sigbjørn SkjæretandGitHub cf91f217f1 convert : add pre-computed hashes first to prevent order mishaps (#14701) 2025-07-16 08:51:12 +02:00
Sigbjørn SkjæretandGitHub 923e3ea2e3 cuda : add set rows for bf16 (#14664) 2025-07-13 15:01:24 +02:00
Sigbjørn SkjæretandGitHub 105554595f llama : remove unintended whitespace (#14592) 2025-07-09 10:19:50 +02:00
Sigbjørn SkjæretandGitHub e1a7059053 llama : fix incorrect minicpm3 v_states shape (#14571) 2025-07-07 23:35:35 +02:00
Sigbjørn SkjæretandGitHub 12f55c302b llama : remove ggml_cont where possible (#14568) 2025-07-07 21:35:08 +02:00
Sigbjørn SkjæretandGitHub ddef99522d server : fix assistant prefilling when content is an array (#14360) 2025-07-05 09:17:14 +02:00
Sigbjørn SkjæretandGitHub 6681688146 opencl: add GELU_ERF (#14476) 2025-07-04 23:24:56 -07:00
Sigbjørn SkjæretandGitHub 28657a8229 ggml : implement GEGLU_ERF and GEGLU_QUICK ops (#14445) 2025-07-03 23:07:22 +02:00
Sigbjørn SkjæretandGitHub e75ba4c043 gguf-py : add support for chat template jinja files (#14508) 
* add support for chat template jinja files

* remove gemma3n hack
 2025-07-02 21:02:35 +02:00