nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 27 Packages Projects Releases Wiki Activity
8,143 Commits 1 Branch 0 Tags
aa6f918c1c786668db530c3a1c3ff8a93da928f7
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Sigbjørn SkjæretandGitHub e8e261699a cli : provide model with text filename (#19783) 2026-02-22 22:33:49 +01:00
Sigbjørn SkjæretandGitHub 9f0684f003 ci : fix rocm archive name [no ci] (#19808) 2026-02-22 16:14:37 +01:00
Sigbjørn SkjæretandGitHub e877ad8bd9 ci : fix rocm release path [no ci] (#19784) 2026-02-22 08:07:46 +01:00
Sigbjørn SkjæretandGitHub e48349a49d ci : bump komac version (#19682) 2026-02-17 09:30:31 +01:00
Sigbjørn SkjæretandGitHub 079feab9e3 convert : ensure all models handle new experts count (#19621) 
* ensure all models handle new experts count

* revert removal for PhiMoeModel, does not inherit from base
 2026-02-14 22:22:32 +01:00
Sigbjørn SkjæretandGitHub 0d00ef65ed convert : store ffn_gate_inp_shexp as F32 (#19606) 2026-02-14 08:17:43 +01:00
Sigbjørn SkjæretandGitHub b2ecc0cdb4 support --verbose-prompt (#19576) 2026-02-13 12:49:10 +01:00
Sigbjørn SkjæretandGitHub 9a5f57795c ci : remove server job from webui and move slow test (#19424) 
* remove server job from webui and move slow test

* use pip-install option
 2026-02-08 01:20:00 +01:00
Sigbjørn SkjæretandGitHub b828e18c75 docker : fix vulkan build (#19352) 2026-02-05 11:10:39 +01:00
Sigbjørn SkjæretandGitHub 11fb327bf3 vendor : add missing llama_add_compile_flags (#19322) 
* add missing llama_add_compile_flags

* disable all warnings for ssl, crypto and fipsmodule
 2026-02-05 02:27:38 +01:00
Sigbjørn SkjæretandGitHub a6fd8ca1fe models : remove unnecessary cont in openelm (#19289) 2026-02-03 14:20:57 +01:00
Sigbjørn SkjæretandGitHub b45ef2702c jinja : do not pass empty tools and add some none filters (#19176) 2026-01-29 14:06:54 +01:00
Sigbjørn SkjæretandGitHub 50e8962f79 ci : find latest release with asset for winget (#19161) 2026-01-28 22:05:39 +01:00
Sigbjørn SkjæretandGitHub 60368e1d73 jinja : undefined should be treated as sequence/iterable (return string/array) by filters/tests (#19147) 
* undefined is treated as iterable (string/array) by filters

`tojson` is not a supported `undefined` filter

* add tests

* add sequence and iterable tests

keep it DRY and fix some types
 2026-01-28 14:40:29 +01:00
Sigbjørn SkjæretGitHubXuan Son Nguyen
 		2b4cbd2834 jinja : implement mixed type object keys (#18955) 
* implement mixed type object keys

* add tests

* refactor

* minor fixes

* massive refactor

* add more tests

* forgotten tuples

* fix array/object is_hashable

* correct (albeit broken) jinja responses

verified with transformers

* improved hashing and equality

* refactor hash function

* more exhausive test case

* clean up

* cont

* cont (2)

* missing cstring

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2026-01-27 19:50:42 +01:00
Sigbjørn SkjæretandGitHub c0204a0893 ci : revert slim runner for winget (#19129) 2026-01-27 11:54:25 +01:00
Sigbjørn SkjæretandGitHub 142cbe2ac6 ci : use new 1vCPU runner for lightweight jobs (#19107) 
* use new 1vCPU runner for lightweight jobs

* pyright is too heavy, look into ty some day

use new pip-install input
 2026-01-26 15:22:49 +01:00
Sigbjørn SkjæretandGitHub 0bf5636938 convert : yield Gemma3N custom_map tensors directly (#19091) 2026-01-25 18:03:34 +01:00
Sigbjørn SkjæretandGitHub 70d860824a convert : fix Gemma3N, GraniteMoe and Ernie4.5Moe (#19084) 
* fix Gemma3N and Ernie4.5Moe

* fix GraniteMoe
 2026-01-25 13:05:05 +01:00
Sigbjørn SkjæretandGitHub 959ecf7f23 jinja : fix undefined keys and attributes and int/float as bool (#18924) 
* fix undefined keys and attributes

* add falsy tests

* as_bool for integers and floats

* more falsy/truthy tests

* --typo
 2026-01-19 20:29:43 +01:00
Sigbjørn SkjæretandGitHub 4037093c66 ci : run test-jinja -py on high perf [no ci] (#18916) 2026-01-19 20:29:15 +01:00
Sigbjørn SkjæretandGitHub bbcdac0189 jinja : fix object item order (and properly implement dictsort) (#18904) 
* fix object item order

* as_ordered_object

* copy whole object
 2026-01-18 03:40:06 +01:00
Sigbjørn SkjæretandGitHub d03c45c9c5 jinja : attribute support for join, map and sort (#18883) 
* support negative array index and default value

* attribute support (int and str) for join, map and sort

* add tests

* update CODEOWNERS

* improve fixme sorting comment
 2026-01-18 02:53:01 +01:00
Sigbjørn SkjæretandGitHub 10c98cbdf6 jinja : add missing tojson filter for bool (#18900) 
* add missing tojson for bool

* add more literal tests
 2026-01-18 01:05:09 +01:00
Sigbjørn SkjæretandGitHub 420960ab92 jinja : fix lexing of float literals with sign (#18901) 
* fix lexing of float literals with sign

* add test

* consume_numeric
 2026-01-18 00:57:51 +01:00
Sigbjørn SkjæretandGitHub 57c0beaed0 ci : add label for jinja changes (#18903) 2026-01-17 21:52:02 +01:00
Sigbjørn SkjæretandGitHub 2a13180100 model-loader : support bool array sliding window pattern (#18850) 2026-01-15 10:12:46 +01:00
Sigbjørn SkjæretandGitHub 7fdc8c893d scripts : follow api redirects in pr2wt.sh (#18739) 2026-01-10 16:04:05 +01:00
Sigbjørn SkjæretandGitHub 5b8844ae53 scripts : fix repos cloned with .git extension (#18669) 2026-01-07 22:35:34 +01:00
Sigbjørn SkjæretandGitHub 7e16fef085 convert : more variants of rope_theta config entries (#18668) 2026-01-07 22:34:51 +01:00
Sigbjørn SkjæretandGitHub 9dfa8ee950 ci : run cann build unconditionally [no ci] (#18659) 2026-01-07 13:07:08 +01:00
Sigbjørn SkjæretandGitHub eadc4184ca llama : refactor rope_freq_base/scale_swa conversion and init (#18553) 
* refactor rope_freq_base/scale_swa conversion and init

* safe defaults for unknowns

* update relevant models

* grammar

* add get_rope_freq_scale to modern-bert

* const

* const

* log swa info
 2026-01-05 09:14:04 +01:00
Sigbjørn SkjæretandGitHub 169ee68ffb model : remove modern-bert iswa template (#18529) 
* remove modern-bert iswa template

* forgotten
 2026-01-02 00:06:42 +01:00
Sigbjørn SkjæretandGitHub 0db8109849 convert : lint fix (#18507) 2025-12-31 14:28:21 +01:00
Sigbjørn SkjæretandGitHub 4849661d98 docker : add CUDA 13.1 image build (#18441) 
* add updated cuda-new.Dockerfile for Ubuntu 24.04 compatibilty

* add cuda13 build
 2025-12-30 22:28:53 +01:00
Sigbjørn SkjæretandGitHub 74e05131e9 ci : remove non-windows zip artifacts (#18201) 
* remove non-windows zip artifacts

* add cuda dll links
 2025-12-19 22:29:46 +01:00
Sigbjørn SkjæretandGitHub f74747d886 ci : only save ccache on master (#18207) 2025-12-19 22:29:37 +01:00
Sigbjørn SkjæretandGitHub ec7b9329ae gguf-py : use copy-on-write mode for localtensor (#18162) 2025-12-18 13:45:38 +01:00
Sigbjørn SkjæretandGitHub 9cff4cc554 convert : sort and use file parts from model index if present (#18043) 
* keep file part order from model index

* treat index as authoritative

* sort index parts
 2025-12-18 07:54:54 +01:00
Sigbjørn SkjæretandGitHub 5166aaf868 convert : force patch_merger tensors to f16/f32 (#18124) 2025-12-17 22:15:53 +01:00
Sigbjørn SkjæretandGitHub 6853bee680 ci : clean up webui jobs (#18116) 
* clean up webui jobs

* refined step control

* forgot dependencies

* apparently always() is needed
 2025-12-17 10:45:40 +01:00
Sigbjørn SkjæretandGitHub d6742125c3 ci : separate webui from server (#18072) 
* separate webui from server

* add public to path
 2025-12-16 08:17:26 +01:00
Sigbjørn SkjæretandGitHub d6a1e18c65 convert : move rope_parameters to TextModel class (#18061) 
* make sure to search text_config for rope parameters

* move rope_parameters to TextModel class
 2025-12-15 22:03:16 +01:00
Sigbjørn SkjæretandGitHub 5c8a717128 convert : refactor rope scaling handling (#18013) 
* refactor rope scaling handling

* ws--

* missed a couple

* use find_hparam
 2025-12-14 16:04:37 +01:00
Sigbjørn SkjæretandGitHub 8e4d678528 common : skip model validation when --completion-bash is requested (#17975) 2025-12-13 08:40:50 +01:00
Sigbjørn SkjæretandGitHub 2bc94e7928 add llama-completion to completion-bash executables (#17976) 2025-12-13 08:35:50 +01:00
Sigbjørn SkjæretandGitHub b7f5f46e03 docker : include legacy llama-completion binary (#17964) 2025-12-12 19:39:23 +01:00
Sigbjørn SkjæretandGitHub dcb7d17758 cann : fix ops broken by circular padding guard (#17825) 2025-12-12 15:49:27 +01:00
Sigbjørn SkjæretandGitHub 45e350e3d3 ci: fix riscv64-native build (#17916) 2025-12-10 23:24:31 +01:00
Sigbjørn SkjæretandGitHub 4df6e859e9 cuda : add missing support check for xielu (#17895) 2025-12-10 16:16:20 +01:00
Sigbjørn SkjæretandGitHub 63391852b0 docs : update cpu and cuda ops (#17890) 
* update cuda ops

* update CPU as well
 2025-12-09 23:31:29 +01:00
Sigbjørn SkjæretandGitHub 86a3f0fad8 ggml : allow fill node alloc inplace (#17870) 2025-12-09 12:23:47 +01:00
Sigbjørn SkjæretandGitHub 42b12b5608 model : nit, DeepSeek V1 MoE is 16B and GigaChat is 20B (#12652) 
* nit, DeepSeek V1 MoE is 16B

* base type on n_ff_exp instead
 2025-12-09 12:15:06 +01:00
Sigbjørn SkjæretandGitHub c8554b66e0 graph : use fill instead of scale_bias in grouped expert selection (#17867) 
* use fill instead of scale_bias in grouped expert selection

* do not explicitly use _inplace
 2025-12-08 21:29:59 +01:00
Sigbjørn SkjæretandGitHub 0a540f9abd ci : add windows-cuda 13.1 release (#17839) 2025-12-07 14:02:04 +01:00
Sigbjørn SkjæretandGitHub 22577583a3 common : change --color to accept on/off/auto, default to auto (#17827) 2025-12-07 03:43:50 +01:00
Sigbjørn SkjæretandGitHub 03d9a77b85 ci : transform release binary root dir in tar to llama-bXXXX (#17773) 
* transform release binary root dir in tar to llama-bXXXX

* bsdtar supports -s instead of --transform
 2025-12-05 01:50:19 +01:00
Sigbjørn SkjæretandGitHub 7dba049b07 ci : disable ggml-ci-x64-amd-* (#17753) 2025-12-04 11:25:08 +01:00
Sigbjørn SkjæretandGitHub b3e3060f4e ci : move release details to the top visible by default (#17719) 2025-12-03 09:22:46 +01:00
Sigbjørn SkjæretandGitHub b61de2b2df convert : allow quantizing lora again (#17453) 2025-11-24 15:50:55 +01:00
Sigbjørn SkjæretandGitHub 96ac5a2329 cuda : support non-contiguous i32 to i32 copy (#17326) 
* support non-contiguous i32 to i32 copy

* add tests

* rename cpy_flt to cpy_scalar and reindent params
 2025-11-23 11:13:34 +01:00
Sigbjørn SkjæretandGitHub 07b0e7a5ac convert : use self.block_count everywhere instead of reading hparams (#17359) 2025-11-19 11:52:38 +01:00
Sigbjørn SkjæretandGitHub 662192e1dc convert : remove unnecessary chat template patching (#17289) 2025-11-15 20:58:59 +01:00
Sigbjørn SkjæretandGitHub 9a8860cf5d convert : use all parts in safetensors index (#17286) 2025-11-15 14:12:39 +01:00
Sigbjørn SkjæretandGitHub 9d3ef4809f convert : set expert gating func in base class (#17279) 2025-11-15 14:06:24 +01:00
Sigbjørn SkjæretandGitHub 7bef684118 models : move build_inp_out_ids outside loop (#17151) 
* move build_inp_out_ids outside loop

* realign
 2025-11-10 22:55:30 +01:00
Sigbjørn SkjæretandGitHub 9008027aa3 hparams : add n_embd_inp() to support extended embed (#16928) 
* add n_embd_full to support extended embed

* don't change output

* rename to n_embd_inp

* restore n_embd where applicable
 2025-11-07 19:27:58 +01:00
Sigbjørn SkjæretandGitHub d945834366 ci : apply model label to models (#16994) 2025-11-04 12:29:39 +01:00
Sigbjørn SkjæretandGitHub b164259bba chore : fix models indent after refactor (#16992) 2025-11-04 12:29:15 +01:00
Sigbjørn SkjæretandGitHub dd52868050 ci : disable failing riscv cross build (#16952) 2025-11-02 23:11:21 +01:00
Sigbjørn SkjæretandGitHub 961660b8c3 common : allow --system-prompt-file for diffusion-cli (#16903) 2025-11-01 11:01:42 +01:00
Sigbjørn SkjæretandGitHub 74fef4129f codeowners : update after refactor (#16905) 2025-11-01 09:55:25 +02:00
Sigbjørn SkjæretandGitHub 6eb208d17e ci : enable free-disk-space on cuda docker build (#16877) 2025-10-31 00:34:27 +01:00
Sigbjørn SkjæretandGitHub 229bf68628 cuda : fix argsort with 64k+ rows (#16849) 2025-10-30 08:56:28 +01:00
Sigbjørn SkjæretandGitHub 144a4ce824 vendor : sync minja (#16500) 
* sync minja.hpp

Adds Call/EndCall support, used in MiniCPM3 and MiniCPM4-MCP.

* remove spurious semicolon

* sync from ochafik/minja
 2025-10-29 14:09:50 +01:00
Sigbjørn SkjæretandGitHub bd562fe4f7 cuda : use fast copy when src and dst are of different type and contiguous (#16789) 
* use fast copy when src and dst are contiguous and same shape

* use int64_t ne and ignore shape
 2025-10-26 21:31:41 +01:00
Sigbjørn SkjæretandGitHub 73a48c9790 convert : enable expert group selection for all models with it (#16691) 2025-10-26 17:21:23 +01:00
Sigbjørn SkjæretandGitHub f696428ce8 graph : add clamping to ffn_moe_weights_sum to avoid div-by-zero (#16655) 
* add missing norm topk bias

* use clamping instead, update number and add comment
 2025-10-26 17:20:32 +01:00
Sigbjørn SkjæretandGitHub 7cce4f8158 model : set res->t_embd in SmallThinker models (#16782) 2025-10-26 16:08:52 +01:00
Sigbjørn SkjæretandGitHub 84bf3c6778 model : add BailingMoeV2 support (#16063) 
* add BailingMoeV2 support

* update llm types

* undo

* undo

* update llm types

* add model collection link

* update

* almost working

* correct group selection and rename n_group_exp

* avoid large top_k and use argmax instead for now

if we had something like argmax2 that would be equivalent, but this works fine until then

* poke

* skip group selection when there are no tokens

* fix 1T conversion

* hopefully fixed expert group selection

third time's the charm?

* make expert group selection generally available

The new LLaDA2Moe model uses this method too, make it generally available regardless of architecture.

* allow n_expert_groups to be 1 (Kimi K2)

* address review suggestions
 2025-10-20 21:38:20 +02:00
Sigbjørn SkjæretandGitHub cec5edbcae ci : avoid manual updates of docs/ops.md (#16663) 2025-10-19 14:03:25 +02:00
Sigbjørn SkjæretandGitHub 3a002afafa ci : refactor sdk caching to minimize storage (#16414) 
* refactor sdk caching to minimize storage

* use correct action

* add myself as owner to /.github/actions/ [no ci]
 2025-10-06 17:40:21 +02:00
Sigbjørn SkjæretandGitHub 946f71ed9a llama : fix shapes for bert/mpt q/k norm (#16409) 2025-10-03 14:40:25 +02:00
Sigbjørn SkjæretGitHubJeff Bolz
 		d64c8104f0 test-barrier : do not use more threads than physically available (#16389) 
* do not use more threads than physically available

* ensure n_threads > 0

Co-authored-by: Jeff Bolz <jbolz@nvidia.com>

---------

Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
 2025-10-02 20:10:12 +02:00
Sigbjørn SkjæretandGitHub 72ee736c44 ci : fix ubuntu-latest-cmake-rpc (disable ccache) (#16388) 2025-10-02 13:51:36 +02:00
Sigbjørn SkjæretandGitHub 1104ca1a1c ci : use registry cache for docker builds (#16366) 2025-10-01 14:09:52 +02:00
Sigbjørn SkjæretandGitHub b2ba81dbe0 ci : fix ccache key for ubuntu-cpu-cmake (#16355) 
* fix ccache key for ubuntu-cpu-cmake

* set it for release as well [no ci]
 2025-09-30 21:41:42 +02:00
Sigbjørn SkjæretandGitHub 2df5bcf357 ci : disable ccache for android (#16348) 2025-09-30 15:38:01 +02:00
Sigbjørn SkjæretandGitHub adc76347d7 ggml : check cuda and metal argsort limits and add test (#16323) 
* check cuda argsort limits and add test

* add metal check
 2025-09-29 11:09:00 +02:00
Sigbjørn SkjæretandGitHub b887d2f341 ggml : fix GGML_F32_VEC_FMA argument order in ggml_vec_mad1_f32 (#16307) 
* fix GGML_F32_VEC_FMA argument order in ggml_vec_mad1_f32

* add test that fails on simd
 2025-09-28 23:15:03 +02:00
Sigbjørn SkjæretandGitHub 835b2b915c model : add GroveMoE support (#15510) 
* add GroveMoE support

* remove constexpr that fails on certain compilers

* revert crude scalar div implementation, use cast

* build_attn_inp_kv_unified -> build_attn_inp_kv

* fix build_attn

* re-apply ffn_exps regex changes
 2025-09-25 19:50:28 +02:00
Sigbjørn SkjæretandGitHub c0c59c1157 codeowners : match all requirements files (#16214) 2025-09-24 08:53:20 +02:00
Sigbjørn SkjæretandGitHub f6b4af3d04 ggml : fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl (#15928) 
* fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl

* change initialization to true
 2025-09-23 10:25:20 +02:00
Sigbjørn SkjæretGitHubGeorgi GerganovJeff Bolz
 		3ecb2f671a ggml : implement set_rows with i32 index (#16159) 
* implement set_rows with i32 index

* template fix

* test quantized path

warnings--

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* forgotten name change

* deduplicate cuda/sycl and test-fix

* indent++

* vulkan: support set_rows with i32 index type (#16162)

* disable i32 index for webgpu for now

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
 2025-09-22 19:13:00 +02:00
Sigbjørn SkjæretandGitHub b2d980fce0 codeowners : claim responsibility for ci, models, gguf-py and convert (#16124) 
* claim responsibility for ci, gguf-py and convert

* add myself to various src/llama- files
 2025-09-22 10:59:05 +03:00
Sigbjørn SkjæretandGitHub ad6bd9083b cuda : add missing F32<->I32 entries in ggml_cuda_cpy_fn (#16060) 2025-09-18 13:28:22 +02:00
Sigbjørn SkjæretandGitHub b8e09f08b9 model : add grok-2 support (#15539) 
* add grok-2 support

* type fix

* type fix

* type fix

* "fix" vocab for invalid sequences

* fix expert tensor mapping and spaces in vocab

* add chat template

* fix norm tensor mapping

* rename layer_out_norm to ffn_post_norm

* ensure ffn_post_norm is mapped

* fix experts merging

* remove erroneous FFN_GATE entry

* concatenate split tensors and add more metadata

* process all expert layers and try cat instead of hstack

* add support for community BPE vocab

* fix expert feed forward length and ffn_down concat

* commit this too

* add ffn_up/gate/down, unsure if sequence is right

* add ffn_gate/down/up to tensor names

* correct residual moe (still not working)

* mess--

* fix embedding scale being applied twice

* add built in chat template

* change beta fast for grok if default value

* remove spm vocab in favor of community bpe vocab

* change attention temp length metadata type to integer

* update attention temp length metadata

* remove comment

* replace M_SQRT2 with std::sqrt(2)

* add yarn metadata, move defaults to hparams
 2025-09-14 23:00:59 +02:00
Sigbjørn SkjæretandGitHub 6c019cb04e server : only attempt to enable thinking if using jinja (#15967) 2025-09-14 21:17:04 +02:00
Sigbjørn SkjæretGitHubGeorgi Gerganov
 		6ab397e12b graph : support non-contiguous Q in build_attn_mha (#15908) 
* support non-contiguous Q in build_attn_mha

* Update src/llama-graph.cpp

ggml-ci

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-09-10 19:08:59 +02:00
Sigbjørn SkjæretandGitHub 5ef22d281d CUDA: non-contiguous src0 not supported for PAD (#15869) 2025-09-08 12:55:44 +03:00