nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 24 Packages Projects Releases Wiki Activity
9,508 Commits 1 Branch 0 Tags
a121232fdc80ad93587adfce2c56452c9052b850
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan-Son NguyenandGitHub a121232fdc agents: refactor, include more guidelines (#24111) 
* agents: refactor, include more guidelines

* better example

* rephrase a bit

* add more examples

* nits
 2026-06-04 13:40:23 +02:00
Xuan-Son NguyenandGitHub c8d6a00636 mtmd: enable non-causal vision for gemma 4 unified (#24082) 2026-06-03 19:05:17 +02:00
Xuan-Son NguyenandGitHub a731805ced mtmd, model: allow skip build_vit() (#24077) 
* add model

* nits
 2026-06-03 17:10:35 +02:00
Xuan-Son NguyenandGitHub 60130d18f9 server: add SSE ping interval (#24013) 2026-06-02 14:14:55 +02:00
Xuan-Son NguyenandGitHub e674b1279b test: (test-llama-archs) log the config name first (#23885) 2026-05-30 12:22:38 +02:00
Xuan-Son NguyenandGitHub 0821c5fcfd server: in SSE mode, send HTTP headers when slot starts (#23884) 
* server: in SSE mode, send HTTP headers when slot starts

* ref to pr

* stream should be false by default
 2026-05-30 00:06:29 +02:00
Xuan-Son NguyenandGitHub b5f52280fb server: remove obsolete scripts (#23870) 2026-05-29 19:47:30 +02:00
Xuan-Son NguyenandGitHub 06d26dfdff download: add option to skip_download (#23059) 
* download: add option to skip_download

* fix

* fix 2

* if file doesn't exist, respect skip_download flag
 2026-05-29 16:30:55 +02:00
Xuan-Son NguyenandGitHub cb47092b00 server: bump timeout to 3600s (#23842) 
* server: bump timeout to 3600s

* nits: change wording
 2026-05-29 10:23:17 +02:00
Xuan-Son NguyenandGitHub 751ebd17a5 mtmd-debug: add color and rainbow mode (#23829) 
* mtmd-debug: add color and rainbow mode

* fix M_PI

* max_dist
 2026-05-28 20:59:14 +02:00
Xuan-Son NguyenandGitHub c8914ad4f4 mtmd: fix gemma 4 projector pre_norm (#23822) 2026-05-28 20:58:55 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		d6be3158e1 mtmd: fix gemma 4 audio rms norm eps (#23815) 
* mtmd: fix gemma 4 audio rms norm eps

* Update tools/mtmd/clip.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-05-28 16:31:37 +02:00
Xuan-Son NguyenandGitHub e2b129e1bf mtmd: fit_params now take into account mmproj (#21489) 
* mtmd: fit_params now take into account mmproj

* rename alloc_compute_meta to reserve_compute_meta

* rm unused functions

* add ggml_backend_dev_t support

* add debug log
 2026-05-20 11:27:44 +02:00
Xuan-Son NguyenandGitHub b64739ea39 server: (router) alloc tmp buffer on heap (#23159) 2026-05-16 23:42:16 +02:00
Xuan-Son NguyenandGitHub 72e60f500d mtmd: add chunks and fix preproc for qwen3a (#23073) 
* mtmd: add chunks and fix preproc for qwen3a

* add attn_mask

* limit mtmd_chunk size (avoid blow up memory)

* correct audio tokens

* re-order the set_input case

* remove attn_mask
 2026-05-15 19:32:47 +02:00
Xuan-Son NguyenandGitHub e75cd5efb5 download: do not exit() on error (#23008) 2026-05-13 15:14:58 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		3796c94bad ci: validate model naming convention (#22680) 
* ci: validate model naming convention

* bring back dedicated ec workflow

* add missing jobs

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-05-13 10:59:37 +02:00
Xuan-Son NguyenandGitHub 7bfe120c21 mtmd, server, common: expose modalities to /v1/models (#22952) 
* mtmd, server, common: expose modalities to /v1/models

* fix build

* rename to mtmd_caps
 2026-05-12 19:08:07 +02:00
Xuan-Son NguyenandGitHub 29debb3a6a server: support Vertex AI compatible API (#22545) 
* server: support Vertex AI compatible API

* a bit safer

* support other AIP_* env var

* various fixes

* if AIP_MODE is unset, do nothing

* fix test case

* fix windows build
 2026-05-08 15:23:04 +02:00
Xuan-Son NguyenandGitHub 9dcf835528 server: (router) expose child model info from router's /v1/models (#22683) 
* server: (router) expose child model info from router's /v1/models

* update docs
 2026-05-08 14:42:15 +02:00
Xuan-Son NguyenandGitHub 935a340292 server: implement /models?reload=1 (#21848) 2026-05-04 16:23:26 +02:00
Xuan-Son NguyenandGitHub 994118a183 model: move load_hparams and load_tensors to per-model definition (#22004) 
* git-friendly migration

* add build_graph

* nits

* exclude old code from build

* wip

* add llm_arch_model_i

* prepare downstream functions

* nits

* nits

* wip

* wip

* add back create_tensor_qkv

* fix files missing include

* enforce one llm_build per arch

* cmake: use glob

* missing model params

* nits

* wip

* wip (2)

* wip (3)

* test-llama-archs is happy

* improve switch case

* move more stuff into llm_arch_model_i

* fix downstream code

* nits

* nits (2)

* fix order

* llama_model_base

* LLAMA_LOAD_LOCALS

* small fix

* fix build errors

* auto

* rm migration script and ifdef
 2026-05-04 12:36:59 +02:00
Xuan-Son NguyenandGitHub 82d3f4d3b2 mtmd: also support LLAMA_ROPE_TYPE_NONE (#22242) 2026-04-22 12:16:29 +02:00
Xuan-Son NguyenandGitHub 17f6245168 server: ignore reasoning content from transcription api (#21905) 2026-04-22 12:10:50 +02:00
Xuan-Son NguyenandGitHub 04fe84b69d server: allow cancel loading model (#21814) 2026-04-22 00:26:09 +02:00
Xuan-Son NguyenandGitHub 9998d88bc8 mtmd: correct mtmd_decode_use_mrope() (#22188) 2026-04-21 10:53:37 +02:00
Xuan-Son NguyenandGitHub 86f8daacfe mtmd: correct get_n_pos / get_decoder_pos (#22175) 2026-04-20 23:29:19 +02:00
Xuan-Son NguyenandGitHub a678916623 mtmd: refactor mtmd_decode_use_mrope (#22161) 2026-04-20 14:45:11 +02:00
Xuan-Son NguyenandGitHub 19124078be mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos (breaking change) (#22082) 
* mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos

* fix build
 2026-04-19 11:57:21 +02:00
Xuan-Son NguyenandGitHub 089dd41fe3 cmake: use glob to collect src/models sources (#22005) 2026-04-16 23:25:16 +02:00
Xuan-Son NguyenandGitHub 4fbdabdc61 model: using single llm_build per arch (#21970) 
* model: using single llm_build per arch

* fix merge

* nits
 2026-04-16 21:10:22 +02:00
Xuan-Son NguyenandGitHub 408225bb1a server: use random media marker (#21962) 
* server: use random media marker

* nits

* remove legacy <__image__> token

* revert special char in random
 2026-04-15 23:52:22 +02:00
Xuan-Son NguyenandGitHub 80d8770804 docs: more extensive RoPE documentation [no ci] (#21953) 
* more extensive ggml_rope documentation

* add more docs

* nits
 2026-04-15 14:45:16 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		fae3a28070 ggml : remove ggml-ext.h (#21869) 
* ggml: correct placement of ggml-ext.h

* ggml : remove ggml-ext.h

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-04-14 17:32:58 +03:00
Xuan-Son NguyenandGitHub 707c0b7a6e mtmd: add mtmd_image_tokens_get_decoder_pos() API (#21851) 
* mtmd: add mtmd_image_tokens_get_decoder_pos() API

* consistent naming

* fix build
 2026-04-14 16:07:41 +02:00
Xuan-Son NguyenandGitHub e489a5ca0e server: support OAI /v1/audio/transcriptions API (#21863) 
* server: support OAI /v1/audio/transcriptions API

* address autoreview comments

* correct default response_format value
 2026-04-14 11:09:52 +02:00
Xuan-Son NguyenandGitHub e974923698 docs: listing qwen3-asr and qwen3-omni as supported (#21857) 
* docs: listing qwen3-asr and qwen3-omni as supported

* nits
 2026-04-13 22:28:17 +02:00
Xuan-Son NguyenandGitHub 920b3e78cb mtmd: use causal attn for gemma 4 audio (#21824) 2026-04-13 09:47:55 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		21a4933042 mtmd: qwen3 audio support (qwen3-omni and qwen3-asr) (#19441) 
* add qwen3a

* wip

* vision ok

* no more deepstack for audio

* convert ASR model ok

* qwen3 asr working

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* nits

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix bad merge

* fix multi inheritance

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-04-12 23:57:25 +02:00
Xuan-Son NguyenandGitHub aa4695c5e5 mtmd: add gemma 4 test (vision + audio) [no ci] (#21806) 
* mtmd: add gemma 4 test (vision + audio)

* add to docs
 2026-04-12 16:29:03 +02:00
Xuan-Son NguyenandGitHub 6313acbef0 docs: add guide on how to add multimodal support (#21778) 
* docs: add guide on how to add multimodal support

* nits
 2026-04-12 13:02:38 +02:00
Xuan-Son NguyenandGitHub 057dba336e model: fix multimodal padding token for gemma3n/gemma4 (#21625) 
* model: fix multimodal padding token for gemma3n/gemma4

* nits
 2026-04-09 12:18:23 +02:00
Xuan-Son NguyenandGitHub 501aeed18f mtmd: support dots.ocr (#17575) 
* convert gguf

* clip impl

* fix conversion

* wip

* corrections

* update docs

* add gguf to test script
 2026-04-09 12:16:38 +02:00
Xuan-Son NguyenandGitHub 3979f2bb08 docs: add hunyuan-ocr gguf, also add test [no ci] (#21490) 2026-04-06 14:02:37 +02:00
Xuan-Son NguyenandGitHub 63f8fe0ef4 model, mtmd: fix gguf conversion for audio/vision mmproj (#21309) 
* fix gguf conversion for audio/vision mmproj

* fix test
 2026-04-02 17:10:32 +02:00
Xuan-Son NguyenandGitHub c30e012253 contrib : rewrite AGENTS.md, make it more clear about project values (#21270) 
* contrib : rewrite AGENTS.md, make it more clear about types of permitted AI usage

* permit AI for writing code
 2026-04-01 23:31:51 +02:00
Xuan-Son NguyenandGitHub 0356e33aaf scripts: add function call test script (#21234) 
* scripts: add function call test script

* add reasoning_content

* fix lint
 2026-04-01 15:31:58 +02:00
Xuan-Son NguyenGitHubAleksander Grygier
 		4a00bbfed6 server: (webui) no more gzip compression (#21073) 
* webui: no more gzip

* try changing a small line

* Revert "try changing a small line"

This reverts commit 0d7a3531593d87b724d404c8727a96becab3ab07.

* fix lint

* fix test

* rebuild

* split into html/css/js

* lint

* chore: update webui build output

* chore: Update git hooks script

* server: update webui build output

* chore: Update pre-commit hook

* refactor: Cleanup

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
 2026-03-31 15:44:26 +02:00
Xuan-Son NguyenGitHubGeorgi GerganovAleksander Grygier
 		abf9a62161 server: wrap headers for mcp proxy (#21072) 
* server: wrap headers for mcp proxy

* Update tools/server/server-cors-proxy.h

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix build

* chore: update webui build output

* chore: update webui build output

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
 2026-03-30 08:59:16 +02:00
Xuan-Son NguyenandGitHub 871f1a2d2f mtmd: add more sanity checks (#21047) 2026-03-27 11:00:52 +01:00
Xuan-Son NguyenandGitHub 20197b6fe3 server: add built-in tools backend support (#20898) 
* wip: server_tools

* refactor

* displayName -> display_name

* snake_case everywhere

* rm redundant field

* change arg to --tools all

* add readme mention

* llama-gen-docs
 2026-03-27 10:07:11 +01:00
Xuan-Son NguyenandGitHub 8c60b8a2be ci: pin external actions to exact commit SHA (#21033) 2026-03-26 20:44:00 +01:00
Xuan-Son NguyenandGitHub a73bbd5d92 mtmd: refactor image preprocessing (#21031) 
* mtmd: refactor image pre-processing

* correct some places

* correct lfm2

* fix deepseek-ocr on server

* add comment to clarify about mtmd_image_preprocessor_dyn_size
 2026-03-26 19:49:20 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		914eb5ff0c jinja: fix macro with kwargs (#20960) 
* jinja: fix macro with kwargs

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix newline problem

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-03-25 12:22:48 +01:00
Xuan-Son NguyenGitHubPiotr Wilkin
 		bd6992180b contrib: add "Requirements" section to PR template (#20841) 
* contrib: add "Requirements" section to PR template

* typo [no ci]

* use h2, add "Additional information"

---------

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>
 2026-03-23 16:59:02 +01:00
Xuan-Son NguyenandGitHub 31a5cf4c3f server: use httplib dynamic threads (#20817) 
* server: use httplib dynamic threads

* change to n_threads_http + 1024
 2026-03-23 12:22:46 +01:00
Xuan-Son NguyenandGitHub 49bfddeca1 server: allow router to report child instances sleep status (#20849) 
* server: allow router to report child instances sleep status

* refactor

* move sleeping to state

* nits
 2026-03-22 18:33:52 +01:00
Xuan-Son NguyenandGitHub 568aec82d2 docs : explicit about banning accounts that violates policy (#19593) 2026-03-21 15:50:16 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		fb78ad29bb server: (doc) clarify in-scope and out-scope features (#20794) 
* server: (doc) clarify in-scope and out-scope features

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-03-20 14:03:50 +01:00
Xuan-Son NguyenandGitHub 1e64534570 mtmd: add clip_graph::build_mm() (#20751) 
* clip: add build_mm()

* apply to all models

* add TODO for bias overload
 2026-03-19 13:11:39 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		d34ff7eb5b model: mistral small 4 support (#20649) 
* model: mistral small 4 support

* fix test

* fix test (2)

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* change newline

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-03-17 00:31:14 +01:00
Xuan-Son NguyenandGitHub 94d0262277 mtmd: add llama-mtmd-debug binary (#20508) 
* mtmd: add llama-mtmd-debug binary

* adapt

* fixes

* fix compile error

* fix windows compile error

* rm legacy clip_debug_encode()

* add MTMD_API to fix build
 2026-03-14 15:52:29 +01:00
Xuan-Son NguyenandGitHub 0842b9b465 model: fix step3.5 n_rot (#20318) 2026-03-09 23:42:24 +01:00
Xuan-Son NguyenandGitHub 59db9a357d llama: dynamic head_dim and n_rot for SWA (#20301) 
* llama: dynamic head_dim and n_rot for SWA

* also add gguf_writer wrappers

* fix build

* build_rope_shift arg reorder
 2026-03-09 22:22:39 +01:00
Xuan-Son NguyenandGitHub c747294b2d scripts: update corpus of compare-logprobs (#19326) 
* scripts: update corpus of compare-logprobs

* fix
 2026-02-25 12:57:34 +01:00
Xuan-Son NguyenandGitHub 5452d736f8 jinja: correct stats for tojson and string filters (#19785) 2026-02-22 21:08:23 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		eeef3cfced model: support GLM-OCR (#19677) 
* model: support GLM-OCR

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-02-18 17:51:40 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		752584d5f5 model: support GLM MoE DSA arch (NOTE: indexer is not yet supported) (#19460) 
* model: support GLM MoE DSA arch

* working version

* pyright

* keep indexer tensors

* add indexer gguf params

* loaded now

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* update

* Update src/llama-model.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* minor fix and cleanup

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-02-13 14:56:53 +01:00
Xuan-Son NguyenandGitHub 9a96352729 test: fix IMROPE perf test case (#19465) 2026-02-10 14:37:50 +01:00
Xuan-Son NguyenandGitHub 98e57ca422 chat: fix case where template accepts type content only (#19419) 
* chat: fix case where template accepts type content only

* rm stray log

* reuse render_message_to_json
 2026-02-09 22:14:12 +01:00
Xuan-Son NguyenandGitHub e0c93af2a0 debug: make common_debug_print_tensor readable (#19331) 
* debug: make common_debug_print_tensor readable

* editorconfig
 2026-02-04 17:55:31 +01:00
Xuan-Son NguyenandGitHub 8abcc70a74 model: (qwen3next) correct vectorized key_gdiff calculation (#19324) 
* model: (qwen3next) correct vectorized key_gdiff calculation

* move transpose to outside of loop
 2026-02-04 13:09:58 +01:00
Xuan-Son NguyenandGitHub 07a7412a3b mtmd: add min/max pixels gguf metadata (#19273) 2026-02-02 20:59:06 +01:00
Xuan-Son NguyenGitHubAldehir Rojas
 		bb02f74c61 chat: fix language input for translategemma (#19052) 
* chat: fix language input for translategemma

* Update common/chat.cpp

Co-authored-by: Aldehir Rojas <hello@alde.dev>

---------

Co-authored-by: Aldehir Rojas <hello@alde.dev>
 2026-01-24 17:58:45 +01:00
Xuan-Son NguyenandGitHub b5b8fa1c8b chat : fix translategemma crash on common_chat_format_example (#19019) 2026-01-23 12:03:42 +01:00
Xuan-Son NguyenandGitHub 51fa458a92 server : support preserving reasoning_content in assistant message (#18994) 
* support reasoning_content input

* report template caps to webui

* add docs

* rm commented code
 2026-01-22 21:30:06 +01:00
Xuan-Son NguyenandGitHub 4e595b250a server: do not log certain endpoints (avoid log spam) (#19028) 2026-01-22 19:24:37 +01:00
Xuan-Son NguyenandGitHub 9eb5bfec1a mtmd : update docs to use llama_model_n_embd_inp (#18999) 2026-01-22 14:36:32 +01:00
Xuan-Son NguyenandGitHub 2c1f199653 cli : fix reasoning responses in CLI (#18961) 
* cli : fix reasoning responses in CLI

* fix build

* fix build (2)
 2026-01-20 18:23:25 +01:00
Xuan-Son NguyenandGitHub 6df686bee6 server : refactor oai_parser_opt, move it to server_chat_params (#18937) 
* server_chat_params

* move chat format into CLI

* use meta whenever possible

* clean up, no more chatml fallback
 2026-01-19 23:28:01 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		fe44d35574 tests : add test-jinja -py option for cross-checking (#18906) 
* tests : add test-jinja -py option or cross-checking

* Update tests/test-jinja.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix + add source

* SandboxedEnvironment

* fix array.map case

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-01-18 08:14:27 +01:00
Xuan-Son NguyenandGitHub f55b033ae6 jinja: correct member access rule (#18905) 2026-01-18 00:48:55 +01:00
Xuan-Son NguyenGitHubAlde RojasSigbjørn SkjæretGeorgi Gerganov
 		c15395f73c common : implement new jinja template engine (#18462) 
* jinja vm

* lexer

* add vm types

* demo

* clean up

* parser ok

* binary_expression::execute

* shadow naming

* bin ops works!

* fix map object

* add string builtins

* add more builtins

* wip

* use mk_val

* eval with is_user_input

* render gemma tmpl ok

* track input string even after transformations

* support binded functions

* keyword arguments and slicing array

* use shared_ptr for values

* add mk_stmt

* allow print source on exception

* fix negate test

* testing more templates

* mostly works

* add filter_statement

* allow func to access ctx

* add jinja-value.cpp

* impl global_from_json

* a lot of fixes

* more tests

* more fix, more tests

* more fixes

* rm workarounds

* demo: type inferrence

* add placeholder for tojson

* improve function args handling

* rm type inference

* no more std::regex

* trailing spaces

* make testing more flexible

* make output a bit cleaner

* (wip) redirect minja calls

* test: add --output

* fix crash on macro kwargs

* add minimal caps system

* add some workarounds

* rm caps_apply_workarounds

* get rid of preprocessing

* more fixes

* fix test-chat-template

* move test-chat-jinja into test-chat-template

* rm test-chat-jinja from cmake

* test-chat-template: use common

* fix build

* fix build (2)

* rename vm --> interpreter

* improve error reporting

* correct lstrip behavior

* add tojson

* more fixes

* disable tests for COMMON_CHAT_FORMAT_GENERIC

* make sure tojson output correct order

* add object.length

* fully functional selectattr / rejectattr

* improve error reporting

* more builtins added, more fixes

* create jinja rendering tests

* fix testing.h path

* adjust whitespace rules

* more fixes

* temporary disable test for ibm-granite

* r/lstrip behavior matched with hf.js

* minimax, glm4.5 ok

* add append and pop

* kimi-k2 ok

* test-chat passed

* fix lstrip_block

* add more jinja tests

* cast to unsigned char

* allow dict key to be numeric

* nemotron: rm windows newline

* tests ok

* fix test

* rename interpreter --> runtime

* fix build

* add more checks

* bring back generic format support

* fix Apertus

* [json.exception.out_of_range.403] key 'content' not found

* rm generic test

* refactor input marking

* add docs

* fix windows build

* clarify error message

* improved tests

* split/rsplit with maxsplit

* non-inverse maxsplit

forgot to change after simplifying

* implement separators for tojson and fix indent

* i like to move it move it

* rename null -- > none

* token::eof

* some nits + comments

* add exception classes for lexer and parser

* null -> none

* rename global -> env

* rm minja

* update docs

* docs: add input marking caveats

* imlement missing jinja-tests functions

* oops

* support trim filter with args, remove bogus to_json reference

* numerous argument fixes

* updated tests

* implement optional strip chars parameter

* use new chars parameter

* float filter also has default

* always leave at least one decimal in float string

* jinja : static analysis + header cleanup + minor fixes

* add fuzz test

* add string.cpp

* fix chat_template_kwargs

* nits

* fix build

* revert

* unrevert

sorry :)

* add fuzz func_args, refactor to be safer

* fix array.map()

* loosen ensure_vals max count condition, add not impl for map(int)

* hopefully fix windows

* check if empty first

* normalize newlines

---------

Co-authored-by: Alde Rojas <hello@alde.dev>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-01-16 11:22:06 +01:00
Xuan-Son NguyenandGitHub a04c2b06a3 server: improve slots scheduling for n_cmpl (#18789) 
* server : make sure children tasks are scheduled to launch with parent

* fix

* add comment pointing to this PR

* fix

* clean up

* more debug messages

* add pop_deferred_task with specific ID version

* improve the logic

* simple approach

* no double move

* correct return type of launch_slots_with_parent_task
 2026-01-15 17:10:28 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		a7e6ddb8bd lora: make sure model keep track of associated adapters (#18490) 
* lora: make sure model keep track of associated adapters

* deprecate llama_adapter_lora_free

* minor : std::unordered_set over std::set

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-01-15 10:24:28 +01:00
Xuan-Son NguyenandGitHub e047f9ee9d mtmd: fix use_non_causal being reported incorrectly (#18793) 
* mtmd: fix use_non_causal being reported incorrectly

* move clip_is_mrope to mtmd_decode_use_mrope

* fix sloppy code ggml_cpy
 2026-01-13 12:19:38 +01:00
Xuan-Son NguyenandGitHub ce3bf9b1a4 server: update docs for sleeping [no ci] (#18777) 2026-01-12 13:01:24 +01:00
Xuan-Son NguyenandGitHub 0c3b7a9efe model: fix qwen3next broken due to #18683 (#18762) 2026-01-11 21:00:10 +01:00
Xuan-Son NguyenandGitHub 4b060bf240 security: make it clear about subtopics in server (#18754) 
* security: make it clear about subtopics in server

* exclude DoS
 2026-01-11 16:51:03 +01:00
Xuan-Son NguyenandGitHub 506bb6e010 model: try to improve Qwen3 Next (#18683) 
* qwen3next: simplify qkvz projection

* use ggml_swiglu_split

* revert swiglu_split, but remove redundant repeat()

* fix missing reshape

* rm 2 redundant transposes

* move mul_mat(k,q) to outside of chunking

* rm redundant cont

* improve g_cs_chunk

* add comments about no cont

* use std::pair instead of ggml_concat

* vectorize key_gdiff calculation

* rm unused tensor

* avoid ggml_concat inside loop

* bring back ggml_concat as it may not work on other backend

* nits
 2026-01-11 12:53:33 +01:00
Xuan-Son NguyenandGitHub 28068af789 security: narrow down the scope of what we consider a vulnerability (#18752) 
* security: narrow down the scope of what we consider a vulnerability

* fix typo
 2026-01-11 12:23:36 +01:00
Xuan-Son NguyenandGitHub 23f82f2420 preset: allow named remote preset (#18728) 
* preset: allow named remote preset

* nits: fix docs

* cont docs
 2026-01-10 15:12:29 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		9ac2693a30 server: fix n_cmpl not skipping processing prompt (#18663) 
* server: fix n_cmpl not skipping processing

* fix infinite loop on empty batch

* cont : init child samplers + modify child logic

* cont : cleanup

* cont : improve n_cmpl logic

- launch the parent task first so it finds the slot with best cache
- parent task waits for child tasks to be launched
- when a child task finishes - remove its cache

* cont : remove redundant function

* cont : reduce parent checks

* fix : nullptr task dereference

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-01-10 00:00:41 +01:00
Xuan-Son NguyenandGitHub 8ece3836b4 common: support remote preset (#18520) 
* arg: support remote preset

* proof reading

* allow one HF repo to point to multiple HF repos

* docs: mention about multiple GGUF use case

* correct clean_file_name

* download: also return HTTP status code

* fix case with cache file used

* fix --offline option
 2026-01-08 22:35:40 +01:00
Xuan-Son NguyenandGitHub 07fbe19f1f arg: use CSV escape style for multiple-value args (#18643) 
* arg: use CSV escape style for multiple-value args

* add test
 2026-01-06 17:51:08 +01:00
Xuan-Son NguyenandGitHub cd78e57c3a lora: count lora nodes in graph_max_nodes (#18469) 
* lora: count lora nodes in graph_max_nodes

* 3 nodes per weight

* 4 nodes

* keep track n_lora_nodes from llama_model

* fix assert

* rm redundant header

* common: load adapters before context creation

* use 6 nodes
 2025-12-30 15:53:12 +01:00
Xuan-Son NguyenandGitHub 51a48720b8 webui: fix prompt progress ETA calculation (#18468) 
* webui: fix prompt progress ETA calculation

* handle case done === 0
 2025-12-29 21:42:11 +01:00
Xuan-Son NguyenGitHubJohannes Gäßler
 		3595ae5963 contributing: tighten AI usage policy (#18388) 
* contributing: tighten AI usage policy

* refactor AGENTS.md

* proofreading

* update contributing

* add claude.md

* add trailing newline

* add note about dishonest practices

* rm point about dishonest

* rm requirement watermarking

* add .gemini/settings.json

* allow initially AI-generated content

* revise

* Update CONTRIBUTING.md

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* improve

* trailing space

* Apply suggestions from code review

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* update

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2025-12-29 16:01:32 +01:00
Xuan-Son NguyenandGitHub cffa5c46ea mtmd: clarify that we no longer accept AI-generated PRs (#18406) 2025-12-28 09:57:04 +01:00
Xuan-Son NguyenandGitHub f5acfb2ffa server: (router) add stop-timeout option (#18350) 
* server: (router) add stop-timeout option

* also allow stop while loading

* add docs

* unload_lru: also wait for unload to complete
 2025-12-24 23:47:49 +01:00