nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 20 Packages Projects Releases Wiki Activity
9,654 Commits 1 Branch 0 Tags
e3cab403bfac17066d7c1a3eb3258e95c648d114
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan-Son NguyenandGitHub e8067a8b36 ui: build-time gzip compression (#24571) 
* ui: keep original file name and path

* fix nocache

* ui: build-time gzip compression
 2026-06-13 16:57:27 +02:00
Xuan-Son NguyenandGitHub 597b6672e8 ui: keep original file name and path (#24568) 
* ui: keep original file name and path

* fix nocache
 2026-06-13 14:31:41 +02:00
Xuan-Son NguyenGitHubAlde Rojas
 		57fe1f07c3 server: clean up static assets handling (#24550) 
* server: clean up static assets handling

* nits

* simplify file name handling, use static file name everywhere

* cmake/ui : bundle UI assets in an archive

* ui : run prettier on post-build.js

---------

Co-authored-by: Alde Rojas <hello@alde.dev>
 2026-06-13 11:51:20 +02:00
Xuan-Son NguyenandGitHub e37abd6b5f mtmd: add batching API (#24384) 
* mtmd: add batching API

* wip

* first working version (gemma4v)

* add arg

* nits

* wire up support_batch()

* fix 0.0 output embd

* fix audio

* nits

* refactor a bit

* nits

* fix non-batching case

* fix comment
 2026-06-13 00:10:29 +02:00
Xuan-Son NguyenandGitHub 18ef86ecec server: skip unused log lines on router mode (#24463) 2026-06-11 11:36:35 +02:00
Xuan-Son NguyenGitHubMikko Juola
 		9682e351b8 mtmd: refactor video subproc handling (#24316) 
* mtmd: refactor video subproc handling

* Update tools/mtmd/mtmd-helper.cpp

Co-authored-by: Mikko Juola <mikjuo@gmail.com>

---------

Co-authored-by: Mikko Juola <mikjuo@gmail.com>
 2026-06-09 13:15:12 +03:00
Xuan-Son NguyenandGitHub 3ebe862b5d docker: install ffmpeg in the released image (#24302) 2026-06-08 16:59:57 +02:00
Xuan-Son NguyenandGitHub 8f83d6c271 mtmd : add video input support (#24269) 
* wip

* ok: lazy bitmap API

* remember to free lazy text

* wip

* add mtmd_helper_video

* support video input on server (base64 input)

* add MTMD_VIDEO config

* add timestamp

* update CLI

* cli: allow auto-completion for video

* add --video arg

* fix build

* update docs

* rename as suggested
 2026-06-08 14:40:12 +03:00
Xuan-Son NguyenandGitHub 715b86a366 cli: fix spinner not show during prompt processing (#24283) 2026-06-08 11:11:45 +02:00
Xuan-Son NguyenandGitHub daf6bc9f2d metal : fix im2col 1D case (audio models) (#24220) 2026-06-08 09:03:18 +03:00
Xuan-Son NguyenGitHubandrewmd5
 		31e82494c0 mtmd: support "frame merge" for qwen-vl-based models (#21858) 
* feat: add video support for Qwen3.5

* various clean up

* revise the design

* fix llava-uhd case

* nits

* nits 2

---------

Co-authored-by: andrewmd5 <1297077+andrewmd5@users.noreply.github.com>
 2026-06-06 21:17:25 +02:00
Xuan-Son NguyenandGitHub f5c6ae1827 mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913) 
* mtmd: add "placeholder bitmap" for counting tokens w/o preprocessing

* fast path skip preproc for placeholder

* fix build

* correct the api

* add server endpoint + tests

* add object name

* update docs

* add proxy handling

* fix build

* fix audio input path

* use is_placeholder in process_mtmd_prompt()

* nits

* nits (2)

* docs: clarify chat/completions/input_tokens is not official

* fix merge problem
 2026-06-06 11:06:51 +02:00
Xuan-Son NguyenandGitHub c4a278d68e model: fix build failed (#24193) 2026-06-05 18:12:27 +02:00
Xuan-Son NguyenandGitHub cc7bef34e2 ui: add ignore-scripts=true to npmrc (#24149) 2026-06-05 14:31:03 +02:00
Xuan-Son NguyenandGitHub 260862b8ca arg: fix double mtp downloads (#24128) 2026-06-04 19:23:48 +03:00
Xuan-Son NguyenandGitHub a121232fdc agents: refactor, include more guidelines (#24111) 
* agents: refactor, include more guidelines

* better example

* rephrase a bit

* add more examples

* nits
 2026-06-04 13:40:23 +02:00
Xuan-Son NguyenandGitHub c8d6a00636 mtmd: enable non-causal vision for gemma 4 unified (#24082) 2026-06-03 19:05:17 +02:00
Xuan-Son NguyenandGitHub a731805ced mtmd, model: allow skip build_vit() (#24077) 
* add model

* nits
 2026-06-03 17:10:35 +02:00
Xuan-Son NguyenandGitHub 60130d18f9 server: add SSE ping interval (#24013) 2026-06-02 14:14:55 +02:00
Xuan-Son NguyenandGitHub e674b1279b test: (test-llama-archs) log the config name first (#23885) 2026-05-30 12:22:38 +02:00
Xuan-Son NguyenandGitHub 0821c5fcfd server: in SSE mode, send HTTP headers when slot starts (#23884) 
* server: in SSE mode, send HTTP headers when slot starts

* ref to pr

* stream should be false by default
 2026-05-30 00:06:29 +02:00
Xuan-Son NguyenandGitHub b5f52280fb server: remove obsolete scripts (#23870) 2026-05-29 19:47:30 +02:00
Xuan-Son NguyenandGitHub 06d26dfdff download: add option to skip_download (#23059) 
* download: add option to skip_download

* fix

* fix 2

* if file doesn't exist, respect skip_download flag
 2026-05-29 16:30:55 +02:00
Xuan-Son NguyenandGitHub cb47092b00 server: bump timeout to 3600s (#23842) 
* server: bump timeout to 3600s

* nits: change wording
 2026-05-29 10:23:17 +02:00
Xuan-Son NguyenandGitHub 751ebd17a5 mtmd-debug: add color and rainbow mode (#23829) 
* mtmd-debug: add color and rainbow mode

* fix M_PI

* max_dist
 2026-05-28 20:59:14 +02:00
Xuan-Son NguyenandGitHub c8914ad4f4 mtmd: fix gemma 4 projector pre_norm (#23822) 2026-05-28 20:58:55 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		d6be3158e1 mtmd: fix gemma 4 audio rms norm eps (#23815) 
* mtmd: fix gemma 4 audio rms norm eps

* Update tools/mtmd/clip.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-05-28 16:31:37 +02:00
Xuan-Son NguyenandGitHub e2b129e1bf mtmd: fit_params now take into account mmproj (#21489) 
* mtmd: fit_params now take into account mmproj

* rename alloc_compute_meta to reserve_compute_meta

* rm unused functions

* add ggml_backend_dev_t support

* add debug log
 2026-05-20 11:27:44 +02:00
Xuan-Son NguyenandGitHub b64739ea39 server: (router) alloc tmp buffer on heap (#23159) 2026-05-16 23:42:16 +02:00
Xuan-Son NguyenandGitHub 72e60f500d mtmd: add chunks and fix preproc for qwen3a (#23073) 
* mtmd: add chunks and fix preproc for qwen3a

* add attn_mask

* limit mtmd_chunk size (avoid blow up memory)

* correct audio tokens

* re-order the set_input case

* remove attn_mask
 2026-05-15 19:32:47 +02:00
Xuan-Son NguyenandGitHub e75cd5efb5 download: do not exit() on error (#23008) 2026-05-13 15:14:58 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		3796c94bad ci: validate model naming convention (#22680) 
* ci: validate model naming convention

* bring back dedicated ec workflow

* add missing jobs

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-05-13 10:59:37 +02:00
Xuan-Son NguyenandGitHub 7bfe120c21 mtmd, server, common: expose modalities to /v1/models (#22952) 
* mtmd, server, common: expose modalities to /v1/models

* fix build

* rename to mtmd_caps
 2026-05-12 19:08:07 +02:00
Xuan-Son NguyenandGitHub 29debb3a6a server: support Vertex AI compatible API (#22545) 
* server: support Vertex AI compatible API

* a bit safer

* support other AIP_* env var

* various fixes

* if AIP_MODE is unset, do nothing

* fix test case

* fix windows build
 2026-05-08 15:23:04 +02:00
Xuan-Son NguyenandGitHub 9dcf835528 server: (router) expose child model info from router's /v1/models (#22683) 
* server: (router) expose child model info from router's /v1/models

* update docs
 2026-05-08 14:42:15 +02:00
Xuan-Son NguyenandGitHub 935a340292 server: implement /models?reload=1 (#21848) 2026-05-04 16:23:26 +02:00
Xuan-Son NguyenandGitHub 994118a183 model: move load_hparams and load_tensors to per-model definition (#22004) 
* git-friendly migration

* add build_graph

* nits

* exclude old code from build

* wip

* add llm_arch_model_i

* prepare downstream functions

* nits

* nits

* wip

* wip

* add back create_tensor_qkv

* fix files missing include

* enforce one llm_build per arch

* cmake: use glob

* missing model params

* nits

* wip

* wip (2)

* wip (3)

* test-llama-archs is happy

* improve switch case

* move more stuff into llm_arch_model_i

* fix downstream code

* nits

* nits (2)

* fix order

* llama_model_base

* LLAMA_LOAD_LOCALS

* small fix

* fix build errors

* auto

* rm migration script and ifdef
 2026-05-04 12:36:59 +02:00
Xuan-Son NguyenandGitHub 82d3f4d3b2 mtmd: also support LLAMA_ROPE_TYPE_NONE (#22242) 2026-04-22 12:16:29 +02:00
Xuan-Son NguyenandGitHub 17f6245168 server: ignore reasoning content from transcription api (#21905) 2026-04-22 12:10:50 +02:00
Xuan-Son NguyenandGitHub 04fe84b69d server: allow cancel loading model (#21814) 2026-04-22 00:26:09 +02:00
Xuan-Son NguyenandGitHub 9998d88bc8 mtmd: correct mtmd_decode_use_mrope() (#22188) 2026-04-21 10:53:37 +02:00
Xuan-Son NguyenandGitHub 86f8daacfe mtmd: correct get_n_pos / get_decoder_pos (#22175) 2026-04-20 23:29:19 +02:00
Xuan-Son NguyenandGitHub a678916623 mtmd: refactor mtmd_decode_use_mrope (#22161) 2026-04-20 14:45:11 +02:00
Xuan-Son NguyenandGitHub 19124078be mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos (breaking change) (#22082) 
* mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos

* fix build
 2026-04-19 11:57:21 +02:00
Xuan-Son NguyenandGitHub 089dd41fe3 cmake: use glob to collect src/models sources (#22005) 2026-04-16 23:25:16 +02:00
Xuan-Son NguyenandGitHub 4fbdabdc61 model: using single llm_build per arch (#21970) 
* model: using single llm_build per arch

* fix merge

* nits
 2026-04-16 21:10:22 +02:00
Xuan-Son NguyenandGitHub 408225bb1a server: use random media marker (#21962) 
* server: use random media marker

* nits

* remove legacy <__image__> token

* revert special char in random
 2026-04-15 23:52:22 +02:00
Xuan-Son NguyenandGitHub 80d8770804 docs: more extensive RoPE documentation [no ci] (#21953) 
* more extensive ggml_rope documentation

* add more docs

* nits
 2026-04-15 14:45:16 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		fae3a28070 ggml : remove ggml-ext.h (#21869) 
* ggml: correct placement of ggml-ext.h

* ggml : remove ggml-ext.h

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-04-14 17:32:58 +03:00
Xuan-Son NguyenandGitHub 707c0b7a6e mtmd: add mtmd_image_tokens_get_decoder_pos() API (#21851) 
* mtmd: add mtmd_image_tokens_get_decoder_pos() API

* consistent naming

* fix build
 2026-04-14 16:07:41 +02:00
Xuan-Son NguyenandGitHub e489a5ca0e server: support OAI /v1/audio/transcriptions API (#21863) 
* server: support OAI /v1/audio/transcriptions API

* address autoreview comments

* correct default response_format value
 2026-04-14 11:09:52 +02:00
Xuan-Son NguyenandGitHub e974923698 docs: listing qwen3-asr and qwen3-omni as supported (#21857) 
* docs: listing qwen3-asr and qwen3-omni as supported

* nits
 2026-04-13 22:28:17 +02:00
Xuan-Son NguyenandGitHub 920b3e78cb mtmd: use causal attn for gemma 4 audio (#21824) 2026-04-13 09:47:55 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		21a4933042 mtmd: qwen3 audio support (qwen3-omni and qwen3-asr) (#19441) 
* add qwen3a

* wip

* vision ok

* no more deepstack for audio

* convert ASR model ok

* qwen3 asr working

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* nits

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix bad merge

* fix multi inheritance

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-04-12 23:57:25 +02:00
Xuan-Son NguyenandGitHub aa4695c5e5 mtmd: add gemma 4 test (vision + audio) [no ci] (#21806) 
* mtmd: add gemma 4 test (vision + audio)

* add to docs
 2026-04-12 16:29:03 +02:00
Xuan-Son NguyenandGitHub 6313acbef0 docs: add guide on how to add multimodal support (#21778) 
* docs: add guide on how to add multimodal support

* nits
 2026-04-12 13:02:38 +02:00
Xuan-Son NguyenandGitHub 057dba336e model: fix multimodal padding token for gemma3n/gemma4 (#21625) 
* model: fix multimodal padding token for gemma3n/gemma4

* nits
 2026-04-09 12:18:23 +02:00
Xuan-Son NguyenandGitHub 501aeed18f mtmd: support dots.ocr (#17575) 
* convert gguf

* clip impl

* fix conversion

* wip

* corrections

* update docs

* add gguf to test script
 2026-04-09 12:16:38 +02:00
Xuan-Son NguyenandGitHub 3979f2bb08 docs: add hunyuan-ocr gguf, also add test [no ci] (#21490) 2026-04-06 14:02:37 +02:00
Xuan-Son NguyenandGitHub 63f8fe0ef4 model, mtmd: fix gguf conversion for audio/vision mmproj (#21309) 
* fix gguf conversion for audio/vision mmproj

* fix test
 2026-04-02 17:10:32 +02:00
Xuan-Son NguyenandGitHub c30e012253 contrib : rewrite AGENTS.md, make it more clear about project values (#21270) 
* contrib : rewrite AGENTS.md, make it more clear about types of permitted AI usage

* permit AI for writing code
 2026-04-01 23:31:51 +02:00
Xuan-Son NguyenandGitHub 0356e33aaf scripts: add function call test script (#21234) 
* scripts: add function call test script

* add reasoning_content

* fix lint
 2026-04-01 15:31:58 +02:00
Xuan-Son NguyenGitHubAleksander Grygier
 		4a00bbfed6 server: (webui) no more gzip compression (#21073) 
* webui: no more gzip

* try changing a small line

* Revert "try changing a small line"

This reverts commit 0d7a3531593d87b724d404c8727a96becab3ab07.

* fix lint

* fix test

* rebuild

* split into html/css/js

* lint

* chore: update webui build output

* chore: Update git hooks script

* server: update webui build output

* chore: Update pre-commit hook

* refactor: Cleanup

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
 2026-03-31 15:44:26 +02:00
Xuan-Son NguyenGitHubGeorgi GerganovAleksander Grygier
 		abf9a62161 server: wrap headers for mcp proxy (#21072) 
* server: wrap headers for mcp proxy

* Update tools/server/server-cors-proxy.h

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix build

* chore: update webui build output

* chore: update webui build output

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
 2026-03-30 08:59:16 +02:00
Xuan-Son NguyenandGitHub 871f1a2d2f mtmd: add more sanity checks (#21047) 2026-03-27 11:00:52 +01:00
Xuan-Son NguyenandGitHub 20197b6fe3 server: add built-in tools backend support (#20898) 
* wip: server_tools

* refactor

* displayName -> display_name

* snake_case everywhere

* rm redundant field

* change arg to --tools all

* add readme mention

* llama-gen-docs
 2026-03-27 10:07:11 +01:00
Xuan-Son NguyenandGitHub 8c60b8a2be ci: pin external actions to exact commit SHA (#21033) 2026-03-26 20:44:00 +01:00
Xuan-Son NguyenandGitHub a73bbd5d92 mtmd: refactor image preprocessing (#21031) 
* mtmd: refactor image pre-processing

* correct some places

* correct lfm2

* fix deepseek-ocr on server

* add comment to clarify about mtmd_image_preprocessor_dyn_size
 2026-03-26 19:49:20 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		914eb5ff0c jinja: fix macro with kwargs (#20960) 
* jinja: fix macro with kwargs

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix newline problem

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-03-25 12:22:48 +01:00
Xuan-Son NguyenGitHubPiotr Wilkin
 		bd6992180b contrib: add "Requirements" section to PR template (#20841) 
* contrib: add "Requirements" section to PR template

* typo [no ci]

* use h2, add "Additional information"

---------

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>
 2026-03-23 16:59:02 +01:00
Xuan-Son NguyenandGitHub 31a5cf4c3f server: use httplib dynamic threads (#20817) 
* server: use httplib dynamic threads

* change to n_threads_http + 1024
 2026-03-23 12:22:46 +01:00
Xuan-Son NguyenandGitHub 49bfddeca1 server: allow router to report child instances sleep status (#20849) 
* server: allow router to report child instances sleep status

* refactor

* move sleeping to state

* nits
 2026-03-22 18:33:52 +01:00
Xuan-Son NguyenandGitHub 568aec82d2 docs : explicit about banning accounts that violates policy (#19593) 2026-03-21 15:50:16 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		fb78ad29bb server: (doc) clarify in-scope and out-scope features (#20794) 
* server: (doc) clarify in-scope and out-scope features

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-03-20 14:03:50 +01:00
Xuan-Son NguyenandGitHub 1e64534570 mtmd: add clip_graph::build_mm() (#20751) 
* clip: add build_mm()

* apply to all models

* add TODO for bias overload
 2026-03-19 13:11:39 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		d34ff7eb5b model: mistral small 4 support (#20649) 
* model: mistral small 4 support

* fix test

* fix test (2)

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* change newline

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-03-17 00:31:14 +01:00
Xuan-Son NguyenandGitHub 94d0262277 mtmd: add llama-mtmd-debug binary (#20508) 
* mtmd: add llama-mtmd-debug binary

* adapt

* fixes

* fix compile error

* fix windows compile error

* rm legacy clip_debug_encode()

* add MTMD_API to fix build
 2026-03-14 15:52:29 +01:00
Xuan-Son NguyenandGitHub 0842b9b465 model: fix step3.5 n_rot (#20318) 2026-03-09 23:42:24 +01:00
Xuan-Son NguyenandGitHub 59db9a357d llama: dynamic head_dim and n_rot for SWA (#20301) 
* llama: dynamic head_dim and n_rot for SWA

* also add gguf_writer wrappers

* fix build

* build_rope_shift arg reorder
 2026-03-09 22:22:39 +01:00
Xuan-Son NguyenandGitHub c747294b2d scripts: update corpus of compare-logprobs (#19326) 
* scripts: update corpus of compare-logprobs

* fix
 2026-02-25 12:57:34 +01:00
Xuan-Son NguyenandGitHub 5452d736f8 jinja: correct stats for tojson and string filters (#19785) 2026-02-22 21:08:23 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		eeef3cfced model: support GLM-OCR (#19677) 
* model: support GLM-OCR

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-02-18 17:51:40 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		752584d5f5 model: support GLM MoE DSA arch (NOTE: indexer is not yet supported) (#19460) 
* model: support GLM MoE DSA arch

* working version

* pyright

* keep indexer tensors

* add indexer gguf params

* loaded now

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* update

* Update src/llama-model.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* minor fix and cleanup

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-02-13 14:56:53 +01:00
Xuan-Son NguyenandGitHub 9a96352729 test: fix IMROPE perf test case (#19465) 2026-02-10 14:37:50 +01:00
Xuan-Son NguyenandGitHub 98e57ca422 chat: fix case where template accepts type content only (#19419) 
* chat: fix case where template accepts type content only

* rm stray log

* reuse render_message_to_json
 2026-02-09 22:14:12 +01:00
Xuan-Son NguyenandGitHub e0c93af2a0 debug: make common_debug_print_tensor readable (#19331) 
* debug: make common_debug_print_tensor readable

* editorconfig
 2026-02-04 17:55:31 +01:00
Xuan-Son NguyenandGitHub 8abcc70a74 model: (qwen3next) correct vectorized key_gdiff calculation (#19324) 
* model: (qwen3next) correct vectorized key_gdiff calculation

* move transpose to outside of loop
 2026-02-04 13:09:58 +01:00
Xuan-Son NguyenandGitHub 07a7412a3b mtmd: add min/max pixels gguf metadata (#19273) 2026-02-02 20:59:06 +01:00
Xuan-Son NguyenGitHubAldehir Rojas
 		bb02f74c61 chat: fix language input for translategemma (#19052) 
* chat: fix language input for translategemma

* Update common/chat.cpp

Co-authored-by: Aldehir Rojas <hello@alde.dev>

---------

Co-authored-by: Aldehir Rojas <hello@alde.dev>
 2026-01-24 17:58:45 +01:00
Xuan-Son NguyenandGitHub b5b8fa1c8b chat : fix translategemma crash on common_chat_format_example (#19019) 2026-01-23 12:03:42 +01:00
Xuan-Son NguyenandGitHub 51fa458a92 server : support preserving reasoning_content in assistant message (#18994) 
* support reasoning_content input

* report template caps to webui

* add docs

* rm commented code
 2026-01-22 21:30:06 +01:00
Xuan-Son NguyenandGitHub 4e595b250a server: do not log certain endpoints (avoid log spam) (#19028) 2026-01-22 19:24:37 +01:00
Xuan-Son NguyenandGitHub 9eb5bfec1a mtmd : update docs to use llama_model_n_embd_inp (#18999) 2026-01-22 14:36:32 +01:00
Xuan-Son NguyenandGitHub 2c1f199653 cli : fix reasoning responses in CLI (#18961) 
* cli : fix reasoning responses in CLI

* fix build

* fix build (2)
 2026-01-20 18:23:25 +01:00
Xuan-Son NguyenandGitHub 6df686bee6 server : refactor oai_parser_opt, move it to server_chat_params (#18937) 
* server_chat_params

* move chat format into CLI

* use meta whenever possible

* clean up, no more chatml fallback
 2026-01-19 23:28:01 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		fe44d35574 tests : add test-jinja -py option for cross-checking (#18906) 
* tests : add test-jinja -py option or cross-checking

* Update tests/test-jinja.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix + add source

* SandboxedEnvironment

* fix array.map case

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-01-18 08:14:27 +01:00
Xuan-Son NguyenandGitHub f55b033ae6 jinja: correct member access rule (#18905) 2026-01-18 00:48:55 +01:00
Xuan-Son NguyenGitHubAlde RojasSigbjørn SkjæretGeorgi Gerganov
 		c15395f73c common : implement new jinja template engine (#18462) 
* jinja vm

* lexer

* add vm types

* demo

* clean up

* parser ok

* binary_expression::execute

* shadow naming

* bin ops works!

* fix map object

* add string builtins

* add more builtins

* wip

* use mk_val

* eval with is_user_input

* render gemma tmpl ok

* track input string even after transformations

* support binded functions

* keyword arguments and slicing array

* use shared_ptr for values

* add mk_stmt

* allow print source on exception

* fix negate test

* testing more templates

* mostly works

* add filter_statement

* allow func to access ctx

* add jinja-value.cpp

* impl global_from_json

* a lot of fixes

* more tests

* more fix, more tests

* more fixes

* rm workarounds

* demo: type inferrence

* add placeholder for tojson

* improve function args handling

* rm type inference

* no more std::regex

* trailing spaces

* make testing more flexible

* make output a bit cleaner

* (wip) redirect minja calls

* test: add --output

* fix crash on macro kwargs

* add minimal caps system

* add some workarounds

* rm caps_apply_workarounds

* get rid of preprocessing

* more fixes

* fix test-chat-template

* move test-chat-jinja into test-chat-template

* rm test-chat-jinja from cmake

* test-chat-template: use common

* fix build

* fix build (2)

* rename vm --> interpreter

* improve error reporting

* correct lstrip behavior

* add tojson

* more fixes

* disable tests for COMMON_CHAT_FORMAT_GENERIC

* make sure tojson output correct order

* add object.length

* fully functional selectattr / rejectattr

* improve error reporting

* more builtins added, more fixes

* create jinja rendering tests

* fix testing.h path

* adjust whitespace rules

* more fixes

* temporary disable test for ibm-granite

* r/lstrip behavior matched with hf.js

* minimax, glm4.5 ok

* add append and pop

* kimi-k2 ok

* test-chat passed

* fix lstrip_block

* add more jinja tests

* cast to unsigned char

* allow dict key to be numeric

* nemotron: rm windows newline

* tests ok

* fix test

* rename interpreter --> runtime

* fix build

* add more checks

* bring back generic format support

* fix Apertus

* [json.exception.out_of_range.403] key 'content' not found

* rm generic test

* refactor input marking

* add docs

* fix windows build

* clarify error message

* improved tests

* split/rsplit with maxsplit

* non-inverse maxsplit

forgot to change after simplifying

* implement separators for tojson and fix indent

* i like to move it move it

* rename null -- > none

* token::eof

* some nits + comments

* add exception classes for lexer and parser

* null -> none

* rename global -> env

* rm minja

* update docs

* docs: add input marking caveats

* imlement missing jinja-tests functions

* oops

* support trim filter with args, remove bogus to_json reference

* numerous argument fixes

* updated tests

* implement optional strip chars parameter

* use new chars parameter

* float filter also has default

* always leave at least one decimal in float string

* jinja : static analysis + header cleanup + minor fixes

* add fuzz test

* add string.cpp

* fix chat_template_kwargs

* nits

* fix build

* revert

* unrevert

sorry :)

* add fuzz func_args, refactor to be safer

* fix array.map()

* loosen ensure_vals max count condition, add not impl for map(int)

* hopefully fix windows

* check if empty first

* normalize newlines

---------

Co-authored-by: Alde Rojas <hello@alde.dev>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-01-16 11:22:06 +01:00
Xuan-Son NguyenandGitHub a04c2b06a3 server: improve slots scheduling for n_cmpl (#18789) 
* server : make sure children tasks are scheduled to launch with parent

* fix

* add comment pointing to this PR

* fix

* clean up

* more debug messages

* add pop_deferred_task with specific ID version

* improve the logic

* simple approach

* no double move

* correct return type of launch_slots_with_parent_task
 2026-01-15 17:10:28 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		a7e6ddb8bd lora: make sure model keep track of associated adapters (#18490) 
* lora: make sure model keep track of associated adapters

* deprecate llama_adapter_lora_free

* minor : std::unordered_set over std::set

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-01-15 10:24:28 +01:00