nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 26 Packages Projects Releases Wiki Activity
9,733 Commits 1 Branch 0 Tags
f449e0553708b895adbd94a301431cef691f632d
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan-Son NguyenandGitHub 2b686a9120 server: refactor child --> router communication (#24821) 
* server: refactor child --> router communication

* fix wakeup case

* add docs

* improve update_status()

* nits
 2026-06-20 01:02:26 +02:00
Xuan-Son NguyenandGitHub e475fa2b5f mtmd, arg: fix utf8 handling on windows (#24779) 
* mtmd, arg: fix utf8 handling on windows

* also fix ggml_fopen

* fix build fail

* also fix CLI
 2026-06-19 22:28:38 +02:00
Xuan-Son NguyenandGitHub 175147e8f6 server: remove all internal mentions about "webui" (#24817) 2026-06-19 22:12:46 +02:00
Xuan-Son NguyenandGitHub 8c2d6f6475 server: add --agent arg, remove redundant webui naming compat (#24801) 
* server: add --agent arg, remove redundant webui naming compat

* corrent env

* fix the test

* llama-gen-docs

* nits: wordings
 2026-06-19 16:06:13 +02:00
Xuan-Son NguyenandGitHub e2e7a9b2d0 mtmd: several bug fixes (#24784) 
* mtmd: several bug fixes

* fix build

* fix gemma4ua

* add sanity check in get_u32()

* fix build (2)

* area() avoid overflow
 2026-06-19 12:18:36 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		159d093a43 server: fix non-bound n_discard value (ctx shifting) (#24786) 
* server: fix non-bound n_discard value

* Update tools/server/server-context.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-06-19 10:53:44 +02:00
Xuan-Son NguyenandGitHub db52540f73 mtmd: add batching support for internvl (#24775) 2026-06-19 01:16:16 +02:00
Xuan-Son NguyenandGitHub a6b3260a42 mtmd: add batching for mtmd-cli, add video tests (#24778) 2026-06-18 21:55:04 +02:00
Xuan-Son NguyenandGitHub 060ce1bf72 mtmd: refactor llava-uhd overview image handling (always use ov_img_first) (#24769) 
* add dedicated "overview" for mtmd_image_preproc_out

* corrections

* correct (again)

* nits

* nits (2)
 2026-06-18 18:53:49 +02:00
Xuan-Son NguyenandGitHub fe7c8b2414 server: (router) fix stopping_thread potentially hang (#24728) 
* server: (router) fix stopping_thread potentially hang

* fix windows build
 2026-06-18 15:41:09 +02:00
Xuan-Son NguyenandGitHub e1efd0991d server: add "schema" and validation (#24150) 
* wip

* working

* correct some limits

* add field name to error message
 2026-06-18 15:40:58 +02:00
Xuan-Son NguyenandGitHub 552258c535 server: (router) rework -hf preset repo (#24739) 
* server: temporary remove HF remote preset

* rework remove preset.ini support

* rm unused get_remote_preset_whitelist()

* print warning

* add docs

* rm stray file
 2026-06-18 12:45:23 +02:00
Xuan-Son NguyenandGitHub 968c43891a server: fix router args not being forwarded to child instances (#24760) 2026-06-18 12:15:46 +02:00
Xuan-Son NguyenandGitHub 24bba7b98e mtmd: refactor preprocessor, add mtmd_image_preproc_out (#24736) 
* add mtmd_image_preproc_out

* add dev docs

* remove unused clip API

* rm unused clip_image_f32_batch::grid

* change preprocess() call signature
 2026-06-18 12:04:39 +02:00
Xuan-Son NguyenandGitHub f3e1828164 mtmd: llava_uhd should no longer use batch dim (#24732) 2026-06-17 22:40:50 +02:00
Xuan-Son NguyenandGitHub 4b4d13ae72 server: (router) add model management API (#23976) 
* wip

* server: (router) add SSE realtime updates API

* nits

* wip

* add download API

* add download api

* update docs

* add delete endpoint

* fix std::terminate

* fix crash

* fix 2

* add tests

* nits
 2026-06-17 18:04:58 +02:00
Xuan-Son NguyenandGitHub e36a602ba3 mtmd: fix miscounting n_tokens (#24656) 2026-06-15 18:07:14 +02:00
Xuan-Son NguyenandGitHub e8067a8b36 ui: build-time gzip compression (#24571) 
* ui: keep original file name and path

* fix nocache

* ui: build-time gzip compression
 2026-06-13 16:57:27 +02:00
Xuan-Son NguyenandGitHub 597b6672e8 ui: keep original file name and path (#24568) 
* ui: keep original file name and path

* fix nocache
 2026-06-13 14:31:41 +02:00
Xuan-Son NguyenGitHubAlde Rojas
 		57fe1f07c3 server: clean up static assets handling (#24550) 
* server: clean up static assets handling

* nits

* simplify file name handling, use static file name everywhere

* cmake/ui : bundle UI assets in an archive

* ui : run prettier on post-build.js

---------

Co-authored-by: Alde Rojas <hello@alde.dev>
 2026-06-13 11:51:20 +02:00
Xuan-Son NguyenandGitHub e37abd6b5f mtmd: add batching API (#24384) 
* mtmd: add batching API

* wip

* first working version (gemma4v)

* add arg

* nits

* wire up support_batch()

* fix 0.0 output embd

* fix audio

* nits

* refactor a bit

* nits

* fix non-batching case

* fix comment
 2026-06-13 00:10:29 +02:00
Xuan-Son NguyenandGitHub 18ef86ecec server: skip unused log lines on router mode (#24463) 2026-06-11 11:36:35 +02:00
Xuan-Son NguyenGitHubMikko Juola
 		9682e351b8 mtmd: refactor video subproc handling (#24316) 
* mtmd: refactor video subproc handling

* Update tools/mtmd/mtmd-helper.cpp

Co-authored-by: Mikko Juola <mikjuo@gmail.com>

---------

Co-authored-by: Mikko Juola <mikjuo@gmail.com>
 2026-06-09 13:15:12 +03:00
Xuan-Son NguyenandGitHub 3ebe862b5d docker: install ffmpeg in the released image (#24302) 2026-06-08 16:59:57 +02:00
Xuan-Son NguyenandGitHub 8f83d6c271 mtmd : add video input support (#24269) 
* wip

* ok: lazy bitmap API

* remember to free lazy text

* wip

* add mtmd_helper_video

* support video input on server (base64 input)

* add MTMD_VIDEO config

* add timestamp

* update CLI

* cli: allow auto-completion for video

* add --video arg

* fix build

* update docs

* rename as suggested
 2026-06-08 14:40:12 +03:00
Xuan-Son NguyenandGitHub 715b86a366 cli: fix spinner not show during prompt processing (#24283) 2026-06-08 11:11:45 +02:00
Xuan-Son NguyenandGitHub daf6bc9f2d metal : fix im2col 1D case (audio models) (#24220) 2026-06-08 09:03:18 +03:00
Xuan-Son NguyenGitHubandrewmd5
 		31e82494c0 mtmd: support "frame merge" for qwen-vl-based models (#21858) 
* feat: add video support for Qwen3.5

* various clean up

* revise the design

* fix llava-uhd case

* nits

* nits 2

---------

Co-authored-by: andrewmd5 <1297077+andrewmd5@users.noreply.github.com>
 2026-06-06 21:17:25 +02:00
Xuan-Son NguyenandGitHub f5c6ae1827 mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913) 
* mtmd: add "placeholder bitmap" for counting tokens w/o preprocessing

* fast path skip preproc for placeholder

* fix build

* correct the api

* add server endpoint + tests

* add object name

* update docs

* add proxy handling

* fix build

* fix audio input path

* use is_placeholder in process_mtmd_prompt()

* nits

* nits (2)

* docs: clarify chat/completions/input_tokens is not official

* fix merge problem
 2026-06-06 11:06:51 +02:00
Xuan-Son NguyenandGitHub c4a278d68e model: fix build failed (#24193) 2026-06-05 18:12:27 +02:00
Xuan-Son NguyenandGitHub cc7bef34e2 ui: add ignore-scripts=true to npmrc (#24149) 2026-06-05 14:31:03 +02:00
Xuan-Son NguyenandGitHub 260862b8ca arg: fix double mtp downloads (#24128) 2026-06-04 19:23:48 +03:00
Xuan-Son NguyenandGitHub a121232fdc agents: refactor, include more guidelines (#24111) 
* agents: refactor, include more guidelines

* better example

* rephrase a bit

* add more examples

* nits
 2026-06-04 13:40:23 +02:00
Xuan-Son NguyenandGitHub c8d6a00636 mtmd: enable non-causal vision for gemma 4 unified (#24082) 2026-06-03 19:05:17 +02:00
Xuan-Son NguyenandGitHub a731805ced mtmd, model: allow skip build_vit() (#24077) 
* add model

* nits
 2026-06-03 17:10:35 +02:00
Xuan-Son NguyenandGitHub 60130d18f9 server: add SSE ping interval (#24013) 2026-06-02 14:14:55 +02:00
Xuan-Son NguyenandGitHub e674b1279b test: (test-llama-archs) log the config name first (#23885) 2026-05-30 12:22:38 +02:00
Xuan-Son NguyenandGitHub 0821c5fcfd server: in SSE mode, send HTTP headers when slot starts (#23884) 
* server: in SSE mode, send HTTP headers when slot starts

* ref to pr

* stream should be false by default
 2026-05-30 00:06:29 +02:00
Xuan-Son NguyenandGitHub b5f52280fb server: remove obsolete scripts (#23870) 2026-05-29 19:47:30 +02:00
Xuan-Son NguyenandGitHub 06d26dfdff download: add option to skip_download (#23059) 
* download: add option to skip_download

* fix

* fix 2

* if file doesn't exist, respect skip_download flag
 2026-05-29 16:30:55 +02:00
Xuan-Son NguyenandGitHub cb47092b00 server: bump timeout to 3600s (#23842) 
* server: bump timeout to 3600s

* nits: change wording
 2026-05-29 10:23:17 +02:00
Xuan-Son NguyenandGitHub 751ebd17a5 mtmd-debug: add color and rainbow mode (#23829) 
* mtmd-debug: add color and rainbow mode

* fix M_PI

* max_dist
 2026-05-28 20:59:14 +02:00
Xuan-Son NguyenandGitHub c8914ad4f4 mtmd: fix gemma 4 projector pre_norm (#23822) 2026-05-28 20:58:55 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		d6be3158e1 mtmd: fix gemma 4 audio rms norm eps (#23815) 
* mtmd: fix gemma 4 audio rms norm eps

* Update tools/mtmd/clip.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-05-28 16:31:37 +02:00
Xuan-Son NguyenandGitHub e2b129e1bf mtmd: fit_params now take into account mmproj (#21489) 
* mtmd: fit_params now take into account mmproj

* rename alloc_compute_meta to reserve_compute_meta

* rm unused functions

* add ggml_backend_dev_t support

* add debug log
 2026-05-20 11:27:44 +02:00
Xuan-Son NguyenandGitHub b64739ea39 server: (router) alloc tmp buffer on heap (#23159) 2026-05-16 23:42:16 +02:00
Xuan-Son NguyenandGitHub 72e60f500d mtmd: add chunks and fix preproc for qwen3a (#23073) 
* mtmd: add chunks and fix preproc for qwen3a

* add attn_mask

* limit mtmd_chunk size (avoid blow up memory)

* correct audio tokens

* re-order the set_input case

* remove attn_mask
 2026-05-15 19:32:47 +02:00
Xuan-Son NguyenandGitHub e75cd5efb5 download: do not exit() on error (#23008) 2026-05-13 15:14:58 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		3796c94bad ci: validate model naming convention (#22680) 
* ci: validate model naming convention

* bring back dedicated ec workflow

* add missing jobs

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-05-13 10:59:37 +02:00
Xuan-Son NguyenandGitHub 7bfe120c21 mtmd, server, common: expose modalities to /v1/models (#22952) 
* mtmd, server, common: expose modalities to /v1/models

* fix build

* rename to mtmd_caps
 2026-05-12 19:08:07 +02:00
Xuan-Son NguyenandGitHub 29debb3a6a server: support Vertex AI compatible API (#22545) 
* server: support Vertex AI compatible API

* a bit safer

* support other AIP_* env var

* various fixes

* if AIP_MODE is unset, do nothing

* fix test case

* fix windows build
 2026-05-08 15:23:04 +02:00
Xuan-Son NguyenandGitHub 9dcf835528 server: (router) expose child model info from router's /v1/models (#22683) 
* server: (router) expose child model info from router's /v1/models

* update docs
 2026-05-08 14:42:15 +02:00
Xuan-Son NguyenandGitHub 935a340292 server: implement /models?reload=1 (#21848) 2026-05-04 16:23:26 +02:00
Xuan-Son NguyenandGitHub 994118a183 model: move load_hparams and load_tensors to per-model definition (#22004) 
* git-friendly migration

* add build_graph

* nits

* exclude old code from build

* wip

* add llm_arch_model_i

* prepare downstream functions

* nits

* nits

* wip

* wip

* add back create_tensor_qkv

* fix files missing include

* enforce one llm_build per arch

* cmake: use glob

* missing model params

* nits

* wip

* wip (2)

* wip (3)

* test-llama-archs is happy

* improve switch case

* move more stuff into llm_arch_model_i

* fix downstream code

* nits

* nits (2)

* fix order

* llama_model_base

* LLAMA_LOAD_LOCALS

* small fix

* fix build errors

* auto

* rm migration script and ifdef
 2026-05-04 12:36:59 +02:00
Xuan-Son NguyenandGitHub 82d3f4d3b2 mtmd: also support LLAMA_ROPE_TYPE_NONE (#22242) 2026-04-22 12:16:29 +02:00
Xuan-Son NguyenandGitHub 17f6245168 server: ignore reasoning content from transcription api (#21905) 2026-04-22 12:10:50 +02:00
Xuan-Son NguyenandGitHub 04fe84b69d server: allow cancel loading model (#21814) 2026-04-22 00:26:09 +02:00
Xuan-Son NguyenandGitHub 9998d88bc8 mtmd: correct mtmd_decode_use_mrope() (#22188) 2026-04-21 10:53:37 +02:00
Xuan-Son NguyenandGitHub 86f8daacfe mtmd: correct get_n_pos / get_decoder_pos (#22175) 2026-04-20 23:29:19 +02:00
Xuan-Son NguyenandGitHub a678916623 mtmd: refactor mtmd_decode_use_mrope (#22161) 2026-04-20 14:45:11 +02:00
Xuan-Son NguyenandGitHub 19124078be mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos (breaking change) (#22082) 
* mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos

* fix build
 2026-04-19 11:57:21 +02:00
Xuan-Son NguyenandGitHub 089dd41fe3 cmake: use glob to collect src/models sources (#22005) 2026-04-16 23:25:16 +02:00
Xuan-Son NguyenandGitHub 4fbdabdc61 model: using single llm_build per arch (#21970) 
* model: using single llm_build per arch

* fix merge

* nits
 2026-04-16 21:10:22 +02:00
Xuan-Son NguyenandGitHub 408225bb1a server: use random media marker (#21962) 
* server: use random media marker

* nits

* remove legacy <__image__> token

* revert special char in random
 2026-04-15 23:52:22 +02:00
Xuan-Son NguyenandGitHub 80d8770804 docs: more extensive RoPE documentation [no ci] (#21953) 
* more extensive ggml_rope documentation

* add more docs

* nits
 2026-04-15 14:45:16 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		fae3a28070 ggml : remove ggml-ext.h (#21869) 
* ggml: correct placement of ggml-ext.h

* ggml : remove ggml-ext.h

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-04-14 17:32:58 +03:00
Xuan-Son NguyenandGitHub 707c0b7a6e mtmd: add mtmd_image_tokens_get_decoder_pos() API (#21851) 
* mtmd: add mtmd_image_tokens_get_decoder_pos() API

* consistent naming

* fix build
 2026-04-14 16:07:41 +02:00
Xuan-Son NguyenandGitHub e489a5ca0e server: support OAI /v1/audio/transcriptions API (#21863) 
* server: support OAI /v1/audio/transcriptions API

* address autoreview comments

* correct default response_format value
 2026-04-14 11:09:52 +02:00
Xuan-Son NguyenandGitHub e974923698 docs: listing qwen3-asr and qwen3-omni as supported (#21857) 
* docs: listing qwen3-asr and qwen3-omni as supported

* nits
 2026-04-13 22:28:17 +02:00
Xuan-Son NguyenandGitHub 920b3e78cb mtmd: use causal attn for gemma 4 audio (#21824) 2026-04-13 09:47:55 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		21a4933042 mtmd: qwen3 audio support (qwen3-omni and qwen3-asr) (#19441) 
* add qwen3a

* wip

* vision ok

* no more deepstack for audio

* convert ASR model ok

* qwen3 asr working

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* nits

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix bad merge

* fix multi inheritance

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-04-12 23:57:25 +02:00
Xuan-Son NguyenandGitHub aa4695c5e5 mtmd: add gemma 4 test (vision + audio) [no ci] (#21806) 
* mtmd: add gemma 4 test (vision + audio)

* add to docs
 2026-04-12 16:29:03 +02:00
Xuan-Son NguyenandGitHub 6313acbef0 docs: add guide on how to add multimodal support (#21778) 
* docs: add guide on how to add multimodal support

* nits
 2026-04-12 13:02:38 +02:00
Xuan-Son NguyenandGitHub 057dba336e model: fix multimodal padding token for gemma3n/gemma4 (#21625) 
* model: fix multimodal padding token for gemma3n/gemma4

* nits
 2026-04-09 12:18:23 +02:00
Xuan-Son NguyenandGitHub 501aeed18f mtmd: support dots.ocr (#17575) 
* convert gguf

* clip impl

* fix conversion

* wip

* corrections

* update docs

* add gguf to test script
 2026-04-09 12:16:38 +02:00
Xuan-Son NguyenandGitHub 3979f2bb08 docs: add hunyuan-ocr gguf, also add test [no ci] (#21490) 2026-04-06 14:02:37 +02:00
Xuan-Son NguyenandGitHub 63f8fe0ef4 model, mtmd: fix gguf conversion for audio/vision mmproj (#21309) 
* fix gguf conversion for audio/vision mmproj

* fix test
 2026-04-02 17:10:32 +02:00
Xuan-Son NguyenandGitHub c30e012253 contrib : rewrite AGENTS.md, make it more clear about project values (#21270) 
* contrib : rewrite AGENTS.md, make it more clear about types of permitted AI usage

* permit AI for writing code
 2026-04-01 23:31:51 +02:00
Xuan-Son NguyenandGitHub 0356e33aaf scripts: add function call test script (#21234) 
* scripts: add function call test script

* add reasoning_content

* fix lint
 2026-04-01 15:31:58 +02:00
Xuan-Son NguyenGitHubAleksander Grygier
 		4a00bbfed6 server: (webui) no more gzip compression (#21073) 
* webui: no more gzip

* try changing a small line

* Revert "try changing a small line"

This reverts commit 0d7a3531593d87b724d404c8727a96becab3ab07.

* fix lint

* fix test

* rebuild

* split into html/css/js

* lint

* chore: update webui build output

* chore: Update git hooks script

* server: update webui build output

* chore: Update pre-commit hook

* refactor: Cleanup

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
 2026-03-31 15:44:26 +02:00
Xuan-Son NguyenGitHubGeorgi GerganovAleksander Grygier
 		abf9a62161 server: wrap headers for mcp proxy (#21072) 
* server: wrap headers for mcp proxy

* Update tools/server/server-cors-proxy.h

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix build

* chore: update webui build output

* chore: update webui build output

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
 2026-03-30 08:59:16 +02:00
Xuan-Son NguyenandGitHub 871f1a2d2f mtmd: add more sanity checks (#21047) 2026-03-27 11:00:52 +01:00
Xuan-Son NguyenandGitHub 20197b6fe3 server: add built-in tools backend support (#20898) 
* wip: server_tools

* refactor

* displayName -> display_name

* snake_case everywhere

* rm redundant field

* change arg to --tools all

* add readme mention

* llama-gen-docs
 2026-03-27 10:07:11 +01:00
Xuan-Son NguyenandGitHub 8c60b8a2be ci: pin external actions to exact commit SHA (#21033) 2026-03-26 20:44:00 +01:00
Xuan-Son NguyenandGitHub a73bbd5d92 mtmd: refactor image preprocessing (#21031) 
* mtmd: refactor image pre-processing

* correct some places

* correct lfm2

* fix deepseek-ocr on server

* add comment to clarify about mtmd_image_preprocessor_dyn_size
 2026-03-26 19:49:20 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		914eb5ff0c jinja: fix macro with kwargs (#20960) 
* jinja: fix macro with kwargs

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix newline problem

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-03-25 12:22:48 +01:00
Xuan-Son NguyenGitHubPiotr Wilkin
 		bd6992180b contrib: add "Requirements" section to PR template (#20841) 
* contrib: add "Requirements" section to PR template

* typo [no ci]

* use h2, add "Additional information"

---------

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>
 2026-03-23 16:59:02 +01:00
Xuan-Son NguyenandGitHub 31a5cf4c3f server: use httplib dynamic threads (#20817) 
* server: use httplib dynamic threads

* change to n_threads_http + 1024
 2026-03-23 12:22:46 +01:00
Xuan-Son NguyenandGitHub 49bfddeca1 server: allow router to report child instances sleep status (#20849) 
* server: allow router to report child instances sleep status

* refactor

* move sleeping to state

* nits
 2026-03-22 18:33:52 +01:00
Xuan-Son NguyenandGitHub 568aec82d2 docs : explicit about banning accounts that violates policy (#19593) 2026-03-21 15:50:16 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		fb78ad29bb server: (doc) clarify in-scope and out-scope features (#20794) 
* server: (doc) clarify in-scope and out-scope features

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-03-20 14:03:50 +01:00
Xuan-Son NguyenandGitHub 1e64534570 mtmd: add clip_graph::build_mm() (#20751) 
* clip: add build_mm()

* apply to all models

* add TODO for bias overload
 2026-03-19 13:11:39 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		d34ff7eb5b model: mistral small 4 support (#20649) 
* model: mistral small 4 support

* fix test

* fix test (2)

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* change newline

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-03-17 00:31:14 +01:00
Xuan-Son NguyenandGitHub 94d0262277 mtmd: add llama-mtmd-debug binary (#20508) 
* mtmd: add llama-mtmd-debug binary

* adapt

* fixes

* fix compile error

* fix windows compile error

* rm legacy clip_debug_encode()

* add MTMD_API to fix build
 2026-03-14 15:52:29 +01:00
Xuan-Son NguyenandGitHub 0842b9b465 model: fix step3.5 n_rot (#20318) 2026-03-09 23:42:24 +01:00
Xuan-Son NguyenandGitHub 59db9a357d llama: dynamic head_dim and n_rot for SWA (#20301) 
* llama: dynamic head_dim and n_rot for SWA

* also add gguf_writer wrappers

* fix build

* build_rope_shift arg reorder
 2026-03-09 22:22:39 +01:00
Xuan-Son NguyenandGitHub c747294b2d scripts: update corpus of compare-logprobs (#19326) 
* scripts: update corpus of compare-logprobs

* fix
 2026-02-25 12:57:34 +01:00
Xuan-Son NguyenandGitHub 5452d736f8 jinja: correct stats for tojson and string filters (#19785) 2026-02-22 21:08:23 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		eeef3cfced model: support GLM-OCR (#19677) 
* model: support GLM-OCR

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-02-18 17:51:40 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		752584d5f5 model: support GLM MoE DSA arch (NOTE: indexer is not yet supported) (#19460) 
* model: support GLM MoE DSA arch

* working version

* pyright

* keep indexer tensors

* add indexer gguf params

* loaded now

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* update

* Update src/llama-model.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* minor fix and cleanup

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-02-13 14:56:53 +01:00