nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 19 Packages Projects Releases Wiki Activity
9,749 Commits 1 Branch 0 Tags
2f89acc2bc614dc121db065a74e503bf88668951
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan-Son NguyenandGitHub 2f89acc2bc mtmd: add load progress callback (#24865) 2026-06-21 13:40:52 +02:00
Xuan-Son NguyenandGitHub bfa3219177 server: add "verbose" field to schema (#24864) 2026-06-21 13:03:14 +02:00
Xuan-Son NguyenandGitHub d6d899580d server: real-time model load progress tracking via /models/sse (#24828) 
* server: real-time model load progress tracking via /models/sse

* update docs

* add mutex for notify_to_router

* correct docs
 2026-06-21 11:58:14 +02:00
Xuan-Son NguyenGitHubPascal
 		75f460ac28 arg: try fixing test-args-parser randomly fails (#24826) 
* arg: try fixing test-args-parser randomly fails

* return ref

* try triggering the workflow

* exception wrapper

* wip

* test

* test 2

* arg: guard win32 utf8 argv override

make_utf8_argv rebuilds argv from GetCommandLineW to fix utf8 handling of
non ascii arguments on windows. the override runs unconditionally inside
common_params_parse, so it also clobbers a programmatic argv passed by a
caller. test-arg-parser builds a synthetic argv but then sees the real
process command line instead, the model argument is never parsed, and the
assert that expects success aborts via fastfail (0xC0000409). this shows up
as a random failure in the openvino windows workflow.

only override argv when its length matches the caller argc, so the utf8
repair still applies to real binaries while a programmatic argv stays intact.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
 2026-06-20 19:45:27 +02:00
Xuan-Son NguyenandGitHub 2b686a9120 server: refactor child --> router communication (#24821) 
* server: refactor child --> router communication

* fix wakeup case

* add docs

* improve update_status()

* nits
 2026-06-20 01:02:26 +02:00
Xuan-Son NguyenandGitHub e475fa2b5f mtmd, arg: fix utf8 handling on windows (#24779) 
* mtmd, arg: fix utf8 handling on windows

* also fix ggml_fopen

* fix build fail

* also fix CLI
 2026-06-19 22:28:38 +02:00
Xuan-Son NguyenandGitHub 175147e8f6 server: remove all internal mentions about "webui" (#24817) 2026-06-19 22:12:46 +02:00
Xuan-Son NguyenandGitHub 8c2d6f6475 server: add --agent arg, remove redundant webui naming compat (#24801) 
* server: add --agent arg, remove redundant webui naming compat

* corrent env

* fix the test

* llama-gen-docs

* nits: wordings
 2026-06-19 16:06:13 +02:00
Xuan-Son NguyenandGitHub e2e7a9b2d0 mtmd: several bug fixes (#24784) 
* mtmd: several bug fixes

* fix build

* fix gemma4ua

* add sanity check in get_u32()

* fix build (2)

* area() avoid overflow
 2026-06-19 12:18:36 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		159d093a43 server: fix non-bound n_discard value (ctx shifting) (#24786) 
* server: fix non-bound n_discard value

* Update tools/server/server-context.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-06-19 10:53:44 +02:00
Xuan-Son NguyenandGitHub db52540f73 mtmd: add batching support for internvl (#24775) 2026-06-19 01:16:16 +02:00
Xuan-Son NguyenandGitHub a6b3260a42 mtmd: add batching for mtmd-cli, add video tests (#24778) 2026-06-18 21:55:04 +02:00
Xuan-Son NguyenandGitHub 060ce1bf72 mtmd: refactor llava-uhd overview image handling (always use ov_img_first) (#24769) 
* add dedicated "overview" for mtmd_image_preproc_out

* corrections

* correct (again)

* nits

* nits (2)
 2026-06-18 18:53:49 +02:00
Xuan-Son NguyenandGitHub fe7c8b2414 server: (router) fix stopping_thread potentially hang (#24728) 
* server: (router) fix stopping_thread potentially hang

* fix windows build
 2026-06-18 15:41:09 +02:00
Xuan-Son NguyenandGitHub e1efd0991d server: add "schema" and validation (#24150) 
* wip

* working

* correct some limits

* add field name to error message
 2026-06-18 15:40:58 +02:00
Xuan-Son NguyenandGitHub 552258c535 server: (router) rework -hf preset repo (#24739) 
* server: temporary remove HF remote preset

* rework remove preset.ini support

* rm unused get_remote_preset_whitelist()

* print warning

* add docs

* rm stray file
 2026-06-18 12:45:23 +02:00
Xuan-Son NguyenandGitHub 968c43891a server: fix router args not being forwarded to child instances (#24760) 2026-06-18 12:15:46 +02:00
Xuan-Son NguyenandGitHub 24bba7b98e mtmd: refactor preprocessor, add mtmd_image_preproc_out (#24736) 
* add mtmd_image_preproc_out

* add dev docs

* remove unused clip API

* rm unused clip_image_f32_batch::grid

* change preprocess() call signature
 2026-06-18 12:04:39 +02:00
Xuan-Son NguyenandGitHub f3e1828164 mtmd: llava_uhd should no longer use batch dim (#24732) 2026-06-17 22:40:50 +02:00
Xuan-Son NguyenandGitHub 4b4d13ae72 server: (router) add model management API (#23976) 
* wip

* server: (router) add SSE realtime updates API

* nits

* wip

* add download API

* add download api

* update docs

* add delete endpoint

* fix std::terminate

* fix crash

* fix 2

* add tests

* nits
 2026-06-17 18:04:58 +02:00
Xuan-Son NguyenandGitHub e36a602ba3 mtmd: fix miscounting n_tokens (#24656) 2026-06-15 18:07:14 +02:00
Xuan-Son NguyenandGitHub e8067a8b36 ui: build-time gzip compression (#24571) 
* ui: keep original file name and path

* fix nocache

* ui: build-time gzip compression
 2026-06-13 16:57:27 +02:00
Xuan-Son NguyenandGitHub 597b6672e8 ui: keep original file name and path (#24568) 
* ui: keep original file name and path

* fix nocache
 2026-06-13 14:31:41 +02:00
Xuan-Son NguyenGitHubAlde Rojas
 		57fe1f07c3 server: clean up static assets handling (#24550) 
* server: clean up static assets handling

* nits

* simplify file name handling, use static file name everywhere

* cmake/ui : bundle UI assets in an archive

* ui : run prettier on post-build.js

---------

Co-authored-by: Alde Rojas <hello@alde.dev>
 2026-06-13 11:51:20 +02:00
Xuan-Son NguyenandGitHub e37abd6b5f mtmd: add batching API (#24384) 
* mtmd: add batching API

* wip

* first working version (gemma4v)

* add arg

* nits

* wire up support_batch()

* fix 0.0 output embd

* fix audio

* nits

* refactor a bit

* nits

* fix non-batching case

* fix comment
 2026-06-13 00:10:29 +02:00
Xuan-Son NguyenandGitHub 18ef86ecec server: skip unused log lines on router mode (#24463) 2026-06-11 11:36:35 +02:00
Xuan-Son NguyenGitHubMikko Juola
 		9682e351b8 mtmd: refactor video subproc handling (#24316) 
* mtmd: refactor video subproc handling

* Update tools/mtmd/mtmd-helper.cpp

Co-authored-by: Mikko Juola <mikjuo@gmail.com>

---------

Co-authored-by: Mikko Juola <mikjuo@gmail.com>
 2026-06-09 13:15:12 +03:00
Xuan-Son NguyenandGitHub 3ebe862b5d docker: install ffmpeg in the released image (#24302) 2026-06-08 16:59:57 +02:00
Xuan-Son NguyenandGitHub 8f83d6c271 mtmd : add video input support (#24269) 
* wip

* ok: lazy bitmap API

* remember to free lazy text

* wip

* add mtmd_helper_video

* support video input on server (base64 input)

* add MTMD_VIDEO config

* add timestamp

* update CLI

* cli: allow auto-completion for video

* add --video arg

* fix build

* update docs

* rename as suggested
 2026-06-08 14:40:12 +03:00
Xuan-Son NguyenandGitHub 715b86a366 cli: fix spinner not show during prompt processing (#24283) 2026-06-08 11:11:45 +02:00
Xuan-Son NguyenandGitHub daf6bc9f2d metal : fix im2col 1D case (audio models) (#24220) 2026-06-08 09:03:18 +03:00
Xuan-Son NguyenGitHubandrewmd5
 		31e82494c0 mtmd: support "frame merge" for qwen-vl-based models (#21858) 
* feat: add video support for Qwen3.5

* various clean up

* revise the design

* fix llava-uhd case

* nits

* nits 2

---------

Co-authored-by: andrewmd5 <1297077+andrewmd5@users.noreply.github.com>
 2026-06-06 21:17:25 +02:00
Xuan-Son NguyenandGitHub f5c6ae1827 mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913) 
* mtmd: add "placeholder bitmap" for counting tokens w/o preprocessing

* fast path skip preproc for placeholder

* fix build

* correct the api

* add server endpoint + tests

* add object name

* update docs

* add proxy handling

* fix build

* fix audio input path

* use is_placeholder in process_mtmd_prompt()

* nits

* nits (2)

* docs: clarify chat/completions/input_tokens is not official

* fix merge problem
 2026-06-06 11:06:51 +02:00
Xuan-Son NguyenandGitHub c4a278d68e model: fix build failed (#24193) 2026-06-05 18:12:27 +02:00
Xuan-Son NguyenandGitHub cc7bef34e2 ui: add ignore-scripts=true to npmrc (#24149) 2026-06-05 14:31:03 +02:00
Xuan-Son NguyenandGitHub 260862b8ca arg: fix double mtp downloads (#24128) 2026-06-04 19:23:48 +03:00
Xuan-Son NguyenandGitHub a121232fdc agents: refactor, include more guidelines (#24111) 
* agents: refactor, include more guidelines

* better example

* rephrase a bit

* add more examples

* nits
 2026-06-04 13:40:23 +02:00
Xuan-Son NguyenandGitHub c8d6a00636 mtmd: enable non-causal vision for gemma 4 unified (#24082) 2026-06-03 19:05:17 +02:00
Xuan-Son NguyenandGitHub a731805ced mtmd, model: allow skip build_vit() (#24077) 
* add model

* nits
 2026-06-03 17:10:35 +02:00
Xuan-Son NguyenandGitHub 60130d18f9 server: add SSE ping interval (#24013) 2026-06-02 14:14:55 +02:00
Xuan-Son NguyenandGitHub e674b1279b test: (test-llama-archs) log the config name first (#23885) 2026-05-30 12:22:38 +02:00
Xuan-Son NguyenandGitHub 0821c5fcfd server: in SSE mode, send HTTP headers when slot starts (#23884) 
* server: in SSE mode, send HTTP headers when slot starts

* ref to pr

* stream should be false by default
 2026-05-30 00:06:29 +02:00
Xuan-Son NguyenandGitHub b5f52280fb server: remove obsolete scripts (#23870) 2026-05-29 19:47:30 +02:00
Xuan-Son NguyenandGitHub 06d26dfdff download: add option to skip_download (#23059) 
* download: add option to skip_download

* fix

* fix 2

* if file doesn't exist, respect skip_download flag
 2026-05-29 16:30:55 +02:00
Xuan-Son NguyenandGitHub cb47092b00 server: bump timeout to 3600s (#23842) 
* server: bump timeout to 3600s

* nits: change wording
 2026-05-29 10:23:17 +02:00
Xuan-Son NguyenandGitHub 751ebd17a5 mtmd-debug: add color and rainbow mode (#23829) 
* mtmd-debug: add color and rainbow mode

* fix M_PI

* max_dist
 2026-05-28 20:59:14 +02:00
Xuan-Son NguyenandGitHub c8914ad4f4 mtmd: fix gemma 4 projector pre_norm (#23822) 2026-05-28 20:58:55 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		d6be3158e1 mtmd: fix gemma 4 audio rms norm eps (#23815) 
* mtmd: fix gemma 4 audio rms norm eps

* Update tools/mtmd/clip.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-05-28 16:31:37 +02:00
Xuan-Son NguyenandGitHub e2b129e1bf mtmd: fit_params now take into account mmproj (#21489) 
* mtmd: fit_params now take into account mmproj

* rename alloc_compute_meta to reserve_compute_meta

* rm unused functions

* add ggml_backend_dev_t support

* add debug log
 2026-05-20 11:27:44 +02:00
Xuan-Son NguyenandGitHub b64739ea39 server: (router) alloc tmp buffer on heap (#23159) 2026-05-16 23:42:16 +02:00
Xuan-Son NguyenandGitHub 72e60f500d mtmd: add chunks and fix preproc for qwen3a (#23073) 
* mtmd: add chunks and fix preproc for qwen3a

* add attn_mask

* limit mtmd_chunk size (avoid blow up memory)

* correct audio tokens

* re-order the set_input case

* remove attn_mask
 2026-05-15 19:32:47 +02:00
Xuan-Son NguyenandGitHub e75cd5efb5 download: do not exit() on error (#23008) 2026-05-13 15:14:58 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		3796c94bad ci: validate model naming convention (#22680) 
* ci: validate model naming convention

* bring back dedicated ec workflow

* add missing jobs

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-05-13 10:59:37 +02:00
Xuan-Son NguyenandGitHub 7bfe120c21 mtmd, server, common: expose modalities to /v1/models (#22952) 
* mtmd, server, common: expose modalities to /v1/models

* fix build

* rename to mtmd_caps
 2026-05-12 19:08:07 +02:00
Xuan-Son NguyenandGitHub 29debb3a6a server: support Vertex AI compatible API (#22545) 
* server: support Vertex AI compatible API

* a bit safer

* support other AIP_* env var

* various fixes

* if AIP_MODE is unset, do nothing

* fix test case

* fix windows build
 2026-05-08 15:23:04 +02:00
Xuan-Son NguyenandGitHub 9dcf835528 server: (router) expose child model info from router's /v1/models (#22683) 
* server: (router) expose child model info from router's /v1/models

* update docs
 2026-05-08 14:42:15 +02:00
Xuan-Son NguyenandGitHub 935a340292 server: implement /models?reload=1 (#21848) 2026-05-04 16:23:26 +02:00
Xuan-Son NguyenandGitHub 994118a183 model: move load_hparams and load_tensors to per-model definition (#22004) 
* git-friendly migration

* add build_graph

* nits

* exclude old code from build

* wip

* add llm_arch_model_i

* prepare downstream functions

* nits

* nits

* wip

* wip

* add back create_tensor_qkv

* fix files missing include

* enforce one llm_build per arch

* cmake: use glob

* missing model params

* nits

* wip

* wip (2)

* wip (3)

* test-llama-archs is happy

* improve switch case

* move more stuff into llm_arch_model_i

* fix downstream code

* nits

* nits (2)

* fix order

* llama_model_base

* LLAMA_LOAD_LOCALS

* small fix

* fix build errors

* auto

* rm migration script and ifdef
 2026-05-04 12:36:59 +02:00
Xuan-Son NguyenandGitHub 82d3f4d3b2 mtmd: also support LLAMA_ROPE_TYPE_NONE (#22242) 2026-04-22 12:16:29 +02:00
Xuan-Son NguyenandGitHub 17f6245168 server: ignore reasoning content from transcription api (#21905) 2026-04-22 12:10:50 +02:00
Xuan-Son NguyenandGitHub 04fe84b69d server: allow cancel loading model (#21814) 2026-04-22 00:26:09 +02:00
Xuan-Son NguyenandGitHub 9998d88bc8 mtmd: correct mtmd_decode_use_mrope() (#22188) 2026-04-21 10:53:37 +02:00
Xuan-Son NguyenandGitHub 86f8daacfe mtmd: correct get_n_pos / get_decoder_pos (#22175) 2026-04-20 23:29:19 +02:00
Xuan-Son NguyenandGitHub a678916623 mtmd: refactor mtmd_decode_use_mrope (#22161) 2026-04-20 14:45:11 +02:00
Xuan-Son NguyenandGitHub 19124078be mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos (breaking change) (#22082) 
* mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos

* fix build
 2026-04-19 11:57:21 +02:00
Xuan-Son NguyenandGitHub 089dd41fe3 cmake: use glob to collect src/models sources (#22005) 2026-04-16 23:25:16 +02:00
Xuan-Son NguyenandGitHub 4fbdabdc61 model: using single llm_build per arch (#21970) 
* model: using single llm_build per arch

* fix merge

* nits
 2026-04-16 21:10:22 +02:00
Xuan-Son NguyenandGitHub 408225bb1a server: use random media marker (#21962) 
* server: use random media marker

* nits

* remove legacy <__image__> token

* revert special char in random
 2026-04-15 23:52:22 +02:00
Xuan-Son NguyenandGitHub 80d8770804 docs: more extensive RoPE documentation [no ci] (#21953) 
* more extensive ggml_rope documentation

* add more docs

* nits
 2026-04-15 14:45:16 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		fae3a28070 ggml : remove ggml-ext.h (#21869) 
* ggml: correct placement of ggml-ext.h

* ggml : remove ggml-ext.h

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-04-14 17:32:58 +03:00
Xuan-Son NguyenandGitHub 707c0b7a6e mtmd: add mtmd_image_tokens_get_decoder_pos() API (#21851) 
* mtmd: add mtmd_image_tokens_get_decoder_pos() API

* consistent naming

* fix build
 2026-04-14 16:07:41 +02:00
Xuan-Son NguyenandGitHub e489a5ca0e server: support OAI /v1/audio/transcriptions API (#21863) 
* server: support OAI /v1/audio/transcriptions API

* address autoreview comments

* correct default response_format value
 2026-04-14 11:09:52 +02:00
Xuan-Son NguyenandGitHub e974923698 docs: listing qwen3-asr and qwen3-omni as supported (#21857) 
* docs: listing qwen3-asr and qwen3-omni as supported

* nits
 2026-04-13 22:28:17 +02:00
Xuan-Son NguyenandGitHub 920b3e78cb mtmd: use causal attn for gemma 4 audio (#21824) 2026-04-13 09:47:55 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		21a4933042 mtmd: qwen3 audio support (qwen3-omni and qwen3-asr) (#19441) 
* add qwen3a

* wip

* vision ok

* no more deepstack for audio

* convert ASR model ok

* qwen3 asr working

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* nits

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix bad merge

* fix multi inheritance

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-04-12 23:57:25 +02:00
Xuan-Son NguyenandGitHub aa4695c5e5 mtmd: add gemma 4 test (vision + audio) [no ci] (#21806) 
* mtmd: add gemma 4 test (vision + audio)

* add to docs
 2026-04-12 16:29:03 +02:00
Xuan-Son NguyenandGitHub 6313acbef0 docs: add guide on how to add multimodal support (#21778) 
* docs: add guide on how to add multimodal support

* nits
 2026-04-12 13:02:38 +02:00
Xuan-Son NguyenandGitHub 057dba336e model: fix multimodal padding token for gemma3n/gemma4 (#21625) 
* model: fix multimodal padding token for gemma3n/gemma4

* nits
 2026-04-09 12:18:23 +02:00
Xuan-Son NguyenandGitHub 501aeed18f mtmd: support dots.ocr (#17575) 
* convert gguf

* clip impl

* fix conversion

* wip

* corrections

* update docs

* add gguf to test script
 2026-04-09 12:16:38 +02:00
Xuan-Son NguyenandGitHub 3979f2bb08 docs: add hunyuan-ocr gguf, also add test [no ci] (#21490) 2026-04-06 14:02:37 +02:00
Xuan-Son NguyenandGitHub 63f8fe0ef4 model, mtmd: fix gguf conversion for audio/vision mmproj (#21309) 
* fix gguf conversion for audio/vision mmproj

* fix test
 2026-04-02 17:10:32 +02:00
Xuan-Son NguyenandGitHub c30e012253 contrib : rewrite AGENTS.md, make it more clear about project values (#21270) 
* contrib : rewrite AGENTS.md, make it more clear about types of permitted AI usage

* permit AI for writing code
 2026-04-01 23:31:51 +02:00
Xuan-Son NguyenandGitHub 0356e33aaf scripts: add function call test script (#21234) 
* scripts: add function call test script

* add reasoning_content

* fix lint
 2026-04-01 15:31:58 +02:00
Xuan-Son NguyenGitHubAleksander Grygier
 		4a00bbfed6 server: (webui) no more gzip compression (#21073) 
* webui: no more gzip

* try changing a small line

* Revert "try changing a small line"

This reverts commit 0d7a3531593d87b724d404c8727a96becab3ab07.

* fix lint

* fix test

* rebuild

* split into html/css/js

* lint

* chore: update webui build output

* chore: Update git hooks script

* server: update webui build output

* chore: Update pre-commit hook

* refactor: Cleanup

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
 2026-03-31 15:44:26 +02:00
Xuan-Son NguyenGitHubGeorgi GerganovAleksander Grygier
 		abf9a62161 server: wrap headers for mcp proxy (#21072) 
* server: wrap headers for mcp proxy

* Update tools/server/server-cors-proxy.h

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix build

* chore: update webui build output

* chore: update webui build output

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
 2026-03-30 08:59:16 +02:00
Xuan-Son NguyenandGitHub 871f1a2d2f mtmd: add more sanity checks (#21047) 2026-03-27 11:00:52 +01:00
Xuan-Son NguyenandGitHub 20197b6fe3 server: add built-in tools backend support (#20898) 
* wip: server_tools

* refactor

* displayName -> display_name

* snake_case everywhere

* rm redundant field

* change arg to --tools all

* add readme mention

* llama-gen-docs
 2026-03-27 10:07:11 +01:00
Xuan-Son NguyenandGitHub 8c60b8a2be ci: pin external actions to exact commit SHA (#21033) 2026-03-26 20:44:00 +01:00
Xuan-Son NguyenandGitHub a73bbd5d92 mtmd: refactor image preprocessing (#21031) 
* mtmd: refactor image pre-processing

* correct some places

* correct lfm2

* fix deepseek-ocr on server

* add comment to clarify about mtmd_image_preprocessor_dyn_size
 2026-03-26 19:49:20 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		914eb5ff0c jinja: fix macro with kwargs (#20960) 
* jinja: fix macro with kwargs

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix newline problem

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-03-25 12:22:48 +01:00
Xuan-Son NguyenGitHubPiotr Wilkin
 		bd6992180b contrib: add "Requirements" section to PR template (#20841) 
* contrib: add "Requirements" section to PR template

* typo [no ci]

* use h2, add "Additional information"

---------

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>
 2026-03-23 16:59:02 +01:00
Xuan-Son NguyenandGitHub 31a5cf4c3f server: use httplib dynamic threads (#20817) 
* server: use httplib dynamic threads

* change to n_threads_http + 1024
 2026-03-23 12:22:46 +01:00
Xuan-Son NguyenandGitHub 49bfddeca1 server: allow router to report child instances sleep status (#20849) 
* server: allow router to report child instances sleep status

* refactor

* move sleeping to state

* nits
 2026-03-22 18:33:52 +01:00
Xuan-Son NguyenandGitHub 568aec82d2 docs : explicit about banning accounts that violates policy (#19593) 2026-03-21 15:50:16 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		fb78ad29bb server: (doc) clarify in-scope and out-scope features (#20794) 
* server: (doc) clarify in-scope and out-scope features

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-03-20 14:03:50 +01:00
Xuan-Son NguyenandGitHub 1e64534570 mtmd: add clip_graph::build_mm() (#20751) 
* clip: add build_mm()

* apply to all models

* add TODO for bias overload
 2026-03-19 13:11:39 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		d34ff7eb5b model: mistral small 4 support (#20649) 
* model: mistral small 4 support

* fix test

* fix test (2)

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* change newline

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-03-17 00:31:14 +01:00
Xuan-Son NguyenandGitHub 94d0262277 mtmd: add llama-mtmd-debug binary (#20508) 
* mtmd: add llama-mtmd-debug binary

* adapt

* fixes

* fix compile error

* fix windows compile error

* rm legacy clip_debug_encode()

* add MTMD_API to fix build
 2026-03-14 15:52:29 +01:00
Xuan-Son NguyenandGitHub 0842b9b465 model: fix step3.5 n_rot (#20318) 2026-03-09 23:42:24 +01:00
Xuan-Son NguyenandGitHub 59db9a357d llama: dynamic head_dim and n_rot for SWA (#20301) 
* llama: dynamic head_dim and n_rot for SWA

* also add gguf_writer wrappers

* fix build

* build_rope_shift arg reorder
 2026-03-09 22:22:39 +01:00