100 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Xuan-Son NguyenandGitHub	501aeed18f	mtmd: support dots.ocr (#17575) ... * convert gguf * clip impl * fix conversion * wip * corrections * update docs * add gguf to test script	2026-04-09 12:16:38 +02:00
Xuan-Son NguyenandGitHub	3979f2bb08	docs: add hunyuan-ocr gguf, also add test [no ci] (#21490)	2026-04-06 14:02:37 +02:00
Xuan-Son NguyenandGitHub	63f8fe0ef4	model, mtmd: fix gguf conversion for audio/vision mmproj (#21309) ... * fix gguf conversion for audio/vision mmproj * fix test	2026-04-02 17:10:32 +02:00
Xuan-Son NguyenandGitHub	c30e012253	contrib : rewrite AGENTS.md, make it more clear about project values (#21270) ... * contrib : rewrite AGENTS.md, make it more clear about types of permitted AI usage * permit AI for writing code	2026-04-01 23:31:51 +02:00
Xuan-Son NguyenandGitHub	0356e33aaf	scripts: add function call test script (#21234) ... * scripts: add function call test script * add reasoning_content * fix lint	2026-04-01 15:31:58 +02:00
3 people	4a00bbfed6	server: (webui) no more gzip compression (#21073) ... * webui: no more gzip * try changing a small line * Revert "try changing a small line" This reverts commit 0d7a3531593d87b724d404c8727a96becab3ab07. * fix lint * fix test * rebuild * split into html/css/js * lint * chore: update webui build output * chore: Update git hooks script * server: update webui build output * chore: Update pre-commit hook * refactor: Cleanup --------- Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>	2026-03-31 15:44:26 +02:00
4 people	abf9a62161	server: wrap headers for mcp proxy (#21072) ... * server: wrap headers for mcp proxy * Update tools/server/server-cors-proxy.h Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * fix build * chore: update webui build output * chore: update webui build output --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>	2026-03-30 08:59:16 +02:00
Xuan-Son NguyenandGitHub	871f1a2d2f	mtmd: add more sanity checks (#21047)	2026-03-27 11:00:52 +01:00
Xuan-Son NguyenandGitHub	20197b6fe3	server: add built-in tools backend support (#20898) ... * wip: server_tools * refactor * displayName -> display_name * snake_case everywhere * rm redundant field * change arg to --tools all * add readme mention * llama-gen-docs	2026-03-27 10:07:11 +01:00
Xuan-Son NguyenandGitHub	8c60b8a2be	ci: pin external actions to exact commit SHA (#21033)	2026-03-26 20:44:00 +01:00
Xuan-Son NguyenandGitHub	a73bbd5d92	mtmd: refactor image preprocessing (#21031) ... * mtmd: refactor image pre-processing * correct some places * correct lfm2 * fix deepseek-ocr on server * add comment to clarify about mtmd_image_preprocessor_dyn_size	2026-03-26 19:49:20 +01:00
3 people	914eb5ff0c	jinja: fix macro with kwargs (#20960) ... * jinja: fix macro with kwargs * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * fix newline problem --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>	2026-03-25 12:22:48 +01:00
3 people	bd6992180b	contrib: add "Requirements" section to PR template (#20841) ... * contrib: add "Requirements" section to PR template * typo [no ci] * use h2, add "Additional information" --------- Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>	2026-03-23 16:59:02 +01:00
Xuan-Son NguyenandGitHub	31a5cf4c3f	server: use httplib dynamic threads (#20817) ... * server: use httplib dynamic threads * change to n_threads_http + 1024	2026-03-23 12:22:46 +01:00
Xuan-Son NguyenandGitHub	49bfddeca1	server: allow router to report child instances sleep status (#20849) ... * server: allow router to report child instances sleep status * refactor * move sleeping to state * nits	2026-03-22 18:33:52 +01:00
Xuan-Son NguyenandGitHub	568aec82d2	docs : explicit about banning accounts that violates policy (#19593)	2026-03-21 15:50:16 +01:00
3 people	fb78ad29bb	server: (doc) clarify in-scope and out-scope features (#20794) ... * server: (doc) clarify in-scope and out-scope features * Apply suggestions from code review Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2026-03-20 14:03:50 +01:00
Xuan-Son NguyenandGitHub	1e64534570	mtmd: add clip_graph::build_mm() (#20751) ... * clip: add build_mm() * apply to all models * add TODO for bias overload	2026-03-19 13:11:39 +01:00
3 people	d34ff7eb5b	model: mistral small 4 support (#20649) ... * model: mistral small 4 support * fix test * fix test (2) * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update convert_hf_to_gguf.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * change newline --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>	2026-03-17 00:31:14 +01:00
Xuan-Son NguyenandGitHub	94d0262277	mtmd: add llama-mtmd-debug binary (#20508) ... * mtmd: add llama-mtmd-debug binary * adapt * fixes * fix compile error * fix windows compile error * rm legacy clip_debug_encode() * add MTMD_API to fix build	2026-03-14 15:52:29 +01:00
Xuan-Son NguyenandGitHub	0842b9b465	model: fix step3.5 n_rot (#20318)	2026-03-09 23:42:24 +01:00
Xuan-Son NguyenandGitHub	59db9a357d	llama: dynamic head_dim and n_rot for SWA (#20301) ... * llama: dynamic head_dim and n_rot for SWA * also add gguf_writer wrappers * fix build * build_rope_shift arg reorder	2026-03-09 22:22:39 +01:00
Xuan-Son NguyenandGitHub	c747294b2d	scripts: update corpus of compare-logprobs (#19326) ... * scripts: update corpus of compare-logprobs * fix	2026-02-25 12:57:34 +01:00
Xuan-Son NguyenandGitHub	5452d736f8	jinja: correct stats for tojson and string filters (#19785)	2026-02-22 21:08:23 +01:00
3 people	eeef3cfced	model: support GLM-OCR (#19677) ... * model: support GLM-OCR * Update convert_hf_to_gguf.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>	2026-02-18 17:51:40 +01:00
3 people	752584d5f5	model: support GLM MoE DSA arch (NOTE: indexer is not yet supported) (#19460) ... * model: support GLM MoE DSA arch * working version * pyright * keep indexer tensors * add indexer gguf params * loaded now * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * update * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * minor fix and cleanup --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>	2026-02-13 14:56:53 +01:00
Xuan-Son NguyenandGitHub	9a96352729	test: fix IMROPE perf test case (#19465)	2026-02-10 14:37:50 +01:00
Xuan-Son NguyenandGitHub	98e57ca422	chat: fix case where template accepts type content only (#19419) ... * chat: fix case where template accepts type content only * rm stray log * reuse render_message_to_json	2026-02-09 22:14:12 +01:00
Xuan-Son NguyenandGitHub	e0c93af2a0	debug: make common_debug_print_tensor readable (#19331) ... * debug: make common_debug_print_tensor readable * editorconfig	2026-02-04 17:55:31 +01:00
Xuan-Son NguyenandGitHub	8abcc70a74	model: (qwen3next) correct vectorized key_gdiff calculation (#19324) ... * model: (qwen3next) correct vectorized key_gdiff calculation * move transpose to outside of loop	2026-02-04 13:09:58 +01:00
Xuan-Son NguyenandGitHub	07a7412a3b	mtmd: add min/max pixels gguf metadata (#19273)	2026-02-02 20:59:06 +01:00
3 people	bb02f74c61	chat: fix language input for translategemma (#19052) ... * chat: fix language input for translategemma * Update common/chat.cpp Co-authored-by: Aldehir Rojas <hello@alde.dev> --------- Co-authored-by: Aldehir Rojas <hello@alde.dev>	2026-01-24 17:58:45 +01:00
Xuan-Son NguyenandGitHub	b5b8fa1c8b	chat : fix translategemma crash on common_chat_format_example (#19019)	2026-01-23 12:03:42 +01:00
Xuan-Son NguyenandGitHub	51fa458a92	server : support preserving reasoning_content in assistant message (#18994) ... * support reasoning_content input * report template caps to webui * add docs * rm commented code	2026-01-22 21:30:06 +01:00
Xuan-Son NguyenandGitHub	4e595b250a	server: do not log certain endpoints (avoid log spam) (#19028)	2026-01-22 19:24:37 +01:00
Xuan-Son NguyenandGitHub	9eb5bfec1a	mtmd : update docs to use llama_model_n_embd_inp (#18999)	2026-01-22 14:36:32 +01:00
Xuan-Son NguyenandGitHub	2c1f199653	cli : fix reasoning responses in CLI (#18961) ... * cli : fix reasoning responses in CLI * fix build * fix build (2)	2026-01-20 18:23:25 +01:00
Xuan-Son NguyenandGitHub	6df686bee6	server : refactor oai_parser_opt, move it to server_chat_params (#18937) ... * server_chat_params * move chat format into CLI * use meta whenever possible * clean up, no more chatml fallback	2026-01-19 23:28:01 +01:00
3 people	fe44d35574	tests : add test-jinja -py option for cross-checking (#18906) ... * tests : add test-jinja -py option or cross-checking * Update tests/test-jinja.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * fix + add source * SandboxedEnvironment * fix array.map case --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>	2026-01-18 08:14:27 +01:00
Xuan-Son NguyenandGitHub	f55b033ae6	jinja: correct member access rule (#18905)	2026-01-18 00:48:55 +01:00
5 people	c15395f73c	common : implement new jinja template engine (#18462) ... * jinja vm * lexer * add vm types * demo * clean up * parser ok * binary_expression::execute * shadow naming * bin ops works! * fix map object * add string builtins * add more builtins * wip * use mk_val * eval with is_user_input * render gemma tmpl ok * track input string even after transformations * support binded functions * keyword arguments and slicing array * use shared_ptr for values * add mk_stmt * allow print source on exception * fix negate test * testing more templates * mostly works * add filter_statement * allow func to access ctx * add jinja-value.cpp * impl global_from_json * a lot of fixes * more tests * more fix, more tests * more fixes * rm workarounds * demo: type inferrence * add placeholder for tojson * improve function args handling * rm type inference * no more std::regex * trailing spaces * make testing more flexible * make output a bit cleaner * (wip) redirect minja calls * test: add --output * fix crash on macro kwargs * add minimal caps system * add some workarounds * rm caps_apply_workarounds * get rid of preprocessing * more fixes * fix test-chat-template * move test-chat-jinja into test-chat-template * rm test-chat-jinja from cmake * test-chat-template: use common * fix build * fix build (2) * rename vm --> interpreter * improve error reporting * correct lstrip behavior * add tojson * more fixes * disable tests for COMMON_CHAT_FORMAT_GENERIC * make sure tojson output correct order * add object.length * fully functional selectattr / rejectattr * improve error reporting * more builtins added, more fixes * create jinja rendering tests * fix testing.h path * adjust whitespace rules * more fixes * temporary disable test for ibm-granite * r/lstrip behavior matched with hf.js * minimax, glm4.5 ok * add append and pop * kimi-k2 ok * test-chat passed * fix lstrip_block * add more jinja tests * cast to unsigned char * allow dict key to be numeric * nemotron: rm windows newline * tests ok * fix test * rename interpreter --> runtime * fix build * add more checks * bring back generic format support * fix Apertus * [json.exception.out_of_range.403] key 'content' not found * rm generic test * refactor input marking * add docs * fix windows build * clarify error message * improved tests * split/rsplit with maxsplit * non-inverse maxsplit forgot to change after simplifying * implement separators for tojson and fix indent * i like to move it move it * rename null -- > none * token::eof * some nits + comments * add exception classes for lexer and parser * null -> none * rename global -> env * rm minja * update docs * docs: add input marking caveats * imlement missing jinja-tests functions * oops * support trim filter with args, remove bogus to_json reference * numerous argument fixes * updated tests * implement optional strip chars parameter * use new chars parameter * float filter also has default * always leave at least one decimal in float string * jinja : static analysis + header cleanup + minor fixes * add fuzz test * add string.cpp * fix chat_template_kwargs * nits * fix build * revert * unrevert sorry :) * add fuzz func_args, refactor to be safer * fix array.map() * loosen ensure_vals max count condition, add not impl for map(int) * hopefully fix windows * check if empty first * normalize newlines --------- Co-authored-by: Alde Rojas <hello@alde.dev> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2026-01-16 11:22:06 +01:00
Xuan-Son NguyenandGitHub	a04c2b06a3	server: improve slots scheduling for n_cmpl (#18789) ... * server : make sure children tasks are scheduled to launch with parent * fix * add comment pointing to this PR * fix * clean up * more debug messages * add pop_deferred_task with specific ID version * improve the logic * simple approach * no double move * correct return type of launch_slots_with_parent_task	2026-01-15 17:10:28 +01:00
3 people	a7e6ddb8bd	lora: make sure model keep track of associated adapters (#18490) ... * lora: make sure model keep track of associated adapters * deprecate llama_adapter_lora_free * minor : std::unordered_set over std::set --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2026-01-15 10:24:28 +01:00
Xuan-Son NguyenandGitHub	e047f9ee9d	mtmd: fix use_non_causal being reported incorrectly (#18793) ... * mtmd: fix use_non_causal being reported incorrectly * move clip_is_mrope to mtmd_decode_use_mrope * fix sloppy code ggml_cpy	2026-01-13 12:19:38 +01:00
Xuan-Son NguyenandGitHub	ce3bf9b1a4	server: update docs for sleeping [no ci] (#18777)	2026-01-12 13:01:24 +01:00
Xuan-Son NguyenandGitHub	0c3b7a9efe	model: fix qwen3next broken due to #18683 (#18762)	2026-01-11 21:00:10 +01:00
Xuan-Son NguyenandGitHub	4b060bf240	security: make it clear about subtopics in server (#18754) ... * security: make it clear about subtopics in server * exclude DoS	2026-01-11 16:51:03 +01:00
Xuan-Son NguyenandGitHub	506bb6e010	model: try to improve Qwen3 Next (#18683) ... * qwen3next: simplify qkvz projection * use ggml_swiglu_split * revert swiglu_split, but remove redundant repeat() * fix missing reshape * rm 2 redundant transposes * move mul_mat(k,q) to outside of chunking * rm redundant cont * improve g_cs_chunk * add comments about no cont * use std::pair instead of ggml_concat * vectorize key_gdiff calculation * rm unused tensor * avoid ggml_concat inside loop * bring back ggml_concat as it may not work on other backend * nits	2026-01-11 12:53:33 +01:00
Xuan-Son NguyenandGitHub	28068af789	security: narrow down the scope of what we consider a vulnerability (#18752) ... * security: narrow down the scope of what we consider a vulnerability * fix typo	2026-01-11 12:23:36 +01:00
Xuan-Son NguyenandGitHub	23f82f2420	preset: allow named remote preset (#18728) ... * preset: allow named remote preset * nits: fix docs * cont docs	2026-01-10 15:12:29 +01:00
3 people	9ac2693a30	server: fix n_cmpl not skipping processing prompt (#18663) ... * server: fix n_cmpl not skipping processing * fix infinite loop on empty batch * cont : init child samplers + modify child logic * cont : cleanup * cont : improve n_cmpl logic - launch the parent task first so it finds the slot with best cache - parent task waits for child tasks to be launched - when a child task finishes - remove its cache * cont : remove redundant function * cont : reduce parent checks * fix : nullptr task dereference --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2026-01-10 00:00:41 +01:00
Xuan-Son NguyenandGitHub	8ece3836b4	common: support remote preset (#18520) ... * arg: support remote preset * proof reading * allow one HF repo to point to multiple HF repos * docs: mention about multiple GGUF use case * correct clean_file_name * download: also return HTTP status code * fix case with cache file used * fix --offline option	2026-01-08 22:35:40 +01:00
Xuan-Son NguyenandGitHub	07fbe19f1f	arg: use CSV escape style for multiple-value args (#18643) ... * arg: use CSV escape style for multiple-value args * add test	2026-01-06 17:51:08 +01:00
Xuan-Son NguyenandGitHub	cd78e57c3a	lora: count lora nodes in graph_max_nodes (#18469) ... * lora: count lora nodes in graph_max_nodes * 3 nodes per weight * 4 nodes * keep track n_lora_nodes from llama_model * fix assert * rm redundant header * common: load adapters before context creation * use 6 nodes	2025-12-30 15:53:12 +01:00
Xuan-Son NguyenandGitHub	51a48720b8	webui: fix prompt progress ETA calculation (#18468) ... * webui: fix prompt progress ETA calculation * handle case done === 0	2025-12-29 21:42:11 +01:00
3 people	3595ae5963	contributing: tighten AI usage policy (#18388) ... * contributing: tighten AI usage policy * refactor AGENTS.md * proofreading * update contributing * add claude.md * add trailing newline * add note about dishonest practices * rm point about dishonest * rm requirement watermarking * add .gemini/settings.json * allow initially AI-generated content * revise * Update CONTRIBUTING.md Co-authored-by: Johannes Gäßler <johannesg@5d6.de> * improve * trailing space * Apply suggestions from code review Co-authored-by: Johannes Gäßler <johannesg@5d6.de> * update --------- Co-authored-by: Johannes Gäßler <johannesg@5d6.de>	2025-12-29 16:01:32 +01:00
Xuan-Son NguyenandGitHub	cffa5c46ea	mtmd: clarify that we no longer accept AI-generated PRs (#18406)	2025-12-28 09:57:04 +01:00
Xuan-Son NguyenandGitHub	f5acfb2ffa	server: (router) add stop-timeout option (#18350) ... * server: (router) add stop-timeout option * also allow stop while loading * add docs * unload_lru: also wait for unload to complete	2025-12-24 23:47:49 +01:00
4 people	4cbafad4f0	model: support MiMo-V2-Flash (#18328) ... * mimov2: convert ok * rename mimov2 --> mimo2 * fix conversion * runnable not incorrect * use sink * add_sliding_window_pattern * add swa and per-layer n_head_kv * correct params * somewhat working * correct gating func * nits * mimo2: wire RMS eps + MoE bias + converter guards * add co-author Co-authored-by: Aaryan-Kapoor <Aaryan-Kapoor@users.noreply.github.com> * use add_rope_freq_base_swa --------- Co-authored-by: Aaryan Kapoor <aaryankapoor2006@gmail.com> Co-authored-by: Aaryan-Kapoor <Aaryan-Kapoor@users.noreply.github.com>	2025-12-24 23:07:08 +01:00
Xuan-Son NguyenandGitHub	5ee4e43f26	server: return_progress to also report 0% processing state (#18305)	2025-12-23 21:49:05 +01:00
Xuan-Son NguyenandGitHub	849d021104	server: fix crash with model not having BOS/EOS (#18321)	2025-12-23 14:39:36 +01:00
Xuan-Son NguyenandGitHub	179fd82a72	gen-docs: automatically update markdown file (#18294) ... * gen-docs: automatically update markdown file * also strip whitespace * do not add extra newline * update TOC	2025-12-22 19:30:19 +01:00
Xuan-Son NguyenandGitHub	6ce863c803	server: prevent data race from HTTP threads (#18263) ... * server: prevent data race from HTTP threads * fix params * fix default_generation_settings * nits: make handle_completions_impl looks less strange * stricter const * fix GGML_ASSERT(idx < states.size()) * move index to be managed by server_response_reader * http: make sure req & res lifecycle are tied together * fix compile * fix index handling buggy * fix data race for lora endpoint * nits: fix shadow variable * nits: revert redundant changes * nits: correct naming for json_webui_settings	2025-12-22 14:23:34 +01:00
Xuan-Son NguyenandGitHub	3997c78e33	server: fix data race in to_json_anthropic (#18283)	2025-12-22 13:21:43 +01:00
Xuan-Son NguyenandGitHub	86af848153	server: (docs) remove mention about extra_args (#18262)	2025-12-22 12:22:01 +01:00
Xuan-Son NguyenandGitHub	5e25ddebff	move copilot instructions to AGENTS.md (#18259) ... * move copilot --> agents.md * agents: add disclose AI usage * refine	2025-12-21 19:09:21 +01:00
Xuan-Son NguyenandGitHub	ddcb75dd8a	server: add auto-sleep after N seconds of idle (#18228) ... * implement sleeping at queue level * implement server-context suspend * add test * add docs * optimization: add fast path * make sure to free llama_init * nits * fix use-after-free * allow /models to be accessed during sleeping, fix use-after-free * don't allow accessing /models during sleep, it is not thread-safe * fix data race on accessing props and model_meta * small clean up * trailing whitespace * rm outdated comments	2025-12-21 02:24:42 +01:00
3 people	9e39a1e6a9	server: support load model on startup, support preset-only options (#18206) ... * server: support autoload model, support preset-only options * add docs * load-on-startup * fix * Update common/arg.cpp Co-authored-by: Pascal <admin@serveurperso.com> --------- Co-authored-by: Pascal <admin@serveurperso.com>	2025-12-20 09:25:27 +01:00
Xuan-Son NguyenandGitHub	98c1c7a7bf	presets: refactor, allow cascade presets from different sources, add global section (#18169) ... * presets: refactor, allow cascade presets from different sources * update docs * fix neg arg handling * fix empty mmproj * also filter out server-controlled args before to_ini() * skip loading custom_models if not specified * fix unset_reserved_args * fix crash on windows	2025-12-19 12:08:20 +01:00
3 people	8ea958d4d9	model : add ASR support for LFM2-Audio-1.5B (conformer) (#18106) ... * ASR with LFM2-Audio-1.5B * Set rope_theta * Fix comment * Remove rope_theta setting * Address PR feedback * rename functions to conformer * remove some redundant ggml_cont * fix missing tensor * add prefix "a." for conv tensors * remove redundant reshape * clean up * add test model --------- Co-authored-by: Tarek Dakhran <tarek@liquid.ai>	2025-12-19 00:18:01 +01:00
Xuan-Son NguyenandGitHub	4d1316c440	arg: fix ASAN error on sampler_type_names empty (#18167)	2025-12-18 14:30:32 +01:00
Xuan-Son NguyenandGitHub	e85e9d7637	server: (router) disable SSL on child process (#18141)	2025-12-17 21:39:08 +01:00
Xuan-Son NguyenandGitHub	bde461de8c	server: (router) allow child process to report status via stdout (#18110) ... * server: (router) allow child process to report status via stdout * apply suggestions	2025-12-17 14:54:11 +01:00
Xuan-Son NguyenandGitHub	ef83fb8601	model: fix LFM2 missing tensors (#18105)	2025-12-16 19:07:43 +01:00
Xuan-Son NguyenandGitHub	7f2b2f3c77	arch: refactor LLM_TENSOR_NAMES (#18051) ... * arch: refactor LLM_TENSOR_NAMES * update docs * typo * fix LLM_ARCH_NEMOTRON_H_MOE * show more meaningful error message on missing tensor * fix and tested LLM_ARCH_NEMOTRON_H_MOE	2025-12-16 13:22:30 +01:00
Xuan-Son NguyenandGitHub	7b1db3d3b7	arg: clarify auto kvu/np being set on server (#17997) ... * arg: clarify auto kvu/np being set on server * improve docs * use invalid_argument	2025-12-16 12:01:27 +01:00
3 people	3d86c6c2b5	model: support GLM4V vision encoder (#18042) ... * convert ok * no deepstack * less new tensors * cgraph ok * add mrope for text model * faster patch merger * add GGML_ROPE_TYPE_MRNORM * add support for metal * move glm4v do dedicated graph * convert: add norm_embd * clip: add debugging fn * working correctly * fix style * use bicubic * fix mrope metal * improve cpu * convert to neox ordering on conversion * revert backend changes * force stop if using old weight * support moe variant * fix conversion * fix convert (2) * Update tools/mtmd/clip-graph.h Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * process mrope_section on TextModel base class * resolve conflict merge --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2025-12-16 11:25:26 +01:00
3 people	96a181a933	mtmd: refactor audio preprocessing (#17978) ... * mtmd: refactor audio preprocessing * refactor Co-authored-by: Tarek <tdakhran@users.noreply.github.com> * wip * wip (2) * improve constructor * fix use_natural_log * fix padding for short input * clean up * remove need_chunking --------- Co-authored-by: Tarek <tdakhran@users.noreply.github.com>	2025-12-15 14:16:52 +01:00
Xuan-Son NguyenandGitHub	52392291b2	preset: handle negated arg, reverse the meaning if needed (#18041)	2025-12-14 22:08:10 +01:00
Xuan-Son NguyenandGitHub	0759b09c90	graph: add f_attn_temp_offset (#18025)	2025-12-14 13:05:59 +01:00
Xuan-Son NguyenandGitHub	c00ff929dc	scripts: add script to compare logprobs of llama.cpp against other frameworks (#17947) ... * scripts: add script to compare logits of llama.cpp against other frameworks * accept custom prompt file * fix code style * clarify endpoint * fix displaying * use abs for diff * fix vllm case * rm output file * rename to compare-logprobs * add "pattern"	2025-12-13 22:33:29 +01:00
Xuan-Son NguyenandGitHub	4d5ae24c0a	arg: fix common_params_parse not accepting negated arg (#17991)	2025-12-13 12:53:37 +01:00
3 people	380b4c984e	common: support negated args (#17919) ... * args: support negated args * update docs * fix typo * add more neg options * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * rm duplicated arg * fix LLAMA_ARG_NO_HOST * add test --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>	2025-12-12 23:58:53 +01:00
Xuan-Son NguyenandGitHub	e39a2ce66d	clip: move model cgraphs into their own files (#17965) ... * clip: move model cgraphs into their own files * more explicit enums * fix linux build * fix naming * missing headers * nits: add comments for contributors	2025-12-12 21:14:48 +01:00
Xuan-Son NguyenandGitHub	17158965ac	mtmd: explicitly forbidden inclusion of private header and libcommon (#17946)	2025-12-12 15:16:06 +01:00
Xuan-Son NguyenandGitHub	54a0fee4b7	arg: add -mm and -mmu as short form of --mmproj and --mmproj-url (#17958) ... * arg: add -mm and -mmu as short form of --mmproj and --mmproj-url * correct order * update docs	2025-12-12 14:06:06 +01:00
Xuan-Son NguyenandGitHub	c6b2c9310c	mtmd: some small clean up (#17909) ... * clip: add support for fused qkv in build_vit * use bulid_ffn whenever possible * fix internvl * mtmd-cli: move image to beginning * test script: support custom args	2025-12-10 22:20:06 +01:00
3 people	34a6d86982	cli: enable jinja by default (#17911) ... * cli: enable jinja by default * Update common/arg.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>	2025-12-10 22:19:42 +01:00
3 people	6c2131773c	cli: new CLI experience (#17824) ... * wip * wip * fix logging, add display info * handle commands * add args * wip * move old cli to llama-completion * rm deprecation notice * move server to a shared library * move ci to llama-completion * add loading animation * add --show-timings arg * add /read command, improve LOG_ERR * add args for speculative decoding, enable show timings by default * add arg --image and --audio * fix windows build * support reasoning_content * fix llama2c workflow * color default is auto * fix merge conflicts * properly fix color problem Co-authored-by: bandoti <bandoti@users.noreply.github.com> * better loading spinner * make sure to clean color on force-exit * also clear input files on "/clear" * simplify common_log_flush * add warning in mtmd-cli * implement console writter * fix data race * add attribute * fix llama-completion and mtmd-cli * add some notes about console::log * fix compilation --------- Co-authored-by: bandoti <bandoti@users.noreply.github.com>	2025-12-10 15:28:59 +01:00
3 people	9e79b0116e	convert: allow using quantized Mistral weight (#17889) ... * convert: allow using quantized Mistral weight * data_torch.ndim * update dequant fn Co-authored-by: compilade <compilade@users.noreply.github.com> --------- Co-authored-by: compilade <compilade@users.noreply.github.com>	2025-12-10 10:26:22 +01:00
3 people	4e842d5120	console: allow using arrow left/right, home/end keys and history mode (#17836) ... * console: allow using arrow left/right to edit the line (with UTF-8 support) * console: fix arrow keys on Windows using private-use Unicode * console: add Home/End key support for Windows and Linux * console: add basic Up/Down history navigation * fix build * console: allow using arrow left/right to edit the line (with UTF-8 support) * console: fix arrow keys on Windows using private-use Unicode * console: add Home/End key support for Windows and Linux * console: add basic Up/Down history navigation * console: remove unreachable wc == 0 check after VK switch * console: add Ctrl+Left/Right word navigation - Add KEY_CTRL_ARROW_LEFT and KEY_CTRL_ARROW_RIGHT codes - Windows: detect CTRL modifier via dwControlKeyState - Linux: parse ANSI sequences with modifier (1;5D/C) - Implement move_word_left/right with space-skipping logic - Refactor escape sequence parsing to accumulate params * console: add Delete key support - Windows: VK_DELETE detection - Linux: ESC[3~ sequence parsing - Forward character deletion with UTF-8 support * console: implement bash-style history editing - Edit any history line during UP/DOWN navigation, edits persist - Pressing Enter appends edited version as new history entry - Original line stay untouched in their positions * clean up * better history impl * fix decode_utf8 --------- Co-authored-by: Pascal <admin@serveurperso.com>	2025-12-09 11:53:59 +01:00
Xuan-Son NguyenandGitHub	951520ddb0	server: delegate result_state creation to server_task (#17835) ... * server: delegate result_state creation to server_task * remove unued states * add more docs	2025-12-08 17:04:38 +01:00
3 people	f896d2c34f	server: improve speed of speculative decoding (#17808) ... * server: improve speed of speculative decoding * fix small draft case * add link to the PR * server : fix generation time measurement * server : fix draft acceptance logs (add SRV_CNT, SLT_CNT macros) * server : add comment * add PR to docs --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2025-12-08 14:35:28 +01:00
Xuan-Son NguyenandGitHub	37a4f63244	server : add development documentation (#17760) ... * first draft * rewrite * update & remove duplicated sections	2025-12-08 13:54:58 +01:00
Xuan-Son NguyenandGitHub	4d3726278b	model: add llama 4 scaling for mistral-large (deepseek arch) (#17744)	2025-12-07 22:29:54 +01:00
Xuan-Son NguyenandGitHub	c42712b056	server: support multiple generations from one prompt (OAI "n" option) (#17775) ... * backend support * server: support multiple generations from one prompt (OAI "n" option) * fix invalid batch * format oai * clean up * disable ctx shift * add test * update comments * fix style * add n_cmpl to docs [no ci] * allowing using both n_cmpl and n	2025-12-06 15:54:38 +01:00
3 people	dbc15a7967	convert: support Mistral 3 Large MoE (#17730) ... * convert: support Mistral 3 Large MoE * filter out vision tensors, add missing keys * handle vocab * add temperature_length * fix mscale_all_dim * clean up * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * fix * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>	2025-12-06 10:49:33 +01:00
Xuan-Son NguyenandGitHub	9d0229967a	server: strip content-length header on proxy (#17734)	2025-12-04 16:32:57 +01:00
3 people	c4c10bfb86	server: move msg diffs tracking to HTTP thread (#17740) ... * server: move msg diffs tracking to HTTP thread * wip * tool call tests ok * minor : style * cont : fix * move states to server_response_reader * add safe-guard * fix * fix 2 --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2025-12-04 15:46:08 +01:00
Xuan-Son NguyenandGitHub	13628d8bdb	server: add --media-path for local media files (#17697) ... * server: add --media-path for local media files * remove unused fn	2025-12-02 22:49:20 +01:00