nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 23 Packages Projects Releases Wiki Activity
8,869 Commits 1 Branch 0 Tags
9998d88bc88c2003aa6790d57226ef33608d57c1
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan-Son NguyenandGitHub 9998d88bc8 mtmd: correct mtmd_decode_use_mrope() (#22188) 2026-04-21 10:53:37 +02:00
Xuan-Son NguyenandGitHub 86f8daacfe mtmd: correct get_n_pos / get_decoder_pos (#22175) 2026-04-20 23:29:19 +02:00
Xuan-Son NguyenandGitHub a678916623 mtmd: refactor mtmd_decode_use_mrope (#22161) 2026-04-20 14:45:11 +02:00
Xuan-Son NguyenandGitHub 19124078be mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos (breaking change) (#22082) 
* mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos

* fix build
 2026-04-19 11:57:21 +02:00
Xuan-Son NguyenandGitHub 089dd41fe3 cmake: use glob to collect src/models sources (#22005) 2026-04-16 23:25:16 +02:00
Xuan-Son NguyenandGitHub 4fbdabdc61 model: using single llm_build per arch (#21970) 
* model: using single llm_build per arch

* fix merge

* nits
 2026-04-16 21:10:22 +02:00
Xuan-Son NguyenandGitHub 408225bb1a server: use random media marker (#21962) 
* server: use random media marker

* nits

* remove legacy <__image__> token

* revert special char in random
 2026-04-15 23:52:22 +02:00
Xuan-Son NguyenandGitHub 80d8770804 docs: more extensive RoPE documentation [no ci] (#21953) 
* more extensive ggml_rope documentation

* add more docs

* nits
 2026-04-15 14:45:16 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		fae3a28070 ggml : remove ggml-ext.h (#21869) 
* ggml: correct placement of ggml-ext.h

* ggml : remove ggml-ext.h

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-04-14 17:32:58 +03:00
Xuan-Son NguyenandGitHub 707c0b7a6e mtmd: add mtmd_image_tokens_get_decoder_pos() API (#21851) 
* mtmd: add mtmd_image_tokens_get_decoder_pos() API

* consistent naming

* fix build
 2026-04-14 16:07:41 +02:00
Xuan-Son NguyenandGitHub e489a5ca0e server: support OAI /v1/audio/transcriptions API (#21863) 
* server: support OAI /v1/audio/transcriptions API

* address autoreview comments

* correct default response_format value
 2026-04-14 11:09:52 +02:00
Xuan-Son NguyenandGitHub e974923698 docs: listing qwen3-asr and qwen3-omni as supported (#21857) 
* docs: listing qwen3-asr and qwen3-omni as supported

* nits
 2026-04-13 22:28:17 +02:00
Xuan-Son NguyenandGitHub 920b3e78cb mtmd: use causal attn for gemma 4 audio (#21824) 2026-04-13 09:47:55 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		21a4933042 mtmd: qwen3 audio support (qwen3-omni and qwen3-asr) (#19441) 
* add qwen3a

* wip

* vision ok

* no more deepstack for audio

* convert ASR model ok

* qwen3 asr working

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* nits

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix bad merge

* fix multi inheritance

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-04-12 23:57:25 +02:00
Xuan-Son NguyenandGitHub aa4695c5e5 mtmd: add gemma 4 test (vision + audio) [no ci] (#21806) 
* mtmd: add gemma 4 test (vision + audio)

* add to docs
 2026-04-12 16:29:03 +02:00
Xuan-Son NguyenandGitHub 6313acbef0 docs: add guide on how to add multimodal support (#21778) 
* docs: add guide on how to add multimodal support

* nits
 2026-04-12 13:02:38 +02:00
Xuan-Son NguyenandGitHub 057dba336e model: fix multimodal padding token for gemma3n/gemma4 (#21625) 
* model: fix multimodal padding token for gemma3n/gemma4

* nits
 2026-04-09 12:18:23 +02:00
Xuan-Son NguyenandGitHub 501aeed18f mtmd: support dots.ocr (#17575) 
* convert gguf

* clip impl

* fix conversion

* wip

* corrections

* update docs

* add gguf to test script
 2026-04-09 12:16:38 +02:00
Xuan-Son NguyenandGitHub 3979f2bb08 docs: add hunyuan-ocr gguf, also add test [no ci] (#21490) 2026-04-06 14:02:37 +02:00
Xuan-Son NguyenandGitHub 63f8fe0ef4 model, mtmd: fix gguf conversion for audio/vision mmproj (#21309) 
* fix gguf conversion for audio/vision mmproj

* fix test
 2026-04-02 17:10:32 +02:00
Xuan-Son NguyenandGitHub c30e012253 contrib : rewrite AGENTS.md, make it more clear about project values (#21270) 
* contrib : rewrite AGENTS.md, make it more clear about types of permitted AI usage

* permit AI for writing code
 2026-04-01 23:31:51 +02:00
Xuan-Son NguyenandGitHub 0356e33aaf scripts: add function call test script (#21234) 
* scripts: add function call test script

* add reasoning_content

* fix lint
 2026-04-01 15:31:58 +02:00
Xuan-Son NguyenGitHubAleksander Grygier
 		4a00bbfed6 server: (webui) no more gzip compression (#21073) 
* webui: no more gzip

* try changing a small line

* Revert "try changing a small line"

This reverts commit 0d7a3531593d87b724d404c8727a96becab3ab07.

* fix lint

* fix test

* rebuild

* split into html/css/js

* lint

* chore: update webui build output

* chore: Update git hooks script

* server: update webui build output

* chore: Update pre-commit hook

* refactor: Cleanup

---------

Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
 2026-03-31 15:44:26 +02:00
Xuan-Son NguyenGitHubGeorgi GerganovAleksander Grygier
 		abf9a62161 server: wrap headers for mcp proxy (#21072) 
* server: wrap headers for mcp proxy

* Update tools/server/server-cors-proxy.h

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* fix build

* chore: update webui build output

* chore: update webui build output

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
 2026-03-30 08:59:16 +02:00
Xuan-Son NguyenandGitHub 871f1a2d2f mtmd: add more sanity checks (#21047) 2026-03-27 11:00:52 +01:00
Xuan-Son NguyenandGitHub 20197b6fe3 server: add built-in tools backend support (#20898) 
* wip: server_tools

* refactor

* displayName -> display_name

* snake_case everywhere

* rm redundant field

* change arg to --tools all

* add readme mention

* llama-gen-docs
 2026-03-27 10:07:11 +01:00
Xuan-Son NguyenandGitHub 8c60b8a2be ci: pin external actions to exact commit SHA (#21033) 2026-03-26 20:44:00 +01:00
Xuan-Son NguyenandGitHub a73bbd5d92 mtmd: refactor image preprocessing (#21031) 
* mtmd: refactor image pre-processing

* correct some places

* correct lfm2

* fix deepseek-ocr on server

* add comment to clarify about mtmd_image_preprocessor_dyn_size
 2026-03-26 19:49:20 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		914eb5ff0c jinja: fix macro with kwargs (#20960) 
* jinja: fix macro with kwargs

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix newline problem

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-03-25 12:22:48 +01:00
Xuan-Son NguyenGitHubPiotr Wilkin
 		bd6992180b contrib: add "Requirements" section to PR template (#20841) 
* contrib: add "Requirements" section to PR template

* typo [no ci]

* use h2, add "Additional information"

---------

Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>
 2026-03-23 16:59:02 +01:00
Xuan-Son NguyenandGitHub 31a5cf4c3f server: use httplib dynamic threads (#20817) 
* server: use httplib dynamic threads

* change to n_threads_http + 1024
 2026-03-23 12:22:46 +01:00
Xuan-Son NguyenandGitHub 49bfddeca1 server: allow router to report child instances sleep status (#20849) 
* server: allow router to report child instances sleep status

* refactor

* move sleeping to state

* nits
 2026-03-22 18:33:52 +01:00
Xuan-Son NguyenandGitHub 568aec82d2 docs : explicit about banning accounts that violates policy (#19593) 2026-03-21 15:50:16 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		fb78ad29bb server: (doc) clarify in-scope and out-scope features (#20794) 
* server: (doc) clarify in-scope and out-scope features

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-03-20 14:03:50 +01:00
Xuan-Son NguyenandGitHub 1e64534570 mtmd: add clip_graph::build_mm() (#20751) 
* clip: add build_mm()

* apply to all models

* add TODO for bias overload
 2026-03-19 13:11:39 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		d34ff7eb5b model: mistral small 4 support (#20649) 
* model: mistral small 4 support

* fix test

* fix test (2)

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* change newline

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-03-17 00:31:14 +01:00
Xuan-Son NguyenandGitHub 94d0262277 mtmd: add llama-mtmd-debug binary (#20508) 
* mtmd: add llama-mtmd-debug binary

* adapt

* fixes

* fix compile error

* fix windows compile error

* rm legacy clip_debug_encode()

* add MTMD_API to fix build
 2026-03-14 15:52:29 +01:00
Xuan-Son NguyenandGitHub 0842b9b465 model: fix step3.5 n_rot (#20318) 2026-03-09 23:42:24 +01:00
Xuan-Son NguyenandGitHub 59db9a357d llama: dynamic head_dim and n_rot for SWA (#20301) 
* llama: dynamic head_dim and n_rot for SWA

* also add gguf_writer wrappers

* fix build

* build_rope_shift arg reorder
 2026-03-09 22:22:39 +01:00
Xuan-Son NguyenandGitHub c747294b2d scripts: update corpus of compare-logprobs (#19326) 
* scripts: update corpus of compare-logprobs

* fix
 2026-02-25 12:57:34 +01:00
Xuan-Son NguyenandGitHub 5452d736f8 jinja: correct stats for tojson and string filters (#19785) 2026-02-22 21:08:23 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		eeef3cfced model: support GLM-OCR (#19677) 
* model: support GLM-OCR

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-02-18 17:51:40 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		752584d5f5 model: support GLM MoE DSA arch (NOTE: indexer is not yet supported) (#19460) 
* model: support GLM MoE DSA arch

* working version

* pyright

* keep indexer tensors

* add indexer gguf params

* loaded now

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* update

* Update src/llama-model.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* minor fix and cleanup

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-02-13 14:56:53 +01:00
Xuan-Son NguyenandGitHub 9a96352729 test: fix IMROPE perf test case (#19465) 2026-02-10 14:37:50 +01:00
Xuan-Son NguyenandGitHub 98e57ca422 chat: fix case where template accepts type content only (#19419) 
* chat: fix case where template accepts type content only

* rm stray log

* reuse render_message_to_json
 2026-02-09 22:14:12 +01:00
Xuan-Son NguyenandGitHub e0c93af2a0 debug: make common_debug_print_tensor readable (#19331) 
* debug: make common_debug_print_tensor readable

* editorconfig
 2026-02-04 17:55:31 +01:00
Xuan-Son NguyenandGitHub 8abcc70a74 model: (qwen3next) correct vectorized key_gdiff calculation (#19324) 
* model: (qwen3next) correct vectorized key_gdiff calculation

* move transpose to outside of loop
 2026-02-04 13:09:58 +01:00
Xuan-Son NguyenandGitHub 07a7412a3b mtmd: add min/max pixels gguf metadata (#19273) 2026-02-02 20:59:06 +01:00
Xuan-Son NguyenGitHubAldehir Rojas
 		bb02f74c61 chat: fix language input for translategemma (#19052) 
* chat: fix language input for translategemma

* Update common/chat.cpp

Co-authored-by: Aldehir Rojas <hello@alde.dev>

---------

Co-authored-by: Aldehir Rojas <hello@alde.dev>
 2026-01-24 17:58:45 +01:00
Xuan-Son NguyenandGitHub b5b8fa1c8b chat : fix translategemma crash on common_chat_format_example (#19019) 2026-01-23 12:03:42 +01:00
Xuan-Son NguyenandGitHub 51fa458a92 server : support preserving reasoning_content in assistant message (#18994) 
* support reasoning_content input

* report template caps to webui

* add docs

* rm commented code
 2026-01-22 21:30:06 +01:00
Xuan-Son NguyenandGitHub 4e595b250a server: do not log certain endpoints (avoid log spam) (#19028) 2026-01-22 19:24:37 +01:00
Xuan-Son NguyenandGitHub 9eb5bfec1a mtmd : update docs to use llama_model_n_embd_inp (#18999) 2026-01-22 14:36:32 +01:00
Xuan-Son NguyenandGitHub 2c1f199653 cli : fix reasoning responses in CLI (#18961) 
* cli : fix reasoning responses in CLI

* fix build

* fix build (2)
 2026-01-20 18:23:25 +01:00
Xuan-Son NguyenandGitHub 6df686bee6 server : refactor oai_parser_opt, move it to server_chat_params (#18937) 
* server_chat_params

* move chat format into CLI

* use meta whenever possible

* clean up, no more chatml fallback
 2026-01-19 23:28:01 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		fe44d35574 tests : add test-jinja -py option for cross-checking (#18906) 
* tests : add test-jinja -py option or cross-checking

* Update tests/test-jinja.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fix + add source

* SandboxedEnvironment

* fix array.map case

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2026-01-18 08:14:27 +01:00
Xuan-Son NguyenandGitHub f55b033ae6 jinja: correct member access rule (#18905) 2026-01-18 00:48:55 +01:00
Xuan-Son NguyenGitHubAlde RojasSigbjørn SkjæretGeorgi Gerganov
 		c15395f73c common : implement new jinja template engine (#18462) 
* jinja vm

* lexer

* add vm types

* demo

* clean up

* parser ok

* binary_expression::execute

* shadow naming

* bin ops works!

* fix map object

* add string builtins

* add more builtins

* wip

* use mk_val

* eval with is_user_input

* render gemma tmpl ok

* track input string even after transformations

* support binded functions

* keyword arguments and slicing array

* use shared_ptr for values

* add mk_stmt

* allow print source on exception

* fix negate test

* testing more templates

* mostly works

* add filter_statement

* allow func to access ctx

* add jinja-value.cpp

* impl global_from_json

* a lot of fixes

* more tests

* more fix, more tests

* more fixes

* rm workarounds

* demo: type inferrence

* add placeholder for tojson

* improve function args handling

* rm type inference

* no more std::regex

* trailing spaces

* make testing more flexible

* make output a bit cleaner

* (wip) redirect minja calls

* test: add --output

* fix crash on macro kwargs

* add minimal caps system

* add some workarounds

* rm caps_apply_workarounds

* get rid of preprocessing

* more fixes

* fix test-chat-template

* move test-chat-jinja into test-chat-template

* rm test-chat-jinja from cmake

* test-chat-template: use common

* fix build

* fix build (2)

* rename vm --> interpreter

* improve error reporting

* correct lstrip behavior

* add tojson

* more fixes

* disable tests for COMMON_CHAT_FORMAT_GENERIC

* make sure tojson output correct order

* add object.length

* fully functional selectattr / rejectattr

* improve error reporting

* more builtins added, more fixes

* create jinja rendering tests

* fix testing.h path

* adjust whitespace rules

* more fixes

* temporary disable test for ibm-granite

* r/lstrip behavior matched with hf.js

* minimax, glm4.5 ok

* add append and pop

* kimi-k2 ok

* test-chat passed

* fix lstrip_block

* add more jinja tests

* cast to unsigned char

* allow dict key to be numeric

* nemotron: rm windows newline

* tests ok

* fix test

* rename interpreter --> runtime

* fix build

* add more checks

* bring back generic format support

* fix Apertus

* [json.exception.out_of_range.403] key 'content' not found

* rm generic test

* refactor input marking

* add docs

* fix windows build

* clarify error message

* improved tests

* split/rsplit with maxsplit

* non-inverse maxsplit

forgot to change after simplifying

* implement separators for tojson and fix indent

* i like to move it move it

* rename null -- > none

* token::eof

* some nits + comments

* add exception classes for lexer and parser

* null -> none

* rename global -> env

* rm minja

* update docs

* docs: add input marking caveats

* imlement missing jinja-tests functions

* oops

* support trim filter with args, remove bogus to_json reference

* numerous argument fixes

* updated tests

* implement optional strip chars parameter

* use new chars parameter

* float filter also has default

* always leave at least one decimal in float string

* jinja : static analysis + header cleanup + minor fixes

* add fuzz test

* add string.cpp

* fix chat_template_kwargs

* nits

* fix build

* revert

* unrevert

sorry :)

* add fuzz func_args, refactor to be safer

* fix array.map()

* loosen ensure_vals max count condition, add not impl for map(int)

* hopefully fix windows

* check if empty first

* normalize newlines

---------

Co-authored-by: Alde Rojas <hello@alde.dev>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-01-16 11:22:06 +01:00
Xuan-Son NguyenandGitHub a04c2b06a3 server: improve slots scheduling for n_cmpl (#18789) 
* server : make sure children tasks are scheduled to launch with parent

* fix

* add comment pointing to this PR

* fix

* clean up

* more debug messages

* add pop_deferred_task with specific ID version

* improve the logic

* simple approach

* no double move

* correct return type of launch_slots_with_parent_task
 2026-01-15 17:10:28 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		a7e6ddb8bd lora: make sure model keep track of associated adapters (#18490) 
* lora: make sure model keep track of associated adapters

* deprecate llama_adapter_lora_free

* minor : std::unordered_set over std::set

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-01-15 10:24:28 +01:00
Xuan-Son NguyenandGitHub e047f9ee9d mtmd: fix use_non_causal being reported incorrectly (#18793) 
* mtmd: fix use_non_causal being reported incorrectly

* move clip_is_mrope to mtmd_decode_use_mrope

* fix sloppy code ggml_cpy
 2026-01-13 12:19:38 +01:00
Xuan-Son NguyenandGitHub ce3bf9b1a4 server: update docs for sleeping [no ci] (#18777) 2026-01-12 13:01:24 +01:00
Xuan-Son NguyenandGitHub 0c3b7a9efe model: fix qwen3next broken due to #18683 (#18762) 2026-01-11 21:00:10 +01:00
Xuan-Son NguyenandGitHub 4b060bf240 security: make it clear about subtopics in server (#18754) 
* security: make it clear about subtopics in server

* exclude DoS
 2026-01-11 16:51:03 +01:00
Xuan-Son NguyenandGitHub 506bb6e010 model: try to improve Qwen3 Next (#18683) 
* qwen3next: simplify qkvz projection

* use ggml_swiglu_split

* revert swiglu_split, but remove redundant repeat()

* fix missing reshape

* rm 2 redundant transposes

* move mul_mat(k,q) to outside of chunking

* rm redundant cont

* improve g_cs_chunk

* add comments about no cont

* use std::pair instead of ggml_concat

* vectorize key_gdiff calculation

* rm unused tensor

* avoid ggml_concat inside loop

* bring back ggml_concat as it may not work on other backend

* nits
 2026-01-11 12:53:33 +01:00
Xuan-Son NguyenandGitHub 28068af789 security: narrow down the scope of what we consider a vulnerability (#18752) 
* security: narrow down the scope of what we consider a vulnerability

* fix typo
 2026-01-11 12:23:36 +01:00
Xuan-Son NguyenandGitHub 23f82f2420 preset: allow named remote preset (#18728) 
* preset: allow named remote preset

* nits: fix docs

* cont docs
 2026-01-10 15:12:29 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		9ac2693a30 server: fix n_cmpl not skipping processing prompt (#18663) 
* server: fix n_cmpl not skipping processing

* fix infinite loop on empty batch

* cont : init child samplers + modify child logic

* cont : cleanup

* cont : improve n_cmpl logic

- launch the parent task first so it finds the slot with best cache
- parent task waits for child tasks to be launched
- when a child task finishes - remove its cache

* cont : remove redundant function

* cont : reduce parent checks

* fix : nullptr task dereference

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-01-10 00:00:41 +01:00
Xuan-Son NguyenandGitHub 8ece3836b4 common: support remote preset (#18520) 
* arg: support remote preset

* proof reading

* allow one HF repo to point to multiple HF repos

* docs: mention about multiple GGUF use case

* correct clean_file_name

* download: also return HTTP status code

* fix case with cache file used

* fix --offline option
 2026-01-08 22:35:40 +01:00
Xuan-Son NguyenandGitHub 07fbe19f1f arg: use CSV escape style for multiple-value args (#18643) 
* arg: use CSV escape style for multiple-value args

* add test
 2026-01-06 17:51:08 +01:00
Xuan-Son NguyenandGitHub cd78e57c3a lora: count lora nodes in graph_max_nodes (#18469) 
* lora: count lora nodes in graph_max_nodes

* 3 nodes per weight

* 4 nodes

* keep track n_lora_nodes from llama_model

* fix assert

* rm redundant header

* common: load adapters before context creation

* use 6 nodes
 2025-12-30 15:53:12 +01:00
Xuan-Son NguyenandGitHub 51a48720b8 webui: fix prompt progress ETA calculation (#18468) 
* webui: fix prompt progress ETA calculation

* handle case done === 0
 2025-12-29 21:42:11 +01:00
Xuan-Son NguyenGitHubJohannes Gäßler
 		3595ae5963 contributing: tighten AI usage policy (#18388) 
* contributing: tighten AI usage policy

* refactor AGENTS.md

* proofreading

* update contributing

* add claude.md

* add trailing newline

* add note about dishonest practices

* rm point about dishonest

* rm requirement watermarking

* add .gemini/settings.json

* allow initially AI-generated content

* revise

* Update CONTRIBUTING.md

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* improve

* trailing space

* Apply suggestions from code review

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* update

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2025-12-29 16:01:32 +01:00
Xuan-Son NguyenandGitHub cffa5c46ea mtmd: clarify that we no longer accept AI-generated PRs (#18406) 2025-12-28 09:57:04 +01:00
Xuan-Son NguyenandGitHub f5acfb2ffa server: (router) add stop-timeout option (#18350) 
* server: (router) add stop-timeout option

* also allow stop while loading

* add docs

* unload_lru: also wait for unload to complete
 2025-12-24 23:47:49 +01:00
Xuan-Son NguyenGitHubAaryan-KapoorAaryan Kapoor
 		4cbafad4f0 model: support MiMo-V2-Flash (#18328) 
* mimov2: convert ok

* rename mimov2 --> mimo2

* fix conversion

* runnable not incorrect

* use sink

* add_sliding_window_pattern

* add swa and per-layer n_head_kv

* correct params

* somewhat working

* correct gating func

* nits

* mimo2: wire RMS eps + MoE bias + converter guards

* add co-author

Co-authored-by: Aaryan-Kapoor <Aaryan-Kapoor@users.noreply.github.com>

* use add_rope_freq_base_swa

---------

Co-authored-by: Aaryan Kapoor <aaryankapoor2006@gmail.com>
Co-authored-by: Aaryan-Kapoor <Aaryan-Kapoor@users.noreply.github.com>
 2025-12-24 23:07:08 +01:00
Xuan-Son NguyenandGitHub 5ee4e43f26 server: return_progress to also report 0% processing state (#18305) 2025-12-23 21:49:05 +01:00
Xuan-Son NguyenandGitHub 849d021104 server: fix crash with model not having BOS/EOS (#18321) 2025-12-23 14:39:36 +01:00
Xuan-Son NguyenandGitHub 179fd82a72 gen-docs: automatically update markdown file (#18294) 
* gen-docs: automatically update markdown file

* also strip whitespace

* do not add extra newline

* update TOC
 2025-12-22 19:30:19 +01:00
Xuan-Son NguyenandGitHub 6ce863c803 server: prevent data race from HTTP threads (#18263) 
* server: prevent data race from HTTP threads

* fix params

* fix default_generation_settings

* nits: make handle_completions_impl looks less strange

* stricter const

* fix GGML_ASSERT(idx < states.size())

* move index to be managed by server_response_reader

* http: make sure req & res lifecycle are tied together

* fix compile

* fix index handling buggy

* fix data race for lora endpoint

* nits: fix shadow variable

* nits: revert redundant changes

* nits: correct naming for json_webui_settings
 2025-12-22 14:23:34 +01:00
Xuan-Son NguyenandGitHub 3997c78e33 server: fix data race in to_json_anthropic (#18283) 2025-12-22 13:21:43 +01:00
Xuan-Son NguyenandGitHub 86af848153 server: (docs) remove mention about extra_args (#18262) 2025-12-22 12:22:01 +01:00
Xuan-Son NguyenandGitHub 5e25ddebff move copilot instructions to AGENTS.md (#18259) 
* move copilot --> agents.md

* agents: add disclose AI usage

* refine
 2025-12-21 19:09:21 +01:00
Xuan-Son NguyenandGitHub ddcb75dd8a server: add auto-sleep after N seconds of idle (#18228) 
* implement sleeping at queue level

* implement server-context suspend

* add test

* add docs

* optimization: add fast path

* make sure to free llama_init

* nits

* fix use-after-free

* allow /models to be accessed during sleeping, fix use-after-free

* don't allow accessing /models during sleep, it is not thread-safe

* fix data race on accessing props and model_meta

* small clean up

* trailing whitespace

* rm outdated comments
 2025-12-21 02:24:42 +01:00
Xuan-Son NguyenGitHubPascal
 		9e39a1e6a9 server: support load model on startup, support preset-only options (#18206) 
* server: support autoload model, support preset-only options

* add docs

* load-on-startup

* fix

* Update common/arg.cpp

Co-authored-by: Pascal <admin@serveurperso.com>

---------

Co-authored-by: Pascal <admin@serveurperso.com>
 2025-12-20 09:25:27 +01:00
Xuan-Son NguyenandGitHub 98c1c7a7bf presets: refactor, allow cascade presets from different sources, add global section (#18169) 
* presets: refactor, allow cascade presets from different sources

* update docs

* fix neg arg handling

* fix empty mmproj

* also filter out server-controlled args before to_ini()

* skip loading custom_models if not specified

* fix unset_reserved_args

* fix crash on windows
 2025-12-19 12:08:20 +01:00
Xuan-Son NguyenGitHubTarek Dakhran
 		8ea958d4d9 model : add ASR support for LFM2-Audio-1.5B (conformer) (#18106) 
* ASR with LFM2-Audio-1.5B

* Set rope_theta

* Fix comment

* Remove rope_theta setting

* Address PR feedback

* rename functions to conformer

* remove some redundant ggml_cont

* fix missing tensor

* add prefix "a." for conv tensors

* remove redundant reshape

* clean up

* add test model

---------

Co-authored-by: Tarek Dakhran <tarek@liquid.ai>
 2025-12-19 00:18:01 +01:00
Xuan-Son NguyenandGitHub 4d1316c440 arg: fix ASAN error on sampler_type_names empty (#18167) 2025-12-18 14:30:32 +01:00
Xuan-Son NguyenandGitHub e85e9d7637 server: (router) disable SSL on child process (#18141) 2025-12-17 21:39:08 +01:00
Xuan-Son NguyenandGitHub bde461de8c server: (router) allow child process to report status via stdout (#18110) 
* server: (router) allow child process to report status via stdout

* apply suggestions
 2025-12-17 14:54:11 +01:00
Xuan-Son NguyenandGitHub ef83fb8601 model: fix LFM2 missing tensors (#18105) 2025-12-16 19:07:43 +01:00
Xuan-Son NguyenandGitHub 7f2b2f3c77 arch: refactor LLM_TENSOR_NAMES (#18051) 
* arch: refactor LLM_TENSOR_NAMES

* update docs

* typo

* fix LLM_ARCH_NEMOTRON_H_MOE

* show more meaningful error message on missing tensor

* fix and tested LLM_ARCH_NEMOTRON_H_MOE
 2025-12-16 13:22:30 +01:00
Xuan-Son NguyenandGitHub 7b1db3d3b7 arg: clarify auto kvu/np being set on server (#17997) 
* arg: clarify auto kvu/np being set on server

* improve docs

* use invalid_argument
 2025-12-16 12:01:27 +01:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		3d86c6c2b5 model: support GLM4V vision encoder (#18042) 
* convert ok

* no deepstack

* less new tensors

* cgraph ok

* add mrope for text model

* faster patch merger

* add GGML_ROPE_TYPE_MRNORM

* add support for metal

* move glm4v do dedicated graph

* convert: add norm_embd

* clip: add debugging fn

* working correctly

* fix style

* use bicubic

* fix mrope metal

* improve cpu

* convert to neox ordering on conversion

* revert backend changes

* force stop if using old weight

* support moe variant

* fix conversion

* fix convert (2)

* Update tools/mtmd/clip-graph.h

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* process mrope_section on TextModel base class

* resolve conflict merge

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-12-16 11:25:26 +01:00
Xuan-Son NguyenGitHubTarek
 		96a181a933 mtmd: refactor audio preprocessing (#17978) 
* mtmd: refactor audio preprocessing

* refactor

Co-authored-by: Tarek <tdakhran@users.noreply.github.com>

* wip

* wip (2)

* improve constructor

* fix use_natural_log

* fix padding for short input

* clean up

* remove need_chunking

---------

Co-authored-by: Tarek <tdakhran@users.noreply.github.com>
 2025-12-15 14:16:52 +01:00
Xuan-Son NguyenandGitHub 52392291b2 preset: handle negated arg, reverse the meaning if needed (#18041) 2025-12-14 22:08:10 +01:00
Xuan-Son NguyenandGitHub 0759b09c90 graph: add f_attn_temp_offset (#18025) 2025-12-14 13:05:59 +01:00
Xuan-Son NguyenandGitHub c00ff929dc scripts: add script to compare logprobs of llama.cpp against other frameworks (#17947) 
* scripts: add script to compare logits of llama.cpp against other frameworks

* accept custom prompt file

* fix code style

* clarify endpoint

* fix displaying

* use abs for diff

* fix vllm case

* rm output file

* rename to compare-logprobs

* add "pattern"
 2025-12-13 22:33:29 +01:00
Xuan-Son NguyenandGitHub 4d5ae24c0a arg: fix common_params_parse not accepting negated arg (#17991) 2025-12-13 12:53:37 +01:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		380b4c984e common: support negated args (#17919) 
* args: support negated args

* update docs

* fix typo

* add more neg options

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* rm duplicated arg

* fix LLAMA_ARG_NO_HOST

* add test

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2025-12-12 23:58:53 +01:00