nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 20 Packages Projects Releases Wiki Activity
6,832 Commits 1 Branch 0 Tags
69e9ff010309a1155d704cf9320bdb3aaf4160ca
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan-Son NguyenandGitHub d0660f237a mtmd-cli : allow using --jinja (#16718) 
* mtmd-cli : allow using --jinja

* support -sys

* implement chat_history

* fix clear memory

* rm -sys support, added TODO
 2025-10-23 15:00:49 +02:00
Xuan-Son NguyenandGitHub 1bb4f43380 mtmd : support home-cooked Mistral Small Omni (#14928) 2025-10-16 19:00:31 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		3e3cb19f64 llama-quant: add support for mmproj (#16592) 
* llama-quant: add support for mmproj

* Update src/llama.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* check prefix instead

* small fix

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-10-15 14:48:08 +02:00
Xuan-Son NguyenandGitHub 05a2458121 codeowners : update ownership for @ngxson and @allozuar (#16128) 2025-09-22 11:10:58 +03:00
Xuan-Son NguyenandGitHub 4b8560ab56 chat : fix build on arm64 (#16101) 2025-09-19 13:02:51 +07:00
Xuan-Son NguyenandGitHub 0dd58b6877 ggml : refactor forward_dup for cpu backend (#16062) 
* ggml : refactor forward_dup for cpu backend

* clean up a bit

* add quant/dequant perf test
 2025-09-19 06:31:56 +02:00
Xuan-Son NguyenandGitHub 8f8f2274ee convert : add Llama4ForCausalLM (#16042) 
* convert : add Llama4ForCausalLM

* handle swa

* half working version

* fix use_kq_norm

* fix use_kq_norm
 2025-09-17 19:18:21 +02:00
Xuan-Son NguyenandGitHub 56920f5665 server : bring back timings_per_token (#15879) 2025-09-08 16:50:05 +02:00
Xuan-Son NguyenandGitHub 9fcb29f22f ggml: allow casting between f32 and i32 (#15783) 
* ggml: allow casting between f32 and i32

* fix cuda

* add vulkan

* fix CPU non-cont

* add non-cont test case

* add note

* extend test number range

* correct note

* add cont version for vulkan
 2025-09-08 12:33:01 +02:00
Xuan-Son NguyenandGitHub 3c3635d2f2 server : speed up tests (#15836) 
* server : speed up tests

* clean up

* restore timeout_seconds in some places

* flake8

* explicit offline
 2025-09-06 14:45:24 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		61bdfd5298 server : implement prompt processing progress report in stream mode (#15827) 
* server : implement `return_progress`

* add timings.cache_n

* add progress.time_ms

* add test

* fix test for chat/completions

* readme: add docs on timings

* use ggml_time_us

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-09-06 13:35:04 +02:00
Xuan-Son NguyenandGitHub a68d914426 server: add exceed_context_size_error type (#15780) 
* server: add exceed_context_size_error type

* change error code to 400
 2025-09-04 11:50:23 +02:00
Xuan-Son NguyenandGitHub 79a546220c mtmd : support Kimi VL model (#15458) 
* convert : fix tensor naming conflict for llama 4 vision

* convert ok

* support kimi vision model

* clean up

* fix style

* fix calc number of output tokens

* refactor resize_position_embeddings

* add test case

* rename build fn

* correct a small bug
 2025-08-26 12:54:19 +02:00
Xuan-Son NguyenandGitHub 945e1f12a6 ggml : fix condition of im2col on Metal backend (#15460) 2025-08-21 08:32:26 +03:00
Xuan-Son NguyenandGitHub e9288e8869 chat : clarify the meaning of reasoning_format (#15408) 
* chat : clarify the meaning of reasoning_format

* add link to this PR
 2025-08-19 10:29:36 +02:00
Xuan-Son NguyenandGitHub f08c4c0d8d mtmd : clean up clip_n_output_tokens (#15391) 2025-08-18 22:53:52 +02:00
Xuan-Son NguyenandGitHub fba5c0d680 chat : hotfix gpt-oss jinja raising an exception (#15243) 
* chat : hotfix gpt-oss jinja raising an exception

* fix
 2025-08-11 15:31:35 +02:00
Xuan-Son NguyenandGitHub 53d0a12658 server : allow specifying reasoning_format in HTTP request (#15238) 2025-08-11 14:48:41 +02:00
Xuan-Son NguyenandGitHub 50aa938901 convert : support non-mxfp4 HF model (#15153) 
* convert : support non-mxfp4 HF model

* rm redundant check

* disable debug check
 2025-08-07 23:26:03 +02:00
Xuan-Son NguyenGitHubSigbjørn Skjæret
 		00fa15fedc mtmd : add support for Voxtral (#14862) 
* mtmd : add support for Voxtral

* clean up

* fix python requirements

* add [BEGIN_AUDIO] token

* also support Devstral conversion

* add docs and tests

* fix regression for ultravox

* minor coding style improvement

* correct project activation fn

* Apply suggestions from code review

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2025-07-28 15:01:48 +02:00
Xuan-Son NguyenandGitHub 820de57d4f chat : fix kimi-k2 chat template (#14852) 2025-07-24 13:59:56 +02:00
Xuan-Son NguyenandGitHub 435a6d10d6 llama : minor coding style fix for smollm3 (#14605) 2025-07-10 10:00:20 +03:00
Xuan-Son NguyenandGitHub cb9178f885 llama : remove llm_graph_input_one (#14603) 2025-07-09 23:09:28 +02:00
Xuan-Son NguyenandGitHub 98bab638fb ggml : add ggml_scale_bias (#14417) 
* ggml : add ggml_scale_bias

* ggml_vec_mad1_f32

* add more simd

* add CUDA

* sycl

* vulkan

* cann (placeholder)

* opencl

* will this fix cpu?

* fix cuda

* suggestions from coderabbit

* fix cann compile error

* vDSP_vsmsa

* rm __ARM_FEATURE_SVE

* use memcpy for op params

* make code looks more consistent

* use scalar for __ARM_FEATURE_SVE

* add x param to ggml_vec_mad1_f32
 2025-07-09 18:16:12 +02:00
Xuan-Son NguyenandGitHub 20b7bf8a32 convert : fix smollm3 jinja template (#14586) 2025-07-09 09:26:13 +03:00
Xuan-Son NguyenGitHubVaibhavs10
 		08382869a2 model : add SmolLM3 (#14581) 
* Init - first pass.

* Model -> ModelBase.

* fix errors in conversion.

* Update the graph.

* up.

* up.

* wip

* cgraph ok

* rm redundant code

---------

Co-authored-by: Vaibhavs10 <vaibhavs10@gmail.com>
 2025-07-08 18:07:01 +02:00
Xuan-Son NguyenGitHubkooshi
 		8f22dc0a53 model : add hunyuan moe (#14425) 
* model : add hunyuan moe

* tokenizer ok

* fix tensor name

* cgraph init

* chat template

* wip

* almost working

* skip embed, fix bos

* cleanup

* yarn scaling

* cleanup

* correct rope type

* failed token fix

* ntk alpha freq_base

* tokenization working

* cleanup and pr changes

* vocab_size sanity check

* ntk alpha generic

* Update convert_hf_to_gguf.py

* Apply suggestions from code review

* fix regression

* fix style

---------

Co-authored-by: kooshi <1934337+kooshi@users.noreply.github.com>
 2025-07-08 11:24:06 +03:00
Xuan-Son NguyenandGitHub 0c2ee38ab7 convert : correct gemma 3n conversion (#14450) 
* convert : correct gemma 3n conversion

* rm redundant code
 2025-07-03 10:03:06 +02:00
Xuan-Son NguyenandGitHub 8846aace49 model : gemma3n text-only (#14400) 
* gemma3n

* add llm_graph_input_one
 2025-06-26 20:34:02 +03:00
Xuan-Son NguyenandGitHub 413977de32 mtmd : refactor llava-uhd preprocessing logic (#14247) 
* mtmd : refactor llava-uhd preprocessing logic

* fix editorconfig
 2025-06-18 10:43:57 +02:00
Xuan-Son NguyenandGitHub 95402553a5 llama-chat : fix multiple system message for gemma, orion (#14246) 2025-06-18 09:58:43 +02:00
Xuan-Son NguyenandGitHub 3ac67535c8 llama-graph : use ggml_repeat_4d (#13998) 2025-06-04 10:11:26 +02:00
Xuan-Son NguyenandGitHub ea1431b0fa docs : add "Quick start" section for new users (#13862) 
* docs : add "Quick start" section for non-technical users

* rm flox

* Update README.md
 2025-06-03 13:09:36 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		bfd322796c mtmd : fix memory leak in mtmd_helper_eval_chunk_single (#13961) 
* mtmd : fix memory in mtmd_helper_eval_chunk_single

* mtmd-cli : fix mem leak

* Update tools/mtmd/mtmd-cli.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-06-02 16:29:28 +02:00
Xuan-Son NguyenandGitHub 51fa76f172 mtmd : drop _shared from libmtmd name, merge helpers into libmtmd (⚠️ breaking change) (#13917) 
* mtmd : fix missing public header

* no object

* apply suggestion from Georgi

* rm mtmd-helper, merge it to mtmd

* missing vendor include dir
 2025-05-31 10:14:29 +02:00
Xuan-Son NguyenandGitHub 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 
* convert : allow partial update to the chkhsh pre-tokenizer list

* code style

* update tokenizer out

* rm inp/out files for models not having gguf

* fixed hash for glm

* skip nomic-bert-moe test

* Update convert_hf_to_gguf_update.py

* fix minerva-7b hash

* rm redundant import
 2025-05-30 12:24:37 +02:00
Xuan-Son NguyenandGitHub 763d06edb7 llama : fix KV shift for qwen2vl (#13870) 
* llama : fix KV shift for qwen2vl

* add ref to the PR
 2025-05-28 22:35:31 +02:00
Xuan-Son NguyenandGitHub 10961339b2 mtmd : move helpers to dedicated library (⚠️ breaking change) (#13866) 
* mtmd : move helpers to dedicated library

* fix server build

* rm leftover cmakelist code
 2025-05-28 22:35:22 +02:00
Xuan-Son NguyenandGitHub a3938fb53d convert : fix qwen omni conversion (#13859) 
* convert : fix qwen omni conversion

* fix typo
 2025-05-28 16:12:35 +02:00
Xuan-Son NguyenandGitHub 26b79b6cb3 convert : fix tensor naming conflict for llama 4 vision (#13836) 
* convert : fix tensor naming conflict for llama 4 vision

* add comment
 2025-05-28 10:05:54 +02:00
Xuan-Son NguyenandGitHub a8ea03d8ad ggml : add ggml_repeat_4d (#13824) 2025-05-27 15:53:55 +02:00
Xuan-Son NguyenandGitHub bc583e3c63 mtmd : support Qwen 2.5 Omni (input audio+vision, no audio output) (#13784) 
* mtmd : allow multiple modalities at the same time

* refactor mtmd tokenizer

* fix compile

* ok, missing SinusoidsPositionEmbedding

* first working version

* fix style

* more strict validate of n_embd

* refactor if..else to switch

* fix regression

* add test for 3B

* update docs

* fix tokenizing with add_special

* add more tests

* fix test case "huge"

* rm redundant code

* set_position_mrope_1d rm n_tokens
 2025-05-27 14:06:10 +02:00
Xuan-Son NguyenandGitHub 2f099b510f webui : bump max upload file size to 500MB (#13779) 2025-05-25 18:02:18 +01:00
Xuan-Son NguyenandGitHub 40aaa8a403 mtmd : add support for Qwen2-Audio and SeaLLM-Audio (#13760) 
* mtmd : add Qwen2-Audio support

* small clean up

* update discussion link

* clarify mtmd_get_output_embd

* clarification in multimodal.md

* fix ultravox bug

* ggml_cont
 2025-05-25 14:06:32 +02:00
Xuan-Son NguyenandGitHub 4c32832c59 ggml : add ggml_gelu_erf() CUDA kernel (#13719) 
* ggml : add ggml_gelu_erf() CUDA kernel

* missing semicolon
 2025-05-24 13:06:47 +02:00
Xuan-Son NguyenandGitHub 8a2afb7520 llama : allow custom list of swa_layers (#13726) 2025-05-23 17:07:04 +02:00
Xuan-Son NguyenandGitHub 9ecf3e66a3 server : support audio input (#13714) 
* server : support audio input

* add audio support on webui
 2025-05-23 11:03:47 +02:00
Xuan-Son NguyenandGitHub e16c4731c7 ggml : fix the order of ggml_unary_op (#13718) 2025-05-23 08:12:48 +02:00
Xuan-Son NguyenandGitHub 797990c4bc mtmd : add ultravox audio input (#13623) 
* convert ok, load ok

* warmup ok

* test

* still does not work?

* fix padding

* temporary give up

* fix merge conflict

* build_ultravox()

* rm test

* fix merge conflict

* add necessary mtmd APIs

* first working version (only 4s of audio)

* will this monster compile?

* fix compile

* please compile

* fPIC

* fix windows

* various fixes

* clean up audio_helpers

* fix conversion

* add some debug stuff

* long audio input ok

* adapt the api

* add --audio arg

* final touch UX

* add miniaudio to readme

* fix typo

* refactor kv metadata

* mtmd_default_marker()
 2025-05-22 20:42:48 +02:00
Xuan-Son NguyenandGitHub cf4cb59e64 ggml : add ggml_gelu_erf() (#13667) 
* ggml : add ggml_gelu_na (not approximated)

* fix naming order

* rename na --> erf

* apply review suggesions

* revert naming order
 2025-05-21 16:26:33 +02:00
Xuan-Son NguyenandGitHub 92ecdcc06a mtmd : add vision support for llama 4 (#13282) 
* wip llama 4 conversion

* rm redundant __init__

* fix conversion

* fix conversion

* test impl

* try this

* reshape patch_embeddings_0

* fix view

* rm ffn_post_norm

* cgraph ok

* f32 for pos embd

* add image marker tokens

* Llama4UnfoldConvolution

* correct pixel shuffle

* fix merge conflicts

* correct

* add debug_graph

* logits matched, but it still preceives the image incorrectly

* fix style

* add image_grid_pinpoints

* handle llama 4 preprocessing

* rm load_image_size

* rm unused line

* fix

* small fix 2

* add test & docs

* fix llava-1.6 test

* test: add notion of huge models

* add comment

* add warn about degraded quality
 2025-05-19 13:04:14 +02:00
Xuan-Son NguyenandGitHub 6aa892ec2a server : do not return error out of context (with ctx shift disabled) (#13577) 2025-05-16 21:50:00 +02:00
Xuan-Son NguyenandGitHub aea9f8b4e7 webui : improve accessibility for visually impaired people (#13551) 
* webui : improve accessibility for visually impaired people

* add a11y for extra contents

* fix some labels being read twice

* add skip to main content
 2025-05-16 21:49:01 +02:00
Xuan-Son NguyenandGitHub 06c1e4abc1 readme : add list of dependencies and their license (#13591) 2025-05-16 20:04:18 +02:00
Xuan-Son NguyenandGitHub c531edfa34 convert : fix conversion for llama 4 (#13567) 2025-05-15 17:40:07 +02:00
Xuan-Son NguyenandGitHub 3cc1f1f1d2 webui : handle PDF input (as text or image) + convert pasted long content to file (#13562) 
* webui : handle PDF input (as text or image)

* handle the case where pdf image + server without mtmd

* fix bug missing pages
 2025-05-15 14:24:50 +02:00
Xuan-Son NguyenandGitHub 360a9c98e1 server : fix cache_tokens bug with no cache_prompt (#13533) 2025-05-14 13:35:07 +02:00
Xuan-Son NguyenandGitHub bb1681fbd5 webui : use fflate for more deterministic gzip compress (#13525) 
* webui : use pako for more deterministic gzip compress

* simpler code

* use fflate instead of pako
 2025-05-14 10:26:12 +02:00
Xuan-Son NguyenandGitHub 71bdbdb587 clip : clip.h become private API (⚠️ breaking change) (#13510) 2025-05-13 17:07:21 +02:00
Xuan-Son NguyenandGitHub b4726345ac mtmd : remove libllava, remove clip-quantize-cli (⚠️ breaking change) (#13460) 
* mtmd : remove libllava, remove clip-quantize-cli

* rm clip_model_quantize
 2025-05-13 15:33:58 +02:00
Xuan-Son NguyenandGitHub de4c07f937 clip : cap max image size 1024 for qwen vl model (#13478) 2025-05-12 15:06:51 +02:00
Xuan-Son NguyenandGitHub df8491922f ggml : add mrope kernel for metal (#13457) 2025-05-12 10:29:13 +02:00
Xuan-Son NguyenandGitHub a634d75d1b mtmd : move helpers to dedicated file (#13442) 
* mtmd : move helpers to dedicated file

* fix windows build

* rm redundant include
 2025-05-11 11:34:23 +02:00
Xuan-Son NguyenandGitHub 15e6125a39 mtmd : add hard limit on image resolution for qwen2vl / qwen2.5vl (#13434) 
* mtmd : add hard limit on image resolution for qwen2vl / qwen2.5vl

* fix typo
 2025-05-10 19:57:54 +02:00
Xuan-Son NguyenandGitHub 3b24d26c22 server : update docs (#13432) 2025-05-10 18:44:49 +02:00
Xuan-Son NguyenandGitHub 053367d149 mtmd : support InternVL 2.5 and 3 (#13422) 
* convert : internvl support

* InternVL3-1B working

* fix regression

* rm mobilevlm from test

* fix conversion

* add test for internvl

* add to list of pre-quant

* restore boi/eoi check

* add clarify comment for norm eps
 2025-05-10 16:26:42 +02:00
Xuan-Son NguyenandGitHub 7fef11766c arg : add env var to control mmproj (#13416) 
* arg : add env var to control mmproj

* small note about -hf --mmproj
 2025-05-10 08:16:29 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		33eff40240 server : vision support via libmtmd (#12898) 
* server : (experimental) vision support via libmtmd

* mtmd : add more api around mtmd_image_tokens

* mtmd : add more api around mtmd_image_tokens

* mtmd : ability to calc image hash

* shared_ptr for mtmd_image_tokens

* move hash to user-define ID (fixed)

* abstract out the batch management

* small fix

* refactor logic adding tokens to batch

* implement hashing image

* use FNV hash, now hash bitmap instead of file data

* allow decoding image embedding to be split into batches

* rm whitespace

* disable some features when mtmd is on

* fix --no-mmproj-offload

* mtmd_context_params no timings

* refactor server_inp to server_tokens

* fix the failing test case

* init

* wip

* working version

* add mtmd::bitmaps

* add test target

* rm redundant define

* test: mtmd_input_chunks_free

* rm outdated comment

* fix merging issue

* explicitly create mtmd::input_chunks

* mtmd_input_chunk_copy

* add clone()

* improve server_input struct

* clip :  fix confused naming ffn_up and ffn_down

* rm ffn_i/o/g naming

* rename n_embd, n_ff

* small fix

* no check n_ff

* fix detokenize

* add const to various places

* add warning about breaking changes

* add c api

* helper: use mtmd_image_tokens_get_n_pos

* fix ctx_shift

* fix name shadowing

* more strict condition

* support remote image_url

* remote image_url log

* add CI test

* do not log base64

* add "has_multimodal" to /props

* remove dangling image

* speculative: use slot.cache_tokens.insert

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* rm can_be_detokenized

* on prmpt processing done, assert cache_tokens.size

* handle_completions_impl returns void

* adapt the new web ui

* update docs and hot topics

* rm assert

* small fix (2)

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-05-09 19:29:37 +02:00
Xuan-Son NguyenandGitHub 2189fd3b63 mtmd : fix batch_view for m-rope (#13397) 
* mtmd : fix batch_view for m-rope

* nits : fix comment
 2025-05-09 11:18:02 +02:00
Xuan-Son NguyenandGitHub 3f96aeff39 llama : one-off chat template fix for Mistral-Small-2503 (#13398) 
* llama : one-off chat template fix for Mistral-Small-2503

* update readme

* add mistral-v7-tekken
 2025-05-09 11:17:51 +02:00
Xuan-Son NguyenandGitHub d9c4accaff server : (webui) rename has_multimodal --> modalities (#13393) 
* server : (webui) rename has_multimodal --> modalities

* allow converting SVG to PNG

* less complicated code
 2025-05-09 09:06:37 +02:00
Xuan-Son NguyenandGitHub ee01d71e58 server : (webui) fix a very small misalignment (#13387) 
* server : (webui) fix a very small misalignment

* restore font-bold
 2025-05-08 18:51:45 +02:00
Xuan-Son NguyenandGitHub 8c83449cb7 server : (webui) revamp the input area, plus many small UI improvements (#13365) 
* rework the input area

* process selected file

* change all icons to heroicons

* fix thought process collapse

* move conversation more menu to sidebar

* sun icon --> moon icon

* rm default system message

* stricter upload file check, only allow image if server has mtmd

* build it

* add renaming

* better autoscroll

* build

* add conversation group

* fix scroll

* extra context first, then user input in the end

* fix <hr> tag

* clean up a bit

* build

* add mb-3 for <pre>

* throttle adjustTextareaHeight to make it less laggy

* (nits) missing padding in sidebar

* rm stray console log
 2025-05-08 15:37:29 +02:00
Xuan-Son NguyenandGitHub 32916a4907 clip : refactor graph builder (#13321) 
* mtmd : refactor graph builder

* fix qwen2vl

* clean up siglip cgraph

* pixtral migrated

* move minicpmv to a dedicated build function

* move max_feature_layer to build_llava

* use build_attn for minicpm resampler

* fix windows build

* add comment for batch_size

* also support tinygemma3 test model

* qwen2vl does not use RMS norm

* fix qwen2vl norm (2)
 2025-05-06 22:40:24 +02:00
Xuan-Son NguyenandGitHub 2f54e348ad llama : fix build_ffn without gate (#13336) 
* llama : fix build_ffn without gate

* fix build on windows

* Revert "fix build on windows"

This reverts commit fc420d3c7eef3481d3d2f313fef2757cb33a7c56.
 2025-05-06 14:25:40 +02:00
Xuan-Son NguyenandGitHub 9b61acf060 mtmd : rename llava directory to mtmd (#13311) 
* mv llava to mtmd

* change ref everywhere
 2025-05-05 16:02:55 +02:00
Xuan-Son NguyenandGitHub 5215b91e93 clip : fix confused naming ffn_up and ffn_down (#13290) 
* clip :  fix confused naming ffn_up and ffn_down

* rm ffn_i/o/g naming

* rename n_embd, n_ff

* small fix

* no check n_ff
 2025-05-05 12:54:44 +02:00
Xuan-Son NguyenandGitHub 27aa259532 mtmd : add C public API (#13184) 
* init

* wip

* working version

* add mtmd::bitmaps

* add test target

* rm redundant define

* test: mtmd_input_chunks_free

* rm outdated comment

* fix merging issue

* explicitly create mtmd::input_chunks

* mtmd_input_chunk_copy

* add clone()

* add const to various places

* add warning about breaking changes

* helper: use mtmd_image_tokens_get_n_pos
 2025-05-04 23:43:42 +02:00
Xuan-Son NguyenandGitHub 36667c8edc clip : revert the change of BOI/EOI token for GLM-edge (⚠️ breaking change) (#13259) 2025-05-03 20:07:54 +02:00
Xuan-Son NguyenandGitHub 074e42ab31 convert : converting mmproj for Qwen2/2.5VL from convert_hf_to_gguf (#13209) 
* wip

* qwen2.5vl ok

* vision: fix models missing "text_config"

* add test

* fix test repo name

* fix 32B model

* Revert "fix 32B model"

This reverts commit 651752f1ae25fe8a01c1e57c18cf2eca80b2774e.

* clarify about 32B

* rm qwen surgery script

* update llava/readme

* move V_ENC_EMBD_PATCH handling to Qwen2VLVisionModel
 2025-05-02 17:17:15 +02:00
Xuan-Son NguyenandGitHub dcf886007d convert : explicitly disable trust_remote_code for AutoConfig (#13246) 2025-05-02 08:45:10 +02:00
Xuan-Son NguyenandGitHub 8936784f7a mtmd : add **vision** support for Mistral Small 3.1 (#13231) 
* convert ok

* load ok, missing patch merger

* ah sheet it works

* update llava/readme

* add test

* fix test
 2025-05-01 17:05:42 +02:00
Xuan-Son NguyenandGitHub 13c9a3319b arg : remove CURLINFO_EFFECTIVE_METHOD (#13228) 2025-05-01 10:23:25 +02:00
Xuan-Son NguyenandGitHub 6f67cf1f48 arg : -hf do not fail if url mismatch (#13219) 
* arg : -hf do not fail if url mismatch

* do not return if cannot parse metadata json
 2025-04-30 21:29:15 +01:00
Xuan-Son NguyenandGitHub 3e168bede4 convert : improve model arch handling (#13122) 
* convert : improve model arch handling

* use AutoConfig

* rm trust_remote_code

* Update convert_hf_to_gguf.py

* fix self.block_count for vision

* fix NomicBertModel
 2025-04-30 16:56:24 +02:00
Xuan-Son NguyenandGitHub 07c2e2f76c convert : correct typo image_mean --> image_std (#13208) 2025-04-30 13:06:15 +02:00
Xuan-Son NguyenandGitHub 5933e6fdc9 arg : allow using -hf offline (#13202) 
* arg : allow using -hf offline

* add more comments in code [no ci]
 2025-04-30 10:46:32 +02:00
Xuan-Son NguyenandGitHub da84c04d8f docker : do not build tests (#13204) 
* docker : do not build tests

* include "ggml-cpu.h"
 2025-04-30 10:44:07 +02:00
Xuan-Son NguyenandGitHub 00e3e5a194 mtmd : add qwen2vl and qwen2.5vl (#13141) 
* llava : add clip_n_output_tokens, deprecate clip_n_patches

* mtmd : add qwen2vl and qwen2.5vl

* decode_embd_batch::set_position_...

* working version

* deprecate llama-qwen2vl-cli

* correct order W, H of clip_embd_nbytes_by_img

* edit existing line in hot topics
 2025-04-29 11:47:04 +02:00
Xuan-Son NguyenandGitHub b6ce7430b7 llama-graph : fix text position for mrope (#13159) 
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
 2025-04-29 09:45:49 +03:00
Xuan-Son NguyenandGitHub eaea325324 clip : fix model size display (#13153) 2025-04-28 21:23:19 +02:00
Xuan-Son NguyenandGitHub 4e87962e34 mtmd : fix glm-edge redundant token count (#13139) 
* mtmd : fix glm-edge redundant token count

* fix chat template

* temporary disable GLMEdge test chat tmpl
 2025-04-28 16:12:56 +02:00
Xuan-Son NguyenandGitHub d2b2031e5f llama : (mrope) allow using normal 1D position for text token (#13138) 
* llama : (mrope) use normal position for text token

* rm n_pos_per_embd from llm_graph_input_attn_temp
 2025-04-28 14:20:56 +02:00
Xuan-Son NguyenandGitHub 5fa9e63be8 clip : refactor set input for cgraph + fix qwen2.5vl input (#13136) 
* clip : refactor set input for cgraph

* more strict assert

* minicpmv : use clip_n_mmproj_embd instead of copying the same code everywhere

* split qwen2 and qwen2.5 code blocks

* minor style fix
 2025-04-28 12:18:59 +02:00
Xuan-Son NguyenandGitHub e5d6c2554e llama-chat : fix typo GML --> GLM (#13143) 2025-04-28 10:11:58 +02:00
Xuan-Son NguyenandGitHub 85f36e5e71 arg : fix unused variable (#13142) 2025-04-28 08:16:59 +03:00
Xuan-Son NguyenandGitHub 2d451c8059 common : add common_remote_get_content (#13123) 
* common : add common_remote_get_content

* support max size and timeout

* add tests
 2025-04-26 22:58:12 +02:00
Xuan-Son NguyenandGitHub 4753791e70 clip : improve projector naming (#13118) 
* clip : improve projector naming

* no more kv has_llava_projector

* rm unused kv

* rm more unused
 2025-04-26 22:39:47 +02:00
Xuan-Son NguyenandGitHub edb18b6e8f clip : fix pixtral on some GPU backends (#13097) 
* clip : fix pixtral on some GPU backends

* refactor inp_raw set

* rm outdated comment

* fix dynamic size

* add TODO
 2025-04-25 14:31:42 +02:00
Xuan-Son NguyenandGitHub 13be08daf9 clip : remove boi/eoi embeddings for GLM-edge model (#13081) 2025-04-24 22:17:04 +02:00