nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 27 Packages Projects Releases Wiki Activity
5,545 Commits 1 Branch 0 Tags
df0c0c7d02f951dc639d48fd536f79200460ac83
Commit Graph
95 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan-Son NguyenandGitHub 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 
* convert : allow partial update to the chkhsh pre-tokenizer list

* code style

* update tokenizer out

* rm inp/out files for models not having gguf

* fixed hash for glm

* skip nomic-bert-moe test

* Update convert_hf_to_gguf_update.py

* fix minerva-7b hash

* rm redundant import
 2025-05-30 12:24:37 +02:00
Xuan-Son NguyenandGitHub 763d06edb7 llama : fix KV shift for qwen2vl (#13870) 
* llama : fix KV shift for qwen2vl

* add ref to the PR
 2025-05-28 22:35:31 +02:00
Xuan-Son NguyenandGitHub 10961339b2 mtmd : move helpers to dedicated library (⚠️ breaking change) (#13866) 
* mtmd : move helpers to dedicated library

* fix server build

* rm leftover cmakelist code
 2025-05-28 22:35:22 +02:00
Xuan-Son NguyenandGitHub a3938fb53d convert : fix qwen omni conversion (#13859) 
* convert : fix qwen omni conversion

* fix typo
 2025-05-28 16:12:35 +02:00
Xuan-Son NguyenandGitHub 26b79b6cb3 convert : fix tensor naming conflict for llama 4 vision (#13836) 
* convert : fix tensor naming conflict for llama 4 vision

* add comment
 2025-05-28 10:05:54 +02:00
Xuan-Son NguyenandGitHub a8ea03d8ad ggml : add ggml_repeat_4d (#13824) 2025-05-27 15:53:55 +02:00
Xuan-Son NguyenandGitHub bc583e3c63 mtmd : support Qwen 2.5 Omni (input audio+vision, no audio output) (#13784) 
* mtmd : allow multiple modalities at the same time

* refactor mtmd tokenizer

* fix compile

* ok, missing SinusoidsPositionEmbedding

* first working version

* fix style

* more strict validate of n_embd

* refactor if..else to switch

* fix regression

* add test for 3B

* update docs

* fix tokenizing with add_special

* add more tests

* fix test case "huge"

* rm redundant code

* set_position_mrope_1d rm n_tokens
 2025-05-27 14:06:10 +02:00
Xuan-Son NguyenandGitHub 2f099b510f webui : bump max upload file size to 500MB (#13779) 2025-05-25 18:02:18 +01:00
Xuan-Son NguyenandGitHub 40aaa8a403 mtmd : add support for Qwen2-Audio and SeaLLM-Audio (#13760) 
* mtmd : add Qwen2-Audio support

* small clean up

* update discussion link

* clarify mtmd_get_output_embd

* clarification in multimodal.md

* fix ultravox bug

* ggml_cont
 2025-05-25 14:06:32 +02:00
Xuan-Son NguyenandGitHub 4c32832c59 ggml : add ggml_gelu_erf() CUDA kernel (#13719) 
* ggml : add ggml_gelu_erf() CUDA kernel

* missing semicolon
 2025-05-24 13:06:47 +02:00
Xuan-Son NguyenandGitHub 8a2afb7520 llama : allow custom list of swa_layers (#13726) 2025-05-23 17:07:04 +02:00
Xuan-Son NguyenandGitHub 9ecf3e66a3 server : support audio input (#13714) 
* server : support audio input

* add audio support on webui
 2025-05-23 11:03:47 +02:00
Xuan-Son NguyenandGitHub e16c4731c7 ggml : fix the order of ggml_unary_op (#13718) 2025-05-23 08:12:48 +02:00
Xuan-Son NguyenandGitHub 797990c4bc mtmd : add ultravox audio input (#13623) 
* convert ok, load ok

* warmup ok

* test

* still does not work?

* fix padding

* temporary give up

* fix merge conflict

* build_ultravox()

* rm test

* fix merge conflict

* add necessary mtmd APIs

* first working version (only 4s of audio)

* will this monster compile?

* fix compile

* please compile

* fPIC

* fix windows

* various fixes

* clean up audio_helpers

* fix conversion

* add some debug stuff

* long audio input ok

* adapt the api

* add --audio arg

* final touch UX

* add miniaudio to readme

* fix typo

* refactor kv metadata

* mtmd_default_marker()
 2025-05-22 20:42:48 +02:00
Xuan-Son NguyenandGitHub cf4cb59e64 ggml : add ggml_gelu_erf() (#13667) 
* ggml : add ggml_gelu_na (not approximated)

* fix naming order

* rename na --> erf

* apply review suggesions

* revert naming order
 2025-05-21 16:26:33 +02:00
Xuan-Son NguyenandGitHub 92ecdcc06a mtmd : add vision support for llama 4 (#13282) 
* wip llama 4 conversion

* rm redundant __init__

* fix conversion

* fix conversion

* test impl

* try this

* reshape patch_embeddings_0

* fix view

* rm ffn_post_norm

* cgraph ok

* f32 for pos embd

* add image marker tokens

* Llama4UnfoldConvolution

* correct pixel shuffle

* fix merge conflicts

* correct

* add debug_graph

* logits matched, but it still preceives the image incorrectly

* fix style

* add image_grid_pinpoints

* handle llama 4 preprocessing

* rm load_image_size

* rm unused line

* fix

* small fix 2

* add test & docs

* fix llava-1.6 test

* test: add notion of huge models

* add comment

* add warn about degraded quality
 2025-05-19 13:04:14 +02:00
Xuan-Son NguyenandGitHub 6aa892ec2a server : do not return error out of context (with ctx shift disabled) (#13577) 2025-05-16 21:50:00 +02:00
Xuan-Son NguyenandGitHub aea9f8b4e7 webui : improve accessibility for visually impaired people (#13551) 
* webui : improve accessibility for visually impaired people

* add a11y for extra contents

* fix some labels being read twice

* add skip to main content
 2025-05-16 21:49:01 +02:00
Xuan-Son NguyenandGitHub 06c1e4abc1 readme : add list of dependencies and their license (#13591) 2025-05-16 20:04:18 +02:00
Xuan-Son NguyenandGitHub c531edfa34 convert : fix conversion for llama 4 (#13567) 2025-05-15 17:40:07 +02:00
Xuan-Son NguyenandGitHub 3cc1f1f1d2 webui : handle PDF input (as text or image) + convert pasted long content to file (#13562) 
* webui : handle PDF input (as text or image)

* handle the case where pdf image + server without mtmd

* fix bug missing pages
 2025-05-15 14:24:50 +02:00
Xuan-Son NguyenandGitHub 360a9c98e1 server : fix cache_tokens bug with no cache_prompt (#13533) 2025-05-14 13:35:07 +02:00
Xuan-Son NguyenandGitHub bb1681fbd5 webui : use fflate for more deterministic gzip compress (#13525) 
* webui : use pako for more deterministic gzip compress

* simpler code

* use fflate instead of pako
 2025-05-14 10:26:12 +02:00
Xuan-Son NguyenandGitHub 71bdbdb587 clip : clip.h become private API (⚠️ breaking change) (#13510) 2025-05-13 17:07:21 +02:00
Xuan-Son NguyenandGitHub b4726345ac mtmd : remove libllava, remove clip-quantize-cli (⚠️ breaking change) (#13460) 
* mtmd : remove libllava, remove clip-quantize-cli

* rm clip_model_quantize
 2025-05-13 15:33:58 +02:00
Xuan-Son NguyenandGitHub de4c07f937 clip : cap max image size 1024 for qwen vl model (#13478) 2025-05-12 15:06:51 +02:00
Xuan-Son NguyenandGitHub df8491922f ggml : add mrope kernel for metal (#13457) 2025-05-12 10:29:13 +02:00
Xuan-Son NguyenandGitHub a634d75d1b mtmd : move helpers to dedicated file (#13442) 
* mtmd : move helpers to dedicated file

* fix windows build

* rm redundant include
 2025-05-11 11:34:23 +02:00
Xuan-Son NguyenandGitHub 15e6125a39 mtmd : add hard limit on image resolution for qwen2vl / qwen2.5vl (#13434) 
* mtmd : add hard limit on image resolution for qwen2vl / qwen2.5vl

* fix typo
 2025-05-10 19:57:54 +02:00
Xuan-Son NguyenandGitHub 3b24d26c22 server : update docs (#13432) 2025-05-10 18:44:49 +02:00
Xuan-Son NguyenandGitHub 053367d149 mtmd : support InternVL 2.5 and 3 (#13422) 
* convert : internvl support

* InternVL3-1B working

* fix regression

* rm mobilevlm from test

* fix conversion

* add test for internvl

* add to list of pre-quant

* restore boi/eoi check

* add clarify comment for norm eps
 2025-05-10 16:26:42 +02:00
Xuan-Son NguyenandGitHub 7fef11766c arg : add env var to control mmproj (#13416) 
* arg : add env var to control mmproj

* small note about -hf --mmproj
 2025-05-10 08:16:29 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		33eff40240 server : vision support via libmtmd (#12898) 
* server : (experimental) vision support via libmtmd

* mtmd : add more api around mtmd_image_tokens

* mtmd : add more api around mtmd_image_tokens

* mtmd : ability to calc image hash

* shared_ptr for mtmd_image_tokens

* move hash to user-define ID (fixed)

* abstract out the batch management

* small fix

* refactor logic adding tokens to batch

* implement hashing image

* use FNV hash, now hash bitmap instead of file data

* allow decoding image embedding to be split into batches

* rm whitespace

* disable some features when mtmd is on

* fix --no-mmproj-offload

* mtmd_context_params no timings

* refactor server_inp to server_tokens

* fix the failing test case

* init

* wip

* working version

* add mtmd::bitmaps

* add test target

* rm redundant define

* test: mtmd_input_chunks_free

* rm outdated comment

* fix merging issue

* explicitly create mtmd::input_chunks

* mtmd_input_chunk_copy

* add clone()

* improve server_input struct

* clip :  fix confused naming ffn_up and ffn_down

* rm ffn_i/o/g naming

* rename n_embd, n_ff

* small fix

* no check n_ff

* fix detokenize

* add const to various places

* add warning about breaking changes

* add c api

* helper: use mtmd_image_tokens_get_n_pos

* fix ctx_shift

* fix name shadowing

* more strict condition

* support remote image_url

* remote image_url log

* add CI test

* do not log base64

* add "has_multimodal" to /props

* remove dangling image

* speculative: use slot.cache_tokens.insert

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* rm can_be_detokenized

* on prmpt processing done, assert cache_tokens.size

* handle_completions_impl returns void

* adapt the new web ui

* update docs and hot topics

* rm assert

* small fix (2)

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-05-09 19:29:37 +02:00
Xuan-Son NguyenandGitHub 2189fd3b63 mtmd : fix batch_view for m-rope (#13397) 
* mtmd : fix batch_view for m-rope

* nits : fix comment
 2025-05-09 11:18:02 +02:00
Xuan-Son NguyenandGitHub 3f96aeff39 llama : one-off chat template fix for Mistral-Small-2503 (#13398) 
* llama : one-off chat template fix for Mistral-Small-2503

* update readme

* add mistral-v7-tekken
 2025-05-09 11:17:51 +02:00
Xuan-Son NguyenandGitHub d9c4accaff server : (webui) rename has_multimodal --> modalities (#13393) 
* server : (webui) rename has_multimodal --> modalities

* allow converting SVG to PNG

* less complicated code
 2025-05-09 09:06:37 +02:00
Xuan-Son NguyenandGitHub ee01d71e58 server : (webui) fix a very small misalignment (#13387) 
* server : (webui) fix a very small misalignment

* restore font-bold
 2025-05-08 18:51:45 +02:00
Xuan-Son NguyenandGitHub 8c83449cb7 server : (webui) revamp the input area, plus many small UI improvements (#13365) 
* rework the input area

* process selected file

* change all icons to heroicons

* fix thought process collapse

* move conversation more menu to sidebar

* sun icon --> moon icon

* rm default system message

* stricter upload file check, only allow image if server has mtmd

* build it

* add renaming

* better autoscroll

* build

* add conversation group

* fix scroll

* extra context first, then user input in the end

* fix <hr> tag

* clean up a bit

* build

* add mb-3 for <pre>

* throttle adjustTextareaHeight to make it less laggy

* (nits) missing padding in sidebar

* rm stray console log
 2025-05-08 15:37:29 +02:00
Xuan-Son NguyenandGitHub 32916a4907 clip : refactor graph builder (#13321) 
* mtmd : refactor graph builder

* fix qwen2vl

* clean up siglip cgraph

* pixtral migrated

* move minicpmv to a dedicated build function

* move max_feature_layer to build_llava

* use build_attn for minicpm resampler

* fix windows build

* add comment for batch_size

* also support tinygemma3 test model

* qwen2vl does not use RMS norm

* fix qwen2vl norm (2)
 2025-05-06 22:40:24 +02:00
Xuan-Son NguyenandGitHub 2f54e348ad llama : fix build_ffn without gate (#13336) 
* llama : fix build_ffn without gate

* fix build on windows

* Revert "fix build on windows"

This reverts commit fc420d3c7eef3481d3d2f313fef2757cb33a7c56.
 2025-05-06 14:25:40 +02:00
Xuan-Son NguyenandGitHub 9b61acf060 mtmd : rename llava directory to mtmd (#13311) 
* mv llava to mtmd

* change ref everywhere
 2025-05-05 16:02:55 +02:00
Xuan-Son NguyenandGitHub 5215b91e93 clip : fix confused naming ffn_up and ffn_down (#13290) 
* clip :  fix confused naming ffn_up and ffn_down

* rm ffn_i/o/g naming

* rename n_embd, n_ff

* small fix

* no check n_ff
 2025-05-05 12:54:44 +02:00
Xuan-Son NguyenandGitHub 27aa259532 mtmd : add C public API (#13184) 
* init

* wip

* working version

* add mtmd::bitmaps

* add test target

* rm redundant define

* test: mtmd_input_chunks_free

* rm outdated comment

* fix merging issue

* explicitly create mtmd::input_chunks

* mtmd_input_chunk_copy

* add clone()

* add const to various places

* add warning about breaking changes

* helper: use mtmd_image_tokens_get_n_pos
 2025-05-04 23:43:42 +02:00
Xuan-Son NguyenandGitHub 36667c8edc clip : revert the change of BOI/EOI token for GLM-edge (⚠️ breaking change) (#13259) 2025-05-03 20:07:54 +02:00
Xuan-Son NguyenandGitHub 074e42ab31 convert : converting mmproj for Qwen2/2.5VL from convert_hf_to_gguf (#13209) 
* wip

* qwen2.5vl ok

* vision: fix models missing "text_config"

* add test

* fix test repo name

* fix 32B model

* Revert "fix 32B model"

This reverts commit 651752f1ae25fe8a01c1e57c18cf2eca80b2774e.

* clarify about 32B

* rm qwen surgery script

* update llava/readme

* move V_ENC_EMBD_PATCH handling to Qwen2VLVisionModel
 2025-05-02 17:17:15 +02:00
Xuan-Son NguyenandGitHub dcf886007d convert : explicitly disable trust_remote_code for AutoConfig (#13246) 2025-05-02 08:45:10 +02:00
Xuan-Son NguyenandGitHub 8936784f7a mtmd : add **vision** support for Mistral Small 3.1 (#13231) 
* convert ok

* load ok, missing patch merger

* ah sheet it works

* update llava/readme

* add test

* fix test
 2025-05-01 17:05:42 +02:00
Xuan-Son NguyenandGitHub 13c9a3319b arg : remove CURLINFO_EFFECTIVE_METHOD (#13228) 2025-05-01 10:23:25 +02:00
Xuan-Son NguyenandGitHub 6f67cf1f48 arg : -hf do not fail if url mismatch (#13219) 
* arg : -hf do not fail if url mismatch

* do not return if cannot parse metadata json
 2025-04-30 21:29:15 +01:00
Xuan-Son NguyenandGitHub 3e168bede4 convert : improve model arch handling (#13122) 
* convert : improve model arch handling

* use AutoConfig

* rm trust_remote_code

* Update convert_hf_to_gguf.py

* fix self.block_count for vision

* fix NomicBertModel
 2025-04-30 16:56:24 +02:00
Xuan-Son NguyenandGitHub 07c2e2f76c convert : correct typo image_mean --> image_std (#13208) 2025-04-30 13:06:15 +02:00
Xuan-Son NguyenandGitHub 5933e6fdc9 arg : allow using -hf offline (#13202) 
* arg : allow using -hf offline

* add more comments in code [no ci]
 2025-04-30 10:46:32 +02:00
Xuan-Son NguyenandGitHub da84c04d8f docker : do not build tests (#13204) 
* docker : do not build tests

* include "ggml-cpu.h"
 2025-04-30 10:44:07 +02:00
Xuan-Son NguyenandGitHub 00e3e5a194 mtmd : add qwen2vl and qwen2.5vl (#13141) 
* llava : add clip_n_output_tokens, deprecate clip_n_patches

* mtmd : add qwen2vl and qwen2.5vl

* decode_embd_batch::set_position_...

* working version

* deprecate llama-qwen2vl-cli

* correct order W, H of clip_embd_nbytes_by_img

* edit existing line in hot topics
 2025-04-29 11:47:04 +02:00
Xuan-Son NguyenandGitHub b6ce7430b7 llama-graph : fix text position for mrope (#13159) 
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
 2025-04-29 09:45:49 +03:00
Xuan-Son NguyenandGitHub eaea325324 clip : fix model size display (#13153) 2025-04-28 21:23:19 +02:00
Xuan-Son NguyenandGitHub 4e87962e34 mtmd : fix glm-edge redundant token count (#13139) 
* mtmd : fix glm-edge redundant token count

* fix chat template

* temporary disable GLMEdge test chat tmpl
 2025-04-28 16:12:56 +02:00
Xuan-Son NguyenandGitHub d2b2031e5f llama : (mrope) allow using normal 1D position for text token (#13138) 
* llama : (mrope) use normal position for text token

* rm n_pos_per_embd from llm_graph_input_attn_temp
 2025-04-28 14:20:56 +02:00
Xuan-Son NguyenandGitHub 5fa9e63be8 clip : refactor set input for cgraph + fix qwen2.5vl input (#13136) 
* clip : refactor set input for cgraph

* more strict assert

* minicpmv : use clip_n_mmproj_embd instead of copying the same code everywhere

* split qwen2 and qwen2.5 code blocks

* minor style fix
 2025-04-28 12:18:59 +02:00
Xuan-Son NguyenandGitHub e5d6c2554e llama-chat : fix typo GML --> GLM (#13143) 2025-04-28 10:11:58 +02:00
Xuan-Son NguyenandGitHub 85f36e5e71 arg : fix unused variable (#13142) 2025-04-28 08:16:59 +03:00
Xuan-Son NguyenandGitHub 2d451c8059 common : add common_remote_get_content (#13123) 
* common : add common_remote_get_content

* support max size and timeout

* add tests
 2025-04-26 22:58:12 +02:00
Xuan-Son NguyenandGitHub 4753791e70 clip : improve projector naming (#13118) 
* clip : improve projector naming

* no more kv has_llava_projector

* rm unused kv

* rm more unused
 2025-04-26 22:39:47 +02:00
Xuan-Son NguyenandGitHub edb18b6e8f clip : fix pixtral on some GPU backends (#13097) 
* clip : fix pixtral on some GPU backends

* refactor inp_raw set

* rm outdated comment

* fix dynamic size

* add TODO
 2025-04-25 14:31:42 +02:00
Xuan-Son NguyenandGitHub 13be08daf9 clip : remove boi/eoi embeddings for GLM-edge model (#13081) 2025-04-24 22:17:04 +02:00
Xuan-Son NguyenandGitHub 7c727fbe39 arg : add --no-mmproj-offload (#13093) 
* arg : add --no-mmproj-offload

* Update common/arg.cpp
 2025-04-24 14:04:14 +02:00
Xuan-Son NguyenandGitHub 80982e815e arg : clean up handling --mmproj with -hf (#13082) 
* arg : clean up handling --mmproj with -hf

* rm change about no_mmproj

* Revert "rm change about no_mmproj"

This reverts commit 2cac8e0efb629d66c612f137e75d562f94bb9e6c.

* handle no_mmproj explicitly

* skip download mmproj on examples not using it
 2025-04-24 12:14:13 +02:00
Xuan-Son NguyenandGitHub ecda2ec4b3 mtmd : Support Pixtral 12B (#13065) 
* add pixtral text model (vision is wip)

* cgraph ok, just missing 2D RoPE

* fix bad rebase

* first working version

* fix problem with img_break token

* support dynamic image size

* update docs

* update test script
 2025-04-23 20:21:59 +02:00
Xuan-Son NguyenandGitHub dc39a5e7a8 mtmd : support SmolVLM (version 1 and 2) (#13050) 
* mtmd : support SmolVLM (version 1 and 2)

* correct chat template

* fix n_patches

* scale_factor is an int

* add more models to test
 2025-04-22 16:24:54 +02:00
Xuan-Son NguyenandGitHub 243453533e llava : update documentations (#13055) 
* llava : update documentations

* fix typo
 2025-04-22 10:37:00 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		84a9bf2fc2 mtmd : merge llava, gemma3 and minicpmv CLI into single llama-mtmd-cli (#13012) 
* mtmd : merge `llava-cli` and `gemma3-cli` into single `mtmd-cli`

* support for minicpmv

* remove cpp files of llava and minicpmv

* update hot topics

* mtmd : add not supported msg for qwen2vl

* Update examples/llava/mtmd.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-04-21 15:32:58 +02:00
Xuan-Son NguyenGitHubcompilade
 		2016f07bd1 convert : experimental support for --mmproj flag (#13023) 
* convert : experimental support for `--mmproj` flag

* fix bad ctrl+f replace

* fix style

* split into subclasses TextModel and VisionModel

* rename Mode --> ModelBase

* small fix

* correct CLIP_VISION arch name (because existing GGUF already use it)

* Apply suggestions from code review

Co-authored-by: compilade <git@compilade.net>

* fix Mistral3Model

* fix typo

Co-authored-by: compilade <git@compilade.net>

---------

Co-authored-by: compilade <git@compilade.net>
 2025-04-20 23:29:36 +02:00
Xuan-Son NguyenandGitHub 37b9f0d29d clip : refactor, add image_manipulation and llava_uhd classes (#13011) 
* clip : refactor, add `image_manipulation` and `llava_uhd`

* refactor llava-1.6 preprocessing

* simplify logic for llava-1.5

* missing include
 2025-04-19 09:15:45 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		35370ba945 server : use std::move whenever possible (#12936) 
* server : use std::move whenever possible

* use r-value ref

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* make task creation scoped

* restore std::move

* fix task_id not set correctly

* apply changes from suggestion

Co-authored-by: ggerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-04-18 19:58:12 +02:00
Xuan-Son NguyenandGitHub b9154ecff9 mtmd : add methods to access mtmd_image_tokens (#12906) 
* mtmd : add more api around mtmd_image_tokens

* mtmd : ability to calc image hash

* shared_ptr for mtmd_image_tokens

* move hash to user-define ID (fixed)

* fix prompt_modified

* rm redundant data member
 2025-04-18 10:04:51 +02:00
Xuan-Son NguyenandGitHub 0c50923944 clip : use smart pointer (⚠️ breaking change) (#12869) 
* clip : use smart pointers

* fix warmup

* add forward declaration

* misisng include

* fix include (2)

* composite

* simplify batch ptr

* fix conflict
 2025-04-11 12:09:39 +02:00
Xuan-Son NguyenGitHubDaniel Han
 		5b1f13cb64 convert : proper tensor name mapping for llama4 (#12870) 
* Llama-4 mapping

* remove hacky renaming

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
 2025-04-11 09:23:37 +02:00
Xuan-Son NguyenandGitHub 8b91d5355a llama : correct rms norm for llama 4 (#12882) 2025-04-11 08:49:50 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		8b9cc7cdd8 llava : introduce libmtmd (#12849) 
* wip llava2

* migrated gemma3 to llava2

* add timings

* correct pre/postfix

* fix missing include

* fix compilation unused var warn

* update llava2_tokenize

* change name llava2 --> mtmd

* improve api

* refine helpers

* Update examples/llava/mtmd.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-04-10 22:57:16 +02:00
Xuan-Son NguyenGitHubFrancis Couture-Harpin
 		64eda5deb9 convert : ability to lazy-load safetensors remotely without downloading to disk (#12820) 
* gguf util : add SafetensorRemote

* fix style

* convert: add --remote option

* convert : allow using lazy remote tensors

It's a bit slow for now since everything is blocking and single-threaded.

* correct metadata.name

* small style fix

* support HF_TOKEN

* convert : use writeable buffer for remote lazy tensors

* convert : fix flake8 lint regarding lamdba assigment

* multithreaded download

* multithread: print debug

* fix style

* Revert "multithreaded download"

This reverts commit 42fc895ace385edc972ad819c76c704aeea61791.

* bring back _get_request_headers

---------

Co-authored-by: Francis Couture-Harpin <git@compilade.net>
 2025-04-10 17:24:44 +02:00
Xuan-Son NguyenandGitHub 65a69e6e1b clip : do not print ftype (#12832) 2025-04-09 10:09:53 +02:00
Xuan-Son NguyenandGitHub 78a1ba0a4f server : fix thread.join() on exit (#12831) 2025-04-08 18:37:06 +02:00
Xuan-Son NguyenandGitHub 1466621e73 llama : Support llama 4 text-only (#12791) 
* llama4 conversion

* initial support, no chat template

* clean up a bit

* fix tokenizer conversion

* correct hparams

* try this

* fix shexp

* ffn_inp_normed

* chat template

* clean up model conversion

* add_bos

* add scale_before_ffn

* fix order

* weight_before_ffn

* llm_graph_input_attn_temp

* add chunk attn mask

* build_inp_attn_scale()

* add comment about ggml_repeat

* clarify comments

* fix build
 2025-04-07 23:06:44 +02:00
Xuan-Son NguyenandGitHub e391d3ee8d ci : no curl on ggml-ci (#12796) 2025-04-07 15:37:28 +03:00
Xuan-Son NguyenandGitHub bd3f59f812 cmake : enable curl by default (#12761) 
* cmake : enable curl by default

* no curl if no examples

* fix build

* fix build-linux-cross

* add windows-setup-curl

* fix

* shell

* fix path

* fix windows-latest-cmake*

* run: include_directories

* LLAMA_RUN_EXTRA_LIBS

* sycl: no llama_curl

* no test-arg-parser on windows

* clarification

* try riscv64 / arm64

* windows: include libcurl inside release binary

* add msg

* fix mac / ios / android build

* will this fix xcode?

* try clearing the cache

* add bunch of licenses

* revert clear cache

* fix xcode

* fix xcode (2)

* fix typo
 2025-04-07 13:35:19 +02:00
Xuan-Son NguyenGitHubGeorgi Gerganov
 		0364178ca2 clip : refactor clip_init, add tests (#12757) 
* refactor clip_init

* fix loading file

* fix style

* test ok

* better test with report

* add missing headers

* clarify

* add KEY_MM_PATCH_MERGE_TYPE

* remove bool has_* pattern

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* Update examples/llava/clip.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* use ggml_soft_max_ext

* refactor logging system

* add minicpm-v-o 2.6 for testing

* use nullptr everywhere

* fix Yi-VL model

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-04-05 17:17:40 +02:00
Xuan-Son NguyenandGitHub 42eb248f46 common : remove json.hpp from common.cpp (#12697) 
* common : remove json.hpp from common.cpp

* fix comment
 2025-04-02 09:58:34 +02:00
Xuan-Son NguyenandGitHub 267c1399f1 common : refactor downloading system, handle mmproj with -hf option (#12694) 
* (wip) refactor downloading system [no ci]

* fix all examples

* fix mmproj with -hf

* gemma3: update readme

* only handle mmproj in llava example

* fix multi-shard download

* windows: fix problem with std::min and std::max

* fix 2
 2025-04-01 23:44:05 +02:00
Xuan-Son NguyenandGeorgi Gerganov 360dc22c00 cpu : rm unused variable (ggml/1166) 2025-03-30 08:33:31 +03:00
Xuan-Son NguyenandGitHub af6ae1efb2 llama : fix non-causal mask for gemma 3 (#12615) 2025-03-30 00:07:37 +01:00
Xuan-Son NguyenandGitHub fbdfefe74e llama : gemma3 : use output tensor if it exists in model weight (#12506) 
* llama : gemma3 : use output tensor if it exists in model weight

* also add to the llm_tensor_names
 2025-03-22 23:28:19 +01:00
Xuan-Son NguyenandGitHub 29fff308c7 llama : support converting Mistral Small text-only (#12450) 2025-03-18 19:16:19 +01:00
Xuan-Son NguyenandGitHub 99aa304fb9 llama : add support for EXAONE tied word embeddings (#12451) 2025-03-18 17:24:33 +01:00
Xuan-Son NguyenandGitHub be7c303410 arg : no n_predict = -2 for examples except for main and infill (#12364) 2025-03-13 12:34:54 +01:00
Xuan-Son NguyenandGitHub 7841fc723e llama : Add Gemma 3 support (+ experimental vision capability) (#12343) 
* llama : Add Gemma 3 text-only support

* fix python coding style

* fix compile on ubuntu

* python: fix style

* fix ubuntu compile

* fix build on ubuntu (again)

* fix ubuntu build, finally

* clip : Experimental support for Gemma 3 vision (#12344)

* clip : Experimental support for Gemma 3 vision

* fix build

* PRId64
 2025-03-12 09:30:24 +01:00