o7si and GitHub
32eddaf2ea
cmake : fix ui build with read-only source ( #24752 )
2026-06-18 18:59:18 +02:00
o7si and GitHub
1bfbdb134e
vocab : adopt leading TemplateProcessing special token as BOS ( #24428 )
2026-06-11 10:37:23 +03:00
68f30663cf
vocab : refactor normalizer flags into options struct, add strip_accents ( #24371 )
...
* vocab : refactor normalizer flags into options struct, add strip_accents
* Update src/llama-vocab.h
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update src/llama-vocab.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-06-11 10:36:50 +03:00
o7si and GitHub
5aba5364d9
vocab: add normalizer.lowercase support to WPM ( #23899 )
...
* vocab : add jina-embeddings-v2-base-zh (whitespace tokenizer)
* vocab : add normalizer.lowercase support to WPM
* vocab : default normalizer.lowercase to false for whitespace pre-tokenizer
2026-06-01 14:26:47 +03:00
d4c8e2c29c
vocab : add tokenizer support for jina-embeddings-v2-base-zh ( #18756 )
...
* vocab : add jina-embeddings-v2-base-zh (whitespace tokenizer)
* lowercase defaults to true
* type fix
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-05-31 12:37:35 +02:00
d0a6a31470
model : add support for JinaBertModel with non-gated ffn ( #18475 )
...
* WIP: Initial commit for fixing JinaBert original FF type support
* convert: add jina-v2-de tokenizer variant for German_Semantic_V3
* convert: fix token collision in BERT phantom vocab conversion
* convert: add feed_forward_type metadata
* model: add feed_forward_type metadata for jina-bert-v2
* model: jina-bert-v2 support standard GELU FFN variant
* model: remove ffn_type, detect FFN variant from tensor dimensions
* Update src/llama-model.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update src/llama-model.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update src/models/bert.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update src/models/bert.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* revert collision fix to be handled in separate PR
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-01-01 18:38:51 +01:00
2b2afade9f
convert : fix encoding of WPM vocab for BERT models ( #18500 )
...
* convert: avoid token collision when stripping ## prefix
* convert: use token types for BERT special tokens check
* Update convert_hf_to_gguf.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-01-01 18:27:07 +01:00
o7si and GitHub
daa242dfc8
common: fix return value check for setpriority ( #18412 )
...
* common: fix return value check for setpriority
* tools: add logging for process priority setting
2025-12-29 11:07:49 +02:00
o7si and GitHub
60f17f56da
rpc: fix segfault on invalid endpoint format ( #18387 )
...
* rpc: fix segfault on invalid endpoint format
* rpc: add error log for failed endpoint connection
2025-12-28 12:34:41 +02:00
o7si and GitHub
4893cc07bb
server : fix crash when seq_rm fails for hybrid/recurrent models ( #18391 )
...
* server : fix crash when seq_rm fails for hybrid/recurrent models
* server : add allow_processing param to clear_slot
2025-12-26 16:35:29 +01:00
o7si and GitHub
3ce7a65c2f
server: fix: /metrics endpoint returning JSON-escaped Prometheus format ( #17386 )
...
* fix: /metrics endpoint returning JSON-escaped Prometheus format
* mod: remove string overload from ok() method
2025-11-28 19:14:00 +01:00
o7si and GitHub
f3a848a3b1
chore: upgrade cpp-httplib from v0.27.0 to v0.28.0 ( #17513 )
2025-11-26 09:21:06 +02:00
5088b435d4
convert : fix TypeError when loading base model remotely in convert_lora_to_gguf ( #17385 )
...
* fix: TypeError when loading base model remotely in convert_lora_to_gguf
* refactor: simplify base model loading using cache_dir from HuggingFace
* Update convert_lora_to_gguf.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* feat: add remote_hf_model_id to trigger lazy mode in LoRA converter
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2025-11-20 12:30:12 +01:00
o7si and GitHub
97cb3fd5ae
fix: resolve undefined variable 'svr' compilation error ( #17348 )
2025-11-18 10:10:47 +02:00
o7si and GitHub
ffb6f3d921
vocab : correct bounds check for UGM XCDA array access ( #17215 )
2025-11-12 23:41:02 +01:00
