nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 9 Packages Projects Releases Wiki Activity

mtmd: Add DeepSeekOCR 2 Support (#20975)

Browse Source 
* mtmd: DeepSeek-OCR 2 support, with multi-tile dynamic resolution

* introduced clip_image_f32::add_viewsep

* address PR review

- drop redundant ggml_cpy ops in both deepseekocr versions build
- drop no-op ggml_cont in build_sam
- assert num_image_tokens deepseekocr2
- view_seperator as (1, n_embd) at conversion (for both versions)
- drop redundant ggml_reshape_2d

* Update tools/mtmd/models/deepseekocr2.cpp

Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>

---------

Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
This commit is contained in:
Saba Fallah
2026-05-29 16:13:51 +02:00
committed by GitHub
co-authored by Xuan-Son Nguyen
parent 6ed481eea4
commit da3f990a47
16 changed files with 505 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/mtmd/models/models.h
+5
View File
@@ -121,6 +121,11 @@ struct clip_graph_deepseekocr : clip_graph {
ggml_tensor * build_sam(ggml_tensor * inp); // build the SAM model
};
struct clip_graph_deepseekocr2 : clip_graph_deepseekocr {
clip_graph_deepseekocr2(clip_ctx * ctx, const clip_image_f32 & img) : clip_graph_deepseekocr(ctx, img) {}
ggml_cgraph * build() override; // reuses build_sam() from base
};
struct clip_graph_conformer : clip_graph {
clip_graph_conformer(clip_ctx * ctx, const clip_image_f32 & img) : clip_graph(ctx, img) {}
ggml_cgraph * build() override;