nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 35 Packages Projects Releases Wiki Activity

mtmd: add mtmd_image_tokens_get_decoder_pos() API (#21851)

Browse Source 
* mtmd: add mtmd_image_tokens_get_decoder_pos() API

* consistent naming

* fix build
This commit is contained in:
Xuan-Son Nguyen
2026-04-14 16:07:41 +02:00
committed by GitHub
parent 1f30ac0cea
commit 707c0b7a6e
5 changed files with 49 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/mtmd/mtmd-helper.h
+4
View File
@@ -47,6 +47,10 @@ MTMD_API size_t mtmd_helper_get_n_tokens(const mtmd_input_chunks * chunks);
// normally, n_pos is equal to n_tokens, but for M-RoPE it is different
MTMD_API llama_pos mtmd_helper_get_n_pos(const mtmd_input_chunks * chunks);
// helper to get the list of relative positions corresponding to the embedding tokens, to be used by M-RoPE
// out_pos must have length == mtmd_helper_get_n_tokens(image)
MTMD_API void mtmd_helper_image_get_decoder_pos(const mtmd_image_tokens * image, mtmd_decoder_pos * out_pos);
// helper function that automatically:
// 1. run llama_decode() on text chunks
// 2. run mtmd_encode() on image chunks, then mtmd_get_output_embd() and then llama_decode()