nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 31 Packages Projects Releases Wiki Activity

mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos (breaking change) (#22082)

Browse Source 
* mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos

* fix build
This commit is contained in:
Xuan-Son Nguyen
2026-04-19 11:57:21 +02:00
committed by GitHub
parent bcdcc1044f
commit 19124078be
5 changed files with 22 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/mtmd/mtmd.h
+3 -1
View File
@@ -196,11 +196,13 @@ struct mtmd_decoder_pos {
uint32_t t;
uint32_t x;
uint32_t y;
uint32_t z; // unused for now, reserved for future use
};
// get position for decoder attention, to be used by M-RoPE models
// i is the index of the embedding token, ranging from 0 to mtmd_image_tokens_get_n_tokens() - 1
// pos_0 is the absolute position of the first token
// return relative position (for example, embedding 0 will have position (0, 0, 0); remember to adjust it to the current absolute position)
MTMD_API struct mtmd_decoder_pos mtmd_image_tokens_get_decoder_pos(const mtmd_image_tokens * image_tokens, size_t i);
MTMD_API struct mtmd_decoder_pos mtmd_image_tokens_get_decoder_pos(const mtmd_image_tokens * image_tokens, llama_pos pos_0, size_t i);
// tokenize an input text prompt and a list of bitmaps (images/audio)
// the prompt must have the input image marker (default: "<__media__>") in it