model: support MiMo-V2-Flash (#18328)

* mimov2: convert ok

* rename mimov2 --> mimo2

* fix conversion

* runnable not incorrect

* use sink

* add_sliding_window_pattern

* add swa and per-layer n_head_kv

* correct params

* somewhat working

* correct gating func

* nits

* mimo2: wire RMS eps + MoE bias + converter guards

* add co-author

Co-authored-by: Aaryan-Kapoor <Aaryan-Kapoor@users.noreply.github.com>

* use add_rope_freq_base_swa

---------

Co-authored-by: Aaryan Kapoor <aaryankapoor2006@gmail.com>
Co-authored-by: Aaryan-Kapoor <Aaryan-Kapoor@users.noreply.github.com>

This commit is contained in:

Xuan-Son Nguyen

2025-12-24 23:07:08 +01:00

committed by

GitHub

co-authored by

Aaryan-Kapoor

Aaryan Kapoor

parent c184284230

commit 4cbafad4f0

11 changed files with 324 additions and 4 deletions

									
										src/llama-arch.h
									
		+1
		
												View File
												
				@@ -119,6 +119,7 @@ enum llm_arch {

				    LLM_ARCH_RND1,

				    LLM_ARCH_PANGU_EMBED,

				    LLM_ARCH_MISTRAL3,

				    LLM_ARCH_MIMO2,

				    LLM_ARCH_LLAMA_EMBED,

				    LLM_ARCH_UNKNOWN,

				};