model: support MiMo-V2-Flash (#18328)

* mimov2: convert ok

* rename mimov2 --> mimo2

* fix conversion

* runnable not incorrect

* use sink

* add_sliding_window_pattern

* add swa and per-layer n_head_kv

* correct params

* somewhat working

* correct gating func

* nits

* mimo2: wire RMS eps + MoE bias + converter guards

* add co-author

Co-authored-by: Aaryan-Kapoor <Aaryan-Kapoor@users.noreply.github.com>

* use add_rope_freq_base_swa

---------

Co-authored-by: Aaryan Kapoor <aaryankapoor2006@gmail.com>
Co-authored-by: Aaryan-Kapoor <Aaryan-Kapoor@users.noreply.github.com>
This commit is contained in:
Xuan-Son Nguyen
2025-12-24 23:07:08 +01:00
committed by GitHub
co-authored by Aaryan-Kapoor Aaryan Kapoor
parent c184284230
commit 4cbafad4f0
11 changed files with 324 additions and 4 deletions
+1
View File
@@ -320,6 +320,7 @@ class TensorNameMap:
MODEL_TENSOR.ATTN_SINKS: (
"model.layers.{bid}.self_attn.sinks", # openai-moe
"model.layers.{bid}.self_attn.attention_sink_bias", # mimov2
),
MODEL_TENSOR.ATTN_GATE: (