nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity

model : clean up and fix EXAONE-MoE configuration (#18840)

Browse Source 
* Fix mismatch of EXAONE-MoE configuration

* ensure gating func is set, cleanup

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
This commit is contained in:
Junwon Hwang
2026-01-14 19:38:21 +01:00
committed by GitHub
co-authored by Sigbjørn Skjæret
parent 516a4ca9b5
commit 8fb7175576
2 changed files with 3 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
convert_hf_to_gguf.py
+1 -5
View File
@@ -8772,11 +8772,7 @@ class ExaoneMoEModel(Exaone4Model):
self.gguf_writer.add_expert_weights_norm(self.hparams["norm_topk_prob"])
n_dense_layer = self.hparams.get("first_k_dense_replace", self.hparams.get("first_last_k_dense_replace", 0))
self.gguf_writer.add_leading_dense_block_count(n_dense_layer)
# For here, we hard-code the number of NextN/MTP layers to 1 for K-EXAONE,
# so that we can convert MTP weights to GGUF format for speculative decoding.
# This is because HF config of K-EXAONE does not have `num_nextn_predict_layers` at now.
# Will be updated when HF config is updated.
self.gguf_writer.add_nextn_predict_layers(self.hparams.get("num_nextn_predict_layers", 1))
self.gguf_writer.add_nextn_predict_layers(self.hparams.get("num_nextn_predict_layers", 0))
self.gguf_writer.add_rope_scaling_type(gguf.RopeScalingType.NONE)