model : full modern bert support (#18330)

* full modern bert support

* added gelu op in rank pooling for modern bert

* still working on stuff, added mean calculation before classifier head

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* first layer is dense, as per modern bert research paper

* Update src/llama-graph.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* fixed set input for mean pooling to check if pooling type is ranking since modern bert does mean & rank

* Update src/llama-graph.cpp

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

This commit is contained in:

Ryan Mangeno

2026-02-19 08:52:21 +01:00

committed by

GitHub

co-authored by

Sigbjørn Skjæret

parent 3bb2fcc856

commit c0d0430340

12 changed files with 54 additions and 22 deletions

									
										src/llama-arch.h
									
		+1
		
												View File
												
				@@ -497,6 +497,7 @@ enum llm_tensor {

				    LLM_TENSOR_ENC_OUTPUT_NORM,

				    LLM_TENSOR_CLS,

				    LLM_TENSOR_CLS_OUT,

				    LLM_TENSOR_CLS_NORM,

				    LLM_TENSOR_CONV1D,

				    LLM_TENSOR_CONVNEXT_DW,

				    LLM_TENSOR_CONVNEXT_NORM,