 Molly SophiaandGitHub
|
7dfad387e3
|
llama: Add support for RWKV v7 architecture (#12412)
* ggml: Add op l2_norm
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* ggml: Add op rwkv_wkv7
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: Add support for RWKV7 and ARWKV7 models
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: fix inference with RWKV6Qwen2
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: add more (a)rwkv7 variants in size
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Apply code-format changes
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* fix MUSA build
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: fix shape error with rwkv using llama-parallel
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
---------
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2025-03-18 07:27:50 +08:00 |
|
|
Molly SophiaandGitHub
|
393fca629e
|
ggml-cpu: Fix build with sve (#12059)
* ggml-cpu: Fix build with sve
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* ggml-cpu: Remove unused variable in sve q3_k vec dot
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
---------
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2025-02-25 19:28:22 +08:00 |
|
|
Molly SophiaandGitHub
|
325afb370a
|
llama: fix missing k_cache store for rwkv6qwen2 (#11445)
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2025-01-29 12:07:21 +08:00 |
|
|
|
ee7136c6d1
|
llama: add support for QRWKV6 model architecture (#11001)
llama: add support for QRWKV6 model architecture (#11001)
* WIP: Add support for RWKV6Qwen2
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* RWKV: Some graph simplification
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Add support for RWKV6Qwen2 with cpu and cuda GLA
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* RWKV6[QWEN2]: Concat lerp weights together to reduce cpu overhead
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Fix some typos
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* code format changes
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Fix wkv test & add gla test
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Fix cuda warning
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Update README.md
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Update ggml/src/ggml-cuda/gla.cu
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* Fix fused lerp weights loading with RWKV6
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* better sanity check skipping for QRWKV6 in llama-quant
thanks @compilade
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
Co-authored-by: compilade <git@compilade.net>
---------
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: compilade <git@compilade.net>
|
2025-01-10 09:58:08 +08:00 |
|
|
|
4b0c638b9a
|
common : disable KV cache shifting automatically for unsupported models (#11053)
* Disable KV cache shifting automatically for unsupported models
instead of exiting directly
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Update common/common.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
---------
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
|
2025-01-03 14:13:18 +02:00 |
|
|
Molly SophiaandGitHub
|
0a11f8b7b5
|
convert : fix RWKV v6 model conversion (#10913)
* Enable --no-context-shift for llama-perplexity example
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* RWKV 6: Fix error in ggml_cuda_op_bin_bcast
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
---------
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2024-12-20 11:44:58 +02:00 |
|
|
|
11d47057a5
|
Rwkv chat template fix (#10001)
* llama: remove useless template matching for rwkv-world
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* converter: Add comment about the hack for rwkv models
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Update src/llama.cpp
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
---------
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
|
2024-10-22 15:22:26 +02:00 |
|
|
Molly SophiaandGitHub
|
4ff7fe1fb3
|
llama : add chat template for RWKV-World + fix EOT (#9968)
* Add chat template for RWKV-World
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* RWKV: Fix the chat template not being used
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* RWKV v6: Set EOT token to ``\n\n``
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* readme: add rwkv into supported model list
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
---------
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2024-10-22 13:33:37 +03:00 |
|
|
Molly SophiaandGitHub
|
10433e8b45
|
llama : add tensor name for "result_norm" (#9907)
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2024-10-16 13:10:21 +03:00 |
|
|
Molly SophiaandGitHub
|
912c331d3d
|
Fix merge error in #9454 (#9589)
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2024-09-22 15:26:50 +02:00 |
|
|
Molly SophiaandGitHub
|
2a63caaa69
|
RWKV v6: RWKV_WKV op CUDA implementation (#9454)
* ggml: CUDA unary op EXP
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* ggml: rwkv_wkv op CUDA impl
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
---------
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2024-09-22 04:29:12 +02:00 |
|
|
Molly SophiaandGitHub
|
39f852f440
|
py : add special tokens in hf_converter for RWKV v6 (#9428)
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2024-09-12 14:25:16 +03:00 |
|
|
Molly SophiaandGitHub
|
0b4ac75772
|
RWKV v6: Add time_mix_decay_w1/w2 in quant exclusion list (#9387)
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2024-09-10 10:02:30 +03:00 |
|
|
|
8f1d81a0b6
|
llama : support RWKV v6 models (#8980)
* convert_hf_to_gguf: Add support for RWKV v6
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Add RWKV tokenization
* Fix build
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Do not use special tokens when matching in RWKV tokenizer
* Fix model loading
* Add (broken) placeholder graph builder for RWKV
* Add workaround for kv cache
* Add logits conversion to rwkv5
* Add rwkv5 layer norms
* Add time mix KVRG & correct merge mistake
* Add remaining time mix parameters
* Add time mix output loading
* Add placeholder llm_build_time_mix
* Fix build
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Load more tensors for rwkv v6
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Fix rwkv tokenizer
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* ggml: Add unary operator Exp
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* RWKV v6 graph building
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Add ``rescale_every_n_layers`` parameter
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Add ``wkv.head_size`` key for RWKV
so it doesn't reuse Mamba ssm parameters
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Fix offloading layers to CUDA
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Fix parallel inferencing for RWKV
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Remove trailing whitespaces
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* build_rwkv: Avoid using inplace operations
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* convert_hf_to_gguf: rwkv: Avoid using ``eval``
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* convert_hf_to_gguf: rwkv tokenizer: Don't escape sequences manually
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Update convert_hf_to_gguf.py
Co-authored-by: compilade <git@compilade.net>
* ggml: Add backward computation for unary op ``exp``
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Update convert_hf_to_gguf.py
Co-authored-by: compilade <git@compilade.net>
* Update convert_hf_to_gguf.py
Co-authored-by: compilade <git@compilade.net>
* Use MODEL_ARCH.RWKV6 instead of MODEL_ARCH.RWKV
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* build_rwkv6: Simplify graph
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Detect model.type
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Fix tensor loading for 7B/14B models
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Fix group_norm assertion failure with Metal
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Clean up
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Add quantization tensor exclusion
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Use the new advanced batch splits
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* Update src/llama.cpp
Co-authored-by: compilade <git@compilade.net>
* llama: rwkv6: Use ``ggml_norm`` instead of ``ggml_group_norm``
Co-authored-by: compilade <git@compilade.net>
* llama: rwkv6: Apply code style and misc changes
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* converter: Use class name ``Rwkv6Model``
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Make use of key ``feed_forward_length``
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Add kv ``time_mix_extra_dim`` and ``time_decay_extra_dim``
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* converter: Match ``new_name`` instead of ``name`` for float32 explicit tensors
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Keep ``time_mix_w1/w2`` as F32
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Remove unused nodes
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Apply code format changes
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* llama: rwkv6: Add lora for some supported tensors
Currently att.key/receptance/value/gate/output, ffn.receptance/key/value, as well as head.weight
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
* rwkv : speed-up tokenization using trie
* minor : style + indentation
* llama: rwkv6: Avoid division by zero
Co-authored-by: compilade <git@compilade.net>
* ggml: rwkv_wkv: Avoid copying the state
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
---------
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
Co-authored-by: Layl Bongers <3094382+LaylBongers@users.noreply.github.com>
Co-authored-by: compilade <git@compilade.net>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
|
2024-09-01 17:38:17 +03:00 |
|
|
Molly SophiaandGitHub
|
2d5dd7bb3f
|
ggml : add epsilon as a parameter for group_norm (#8818)
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
|
2024-08-06 10:26:46 +03:00 |
|