nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 23 Packages Projects Releases Wiki Activity
4,095 Commits 1 Branch 0 Tags
89e4caaaf081f4712af61a3e08cb67b406c02b80
Commit Graph
4095 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuan Son NguyenandGitHub d4c3c10fad lora : raise error if lm_head is ignored (#9103) 
* lora : raise error if lm_head is ignored

* fix style

* clarify comment
 2024-09-12 14:33:57 +03:00
Michael PodvitskiyandGitHub 2a825116b6 cmake : fix for builds without GGML_CDEF_PUBLIC (#9338) 
* `GGML_TARGET_DEFINES-NOTFOUND` fix for builds without `GGML_CDEF_PUBLIC`

* Update CMakeLists.txt, spaces fix
 2024-09-12 14:30:01 +03:00
Huang QiandGitHub 4dc4f5f14a ci : update HIP SDK to 24.Q3 (ROCm 6.1) (#9329) 2024-09-12 14:28:43 +03:00
daminhoandGitHub c837981bba py : add Phi-1.5/Phi-2 tokenizer (#9361) 
* add phi2 tokenizer

* add phi name to convert_hf_to_gguf_update.py

* make tokenizer_pre consistent; llama.cpp work
 2024-09-12 14:28:20 +03:00
Trivikram KamatandGitHub 3c26a1644d ci : bump actions/checkout to v4 (#9377) 2024-09-12 14:27:45 +03:00
Michael PodvitskiyandGitHub ff76e18516 cmake : fixed the order of linking libraries for llama-quantize (#9450) 2024-09-12 14:27:14 +03:00
Molly SophiaandGitHub 39f852f440 py : add special tokens in hf_converter for RWKV v6 (#9428) 
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
 2024-09-12 14:25:16 +03:00
Ahmad TameemandGitHub 2b00fa7997 riscv : modify Makefile and add a RISCV_VECT to print log info (#9442) 
- Added ggml_cpu_has_riscv_v() in GGML to print system info in log
- Modified Makefile to only use flag when cross compiling for RISC-V
 2024-09-12 14:24:31 +03:00
Georgi GerganovandGitHub d6a04f872d ggml : hide ggml_object, ggml_cgraph, ggml_hash_set (#9408) 
* ggml : hide ggml_object, ggml_cgraph, ggml_hash_set

ggml-ci

* ggml : add ggml-impl.h to backends

* ggml : fix compiler warnings

ggml-ci

* ggml : add assert upon adding nodes
 2024-09-12 14:23:49 +03:00
Neo Zhang JianyuGitHubarthw
 		c9c8575a1a enhance run script to be easy to change the parameters (#9448) 
Co-authored-by: arthw <14088817+arthw@users.noreply.github.com>
 2024-09-12 17:44:17 +08:00
Xinpeng DouandGitHub df4b7945ae cann: Fix error when running a non-exist op (#9424) 2024-09-12 09:02:35 +08:00
Faisal ZaghloulGitHubfmz
 		449ccfb6f5 Add Jais to list of supported models (#9439) 
Co-authored-by: fmz <quic_fzaghlou@quic.com>
 2024-09-12 02:29:53 +02:00
slarenandGitHub 1b28061400 llama : skip token bounds check when evaluating embeddings (#9437) 2024-09-11 17:52:13 +02:00
Pavel ZloiandGitHub 8db003a19d py : support converting local models (#7547) 
* Support of converting local models added to convert-hf-to-gguf-update.py

* Description fixed

* shutil added to imports
 2024-09-11 15:29:51 +03:00
Xuan Son NguyenandGitHub 0996c5597f llava : correct args for minicpmv-cli (#9429) 2024-09-11 12:59:13 +02:00
Xuan Son NguyenandGitHub 5bb2c5dbd2 files : remove accidentally added lora_test submodule (#9430) 2024-09-11 13:02:09 +03:00
Farbod BijaryGitHubXuan Son NguyenfarbodslarenXuan Son Nguyen
 		67155ab7f5 feat: Implements retrying logic for downloading models using --model-url flag (#9255) 
* feat: Implements retrying logic for downloading models using --model-url flag

* Update common/common.cpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* Update common/common.cpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* apply comments

* implements a retry function to avoid duplication

* fix editorconfig

* change function name

---------

Co-authored-by: farbod <farbod.bjary82@gmail.com>
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
Co-authored-by: slaren <slarengh@gmail.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2024-09-11 11:22:37 +02:00
Johannes GäßlerandGitHub 5af118efda CUDA: fix --split-mode row race condition (#9413) 2024-09-11 10:22:40 +02:00
Georgi GerganovandGitHub d2b496bff4 batched-bench : remove unused code (#9305) 2024-09-11 10:03:54 +03:00
R0CKSTARandGitHub b34e023480 musa: remove Clang builtins mapping (#9421) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2024-09-11 03:46:55 +02:00
Alberto Cabrera PérezandGitHub 51b6038636 sycl : update support conditions (#9394) 
* sycl : update support condition to im2col

Signed-off-by: Alberto Cabrera <alberto.cabrera@codeplay.com>

* Added TODO to remind supporting FP32 im2col

---------

Signed-off-by: Alberto Cabrera <alberto.cabrera@codeplay.com>
 2024-09-11 08:53:42 +08:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		cb9c933eb2 flake.lock: Update (#9360) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/af510d4a62d071ea13925ce41c95e3dec816c01d?narHash=sha256-ODYRm8zHfLTH3soTFWE452ydPYz2iTvr9T8ftDMUQ3E%3D' (2024-08-30)
  → 'github:hercules-ci/flake-parts/567b938d64d4b4112ee253b9274472dc3a346eb6?narHash=sha256-%2Bebgonl3NbiKD2UD0x4BszCZQ6sTfL4xioaM49o5B3Y%3D' (2024-09-01)
• Updated input 'flake-parts/nixpkgs-lib':
    'https://github.com/NixOS/nixpkgs/archive/a5d394176e64ab29c852d03346c1fc9b0b7d33eb.tar.gz?narHash=sha256-uFf2QeW7eAHlYXuDktm9c25OxOyCoUOQmh5SZ9amE5Q%3D' (2024-08-01)
  → 'https://github.com/NixOS/nixpkgs/archive/356624c12086a18f2ea2825fed34523d60ccc4e3.tar.gz?narHash=sha256-Ss8QWLXdr2JCBPcYChJhz4xJm%2Bh/xjl4G0c0XlP6a74%3D' (2024-09-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/71e91c409d1e654808b2621f28a327acfdad8dc2?narHash=sha256-GnR7/ibgIH1vhoy8cYdmXE6iyZqKqFxQSVkFgosBh6w%3D' (2024-08-28)
  → 'github:NixOS/nixpkgs/574d1eac1c200690e27b8eb4e24887f8df7ac27c?narHash=sha256-v3rIhsJBOMLR8e/RNWxr828tB%2BWywYIoajrZKFM%2B0Gg%3D' (2024-09-06)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-10 15:46:59 -07:00
Xuan Son NguyenandGitHub 6cd4e03444 arg : bring back missing ifdef (#9411) 
* arg : bring back missing ifdef

* replace with llama_supports_gpu_offload
 2024-09-10 22:41:29 +02:00
matteoGitHubmatteo serva
 		8d300bd35f enable --special arg for llama-server (#9419) 
Co-authored-by: matteo serva <matteo.serva@gmail.com>
 2024-09-10 22:40:59 +02:00
slarenandGitHub 49006c67b4 llama : move random seed generation to the samplers (#9398) 
* llama_sampler_penalties : clamp penalty_last_n to zero
 2024-09-10 18:04:25 +02:00
Georgi Gerganov 00ba2ff781 metal : fix compile warning with GGML_METAL_NDEBUG (#0) 2024-09-10 10:17:43 +03:00
Daniel BeveniusandGitHub 83008b7cfe llama : update llm_build_copy_mask_state comment [no ci] (#9385) 
This commit updates the comment, which seems to contain a typo or be an
outdated comment, in the copy_mask_state function changing the variable
n_rs to n_kv.

I believe this change is correct and what the comment wants to
convey is to copy the states that are not going to be used in the
upcoming processing, which are the tokens states from n_seqs up to
the number of possible token states n_kv.
 2024-09-10 10:03:21 +03:00
Molly SophiaandGitHub 0b4ac75772 RWKV v6: Add time_mix_decay_w1/w2 in quant exclusion list (#9387) 
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
 2024-09-10 10:02:30 +03:00
slarenandGitHub fb3f249815 make : do not run llama-gen-docs when building (#9399) 2024-09-10 09:23:33 +03:00
Xuan Son NguyenGitHubGeorgi Gerganov
 		bfe76d4a17 common : move arg parser code to arg.cpp (#9388) 
* common : move arg parser to arg.cpp

* better categorize args

* add cmake

* missing climits

* missing cstdarg

* common : more explicit includes

* fix build

* refactor gpt_params_parse

* update server readme

* fix test

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-09-09 23:36:09 +02:00
Radoslav GerganovGitHubslaren
 		293bebe077 rpc : fix segfault with nkvo (#9389) 
* rpc : fix nkvo

* rpc : buf_size must not be static

ref: #9337

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-09-09 18:40:10 +03:00
Prashant VithuleGitHubGeorgi Gerganov
 		5fac4d5764 ggml : vector length agnostic SVE support (#9290) 
* Implemented vector length agnostic SVE using switch case for 512-bit, 256-bit, 128-bit vector lengths

* Implemented vector length agnostic SVE using switch case for 512-bit, 256-bit, 128-bit vector lengths

* Removed WhiteSpaces

* ggml : style changes + fix 512-bit nb loop check

- fix local scope in switch cases
- consistent predicate names
- empty lines when necessary
- opening braces, spaces
- const-correctness
- add asserts

* Update ggml/src/ggml-quants.c

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-09-09 18:37:18 +03:00
slarenandGitHub 5fb5e24811 llama : minor sampling refactor (2) (#9386) 2024-09-09 17:10:46 +02:00
Georgi GerganovandGitHub 38ca6f644b readme : update hot topics 2024-09-09 15:51:37 +03:00
Johannes GäßlerandGitHub 8e6e2fbe14 CUDA: fix variable name conflict for Windows build (#9382) 2024-09-09 14:22:53 +02:00
Antonis MakropoulosandGitHub 5ed087573e readme : add LLMUnity to UI projects (#9381) 
* add LLMUnity to UI projects

* add newline to examples/rpc/README.md to fix editorconfig-checker unit test
 2024-09-09 14:21:38 +03:00
Radoslav GerganovandGitHub 54f376d0b9 rpc : update README [no ci] (#9320) 
Update README with instructions how to offload model layers to both
local and remote devices
 2024-09-09 11:04:39 +03:00
Dan JohanssonandGitHub b2e89a3274 Arm AArch64: Documentation updates (#9321) 
* Arm AArch64: Documentation updates

* Update docs/build.md to include information on how to enable the Arm optimized gemm/gemv kernels

* Update examples/quantize/README.md with information on the Q4_0_4_4, Q4_0_4_8 and Q4_0_8_8 formats

* Add newline to the end of docs/build.md
 2024-09-09 10:02:45 +03:00
Markus TavenrathandGitHub daa9623ab0 Overlap cmdbuffer creation and cmdbuffer execution in Vulkan backend by submitting smaller cmdbuffers early. (#9118) 
* Overlap cmdbuffer creation and cmdbuffer execution in Vulkan backend by submitting smaller cmdbuffers early.

* fix compile issues

* Fix issues where the last submit wasn't executed or handled properly.

* remove trailing whitespace

* Repair GGML_VULKAN_CHECK_RESULTS

* Increase submit counter only if actual work has been submitted and increase submit count to 100.

* Fix some nodes are not checked with GGML_VULKAN_CHECK_RESULTS enabled.
 2024-09-08 21:43:48 +02:00
Georgi GerganovandGitHub e079bffb66 cuda : fix FA Q src index (1 -> 0) (#9374) 2024-09-08 22:01:02 +03:00
Xuan Son NguyenandGitHub 3f7ccfd649 common : bring back missing args, add env var duplication check (#9375) 
* common : bring back missing args

* move duplication check to test-arg-parser

* add check for duplicated env var

* correct default values
 2024-09-08 18:08:55 +02:00
slarenandGitHub a249843d89 common : restore --n-gpu-layers (#9371) 2024-09-08 16:44:42 +02:00
slarenandGitHub 19f4a7b296 llama : refactor samplers internal implementation (#9370) 2024-09-08 15:52:07 +02:00
Neo Zhang JianyuGitHubarthw
 		2a358fb0c4 [SYCL] add check malloc result on device (#9346) 
* add check malloc result on device

* update for review comments, check all malloc_device() result

---------

Co-authored-by: arthw <14088817+arthw@users.noreply.github.com>
 2024-09-08 19:05:29 +08:00
slarenandGitHub eae597182c llama : sanitize tokens in the upper bound (#9359) 2024-09-08 12:41:51 +02:00
Xuan Son NguyenandGitHub 00b02bb249 imatrix : fix arg parser for imatrix (#9366) 
* imatrix : fix arg parser

* beautify printing first arg
 2024-09-08 12:12:17 +02:00
Georgi Gerganov a876861455 metal : update support condition for im2col + fix warning (#0) 2024-09-08 11:05:55 +03:00
Georgi Gerganov 385decbd63 sync : ggml 2024-09-08 11:05:55 +03:00
Georgi Gerganov 60a3107ccd scripts : option to increase git patch context 2024-09-08 11:05:55 +03:00
Salvatore MesoracaandGeorgi Gerganov 406c1a32a1 vulkan: add dryrun support to sin and cos ops (ggml/947) 
sin and cos failed test-backend-ops because they
tried to dereference a context pointer that is null
on dry runs.

This commit prevents that segfault.

Signed-off-by: Salvatore Mesoraca <s.mesoraca16@gmail.com>
 2024-09-08 11:05:55 +03:00