nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity
528 Commits 1 Branch 0 Tags
b608b55a3ea8e4760c617418538465449175bdb8
Commit Graph
30 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Stephan WalterandGitHub f0d70f147d Various fixes to mat_mul benchmark (#1253) 2023-04-30 12:32:37 +00:00
Stephan WalterandGitHub 36d19a603b Remove Q4_3 which is no better than Q5 (#1218) 2023-04-28 23:10:43 +00:00
Stephan WalterandGitHub 0b2da20538 ggml : slightly faster AVX2 implementation for Q5 (#1197) 2023-04-26 23:26:42 +03:00
Stephan WalterGitHubPavol Rusnak
 		4afcc37869 Update SHA256SUMS after quantization change (#1181) 
Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
 2023-04-25 23:41:56 +02:00
Stephan WalterandGitHub 2ec83428de Fix build for gcc 8 and test in CI (#1154) 2023-04-24 15:38:26 +00:00
Stephan WalterandGitHub 53c8434398 Improve AVX2 for vec_dot_q4_3_q8_0 (#1138) 2023-04-23 11:01:03 +00:00
Stephan WalterandGitHub 857308d1e8 ci : trigger CI for drafts, but not most PR actions (#1125) 2023-04-22 16:12:29 +03:00
Stephan WalterandGitHub c50b628810 Fix CI: ARM NEON, quantization unit tests, editorconfig (#1122) 2023-04-22 10:54:13 +00:00
Stephan WalterandGitHub c5aa5e5777 ggml : AVX2 optimization for vec_dot_q4_3_q8_0 and refactoring (#1099) 
* AVX2 optimization for vec_dot_q4_3_q8_0 and refactoring

* finish AVX vectorization of quantize_row_q8_0

* Rename hsum_int_8 to hsum_i32_8
 2023-04-22 10:37:05 +03:00
Stephan WalterGitHubPavol Rusnak
 		2510c1831f Add ggml-model-*.bin checksums for 7B, 13B, 30B, 65B (#1088) 
* Add ggml-model-*.bin checksums for 7B, 13B, 30B
* Add ggml-model-*.bin checksums for 65B

---------

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
 2023-04-20 23:56:44 +02:00
Stephan WalterandGitHub c8c2c52482 AVX2 optimization for vec_dot_q4_2_q8_0 (#1068) 2023-04-20 08:45:41 +02:00
Stephan WalterandGitHub f3d4edf504 ggml : Q4 cleanup - remove 4-bit dot product code (#1061) 
* Q4 cleanup

* Remove unused AVX512 Q4_0 code
 2023-04-19 19:06:37 +03:00
Stephan WalterandGitHub 2f7c8e014e Fix potential int8 overflow in non-SIMD vec_dot (#986) 2023-04-15 18:28:56 +00:00
Stephan WalterandGitHub 0ad964631f Refactor ggml.c for future tensor types (#1001) 2023-04-15 16:25:38 +00:00
Stephan WalterandGitHub 93265e988a make : fix dependencies, use auto variables (#983) 2023-04-14 22:39:48 +03:00
Stephan WalterandGitHub 6232f2d7fd ggml : optimize non-SIMD Q4_0 vector dot product (#703) 2023-04-13 17:59:50 +03:00
Stephan WalterandGitHub e7f6997f89 Don't crash on ftype (formerly f16) == 4 (#917) 2023-04-12 15:06:16 +00:00
Stephan WalterandGitHub 3e6e70d8e8 Add enum llama_ftype, sync ggml_type to model files (#709) 2023-04-11 15:03:51 +00:00
Stephan WalterandGitHub 81040f10aa llama : do not allocate KV cache for "vocab_only == true" (#682) 
Fixes sanitizer CI
 2023-04-02 10:18:53 +03:00
Stephan WalterandGitHub 3525899277 Enable -std= for cmake builds, fix warnings (#598) 2023-03-31 19:19:16 +00:00
Stephan WalterandGitHub 3bcc129ba8 cmake : properly invoke CTest (#629) 2023-03-30 20:56:59 +03:00
Stephan WalterGitHubGeorgi Gerganov
 		99c5b27654 ggml : refactor quantized processing functions (#509) 
* Refactor quantized processing functions

* ggml : minor

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-28 20:13:01 +03:00
Stephan WalterGitHubGeorgi Gerganov
 		436e561931 all : be more strict about converting float to double (#458) 
* Be more strict about converting float to double

* Test equivalence of round, SILU implementations

Test module is commented out in CMakeLists.txt because the tests may
take a long time, depending on how much the compiler optimizes.

* Fix softmax in perplexity.cpp

* all : prefer float over double where appropriate

* perplexity : add <cmath>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-28 19:48:20 +03:00
Stephan WalterGitHubGeorgi Gerganov
 		c1f885067c ggml : introduce structs for the q4 data blocks (#356) 
* Introduce structs for the q4 data blocks

* ggml : rename quant struct variables + fix ARM_NEON

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-28 18:56:03 +03:00
Stephan WalterandGitHub 939ad2d3a5 Fix undefined variables in debug build, remove unused variables (#531) 2023-03-26 15:34:02 +00:00
Stephan WalterandGitHub b391579db9 Update README and comments for standalone perplexity tool (#525) 2023-03-26 16:14:01 +03:00
Stephan WalterGitHubPavol Rusnak
 		a50e39c6fe Revert "Delete SHA256SUMS for now" (#429) 
* Revert "Delete SHA256SUMS for now (#416)"

This reverts commit 8eea5ae0e5.

* Remove ggml files until they can be verified
* Remove alpaca json
* Add also model/tokenizer.model to SHA256SUMS + update README

---------

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
 2023-03-23 15:15:48 +01:00
Stephan WalterGitHubGeorgi Gerganov
 		69c92298a9 Deduplicate q4 quantization functions (#383) 
* Deduplicate q4 quantization functions

* Use const; add basic test

* Re-enable quantization test

* Disable AVX2 flags in CI

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-22 19:29:06 +02:00
Stephan WalterandGitHub 5cb63e2493 Add tqdm to Python requirements (#293) 
* Add tqdm to Python requirements
* Remove torchvision torchaudio, add requests
 2023-03-20 09:24:11 +01:00
Stephan WalterandGitHub 367946c668 Don't tell users to use a bad number of threads (#243) 
The readme tells people to use the command line option "-t 8", causing 8
threads to be started. On systems with fewer than 8 cores, this causes a
significant slowdown. Remove the option from the example command lines
and use /proc/cpuinfo on Linux to determine a sensible default.
 2023-03-17 19:47:35 +02:00