nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 34 Packages Projects Releases Wiki Activity
196 Commits 1 Branch 0 Tags
ab77d7631211b299cb734bea6ad1f74324154150
Commit Graph
82 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Georgi Gerganov ab77d76312 Add longer DAN prompt for testing big batch numbers 2023-03-25 16:49:09 +02:00
Georgi GerganovandGitHub 4a7129acd2 Remove obsolete information from README 2023-03-25 16:30:32 +02:00
Georgi Gerganov 6b6dbc8910 Remove obsolete assert and fix compiler warning 2023-03-25 16:22:05 +02:00
Georgi Gerganov 2a2e63ce05 Fix nasty bug in ggml_compute_forward_mul_mat_f32() and reenable BLAS 2023-03-25 16:10:14 +02:00
Georgi Gerganov 8520fc310e Disable BLAS altogether - the bug is not just for qunatized mat mul 2023-03-24 23:47:06 +02:00
Georgi Gerganov b3f460e941 Disable BLAS branch in mul_mat - seems there is a bug 2023-03-24 23:39:17 +02:00
Georgi GerganovandGitHub 04c6f5ed6f Immediately start processing the prompt before user input has been provided (#476) 2023-03-24 23:17:58 +02:00
Georgi GerganovandGitHub 7a9b6c3a8b Reduce memory usage and allocate enough memory for largest context (#473) 
* Reduce memory usage and allocate enough memory for large contexts

* Simpler scratch buffer usage

* Reenable BLAS for quantized mul_mat

* Fix number of layers in 30B and 65B

* Fix KV cache size for F32
 2023-03-24 23:17:37 +02:00
Georgi Gerganov 31572d9665 Temporary bump the memory buffer size - hopefully fix issues from 483bab2e 2023-03-24 18:23:56 +02:00
Georgi Gerganov afd220d9c6 Properly free llama_context on failure 2023-03-24 17:21:01 +02:00
Georgi GerganovandGitHub b6b268d441 Add link to Roadmap discussion 2023-03-24 09:13:35 +02:00
Georgi Gerganov 3cd8dde0d1 Revert "Fix memory allocation issues and seg faults" 
This reverts commit 4870e455b3.

Will provide the correct fix later
 2023-03-24 06:22:28 +02:00
Georgi Gerganov 4870e455b3 Fix memory allocation issues and seg faults 2023-03-24 00:11:53 +02:00
Georgi GerganovandGitHub 483bab2e3d Avoid the transposed X branch in the Z = X * Y matrix multiplication (#439) 
Should make results reproducible for different number of threads and batch sizes
 2023-03-23 23:22:01 +02:00
Georgi Gerganov 4cc053b6d5 Remove oboslete command from Docker script 2023-03-23 22:39:44 +02:00
Georgi Gerganov 0ba5a3a9a5 Obsolete 2023-03-23 22:32:21 +02:00
Georgi GerganovandGitHub 93208cfb92 Adjust repetition penalty .. 2023-03-23 10:46:58 +02:00
Georgi GerganovandGitHub 03ace14cfd Add link to recent podcast about whisper.cpp and llama.cpp 2023-03-23 09:48:51 +02:00
Georgi Gerganov ae44e23ee3 When seed <= 0 - use the clock to generate one 2023-03-22 07:47:15 +02:00
Georgi Gerganov 928480ef5b Init llama_context_params properly from CLI (#370) 2023-03-22 07:45:14 +02:00
Georgi GerganovandGitHub 56817b1f88 Remove temporary notice and update hot topics 2023-03-22 07:34:02 +02:00
Georgi GerganovandGitHub f5a77a629b Introduce C-style API (#370) 
* Major refactoring - introduce C-style API

* Clean up

* Add <cassert>

* Add <iterator>

* Add <algorithm> ....

* Fix timing reporting and accumulation

* Measure eval time only for single-token calls

* Change llama_tokenize return meaning
 2023-03-22 07:32:36 +02:00
Georgi GerganovandGitHub 3366853e41 Add notice about pending change 2023-03-21 22:57:35 +02:00
Georgi GerganovandGitHub 0f61352708 Update issue templates 2023-03-21 19:47:27 +02:00
Georgi GerganovandGitHub 1daf4dd712 Minor style changes 2023-03-21 18:10:32 +02:00
Georgi Gerganov dc6a845b85 Add chat.sh script 2023-03-21 18:09:46 +02:00
Georgi Gerganov 3bfa3b43b7 Fix convert script, warnings alpaca instructions, default params 2023-03-21 17:59:16 +02:00
Georgi Gerganov 8f644a0a85 Change default repeat_penalty to 1.0 
I feel this penalty is not really helping.
Especially for the example from the README it makes results pretty bad
 2023-03-21 17:32:14 +02:00
Georgi GerganovandGitHub eb34620aec Add tokenizer test + revert to C++11 (#355) 
* Add test-tokenizer-0 to do a few tokenizations - feel free to expand
* Added option to convert-pth-to-ggml.py script to dump just the vocabulary
* Added ./models/ggml-vocab.bin containing just LLaMA vocab data (used for tests)
* Added utility to load vocabulary file from previous point (temporary implementation)
* Avoid using std::string_view and drop back to C++11 (hope I didn't break something)
* Rename gpt_vocab -> llama_vocab
* All CMake binaries go into ./bin/ now
 2023-03-21 17:29:41 +02:00
Georgi Gerganov 4545539d71 Rename script 2023-03-19 21:58:51 +02:00
Georgi Gerganov edeba28366 Add temporary helper script for Alpaca chat 2023-03-19 21:57:48 +02:00
Georgi GerganovandGitHub 160bfb217d Update hot topics to mention Alpaca support 2023-03-19 19:51:55 +02:00
Georgi Gerganov c494ed5b94 Fix off-by-one bug (#115) 2023-03-19 19:46:32 +02:00
Georgi Gerganov c1c7026b47 Fix python stuff (#109) 2023-03-19 19:33:18 +02:00
Georgi Gerganov 70f01cb863 Drop trailing new line from file prompts (#80) 2023-03-19 19:05:04 +02:00
Georgi GerganovandGitHub a4e63b73df Add instruction for using Alpaca (#240) 2023-03-19 18:49:50 +02:00
Georgi Gerganov 9e1707218a Add "--instruct" argument for usage with Alpaca (#240) 
Also start adding prompts in "./prompts"
 2023-03-19 18:37:02 +02:00
Georgi Gerganov 22213a17b5 Change RMSNorm eps to 1e-6 (#173) 
I think this is what is used in the Python code
 2023-03-19 17:30:00 +02:00
Georgi Gerganov 4f54609110 Default to 4 threads (#243) 2023-03-17 21:46:46 +02:00
Georgi GerganovandGitHub e81b9c81c1 Update Contributing section 2023-03-17 20:30:04 +02:00
Georgi GerganovandGitHub 721311070e Update README.md 2023-03-16 15:00:09 +02:00
Georgi GerganovandGitHub ac15de7895 Expand "Contributing" section 2023-03-16 08:55:13 +02:00
Georgi GerganovandGitHub 273abc47ff Update hot topics - RMSnorm 2023-03-16 07:12:12 +02:00
Georgi GerganovandGitHub 97ab2b2578 Add Misc section + update hot topics + minor fixes 2023-03-14 09:43:52 +02:00
Georgi Gerganov c09a9cfb06 CMake build in Release by default (#75) 2023-03-13 21:22:15 +02:00
Georgi GerganovandGitHub 7ec903d3c1 Update contribution section, hot topics, limitations, etc. 2023-03-13 19:21:51 +02:00
Georgi Gerganov 4497ad819c Print system information 2023-03-13 19:15:08 +02:00
Georgi GerganovandGitHub 84d9015c4a Use vdotq_s32 to improve performance (#67) 
* 10% performance boost on ARM

* Back to original change
 2023-03-13 18:36:44 +02:00
Georgi GerganovandGitHub 1808ee0500 Add initial contribution guidelines 2023-03-13 09:42:26 +02:00
Georgi Gerganov c80e2a8f2a Revert "10% performance boost on ARM" 
This reverts commit 113a9e83eb.

There are some reports for illegal instruction.
Moved this stuff to vdotq_s32 branch until resolve
 2023-03-13 01:28:08 +02:00
Georgi Gerganov 54a0e66ea0 Check for vdotq_s32 availability 2023-03-13 01:21:03 +02:00
Georgi Gerganov 543c57e991 Ammend to previous commit - forgot to update non-QRDMX branch 2023-03-13 01:05:24 +02:00
Georgi Gerganov 113a9e83eb 10% performance boost on ARM 2023-03-13 00:56:10 +02:00
Georgi GerganovandGitHub 1a0a74300f Update README.md 2023-03-12 23:39:01 +02:00
Georgi GerganovandGitHub 7027a97837 Update README.md 2023-03-12 22:09:26 +02:00
Georgi GerganovandGitHub 2d555e5b42 Add CI (#60) 2023-03-12 22:08:24 +02:00
Georgi Gerganov 7c9e54e55e Revert "weights_only" arg - this causing more trouble than help 2023-03-12 20:59:01 +02:00
Georgi GerganovandGitHub 702fddf5c5 Clarify meaning of hacking 2023-03-12 09:03:25 +02:00
Georgi GerganovandGitHub 7d86e25bf6 README: add "Supported platforms" + update hot topics 2023-03-12 08:41:54 +02:00
Georgi GerganovandGitHub da1a4ff01f Update README.md 2023-03-12 01:26:32 +02:00
Georgi GerganovandGitHub 4235e3d5b3 Update README.md 2023-03-11 18:10:18 +02:00
Georgi Gerganov f1eaff4721 Add AVX2 support for x86 architectures thanks to @Const-me ! 2023-03-11 18:04:25 +02:00
Georgi Gerganov a9e58529ea Fix un-initialized FP16 tables on x86 (#15, #2) 2023-03-11 17:40:14 +02:00
Georgi Gerganov 7d9ed7b25f Bump memory buffer 2023-03-11 12:45:01 +02:00
Georgi GerganovandGitHub 0c6803321c Update README.md 2023-03-11 12:31:21 +02:00
Georgi Gerganov f60fa9e50a .gitignore models/ 2023-03-11 12:27:02 +02:00
Georgi Gerganov 7211862c94 Update Makefile var + add comment 2023-03-11 12:27:02 +02:00
Georgi GerganovandGitHub a5c5ae2f54 Update README.md 2023-03-11 11:34:25 +02:00
Georgi GerganovandGitHub ea977e85ec Update README.md 2023-03-11 11:34:11 +02:00
Georgi Gerganov 007a8f6f45 Support all LLaMA models + change Q4_0 quantization storage 2023-03-11 11:28:30 +02:00
Georgi GerganovandGitHub 73c6ed5e87 Update README.md 2023-03-11 01:30:47 +02:00
Georgi GerganovandGitHub 01eeed8fb1 Update README.md 2023-03-11 01:22:58 +02:00
Georgi GerganovandGitHub 6da2df34ee Update README.md 2023-03-11 01:18:10 +02:00
Georgi GerganovandGitHub 920a7fe2d9 Update README.md 2023-03-11 00:55:22 +02:00
Georgi GerganovandGitHub 3a57ee59de Update README.md 2023-03-11 00:51:46 +02:00
Georgi GerganovandGitHub b85028522d Update README.md 2023-03-11 00:09:19 +02:00
Georgi GerganovandGitHub 8a01f565ff Update README.md 2023-03-10 23:53:11 +02:00
Georgi Gerganov 70bc0b8b15 Fix a bug in the rope calculation 2023-03-10 23:46:57 +02:00
Georgi GerganovandGitHub 18ebda34d6 Update README.md 2023-03-10 21:52:27 +02:00
Georgi Gerganov 319cdb3e1f Final touches 2023-03-10 21:50:46 +02:00
Georgi GerganovandGitHub 775328064e Create README.md 2023-03-10 21:47:46 +02:00
Georgi Gerganov 26c0846629 Initial release 2023-03-10 20:56:40 +02:00