Erik Scholz and GitHub
80c41ddd8f
CUDA: compress mode option and default to size ( #12029 )
...
cuda 12.8 added the option to specify stronger compression for binaries, so we now default to "size".
2025-03-01 12:57:22 +01:00
f34432ca1e
fix : cuda order of synchronization when setting a buffer (ggml/679)
...
* fix : cuda order of synchronization when setting a buffer
* also sync before memcpy
---------
Co-authored-by: slaren <slarengh@gmail.com >
2024-01-11 09:39:05 +02:00
Erik Scholz and GitHub
ff3bad83e2
flake : update flake.lock for newer transformers version + provide extra dev shell ( #3797 )
...
* flake : update flake.lock for newer transformers version + provide extra dev shell with torch and transformers (for most convert-xxx.py scripts)
2023-10-28 16:41:07 +02:00
Erik Scholz and GitHub
a98b1633d5
nix : add cuda, use a symlinked toolkit for cmake ( #3202 )
2023-09-25 13:48:30 +02:00
Erik Scholz and GitHub
7ddf185537
ci : switch cudatoolkit install on windows to networked ( #3236 )
2023-09-18 02:21:47 +02:00
Erik Scholz and GitHub
6eeb4d9083
convert: remove most of the n_mult usage in convert.py ( #3098 )
2023-09-10 11:06:53 -04:00
c9c3220c48
convert: fix convert.py not working with int filename_stem ( #3028 )
...
* fix implicit int to string conversion
* convert : remove an obsolete pyright comment
---------
Co-authored-by: Cebtenzzre <cebtenzzre@gmail.com >
2023-09-05 19:41:00 +02:00
Erik Scholz and GitHub
698efad5fb
CI: make the brew update temporarily optional. ( #2092 )
...
until they decide to fix the brew installation in the macos runners.
see the open issues. eg https://github.com/actions/runner-images/pull/7710
2023-07-04 01:50:12 +02:00
Erik Scholz and GitHub
9d23589d63
fix pthreads setaffinity usage on android ( #2020 )
2023-06-27 19:06:33 +02:00
Erik Scholz and GitHub
7487137227
rework convert.py to read hyper-parameters from config.json ( #1958 )
...
* Read hyper-parameters from HuggingFace-transformer config.json, if they exist, and fall back to guessing, like before otherwise.
This allows converting open_llama 3B and other non-standard model designs.
2023-06-22 14:20:47 +02:00
Erik Scholz and GitHub
5ea4339273
make kv_f16 the default for api users ( #1517 )
2023-05-18 19:31:01 +02:00
Erik Scholz and GitHub
a3b85b28da
ci : add cublas to windows release ( #1271 )
2023-05-05 22:56:09 +02:00
Erik Scholz and GitHub
34c1072e49
ci: add debug build to sanitizer build matrix ( #527 )
2023-03-26 15:48:40 +00:00
Erik Scholz and GitHub
4122dffff9
cmake: make llama an actual library ( #392 )
2023-03-22 18:37:10 +02:00
Erik Scholz and GitHub
56e659a0b2
fix perplexity after c-api refactor ( #390 )
...
* preallocate a buffer of fitting size for tokenization (utils.cpp)
* don't create a new std::string (especially here, where it's usually large)
2023-03-22 18:09:38 +02:00
Erik Scholz and GitHub
01a297b099
specify build type for ctest on windows ( #371 )
2023-03-21 23:34:25 +02:00
0b366e7357
Command line switch to use F16 for memory_k and memory_v (refactor of #154 ) ( #294 )
...
* Use F16 for memory_k and memory_v
* add command line switch to use f16 instead of f32 for memory k+v
---------
Co-authored-by: Ty Everett <ty@tyweb.us >
2023-03-19 19:57:00 +02:00
