Howard Su and GitHub
32c5411631
Revert "Support using mmap when applying LoRA ( #2095 )" ( #2206 )
...
Has perf regression when mlock is used.
This reverts commit 2347463201
2023-07-13 21:58:25 +08:00
Howard Su and GitHub
ff5d58faec
Fix compile error on Windows CUDA ( #2207 )
2023-07-13 21:58:09 +08:00
4e7464ef88
FP16 is supported in CM=6.0 ( #2177 )
...
* FP16 is supported in CM=6.0
* Building PTX code for both of 60 and 61
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
2023-07-12 20:18:40 +08:00
Howard Su and GitHub
2347463201
Support using mmap when applying LoRA ( #2095 )
...
* Support using mmap when applying LoRA
* Fix Linux
* Update comment to reflect the support lora with mmap
2023-07-11 22:37:01 +08:00
Howard Su and GitHub
481f793acc
Fix opencl by wrap #if-else-endif with \n ( #2086 )
2023-07-07 05:34:18 +02:00
Howard Su and GitHub
051c70dcd5
llama: Don't double count the sampling time ( #2107 )
2023-07-05 18:31:23 +08:00
Howard Su and GitHub
cc45a7feb8
Fix crash of test-tokenizer-0 under Debug build ( #2064 )
...
* Fix crash of test-tokenizer-0 under Debug build
* Change per comment
2023-07-03 20:43:55 +02:00
Howard Su and GitHub
55dbb915cc
[llama] No need to check file version when loading vocab score ( #2079 )
2023-07-03 19:58:58 +08:00
b8c8dda75f
Use unsigned for random seed ( #2006 )
...
* Use unsigned for random seed. Keep -1 as the value to use a time based seed.
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2023-06-29 06:15:15 -07:00
Howard Su and GitHub
b922bc351b
llama : remove shards weight file support ( #2000 )
...
* Remove multiple shards
* Remove multiple file loaders
* Remove llama_load_tensor_shard class
* Simplify load logic
* Remove dead code guess_n_parts function
* Remove vocab_only from constructor of llama_model_loader
* Remove alignment_prevents_mmap which is not more needed.
* Remove useless check
2023-06-28 20:13:02 +03:00
Howard Su and GitHub
0be54f75a6
baby-llama : fix build after ggml_rope change ( #2016 )
2023-06-27 08:07:13 +03:00
Howard Su and GitHub
cbebf61ca7
Fix assert when free invalid cuda pointer ( #2005 )
...
Fix assert via initializing extra structure always.
CUDA error 1 at C:\GPT\llama.cpp\ggml-cuda.cu:2536: invalid argument
2023-06-26 23:15:47 +08:00
1e3abfcef0
cmake : fix build shared ggml when CUDA is enabled ( #1929 )
...
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
2023-06-19 18:10:37 +03:00
Howard Su and GitHub
57cd69460f
cmake : add CUDA_ARCHITECTURES to new target ggml_static ( #1917 )
2023-06-18 07:29:47 +03:00
Howard Su and GitHub
3d59ec5935
ggml : fix warnings under MSVC ( #1908 )
2023-06-17 18:46:15 +03:00
Howard Su and GitHub
64cc19b4fe
Fix the validation of main device ( #1872 )
2023-06-15 19:29:59 +02:00
58970a4c39
Leverage mmap for offloading tensors to GPU ( #1597 )
...
* Rebase to latest
* Show progress
* Add assert to make sure we only allocate temp buffer for non-CPU backend tensor
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
---------
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
2023-06-12 14:44:16 +02:00
Howard Su and GitHub
bb051d9723
opencl : no need to allocate cl_mem on heap ( #1612 )
2023-05-28 20:13:36 +03:00
Howard Su and GitHub
ca74884f66
opencl : use strstr to check if fp16 supported ( #1611 )
...
* Use strstr to check if fp16 supported
* Ensure ext_buffer is null terminated
2023-05-28 20:09:56 +03:00
Howard Su and GitHub
7e312f165c
cmake : fix build under Windows when enable BUILD_SHARED_LIBS ( #1100 )
...
* Fix build under Windows when enable BUILD_SHARED_LIBS
* Make AVX512 test on Windows to build the shared libs
2023-04-22 11:18:20 +03:00
Howard Su and GitHub
c5d70f5c9e
ggml : optimize rope function to avoid call powf in the tight loop ( #807 )
2023-04-14 09:24:52 +03:00
