nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 28 Packages Projects Releases Wiki Activity
3,992 Commits 1 Branch 0 Tags
79a2bc042dcacaad59306865208a8c8c3149e3ea
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Georgi GerganovandGitHub 8d8ff71536 llama : remove Tail-Free sampling (#10071) 
ggml-ci
 2024-10-29 10:42:05 +02:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		07028f9d74 flake.lock: Update (#10063) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/4c2fcb090b1f3e5b47eaa7bd33913b574a11e0a0?narHash=sha256-/uilDXvCIEs3C9l73JTACm4quuHUsIHcns1c%2BcHUJwA%3D' (2024-10-18)
  → 'github:NixOS/nixpkgs/2768c7d042a37de65bb1b5b3268fc987e534c49d?narHash=sha256-AlcmCXJZPIlO5dmFzV3V2XF6x/OpNWUV8Y/FMPGd8Z4%3D' (2024-10-23)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-10-28 08:41:24 -07:00
Georgi GerganovandGitHub 8125e6cbfc server : don't overfill the batch during infill (#10018) 
ggml-ci
 2024-10-28 08:49:32 +02:00
Georgi GerganovandGitHub 8841ce3f43 llama : switch KQ multiplication to F32 precision by default (#10015) 
ggml-ci
 2024-10-27 20:59:58 +02:00
Georgi Gerganov cc2983d375 sync : ggml 2024-10-26 10:34:08 +03:00
Georgi Gerganov 9e4a2563ea scripts : fix amx sync [no ci] 2024-10-26 10:33:31 +03:00
Georgi GerganovandGitHub 668750357e metal : support permuted matrix multiplicaions (#10033) 
* metal : support permuted matrix multiplicaions

ggml-ci

* cont : use nb01 directly for row steps

ggml-ci

* cont : add comments [no ci]

* metal : minor refactor

* metal : minor
 2024-10-25 22:26:15 +03:00
Georgi GerganovandGitHub bc5ba007b2 server : check that the prompt fits in the slot's context (#10030) 
ggml-ci
 2024-10-25 10:13:46 +03:00
Georgi Gerganov 40f2555797 ci : fix cmake flags for SYCL 2024-10-24 21:23:33 +03:00
Georgi Gerganov 190a37d797 sync : ggml 2024-10-23 17:23:55 +03:00
Georgi Gerganov 2d3aba9ee8 llama.vim : bump generation time limit to 3s [no ci] 2024-10-23 17:16:56 +03:00
Georgi Gerganov e94a138d64 llama.vim : fix info text display [no ci] (#9787) 2024-10-22 00:37:55 +03:00
Georgi Gerganov e01c67affe llama.vim : move info to the right of screen [no ci] (#9787) 
'eol' messes up the rendering with nvim v0.10.2 for some reason
 2024-10-21 22:53:18 +03:00
Georgi GerganovandGitHub dbd5f2f573 llama.vim : plugin for Neovim (#9787) 2024-10-21 20:25:02 +03:00
Georgi GerganovandGitHub f594bc80ba ggml : add asserts for type conversion in fattn kernels (#9971) 
ggml-ci
 2024-10-21 16:20:46 +03:00
Georgi GerganovandGitHub 55e47786e3 llama : default sampling changes + greedy update (#9897) 
* llama : deprecate softmax sampler + fix dist sampler

ggml-ci

* tests : replace macros with functions

ggml-ci

* sampling : change temperature sampler logic

For t <= 0.0f, keep the max logit intact and set the rest to -inf

* cont : no need for special "greedy" logic

top-k == 1 is the same

* tests : init prob correctly

* llama : handle temp <= 0.0 in the temp_ext sampler too

ggml-ci

* cont : avoid extra loop in temperature sampler for sub-zero temp

ggml-ci
 2024-10-21 09:46:40 +03:00
Georgi GerganovandGitHub bc21975084 speculative : fix handling of some input params (#9963) 
* speculative : fix batch sizes at initialization

ggml-ci

* speculative : handle params.n_predict == -1

* speculative : limit batch size to llama_n_batch
 2024-10-21 09:37:12 +03:00
Georgi GerganovandGitHub 8901755ba3 server : add n_indent parameter for line indentation requirement (#9929) 
ggml-ci
 2024-10-18 07:32:19 +03:00
Georgi Gerganov 17bb928080 readme : remove --memory-f32 references (#9925) 2024-10-17 23:43:05 +03:00
Georgi Gerganov 9f45fc1e99 llama : change warning to debug log 2024-10-17 23:27:42 +03:00
Georgi GerganovandGitHub 99bd4ac28c llama : infill sampling handle very long tokens (#9924) 
* llama : infill sampling handle very long tokens

ggml-ci

* cont : better indices

ggml-ci
 2024-10-17 22:32:47 +03:00
Georgi Gerganov 0e41b300ed sync : ggml 2024-10-16 11:28:14 +03:00
Georgi GerganovandGitHub 755a9b2bf0 llama : add infill sampler (#9896) 
ggml-ci
 2024-10-15 16:35:33 +03:00
Georgi GerganovandGitHub 223c25a72f server : improve infill context reuse (#9894) 
ggml-ci
 2024-10-15 16:28:55 +03:00
Georgi GerganovandGitHub dcdd535302 server : update preact (#9895) 2024-10-15 12:48:44 +03:00
Georgi GerganovandGitHub d4c19c0f5c server : accept extra_context for the infill endpoint (#9874) 
* server : accept extra_context for the infill endpoint

ggml-ci

* server : update readme [no ci]

* server : use repo-level FIM pattern if possible

ggml-ci
 2024-10-13 21:31:35 +03:00
Georgi GerganovandGitHub c7181bd294 server : reuse cached context chunks (#9866) 
ggml-ci
 2024-10-13 18:52:48 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		92be9f1216 flake.lock: Update (#9870) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/bc947f541ae55e999ffdb4013441347d83b00feb?narHash=sha256-NOiTvBbRLIOe5F6RbHaAh6%2B%2BBNjsb149fGZd1T4%2BKBg%3D' (2024-10-04)
  → 'github:NixOS/nixpkgs/5633bcff0c6162b9e4b5f1264264611e950c8ec7?narHash=sha256-9UTxR8eukdg%2BXZeHgxW5hQA9fIKHsKCdOIUycTryeVw%3D' (2024-10-09)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-10-12 20:11:26 -07:00
Georgi GerganovandGitHub edc265661c server : add option to time limit the generation phase (#9865) 
ggml-ci
 2024-10-12 16:14:27 +03:00
Georgi GerganovandGitHub 1bde94dd02 server : remove self-extend features (#9860) 
* server : remove self-extend

ggml-ci

* server : fix context limit check to use slot.n_past

ggml-ci
 2024-10-12 16:06:31 +03:00
Georgi GerganovandGitHub 95c76e8e92 server : remove legacy system_prompt feature (#9857) 
* server : remove legacy system_prompt feature

ggml-ci

* readme : update [no ci]

* server : fix non-transformer logic + remove response from /props
 2024-10-12 14:51:54 +03:00
Georgi GerganovandGitHub 11ac9800af llama : improve infill support and special token detection (#9798) 
* llama : improve infill support

ggml-ci

* llama : add more FIM token strings

ggml-ci

* server : update prompt on slot restore (#9800)

* gguf : deprecate old FIM token KVs
 2024-10-12 08:21:51 +03:00
Georgi GerganovandGitHub e7022064ab perplexity : fix integer overflow (#9783) 
* perplexity : fix integer overflow

ggml-ci

* perplexity : keep n_vocab as int and make appropriate casts

ggml-ci
 2024-10-09 17:00:18 +03:00
Georgi Gerganov 3dc48fe75a examples : remove llama.vim 
An updated version will be added in #9787
 2024-10-09 10:55:42 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		6279dac039 flake.lock: Update (#9753) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/bcef6817a8b2aa20a5a6dbb19b43e63c5bf8619a?narHash=sha256-HO4zgY0ekfwO5bX0QH/3kJ/h4KvUDFZg8YpkNwIbg1U%3D' (2024-09-12)
  → 'github:hercules-ci/flake-parts/3d04084d54bedc3d6b8b736c70ef449225c361b1?narHash=sha256-K5ZLCyfO/Zj9mPFldf3iwS6oZStJcU4tSpiXTMYaaL0%3D' (2024-10-01)
• Updated input 'flake-parts/nixpkgs-lib':
    'https://github.com/NixOS/nixpkgs/archive/356624c12086a18f2ea2825fed34523d60ccc4e3.tar.gz?narHash=sha256-Ss8QWLXdr2JCBPcYChJhz4xJm%2Bh/xjl4G0c0XlP6a74%3D' (2024-09-01)
  → 'https://github.com/NixOS/nixpkgs/archive/fb192fec7cc7a4c26d51779e9bab07ce6fa5597a.tar.gz?narHash=sha256-0xHYkMkeLVQAMa7gvkddbPqpxph%2BhDzdu1XdGPJR%2BOs%3D' (2024-10-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/1925c603f17fc89f4c8f6bf6f631a802ad85d784?narHash=sha256-J%2BPeFKSDV%2BpHL7ukkfpVzCOO7mBSrrpJ3svwBFABbhI%3D' (2024-09-26)
  → 'github:NixOS/nixpkgs/bc947f541ae55e999ffdb4013441347d83b00feb?narHash=sha256-NOiTvBbRLIOe5F6RbHaAh6%2B%2BBNjsb149fGZd1T4%2BKBg%3D' (2024-10-04)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-10-07 09:35:42 -07:00
Georgi GerganovGitHubslaren
 		d5ac8cf2f2 ggml : add metal backend registry / device (#9713) 
* ggml : add metal backend registry / device

ggml-ci

* metal : fix names [no ci]

* metal : global registry and device instances

ggml-ci

* cont : alternative initialization of global objects

ggml-ci

* llama : adapt to backend changes

ggml-ci

* fixes

* metal : fix indent

* metal : fix build when MTLGPUFamilyApple3 is not available

ggml-ci

* fix merge

* metal : avoid unnecessary singleton accesses

ggml-ci

* metal : minor fix [no ci]

* metal : g_state -> g_ggml_ctx_dev_main [no ci]

* metal : avoid reference of device context in the backend context

ggml-ci

* metal : minor [no ci]

* metal : fix maxTransferRate check

* metal : remove transfer rate stuff

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-10-07 18:27:51 +03:00
Georgi GerganovandGitHub d5cb86844f contrib : simplify + minor edits [no ci] 2024-10-06 14:15:27 +03:00
Georgi GerganovandGitHub f4b2dcdf49 readme : fix typo [no ci] 2024-10-06 13:49:41 +03:00
Georgi Gerganov b6d6c5289f sync : llama.cpp 2024-10-06 12:53:28 +03:00
Georgi GerganovandGitHub 8c475b97b8 rerank : use [SEP] token instead of [BOS] (#9737) 
* rerank : use [SEP] token instead of [BOS]

ggml-ci

* common : sanity check for non-NULL tokens

ggml-ci

* ci : adjust rank score interval

ggml-ci

* ci : add shebang to run.sh

ggml-ci
 2024-10-05 15:55:04 +03:00
Georgi Gerganov 58b16695e1 sync : ggml 2024-10-05 15:53:49 +03:00
Georgi Gerganov 905f5485b2 metal : zero-init buffer contexts (whisper/0) 2024-10-05 15:53:00 +03:00
Georgi Gerganov 17880771ad sync : ggml 2024-10-04 18:50:25 +03:00
Georgi Gerganov d5ed2b929d metal : remove abort (skip) (ggml/0) 2024-10-03 21:18:19 +03:00
Georgi Gerganov 1bb8a64ebf sync : ggml 2024-10-03 21:17:49 +03:00
Georgi GerganovandGitHub 148844fe97 examples : remove benchmark (#9704) 
ggml-ci
 2024-10-02 10:14:44 +03:00
Georgi Gerganov f1b8c42711 sync : ggml 2024-10-01 16:09:42 +03:00
Georgi GerganovandGitHub cad341d889 metal : reduce command encoding overhead (#9698) 
* metal : reduce command encoding overhead

ggml-ci

* metal : add comments
 2024-10-01 16:00:25 +03:00
Georgi Gerganov a90484c6d9 llama : print correct model type for Llama 3.2 1B and 3B 2024-10-01 11:42:01 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		ace4f4be37 flake.lock: Update (#9680) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/c04d5652cfa9742b1d519688f65d1bbccea9eb7e?narHash=sha256-PmUr/2GQGvFTIJ6/Tvsins7Q43KTMvMFhvG6oaYK%2BWk%3D' (2024-09-19)
  → 'github:NixOS/nixpkgs/1925c603f17fc89f4c8f6bf6f631a802ad85d784?narHash=sha256-J%2BPeFKSDV%2BpHL7ukkfpVzCOO7mBSrrpJ3svwBFABbhI%3D' (2024-09-26)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-30 07:48:49 -07:00
Georgi GerganovGitHubWilly Tarreau
 		c919d5db39 ggml : define missing HWCAP flags (#9684) 
ggml-ci

Co-authored-by: Willy Tarreau <w@1wt.eu>
 2024-09-29 21:18:23 +03:00
Georgi Gerganov d0b1d663e4 sync : ggml 2024-09-29 21:16:07 +03:00
Georgi Gerganov 6084bfb261 ggml : fix GGML_MAX_N_THREADS + improve formatting (ggml/969) 2024-09-29 21:15:35 +03:00
Georgi GerganovandGitHub 589b48d41e contrib : add Resources section (#9675) 2024-09-29 14:38:18 +03:00
Georgi GerganovGitHubXuan Son NguyenXuan Son Nguyen
 		f4d2b8846a llama : add reranking support (#9510) 
* py : add XLMRobertaForSequenceClassification [no ci]

* py : fix scalar-tensor conversion [no ci]

* py : fix position embeddings chop [no ci]

* llama : read new cls tensors [no ci]

* llama : add classigication head (wip) [no ci]

* llama : add "rank" pooling type

ggml-ci

* server : add rerank endpoint

ggml-ci

* llama : aboud ggml_repeat during classification

* rerank : cleanup + comments

* server : accept /rerank endpoint in addition to /v1/rerank [no ci]

* embedding : parse special tokens

* jina : support v1 reranker

* vocab : minor style

ggml-ci

* server : initiate tests for later

ggml-ci

* server : add docs

* llama : add comment [no ci]

* llama : fix uninitialized tensors

* ci : add rerank tests

ggml-ci

* add reranking test

* change test data

* Update examples/server/server.cpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* add `--reranking` argument

* update server docs

* llama : fix comment [no ci]

ggml-ci

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-09-28 17:42:03 +03:00
Georgi Gerganov 739842703e llama : add comment about thread-safety [no ci] (#9449) 2024-09-28 15:13:42 +03:00
Georgi GerganovandGitHub b5de3b74a5 readme : update hot topics 2024-09-27 20:57:51 +03:00
Georgi Gerganov bb5f819975 sync : ggml 2024-09-24 11:01:18 +03:00
Georgi Gerganov c038931615 examples : adapt to ggml.h changes (ggml/0) 
ggml-ci
 2024-09-24 11:00:52 +03:00
Georgi GerganovandGitHub 31ac5834fe llama : keep track of all EOG tokens in the vocab (#9609) 
ggml-ci
 2024-09-24 10:16:06 +03:00
Georgi GerganovandGitHub cea1486ecf log : add CONT level for continuing previous log entry (#9610) 2024-09-24 10:15:35 +03:00
Georgi GerganovGitHubslaren
 		b0f27361f3 sampling : avoid expensive softmax during greedy sampling (#9605) 
* sampling : avoid expensive softmax during greedy sampling

ggml-ci

* speculative : fix default RNG seed + set sparams.n_probs

* Update tests/test-sampling.cpp

Co-authored-by: slaren <slarengh@gmail.com>

* sampling : add clarifying comment [no ci]

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-09-24 09:03:17 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		f3979df762 flake.lock: Update (#9586) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/4f807e8940284ad7925ebd0a0993d2a1791acb2f?narHash=sha256-IiA3jfbR7K/B5%2B9byVi9BZGWTD4VSbWe8VLpp9B/iYk%3D' (2024-09-11)
  → 'github:NixOS/nixpkgs/c04d5652cfa9742b1d519688f65d1bbccea9eb7e?narHash=sha256-PmUr/2GQGvFTIJ6/Tvsins7Q43KTMvMFhvG6oaYK%2BWk%3D' (2024-09-19)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-23 08:43:40 -07:00
Georgi GerganovandGitHub 37f8c7b4c9 perplexity : remove extra new lines after chunks (#9596) 2024-09-23 11:28:02 +03:00
Georgi GerganovandGitHub bf9c1013ac metal : use F32 prec for K*Q in vec FA (#9595) 
ggml-ci
 2024-09-23 11:27:47 +03:00
Georgi Gerganov d13edb17ed ggml : fix builds (#0) 
ggml-ci
 2024-09-20 21:15:05 +03:00
Georgi Gerganov 27609c49b9 ggml : fix trailing whitespace (#0) 
ggml-ci
 2024-09-20 21:15:05 +03:00
Georgi Gerganov 4301535326 sync : ggml 
ggml-ci
 2024-09-20 21:15:05 +03:00
Georgi Gerganov a6809c6a2e examples : add null threadpool args where needed (ggml/0) 
ggml-ci
 2024-09-20 21:15:05 +03:00
Georgi GerganovandGitHub d39e26741f examples : flush log upon ctrl+c (#9559) 2024-09-20 11:46:56 +03:00
Georgi GerganovandGitHub 6026da52d6 server : clean-up completed tasks from waiting list (#9531) 
ggml-ci
 2024-09-19 12:44:53 +03:00
Georgi GerganovandGitHub 0d2f22e45c scripts : verify py deps at the start of compare (#9520) 2024-09-18 18:34:32 +03:00
Georgi GerganovandGitHub 19514d632e cmake : do not hide GGML options + rename option (#9465) 
* cmake : do not hide GGML options

ggml-ci

* build : rename flag GGML_CUDA_USE_GRAPHS -> GGML_CUDA_GRAPHS

for consistency

ggml-ci
 2024-09-16 10:27:50 +03:00
Georgi GerganovandGitHub c4965a64f7 metal : handle zero-sized allocs (#9466) 2024-09-16 09:05:56 +03:00
Georgi GerganovandGitHub 90a2fff0e7 flake.lock: Update (#9488) 2024-09-15 19:14:23 -07:00
Georgi GerganovandGitHub 6262d13e0b common : reimplement logging (#9418) 
https://github.com/ggerganov/llama.cpp/pull/9418
 2024-09-15 20:46:12 +03:00
Georgi GerganovGitHubMichael Podvitskiy
 		1f4111e540 cmake : use list(APPEND ...) instead of set() + dedup linker (#9463) 
* cmake : use list(APPEND ...) instead of set() + dedup linker

ggml-ci

* cmake : try fix sycl

* cmake : try to fix sycl 2

* cmake : fix sycl build (#9469)

* try fix sycl build

* use CMAKE_CXX_FLAGS as a string variable

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* one more CMAKE_CXX_FLAGS fix (#9471)

---------

Co-authored-by: Michael Podvitskiy <podvitskiymichael@gmail.com>
 2024-09-14 10:55:05 +03:00
Georgi GerganovGitHubXuan Son Nguyen
 		0abc6a2c25 llama : llama_perf + option to disable timings during decode (#9355) 
* llama : llama_perf + option to disable timings during decode

ggml-ci

* common : add llama_arg

* Update src/llama.cpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* perf : separate functions in the API

ggml-ci

* perf : safer pointer handling + naming update

ggml-ci

* minor : better local var name

* perf : abort on invalid sampler pointer

ggml-ci

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-09-13 09:53:38 +03:00
Georgi GerganovandGitHub d6a04f872d ggml : hide ggml_object, ggml_cgraph, ggml_hash_set (#9408) 
* ggml : hide ggml_object, ggml_cgraph, ggml_hash_set

ggml-ci

* ggml : add ggml-impl.h to backends

* ggml : fix compiler warnings

ggml-ci

* ggml : add assert upon adding nodes
 2024-09-12 14:23:49 +03:00
Georgi GerganovandGitHub d2b496bff4 batched-bench : remove unused code (#9305) 2024-09-11 10:03:54 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		cb9c933eb2 flake.lock: Update (#9360) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/af510d4a62d071ea13925ce41c95e3dec816c01d?narHash=sha256-ODYRm8zHfLTH3soTFWE452ydPYz2iTvr9T8ftDMUQ3E%3D' (2024-08-30)
  → 'github:hercules-ci/flake-parts/567b938d64d4b4112ee253b9274472dc3a346eb6?narHash=sha256-%2Bebgonl3NbiKD2UD0x4BszCZQ6sTfL4xioaM49o5B3Y%3D' (2024-09-01)
• Updated input 'flake-parts/nixpkgs-lib':
    'https://github.com/NixOS/nixpkgs/archive/a5d394176e64ab29c852d03346c1fc9b0b7d33eb.tar.gz?narHash=sha256-uFf2QeW7eAHlYXuDktm9c25OxOyCoUOQmh5SZ9amE5Q%3D' (2024-08-01)
  → 'https://github.com/NixOS/nixpkgs/archive/356624c12086a18f2ea2825fed34523d60ccc4e3.tar.gz?narHash=sha256-Ss8QWLXdr2JCBPcYChJhz4xJm%2Bh/xjl4G0c0XlP6a74%3D' (2024-09-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/71e91c409d1e654808b2621f28a327acfdad8dc2?narHash=sha256-GnR7/ibgIH1vhoy8cYdmXE6iyZqKqFxQSVkFgosBh6w%3D' (2024-08-28)
  → 'github:NixOS/nixpkgs/574d1eac1c200690e27b8eb4e24887f8df7ac27c?narHash=sha256-v3rIhsJBOMLR8e/RNWxr828tB%2BWywYIoajrZKFM%2B0Gg%3D' (2024-09-06)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-10 15:46:59 -07:00
Georgi Gerganov 00ba2ff781 metal : fix compile warning with GGML_METAL_NDEBUG (#0) 2024-09-10 10:17:43 +03:00
Georgi GerganovandGitHub 38ca6f644b readme : update hot topics 2024-09-09 15:51:37 +03:00
Georgi GerganovandGitHub e079bffb66 cuda : fix FA Q src index (1 -> 0) (#9374) 2024-09-08 22:01:02 +03:00
Georgi Gerganov a876861455 metal : update support condition for im2col + fix warning (#0) 2024-09-08 11:05:55 +03:00
Georgi Gerganov 385decbd63 sync : ggml 2024-09-08 11:05:55 +03:00
Georgi Gerganov 60a3107ccd scripts : option to increase git patch context 2024-09-08 11:05:55 +03:00
Georgi Gerganov ba1cf846ed cann : fix doxy (ggml/0) 2024-09-08 11:05:55 +03:00
Georgi Gerganov 51d964a4ef cuda : mark BF16 CONT as unsupported 2024-09-08 11:05:55 +03:00
Georgi GerganovandGitHub a5b5d9a101 llama.android : fix build (#9350) 2024-09-08 00:33:50 +03:00
Georgi GerganovandGitHub f12295b8a9 llama : fix empty ring buffer push (#9358) 2024-09-08 00:33:33 +03:00
Georgi GerganovandGitHub faf69d4237 llama : sanitize invalid tokens (#9357) 
* common : do not add null tokens during warmup

ggml-ci

* llama : check that the input tokens are valid

ggml-ci

* tests : fix batch size of bert model

ggml-ci
 2024-09-08 00:33:13 +03:00
Georgi GerganovandGitHub df270ef745 llama : refactor sampling v2 (#9294) 
- Add `struct llama_sampler` and `struct llama_sampler_i`
- Add `llama_sampler_` API
- Add `llama_sampler_chain_` API for chaining multiple samplers
- Remove `LLAMA_API_INTERNAL`
- Add `llama_perf_` API and remove old `llama_print_timings` and `llama_reset_timings`
 2024-09-07 15:16:19 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		7605ae7daf flake.lock: Update (#9261) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/8471fe90ad337a8074e957b69ca4d0089218391d?narHash=sha256-XOQkdLafnb/p9ij77byFQjDf5m5QYl9b2REiVClC%2Bx4%3D' (2024-08-01)
  → 'github:hercules-ci/flake-parts/af510d4a62d071ea13925ce41c95e3dec816c01d?narHash=sha256-ODYRm8zHfLTH3soTFWE452ydPYz2iTvr9T8ftDMUQ3E%3D' (2024-08-30)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/c374d94f1536013ca8e92341b540eba4c22f9c62?narHash=sha256-Z/ELQhrSd7bMzTO8r7NZgi9g5emh%2BaRKoCdaAv5fiO0%3D' (2024-08-21)
  → 'github:NixOS/nixpkgs/71e91c409d1e654808b2621f28a327acfdad8dc2?narHash=sha256-GnR7/ibgIH1vhoy8cYdmXE6iyZqKqFxQSVkFgosBh6w%3D' (2024-08-28)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-03 16:36:43 -07:00
Georgi GerganovandGitHub b69a480af4 readme : refactor API section + remove old hot topics 2024-09-03 10:00:36 +03:00
Georgi Gerganov c6d4cb4655 llama : minor style 2024-09-02 11:52:37 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		1d1ccce676 flake.lock: Update (#9162) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/c3aa7b8938b17aebd2deecf7be0636000d62a2b9?narHash=sha256-med8%2B5DSWa2UnOqtdICndjDAEjxr5D7zaIiK4pn0Q7c%3D' (2024-08-14)
  → 'github:NixOS/nixpkgs/c374d94f1536013ca8e92341b540eba4c22f9c62?narHash=sha256-Z/ELQhrSd7bMzTO8r7NZgi9g5emh%2BaRKoCdaAv5fiO0%3D' (2024-08-21)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-08-28 21:28:14 -07:00
Georgi Gerganov 20f1789dfb vulkan : fix build (#0) 
ggml-ci
 2024-08-27 22:41:27 +03:00
Georgi Gerganov 231cff5f6f sync : ggml 2024-08-27 22:41:27 +03:00
Georgi GerganovandGitHub 06658ad7c3 metal : separate scale and mask from QKT in FA kernel (#9189) 
* metal : separate scale and mask from QKT in FA kernel

* metal : ne01 check no longer necessary

* metal : keep data in local memory
 2024-08-26 18:31:02 +03:00