nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 23 Packages Projects Releases Wiki Activity
4,165 Commits 1 Branch 0 Tags
a9a678a6b2264e5895533217b0cf8f250534cc58
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Georgi GerganovandGitHub 9ca2e67762 server : add speculative decoding support (#10455) 
* server : add speculative decoding support

ggml-ci

* server : add helper function slot.can_speculate()

ggml-ci
 2024-11-25 16:31:38 +02:00
Georgi Gerganov f6d12e7df8 tests : fix compile warning 2024-11-25 15:17:32 +02:00
Georgi Gerganov b756441104 metal : minor code formatting 2024-11-25 15:08:04 +02:00
Georgi GerganovandGitHub d9d54e498d speculative : refactor and add a simpler example (#10362) 
* speculative : refactor and add a simpler example

ggml-ci

* speculative : clean-up and add comments and TODOs [no ci]

* speculative : manage context in common_speculative

ggml-ci

* speculative : simplify

ggml-ci

* speculative : simplify (cont)

ggml-ci

* speculative : add --draft-min CLI arg

* speculative : minor fixup

* make : build fixes

* speculative : do not redraft previous drafts

ggml-ci

* speculative : fix the draft sampling

ggml-ci

* speculative : fix compile warning

* common : refactor args

ggml-ci

* common : change defaults [no ci]

* common : final touches

ggml-ci
 2024-11-25 09:58:41 +02:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		cce5a90075 flake.lock: Update (#10470) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/5e4fbfb6b3de1aa2872b76d49fafc942626e2add?narHash=sha256-OZiZ3m8SCMfh3B6bfGC/Bm4x3qc1m2SVEAlkV6iY7Yg%3D' (2024-11-15)
  → 'github:NixOS/nixpkgs/23e89b7da85c3640bbc2173fe04f4bd114342367?narHash=sha256-y/MEyuJ5oBWrWAic/14LaIr/u5E0wRVzyYsouYY3W6w%3D' (2024-11-19)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-11-24 08:03:25 -08:00
Georgi GerganovandGitHub 1bb30bf28c llama : handle KV shift for recurrent models (#10402) 
ggml-ci
 2024-11-21 10:22:47 +02:00
Georgi Gerganov 87a533be57 sync : ggml 2024-11-21 09:22:11 +02:00
Georgi Gerganov 9fe0fb0626 sync : ggml 2024-11-19 20:03:21 +02:00
Georgi GerganovandGitHub 8e752a777b llama : add check for KV cache shifts (#10401) 
ggml-ci
 2024-11-19 13:29:26 +02:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		2eb76b2a5e flake.lock: Update (#10346) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/4aa36568d413aca0ea84a1684d2d46f55dbabad7?narHash=sha256-Zwl8YgTVJTEum%2BL%2B0zVAWvXAGbWAuXHax3KzuejaDyo%3D' (2024-11-05)
  → 'github:NixOS/nixpkgs/5e4fbfb6b3de1aa2872b76d49fafc942626e2add?narHash=sha256-OZiZ3m8SCMfh3B6bfGC/Bm4x3qc1m2SVEAlkV6iY7Yg%3D' (2024-11-15)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-11-18 06:08:20 -08:00
Georgi Gerganov 20a780c7b6 gitignore : ignore local run scripts [no ci] 2024-11-17 13:12:22 +02:00
Georgi GerganovandGitHub cf32a9b93a metal : refactor kernel args into structs (#10238) 
* metal : add kernel arg structs (wip)

* metal : fattn args

ggml-ci

* metal : cont + avoid potential int overflow [no ci]

* metal : mul mat struct (wip)

* cont : mul mat vec

* cont : pass by reference

* cont : args is first argument

* cont : use char ptr

* cont : shmem style

* cont : thread counters style

* cont : mul mm id

ggml-ci

* cont : int safety + register optimizations

ggml-ci

* metal : GGML_OP_CONCAT

ggml-ci

* metal : GGML_OP_ADD, GGML_OP_SUB, GGML_OP_MUL, GGML_OP_DIV

* metal : GGML_OP_REPEAT

* metal : GGML_OP_CPY

* metal : GGML_OP_RMS_NORM

* metal : GGML_OP_NORM

* metal : add TODOs for rest of ops

* ggml : add ggml-metal-impl.h

ggml-ci
 2024-11-17 11:23:01 +02:00
Georgi Gerganov 24203e9dd7 ggml : inttypes.h -> cinttypes (#0) 
ggml-ci
 2024-11-17 08:30:29 +02:00
Georgi Gerganov 5d9e59979c ggml : adapt AMX to tensor->grad removal (#0) 
ggml-ci
 2024-11-17 08:30:29 +02:00
Georgi Gerganov a4200cafad make : add ggml-opt (#0) 
ggml-ci
 2024-11-17 08:30:29 +02:00
Georgi Gerganov 84274a10c3 tests : remove test-grad0 2024-11-17 08:30:29 +02:00
Georgi Gerganov 68fcb4759c ggml : fix compile warnings (#0) 
ggml-ci
 2024-11-17 08:30:29 +02:00
Georgi Gerganov 5c9a8b22b1 scripts : update sync 2024-11-17 08:30:29 +02:00
Georgi Gerganov db4cfd5dbc llamafile : fix include path (#0) 
ggml-ci
 2024-11-16 20:36:26 +02:00
Georgi Gerganov 8ee0d09ae6 make : auto-determine dependencies (#0) 2024-11-16 20:36:26 +02:00
Georgi GerganovandGitHub f245cc28d4 scripts : fix missing key in compare-llama-bench.py (#10332) 2024-11-16 10:32:50 +02:00
Georgi Gerganov 09ecbcb596 cmake : fix ppc64 check (whisper/0) 
ggml-ci
 2024-11-15 15:44:06 +02:00
Georgi Gerganov cbf5541a82 sync : ggml 2024-11-15 15:44:06 +02:00
Georgi Gerganov 4802ad350b scripts : fix regex in sync [no ci] 2024-11-15 08:38:43 +02:00
Georgi GerganovandGitHub 2a82891a85 speculative : fix out-of-bounds access (#10289) 2024-11-14 11:44:15 +02:00
Georgi Gerganov 5ea926dad7 sync : ggml 2024-11-13 18:11:54 +02:00
Georgi GerganovandGitHub b0cefea58a metal : more precise Q*K in FA vec kernel (#10247) 2024-11-11 08:39:13 +02:00
Georgi GerganovandGitHub b141e5f6ef server : enable KV cache defrag by default (#10233) 
ggml-ci
 2024-11-11 08:38:43 +02:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		4b3a9212b6 flake.lock: Update (#10243) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/807e9154dcb16384b1b765ebe9cd2bba2ac287fd?narHash=sha256-l253w0XMT8nWHGXuXqyiIC/bMvh1VRszGXgdpQlfhvU%3D' (2024-10-29)
  → 'github:NixOS/nixpkgs/4aa36568d413aca0ea84a1684d2d46f55dbabad7?narHash=sha256-Zwl8YgTVJTEum%2BL%2B0zVAWvXAGbWAuXHax3KzuejaDyo%3D' (2024-11-05)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-11-10 11:45:25 -08:00
Georgi GerganovandGitHub 6423c65aa8 metal : reorder write loop in mul mat kernel + style (#10231) 
* metal : reorder write loop

* metal : int -> short, style

ggml-ci
 2024-11-09 11:53:13 +02:00
Georgi GerganovandGitHub 39a334a9aa metal : fix build and some more comments (#10229) 2024-11-09 11:53:02 +02:00
Georgi GerganovandGitHub bb38cdd8ba metal : fix F32 accumulation in FA vec kernel (#10232) 2024-11-09 11:52:45 +02:00
Georgi Gerganov f018acba22 llama : fix Qwen model type strings 2024-11-09 11:26:34 +02:00
Georgi Gerganov 46323fa9ef metal : hide debug messages from normal log 2024-11-09 11:21:49 +02:00
Georgi GerganovandGitHub ec450d3bbf metal : opt-in compile flag for BF16 (#10218) 
* metal : opt-in compile flag for BF16

ggml-ci

* ci : use BF16

ggml-ci

* swift : switch back to v12

* metal : has_float -> use_float

ggml-ci

* metal : fix BF16 check in MSL

ggml-ci
 2024-11-08 21:59:46 +02:00
Georgi Gerganov 695ad752b2 metal : improve clarity (minor) (#10171) 2024-11-08 18:37:41 +02:00
Georgi GerganovandGitHub 841f27abdb metal : optimize FA kernels (#10171) 
* ggml : add ggml_flash_attn_ext_get_prec

* metal : use F16 precision in FA kernels

ggml-ci

* metal : minor clean-up

* metal : compile-guard bf16 FA kernels

ggml-ci

* build : remove obsolete compile flag [no ci]

* metal : prevent int overflows [no ci]

* cuda : disable BF16 FA

ggml-ci

* metal : fix BF16 requirement for FA kernels

ggml-ci

* make : clean-up [no ci]
 2024-11-08 13:47:22 +02:00
Georgi Gerganov eec4d71737 scripts : add amx to sync-ggml.sh [no ci] 2024-11-07 23:11:36 +02:00
Georgi Gerganov 3b08828674 sync : ggml 2024-11-07 23:08:24 +02:00
Georgi Gerganov a2c6fd747c scripts : sync update 2024-11-07 23:07:55 +02:00
Georgi GerganovandGitHub 5c333e0140 metal : add BF16 support (#8439) 
* ggml : add initial BF16 support

ggml-ci

* metal : add mul_mat_id BF16 support

ggml-ci

* metal : check for bfloat support on the Metal device

ggml-ci

* metal : better var names [no ci]

* metal : do not build bfloat kernels when not supported

ggml-ci

* metal : try to fix BF16 support check

ggml-ci

* metal : this should correctly check bfloat support
 2024-11-06 19:53:51 +02:00
Georgi GerganovandGitHub b11f9ba9b8 server : remove hack for extra parallel slot (#10187) 
ggml-ci
 2024-11-06 13:29:01 +02:00
Georgi GerganovandGitHub 1dc04b2dee ggml : adjust is_first_call init value (#10193) 
ggml-ci
 2024-11-06 11:20:10 +02:00
Georgi GerganovandGitHub a1eaf6a960 metal : add quantized FA support (#10149) 
* metal : add quantized FA (vec) support

ggml-ci

* metal : add quantized FA (non-vec) support

* metal : fix support check

ggml-ci

* metal : clean-up

* metal : clean-up (cont)

* metal : fix shared memory calc + reduce smem + comments

* metal : float-correctness

* metal : minor [no ci]
 2024-11-06 10:24:23 +02:00
Georgi Gerganov 05697f670b metal : simplify f16 and f32 dequant kernels (#0) 2024-11-04 13:49:34 +02:00
Georgi Gerganov f8e58135cf metal : move dequantize templates to beginning of MSL source (#0) 2024-11-04 13:44:06 +02:00
Georgi Gerganov ce027adfb3 sync : ggml 2024-11-04 10:33:37 +02:00
Georgi GerganovandGitHub 08828a6d7d metal : minor fixup in FA kernel (#10143) 
* metal : minor fixup in FA kernel

ggml-ci

* metal : use the unrolled loop variable

* metal : remove unused var
 2024-11-03 15:18:40 +02:00
Georgi GerganovandGitHub 1839f69130 flake.lock: Update (#10146) 2024-11-03 05:14:15 -08:00
Georgi GerganovandGitHub 45950415ed server : fix endpoint checks (#10135) 
ggml-ci
 2024-11-02 18:34:00 +02:00
Georgi GerganovandGitHub 1926d6e39d llama : adjust default context size + print warnings (#10136) 
* llama : adjust default context size + print warnings

ggml-ci

* ggml-ci : add missing gpu-layers + adjust context sizes
 2024-11-02 15:18:56 +02:00
Georgi GerganovandGitHub ba6f62eb79 readme : update hot topics 2024-11-01 17:31:51 +02:00
Georgi GerganovandGitHub 1804adb0cf ggml : remove ggml_scratch (#10121) 
ggml-ci
 2024-11-01 12:58:45 +02:00
Georgi Gerganov 815fe72adc sync : ggml 2024-11-01 10:28:24 +02:00
Georgi Gerganov f221d56220 ggml : alloc ggml_contexts on the heap (whisper/2525) 2024-11-01 10:24:50 +02:00
Georgi GerganovandGitHub 8d8ff71536 llama : remove Tail-Free sampling (#10071) 
ggml-ci
 2024-10-29 10:42:05 +02:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		07028f9d74 flake.lock: Update (#10063) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/4c2fcb090b1f3e5b47eaa7bd33913b574a11e0a0?narHash=sha256-/uilDXvCIEs3C9l73JTACm4quuHUsIHcns1c%2BcHUJwA%3D' (2024-10-18)
  → 'github:NixOS/nixpkgs/2768c7d042a37de65bb1b5b3268fc987e534c49d?narHash=sha256-AlcmCXJZPIlO5dmFzV3V2XF6x/OpNWUV8Y/FMPGd8Z4%3D' (2024-10-23)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-10-28 08:41:24 -07:00
Georgi GerganovandGitHub 8125e6cbfc server : don't overfill the batch during infill (#10018) 
ggml-ci
 2024-10-28 08:49:32 +02:00
Georgi GerganovandGitHub 8841ce3f43 llama : switch KQ multiplication to F32 precision by default (#10015) 
ggml-ci
 2024-10-27 20:59:58 +02:00
Georgi Gerganov cc2983d375 sync : ggml 2024-10-26 10:34:08 +03:00
Georgi Gerganov 9e4a2563ea scripts : fix amx sync [no ci] 2024-10-26 10:33:31 +03:00
Georgi GerganovandGitHub 668750357e metal : support permuted matrix multiplicaions (#10033) 
* metal : support permuted matrix multiplicaions

ggml-ci

* cont : use nb01 directly for row steps

ggml-ci

* cont : add comments [no ci]

* metal : minor refactor

* metal : minor
 2024-10-25 22:26:15 +03:00
Georgi GerganovandGitHub bc5ba007b2 server : check that the prompt fits in the slot's context (#10030) 
ggml-ci
 2024-10-25 10:13:46 +03:00
Georgi Gerganov 40f2555797 ci : fix cmake flags for SYCL 2024-10-24 21:23:33 +03:00
Georgi Gerganov 190a37d797 sync : ggml 2024-10-23 17:23:55 +03:00
Georgi Gerganov 2d3aba9ee8 llama.vim : bump generation time limit to 3s [no ci] 2024-10-23 17:16:56 +03:00
Georgi Gerganov e94a138d64 llama.vim : fix info text display [no ci] (#9787) 2024-10-22 00:37:55 +03:00
Georgi Gerganov e01c67affe llama.vim : move info to the right of screen [no ci] (#9787) 
'eol' messes up the rendering with nvim v0.10.2 for some reason
 2024-10-21 22:53:18 +03:00
Georgi GerganovandGitHub dbd5f2f573 llama.vim : plugin for Neovim (#9787) 2024-10-21 20:25:02 +03:00
Georgi GerganovandGitHub f594bc80ba ggml : add asserts for type conversion in fattn kernels (#9971) 
ggml-ci
 2024-10-21 16:20:46 +03:00
Georgi GerganovandGitHub 55e47786e3 llama : default sampling changes + greedy update (#9897) 
* llama : deprecate softmax sampler + fix dist sampler

ggml-ci

* tests : replace macros with functions

ggml-ci

* sampling : change temperature sampler logic

For t <= 0.0f, keep the max logit intact and set the rest to -inf

* cont : no need for special "greedy" logic

top-k == 1 is the same

* tests : init prob correctly

* llama : handle temp <= 0.0 in the temp_ext sampler too

ggml-ci

* cont : avoid extra loop in temperature sampler for sub-zero temp

ggml-ci
 2024-10-21 09:46:40 +03:00
Georgi GerganovandGitHub bc21975084 speculative : fix handling of some input params (#9963) 
* speculative : fix batch sizes at initialization

ggml-ci

* speculative : handle params.n_predict == -1

* speculative : limit batch size to llama_n_batch
 2024-10-21 09:37:12 +03:00
Georgi GerganovandGitHub 8901755ba3 server : add n_indent parameter for line indentation requirement (#9929) 
ggml-ci
 2024-10-18 07:32:19 +03:00
Georgi Gerganov 17bb928080 readme : remove --memory-f32 references (#9925) 2024-10-17 23:43:05 +03:00
Georgi Gerganov 9f45fc1e99 llama : change warning to debug log 2024-10-17 23:27:42 +03:00
Georgi GerganovandGitHub 99bd4ac28c llama : infill sampling handle very long tokens (#9924) 
* llama : infill sampling handle very long tokens

ggml-ci

* cont : better indices

ggml-ci
 2024-10-17 22:32:47 +03:00
Georgi Gerganov 0e41b300ed sync : ggml 2024-10-16 11:28:14 +03:00
Georgi GerganovandGitHub 755a9b2bf0 llama : add infill sampler (#9896) 
ggml-ci
 2024-10-15 16:35:33 +03:00
Georgi GerganovandGitHub 223c25a72f server : improve infill context reuse (#9894) 
ggml-ci
 2024-10-15 16:28:55 +03:00
Georgi GerganovandGitHub dcdd535302 server : update preact (#9895) 2024-10-15 12:48:44 +03:00
Georgi GerganovandGitHub d4c19c0f5c server : accept extra_context for the infill endpoint (#9874) 
* server : accept extra_context for the infill endpoint

ggml-ci

* server : update readme [no ci]

* server : use repo-level FIM pattern if possible

ggml-ci
 2024-10-13 21:31:35 +03:00
Georgi GerganovandGitHub c7181bd294 server : reuse cached context chunks (#9866) 
ggml-ci
 2024-10-13 18:52:48 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		92be9f1216 flake.lock: Update (#9870) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/bc947f541ae55e999ffdb4013441347d83b00feb?narHash=sha256-NOiTvBbRLIOe5F6RbHaAh6%2B%2BBNjsb149fGZd1T4%2BKBg%3D' (2024-10-04)
  → 'github:NixOS/nixpkgs/5633bcff0c6162b9e4b5f1264264611e950c8ec7?narHash=sha256-9UTxR8eukdg%2BXZeHgxW5hQA9fIKHsKCdOIUycTryeVw%3D' (2024-10-09)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-10-12 20:11:26 -07:00
Georgi GerganovandGitHub edc265661c server : add option to time limit the generation phase (#9865) 
ggml-ci
 2024-10-12 16:14:27 +03:00
Georgi GerganovandGitHub 1bde94dd02 server : remove self-extend features (#9860) 
* server : remove self-extend

ggml-ci

* server : fix context limit check to use slot.n_past

ggml-ci
 2024-10-12 16:06:31 +03:00
Georgi GerganovandGitHub 95c76e8e92 server : remove legacy system_prompt feature (#9857) 
* server : remove legacy system_prompt feature

ggml-ci

* readme : update [no ci]

* server : fix non-transformer logic + remove response from /props
 2024-10-12 14:51:54 +03:00
Georgi GerganovandGitHub 11ac9800af llama : improve infill support and special token detection (#9798) 
* llama : improve infill support

ggml-ci

* llama : add more FIM token strings

ggml-ci

* server : update prompt on slot restore (#9800)

* gguf : deprecate old FIM token KVs
 2024-10-12 08:21:51 +03:00
Georgi GerganovandGitHub e7022064ab perplexity : fix integer overflow (#9783) 
* perplexity : fix integer overflow

ggml-ci

* perplexity : keep n_vocab as int and make appropriate casts

ggml-ci
 2024-10-09 17:00:18 +03:00
Georgi Gerganov 3dc48fe75a examples : remove llama.vim 
An updated version will be added in #9787
 2024-10-09 10:55:42 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		6279dac039 flake.lock: Update (#9753) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/bcef6817a8b2aa20a5a6dbb19b43e63c5bf8619a?narHash=sha256-HO4zgY0ekfwO5bX0QH/3kJ/h4KvUDFZg8YpkNwIbg1U%3D' (2024-09-12)
  → 'github:hercules-ci/flake-parts/3d04084d54bedc3d6b8b736c70ef449225c361b1?narHash=sha256-K5ZLCyfO/Zj9mPFldf3iwS6oZStJcU4tSpiXTMYaaL0%3D' (2024-10-01)
• Updated input 'flake-parts/nixpkgs-lib':
    'https://github.com/NixOS/nixpkgs/archive/356624c12086a18f2ea2825fed34523d60ccc4e3.tar.gz?narHash=sha256-Ss8QWLXdr2JCBPcYChJhz4xJm%2Bh/xjl4G0c0XlP6a74%3D' (2024-09-01)
  → 'https://github.com/NixOS/nixpkgs/archive/fb192fec7cc7a4c26d51779e9bab07ce6fa5597a.tar.gz?narHash=sha256-0xHYkMkeLVQAMa7gvkddbPqpxph%2BhDzdu1XdGPJR%2BOs%3D' (2024-10-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/1925c603f17fc89f4c8f6bf6f631a802ad85d784?narHash=sha256-J%2BPeFKSDV%2BpHL7ukkfpVzCOO7mBSrrpJ3svwBFABbhI%3D' (2024-09-26)
  → 'github:NixOS/nixpkgs/bc947f541ae55e999ffdb4013441347d83b00feb?narHash=sha256-NOiTvBbRLIOe5F6RbHaAh6%2B%2BBNjsb149fGZd1T4%2BKBg%3D' (2024-10-04)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-10-07 09:35:42 -07:00
Georgi GerganovGitHubslaren
 		d5ac8cf2f2 ggml : add metal backend registry / device (#9713) 
* ggml : add metal backend registry / device

ggml-ci

* metal : fix names [no ci]

* metal : global registry and device instances

ggml-ci

* cont : alternative initialization of global objects

ggml-ci

* llama : adapt to backend changes

ggml-ci

* fixes

* metal : fix indent

* metal : fix build when MTLGPUFamilyApple3 is not available

ggml-ci

* fix merge

* metal : avoid unnecessary singleton accesses

ggml-ci

* metal : minor fix [no ci]

* metal : g_state -> g_ggml_ctx_dev_main [no ci]

* metal : avoid reference of device context in the backend context

ggml-ci

* metal : minor [no ci]

* metal : fix maxTransferRate check

* metal : remove transfer rate stuff

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-10-07 18:27:51 +03:00
Georgi GerganovandGitHub d5cb86844f contrib : simplify + minor edits [no ci] 2024-10-06 14:15:27 +03:00
Georgi GerganovandGitHub f4b2dcdf49 readme : fix typo [no ci] 2024-10-06 13:49:41 +03:00
Georgi Gerganov b6d6c5289f sync : llama.cpp 2024-10-06 12:53:28 +03:00
Georgi GerganovandGitHub 8c475b97b8 rerank : use [SEP] token instead of [BOS] (#9737) 
* rerank : use [SEP] token instead of [BOS]

ggml-ci

* common : sanity check for non-NULL tokens

ggml-ci

* ci : adjust rank score interval

ggml-ci

* ci : add shebang to run.sh

ggml-ci
 2024-10-05 15:55:04 +03:00
Georgi Gerganov 58b16695e1 sync : ggml 2024-10-05 15:53:49 +03:00
Georgi Gerganov 905f5485b2 metal : zero-init buffer contexts (whisper/0) 2024-10-05 15:53:00 +03:00
Georgi Gerganov 17880771ad sync : ggml 2024-10-04 18:50:25 +03:00
Georgi Gerganov d5ed2b929d metal : remove abort (skip) (ggml/0) 2024-10-03 21:18:19 +03:00
Georgi Gerganov 1bb8a64ebf sync : ggml 2024-10-03 21:17:49 +03:00