Georgi Gerganov and GitHub
d5cb86844f
contrib : simplify + minor edits [no ci]
2024-10-06 14:15:27 +03:00
Georgi Gerganov and GitHub
f4b2dcdf49
readme : fix typo [no ci]
2024-10-06 13:49:41 +03:00
Georgi Gerganov
b6d6c5289f
sync : llama.cpp
2024-10-06 12:53:28 +03:00
Georgi Gerganov and GitHub
8c475b97b8
rerank : use [SEP] token instead of [BOS] ( #9737 )
...
* rerank : use [SEP] token instead of [BOS]
ggml-ci
* common : sanity check for non-NULL tokens
ggml-ci
* ci : adjust rank score interval
ggml-ci
* ci : add shebang to run.sh
ggml-ci
2024-10-05 15:55:04 +03:00
Georgi Gerganov
58b16695e1
sync : ggml
2024-10-05 15:53:49 +03:00
Georgi Gerganov
905f5485b2
metal : zero-init buffer contexts (whisper/0)
2024-10-05 15:53:00 +03:00
Georgi Gerganov
17880771ad
sync : ggml
2024-10-04 18:50:25 +03:00
Georgi Gerganov
d5ed2b929d
metal : remove abort (skip) (ggml/0)
2024-10-03 21:18:19 +03:00
Georgi Gerganov
1bb8a64ebf
sync : ggml
2024-10-03 21:17:49 +03:00
Georgi Gerganov and GitHub
148844fe97
examples : remove benchmark ( #9704 )
...
ggml-ci
2024-10-02 10:14:44 +03:00
Georgi Gerganov
f1b8c42711
sync : ggml
2024-10-01 16:09:42 +03:00
Georgi Gerganov and GitHub
cad341d889
metal : reduce command encoding overhead ( #9698 )
...
* metal : reduce command encoding overhead
ggml-ci
* metal : add comments
2024-10-01 16:00:25 +03:00
Georgi Gerganov
a90484c6d9
llama : print correct model type for Llama 3.2 1B and 3B
2024-10-01 11:42:01 +03:00
ace4f4be37
flake.lock: Update ( #9680 )
...
Flake lock file updates:
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/c04d5652cfa9742b1d519688f65d1bbccea9eb7e?narHash=sha256-PmUr/2GQGvFTIJ6/Tvsins7Q43KTMvMFhvG6oaYK%2BWk%3D' (2024-09-19)
→ 'github:NixOS/nixpkgs/1925c603f17fc89f4c8f6bf6f631a802ad85d784?narHash=sha256-J%2BPeFKSDV%2BpHL7ukkfpVzCOO7mBSrrpJ3svwBFABbhI%3D' (2024-09-26)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-30 07:48:49 -07:00
c919d5db39
ggml : define missing HWCAP flags ( #9684 )
...
ggml-ci
Co-authored-by: Willy Tarreau <w@1wt.eu >
2024-09-29 21:18:23 +03:00
Georgi Gerganov
d0b1d663e4
sync : ggml
2024-09-29 21:16:07 +03:00
Georgi Gerganov
6084bfb261
ggml : fix GGML_MAX_N_THREADS + improve formatting (ggml/969)
2024-09-29 21:15:35 +03:00
Georgi Gerganov and GitHub
589b48d41e
contrib : add Resources section ( #9675 )
2024-09-29 14:38:18 +03:00
f4d2b8846a
llama : add reranking support ( #9510 )
...
* py : add XLMRobertaForSequenceClassification [no ci]
* py : fix scalar-tensor conversion [no ci]
* py : fix position embeddings chop [no ci]
* llama : read new cls tensors [no ci]
* llama : add classigication head (wip) [no ci]
* llama : add "rank" pooling type
ggml-ci
* server : add rerank endpoint
ggml-ci
* llama : aboud ggml_repeat during classification
* rerank : cleanup + comments
* server : accept /rerank endpoint in addition to /v1/rerank [no ci]
* embedding : parse special tokens
* jina : support v1 reranker
* vocab : minor style
ggml-ci
* server : initiate tests for later
ggml-ci
* server : add docs
* llama : add comment [no ci]
* llama : fix uninitialized tensors
* ci : add rerank tests
ggml-ci
* add reranking test
* change test data
* Update examples/server/server.cpp
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com >
* add `--reranking` argument
* update server docs
* llama : fix comment [no ci]
ggml-ci
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co >
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com >
2024-09-28 17:42:03 +03:00
Georgi Gerganov
739842703e
llama : add comment about thread-safety [no ci] ( #9449 )
2024-09-28 15:13:42 +03:00
Georgi Gerganov and GitHub
b5de3b74a5
readme : update hot topics
2024-09-27 20:57:51 +03:00
Georgi Gerganov
bb5f819975
sync : ggml
2024-09-24 11:01:18 +03:00
Georgi Gerganov
c038931615
examples : adapt to ggml.h changes (ggml/0)
...
ggml-ci
2024-09-24 11:00:52 +03:00
Georgi Gerganov and GitHub
31ac5834fe
llama : keep track of all EOG tokens in the vocab ( #9609 )
...
ggml-ci
2024-09-24 10:16:06 +03:00
Georgi Gerganov and GitHub
cea1486ecf
log : add CONT level for continuing previous log entry ( #9610 )
2024-09-24 10:15:35 +03:00
b0f27361f3
sampling : avoid expensive softmax during greedy sampling ( #9605 )
...
* sampling : avoid expensive softmax during greedy sampling
ggml-ci
* speculative : fix default RNG seed + set sparams.n_probs
* Update tests/test-sampling.cpp
Co-authored-by: slaren <slarengh@gmail.com >
* sampling : add clarifying comment [no ci]
---------
Co-authored-by: slaren <slarengh@gmail.com >
2024-09-24 09:03:17 +03:00
f3979df762
flake.lock: Update ( #9586 )
...
Flake lock file updates:
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/4f807e8940284ad7925ebd0a0993d2a1791acb2f?narHash=sha256-IiA3jfbR7K/B5%2B9byVi9BZGWTD4VSbWe8VLpp9B/iYk%3D' (2024-09-11)
→ 'github:NixOS/nixpkgs/c04d5652cfa9742b1d519688f65d1bbccea9eb7e?narHash=sha256-PmUr/2GQGvFTIJ6/Tvsins7Q43KTMvMFhvG6oaYK%2BWk%3D' (2024-09-19)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-23 08:43:40 -07:00
Georgi Gerganov and GitHub
37f8c7b4c9
perplexity : remove extra new lines after chunks ( #9596 )
2024-09-23 11:28:02 +03:00
Georgi Gerganov and GitHub
bf9c1013ac
metal : use F32 prec for K*Q in vec FA ( #9595 )
...
ggml-ci
2024-09-23 11:27:47 +03:00
Georgi Gerganov
d13edb17ed
ggml : fix builds ( #0 )
...
ggml-ci
2024-09-20 21:15:05 +03:00
Georgi Gerganov
27609c49b9
ggml : fix trailing whitespace ( #0 )
...
ggml-ci
2024-09-20 21:15:05 +03:00
Georgi Gerganov
4301535326
sync : ggml
...
ggml-ci
2024-09-20 21:15:05 +03:00
Georgi Gerganov
a6809c6a2e
examples : add null threadpool args where needed (ggml/0)
...
ggml-ci
2024-09-20 21:15:05 +03:00
Georgi Gerganov and GitHub
d39e26741f
examples : flush log upon ctrl+c ( #9559 )
2024-09-20 11:46:56 +03:00
Georgi Gerganov and GitHub
6026da52d6
server : clean-up completed tasks from waiting list ( #9531 )
...
ggml-ci
2024-09-19 12:44:53 +03:00
Georgi Gerganov and GitHub
0d2f22e45c
scripts : verify py deps at the start of compare ( #9520 )
2024-09-18 18:34:32 +03:00
Georgi Gerganov and GitHub
19514d632e
cmake : do not hide GGML options + rename option ( #9465 )
...
* cmake : do not hide GGML options
ggml-ci
* build : rename flag GGML_CUDA_USE_GRAPHS -> GGML_CUDA_GRAPHS
for consistency
ggml-ci
2024-09-16 10:27:50 +03:00
Georgi Gerganov and GitHub
c4965a64f7
metal : handle zero-sized allocs ( #9466 )
2024-09-16 09:05:56 +03:00
Georgi Gerganov and GitHub
90a2fff0e7
flake.lock: Update ( #9488 )
2024-09-15 19:14:23 -07:00
Georgi Gerganov and GitHub
6262d13e0b
common : reimplement logging ( #9418 )
...
https://github.com/ggerganov/llama.cpp/pull/9418
2024-09-15 20:46:12 +03:00
1f4111e540
cmake : use list(APPEND ...) instead of set() + dedup linker ( #9463 )
...
* cmake : use list(APPEND ...) instead of set() + dedup linker
ggml-ci
* cmake : try fix sycl
* cmake : try to fix sycl 2
* cmake : fix sycl build (#9469 )
* try fix sycl build
* use CMAKE_CXX_FLAGS as a string variable
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com >
* one more CMAKE_CXX_FLAGS fix (#9471 )
---------
Co-authored-by: Michael Podvitskiy <podvitskiymichael@gmail.com >
2024-09-14 10:55:05 +03:00
0abc6a2c25
llama : llama_perf + option to disable timings during decode ( #9355 )
...
* llama : llama_perf + option to disable timings during decode
ggml-ci
* common : add llama_arg
* Update src/llama.cpp
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com >
* perf : separate functions in the API
ggml-ci
* perf : safer pointer handling + naming update
ggml-ci
* minor : better local var name
* perf : abort on invalid sampler pointer
ggml-ci
---------
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com >
2024-09-13 09:53:38 +03:00
Georgi Gerganov and GitHub
d6a04f872d
ggml : hide ggml_object, ggml_cgraph, ggml_hash_set ( #9408 )
...
* ggml : hide ggml_object, ggml_cgraph, ggml_hash_set
ggml-ci
* ggml : add ggml-impl.h to backends
* ggml : fix compiler warnings
ggml-ci
* ggml : add assert upon adding nodes
2024-09-12 14:23:49 +03:00
Georgi Gerganov and GitHub
d2b496bff4
batched-bench : remove unused code ( #9305 )
2024-09-11 10:03:54 +03:00
cb9c933eb2
flake.lock: Update ( #9360 )
...
Flake lock file updates:
• Updated input 'flake-parts':
'github:hercules-ci/flake-parts/af510d4a62d071ea13925ce41c95e3dec816c01d?narHash=sha256-ODYRm8zHfLTH3soTFWE452ydPYz2iTvr9T8ftDMUQ3E%3D' (2024-08-30)
→ 'github:hercules-ci/flake-parts/567b938d64d4b4112ee253b9274472dc3a346eb6?narHash=sha256-%2Bebgonl3NbiKD2UD0x4BszCZQ6sTfL4xioaM49o5B3Y%3D' (2024-09-01)
• Updated input 'flake-parts/nixpkgs-lib':
'https://github.com/NixOS/nixpkgs/archive/a5d394176e64ab29c852d03346c1fc9b0b7d33eb.tar.gz?narHash=sha256-uFf2QeW7eAHlYXuDktm9c25OxOyCoUOQmh5SZ9amE5Q%3D ' (2024-08-01)
→ 'https://github.com/NixOS/nixpkgs/archive/356624c12086a18f2ea2825fed34523d60ccc4e3.tar.gz?narHash=sha256-Ss8QWLXdr2JCBPcYChJhz4xJm%2Bh/xjl4G0c0XlP6a74%3D ' (2024-09-01)
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/71e91c409d1e654808b2621f28a327acfdad8dc2?narHash=sha256-GnR7/ibgIH1vhoy8cYdmXE6iyZqKqFxQSVkFgosBh6w%3D' (2024-08-28)
→ 'github:NixOS/nixpkgs/574d1eac1c200690e27b8eb4e24887f8df7ac27c?narHash=sha256-v3rIhsJBOMLR8e/RNWxr828tB%2BWywYIoajrZKFM%2B0Gg%3D' (2024-09-06)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-10 15:46:59 -07:00
Georgi Gerganov
00ba2ff781
metal : fix compile warning with GGML_METAL_NDEBUG ( #0 )
2024-09-10 10:17:43 +03:00
Georgi Gerganov and GitHub
38ca6f644b
readme : update hot topics
2024-09-09 15:51:37 +03:00
Georgi Gerganov and GitHub
e079bffb66
cuda : fix FA Q src index (1 -> 0) ( #9374 )
2024-09-08 22:01:02 +03:00
Georgi Gerganov
a876861455
metal : update support condition for im2col + fix warning ( #0 )
2024-09-08 11:05:55 +03:00
Georgi Gerganov
385decbd63
sync : ggml
2024-09-08 11:05:55 +03:00
Georgi Gerganov
60a3107ccd
scripts : option to increase git patch context
2024-09-08 11:05:55 +03:00
Georgi Gerganov
ba1cf846ed
cann : fix doxy (ggml/0)
2024-09-08 11:05:55 +03:00
Georgi Gerganov
51d964a4ef
cuda : mark BF16 CONT as unsupported
2024-09-08 11:05:55 +03:00
Georgi Gerganov and GitHub
a5b5d9a101
llama.android : fix build ( #9350 )
2024-09-08 00:33:50 +03:00
Georgi Gerganov and GitHub
f12295b8a9
llama : fix empty ring buffer push ( #9358 )
2024-09-08 00:33:33 +03:00
Georgi Gerganov and GitHub
faf69d4237
llama : sanitize invalid tokens ( #9357 )
...
* common : do not add null tokens during warmup
ggml-ci
* llama : check that the input tokens are valid
ggml-ci
* tests : fix batch size of bert model
ggml-ci
2024-09-08 00:33:13 +03:00
Georgi Gerganov and GitHub
df270ef745
llama : refactor sampling v2 ( #9294 )
...
- Add `struct llama_sampler` and `struct llama_sampler_i`
- Add `llama_sampler_` API
- Add `llama_sampler_chain_` API for chaining multiple samplers
- Remove `LLAMA_API_INTERNAL`
- Add `llama_perf_` API and remove old `llama_print_timings` and `llama_reset_timings`
2024-09-07 15:16:19 +03:00
7605ae7daf
flake.lock: Update ( #9261 )
...
Flake lock file updates:
• Updated input 'flake-parts':
'github:hercules-ci/flake-parts/8471fe90ad337a8074e957b69ca4d0089218391d?narHash=sha256-XOQkdLafnb/p9ij77byFQjDf5m5QYl9b2REiVClC%2Bx4%3D' (2024-08-01)
→ 'github:hercules-ci/flake-parts/af510d4a62d071ea13925ce41c95e3dec816c01d?narHash=sha256-ODYRm8zHfLTH3soTFWE452ydPYz2iTvr9T8ftDMUQ3E%3D' (2024-08-30)
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/c374d94f1536013ca8e92341b540eba4c22f9c62?narHash=sha256-Z/ELQhrSd7bMzTO8r7NZgi9g5emh%2BaRKoCdaAv5fiO0%3D' (2024-08-21)
→ 'github:NixOS/nixpkgs/71e91c409d1e654808b2621f28a327acfdad8dc2?narHash=sha256-GnR7/ibgIH1vhoy8cYdmXE6iyZqKqFxQSVkFgosBh6w%3D' (2024-08-28)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-03 16:36:43 -07:00
Georgi Gerganov and GitHub
b69a480af4
readme : refactor API section + remove old hot topics
2024-09-03 10:00:36 +03:00
Georgi Gerganov
c6d4cb4655
llama : minor style
2024-09-02 11:52:37 +03:00
1d1ccce676
flake.lock: Update ( #9162 )
...
Flake lock file updates:
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/c3aa7b8938b17aebd2deecf7be0636000d62a2b9?narHash=sha256-med8%2B5DSWa2UnOqtdICndjDAEjxr5D7zaIiK4pn0Q7c%3D' (2024-08-14)
→ 'github:NixOS/nixpkgs/c374d94f1536013ca8e92341b540eba4c22f9c62?narHash=sha256-Z/ELQhrSd7bMzTO8r7NZgi9g5emh%2BaRKoCdaAv5fiO0%3D' (2024-08-21)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-28 21:28:14 -07:00
Georgi Gerganov
20f1789dfb
vulkan : fix build ( #0 )
...
ggml-ci
2024-08-27 22:41:27 +03:00
Georgi Gerganov
231cff5f6f
sync : ggml
2024-08-27 22:41:27 +03:00
Georgi Gerganov and GitHub
06658ad7c3
metal : separate scale and mask from QKT in FA kernel ( #9189 )
...
* metal : separate scale and mask from QKT in FA kernel
* metal : ne01 check no longer necessary
* metal : keep data in local memory
2024-08-26 18:31:02 +03:00
Georgi Gerganov and GitHub
fc18425b6a
ggml : add SSM Metal kernels ( #8546 )
...
* ggml : add ggml_ssm_conv metal impl
* ggml : add ssm_scan metal impl
ggml-ci
2024-08-26 17:55:36 +03:00
Georgi Gerganov and GitHub
879275ac98
tests : fix compile warnings for unreachable code ( #9185 )
...
ggml-ci
2024-08-26 16:30:25 +03:00
Georgi Gerganov and GitHub
7a3df798fc
ci : add VULKAN support to ggml-ci ( #9055 )
2024-08-26 12:19:39 +03:00
Georgi Gerganov and GitHub
e5edb210cd
server : update deps ( #9183 )
2024-08-26 12:16:57 +03:00
Georgi Gerganov and GitHub
554b049068
flake.lock: Update ( #9068 )
2024-08-18 07:43:32 -07:00
Georgi Gerganov and GitHub
d5492f0525
ci : disable bench workflow ( #9010 )
2024-08-15 10:11:11 +03:00
Georgi Gerganov and GitHub
43bdd3ce18
cmake : remove unused option GGML_CURL ( #9011 )
2024-08-14 09:14:49 +03:00
Georgi Gerganov and GitHub
d3ae0ee8d7
py : fix requirements check '==' -> '~=' ( #8982 )
...
* py : fix requirements check '==' -> '~='
* cont : fix the fix
* ci : run on all requirements.txt
2024-08-12 11:02:01 +03:00
Georgi Gerganov and GitHub
5ef07e25ac
server : handle models with missing EOS token ( #8997 )
...
ggml-ci
2024-08-12 10:21:50 +03:00
Georgi Gerganov and GitHub
8cd1bcfd3f
flake.lock: Update ( #8979 )
2024-08-11 06:58:58 -07:00
Georgi Gerganov and GitHub
b72942fac9
Merge commit from fork
2024-08-09 23:03:21 +03:00
Georgi Gerganov and GitHub
272e3bd95e
make : fix llava obj file race ( #8946 )
...
ggml-ci
2024-08-09 18:24:30 +03:00
Georgi Gerganov and GitHub
45a55b91aa
llama : better replace_all (cont) ( #8926 )
...
* llama : better replace_all (cont)
ggml-ci
* code : deduplicate replace_all
ggml-ci
2024-08-09 18:23:52 +03:00
Georgi Gerganov
4305b57c80
sync : ggml
2024-08-09 10:03:48 +03:00
Georgi Gerganov
afd27f01fe
scripts : sync cann files ( #0 )
2024-08-08 14:56:52 +03:00
Georgi Gerganov
366d486c16
scripts : fix sync filenames ( #0 )
2024-08-08 14:40:12 +03:00
Georgi Gerganov
e44a561ab0
sync : ggml
2024-08-08 13:19:47 +03:00
Georgi Gerganov
5b33ea1ee7
metal : fix struct name (ggml/912)
...
ggml-ci
2024-08-08 13:19:31 +03:00
Georgi Gerganov and GitHub
0bf16de07b
contributing : add note about write access
2024-08-06 11:48:01 +03:00
Georgi Gerganov and GitHub
f1ea5146d7
llama : better replace_all ( #8852 )
2024-08-05 08:53:39 +03:00
Georgi Gerganov
5587e57a76
sync : ggml
...
ggml-ci
2024-08-05 08:50:57 +03:00
Georgi Gerganov and GitHub
4b77ea95f5
flake.lock: Update ( #8847 )
2024-08-03 19:53:20 -07:00
Georgi Gerganov and GitHub
140074bb86
flake.lock: Update ( #8729 )
2024-07-30 05:58:57 -07:00
Georgi Gerganov
5e2727fe03
scripts : sync vulkan-shaders ( #0 )
2024-07-27 18:08:47 +03:00
Georgi Gerganov
56f20aa25d
scripts : sync ggml-aarch64 sources
2024-07-27 18:07:33 +03:00
Georgi Gerganov
345c8c0c87
ggml : add missing semicolon ( #0 )
...
ggml-ci
2024-07-27 17:43:44 +03:00
Georgi Gerganov
ae7985cd7b
sync : ggml
...
ggml-ci
2024-07-27 17:43:44 +03:00
Georgi Gerganov and GitHub
92090eca21
llama : add function for model-based max number of graph nodes ( #8622 )
...
* llama : model-based max number of graph nodes
ggml-ci
* llama : disable 405B max_nodes path due to lack of complaints
ggml-ci
2024-07-27 14:59:29 +03:00
Georgi Gerganov and GitHub
4226a8d10e
llama : fix build + fix fabs compile warnings ( #8683 )
...
ggml-ci
2024-07-25 19:57:31 +03:00
Georgi Gerganov
88954f7fbd
tests : fix printfs ( #8068 )
2024-07-25 18:58:04 +03:00
Georgi Gerganov and GitHub
eddcb5238b
ggml : add and use ggml_cpu_has_llamafile() ( #8664 )
2024-07-25 12:37:42 +03:00
Georgi Gerganov and GitHub
938943cdbf
llama : move vocab, grammar and sampling into separate files ( #8508 )
...
* llama : move sampling code into llama-sampling
ggml-ci
* llama : move grammar code into llama-grammar
ggml-ci
* cont
ggml-ci
* cont : pre-fetch rules
* cont
ggml-ci
* llama : deprecate llama_sample_grammar
* llama : move tokenizers into llama-vocab
ggml-ci
* make : update llama.cpp deps [no ci]
* llama : redirect external API to internal APIs
ggml-ci
* llama : suffix the internal APIs with "_impl"
ggml-ci
* llama : clean-up
2024-07-23 13:10:17 +03:00
Georgi Gerganov and GitHub
e7e6487ba0
contrib : clarify PR squashing + module names ( #8630 )
...
* contrib : clarify PR squashing
* contrib : fix typo + add list of modules
2024-07-23 11:28:38 +03:00
Georgi Gerganov and GitHub
6f11a83e4e
llama : allow overrides for tokenizer flags ( #8614 )
...
ggml-ci
2024-07-22 13:33:22 +03:00
Georgi Gerganov and GitHub
e093dd2382
tests : re-enable tokenizer tests ( #8611 )
...
* models : remove duplicated gpt-2 vocab
* models : remove old stablelm vocab
* tests : re-enable MPT tokenizer tests
* tests : re-enable DeepSeek tokenizer tests
* cmake : sort
ggml-ci
2024-07-22 13:32:49 +03:00
Georgi Gerganov and GitHub
45f2c19cc5
flake.lock: Update ( #8610 )
2024-07-21 06:45:10 -07:00