nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 26 Packages Projects Releases Wiki Activity
3,869 Commits 1 Branch 0 Tags
5639971466ed74386a1811938022f0c333007b55
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Georgi GerganovandGitHub 148844fe97 examples : remove benchmark (#9704) 
ggml-ci
 2024-10-02 10:14:44 +03:00
Georgi Gerganov f1b8c42711 sync : ggml 2024-10-01 16:09:42 +03:00
Georgi GerganovandGitHub cad341d889 metal : reduce command encoding overhead (#9698) 
* metal : reduce command encoding overhead

ggml-ci

* metal : add comments
 2024-10-01 16:00:25 +03:00
Georgi Gerganov a90484c6d9 llama : print correct model type for Llama 3.2 1B and 3B 2024-10-01 11:42:01 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		ace4f4be37 flake.lock: Update (#9680) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/c04d5652cfa9742b1d519688f65d1bbccea9eb7e?narHash=sha256-PmUr/2GQGvFTIJ6/Tvsins7Q43KTMvMFhvG6oaYK%2BWk%3D' (2024-09-19)
  → 'github:NixOS/nixpkgs/1925c603f17fc89f4c8f6bf6f631a802ad85d784?narHash=sha256-J%2BPeFKSDV%2BpHL7ukkfpVzCOO7mBSrrpJ3svwBFABbhI%3D' (2024-09-26)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-30 07:48:49 -07:00
Georgi GerganovGitHubWilly Tarreau
 		c919d5db39 ggml : define missing HWCAP flags (#9684) 
ggml-ci

Co-authored-by: Willy Tarreau <w@1wt.eu>
 2024-09-29 21:18:23 +03:00
Georgi Gerganov d0b1d663e4 sync : ggml 2024-09-29 21:16:07 +03:00
Georgi Gerganov 6084bfb261 ggml : fix GGML_MAX_N_THREADS + improve formatting (ggml/969) 2024-09-29 21:15:35 +03:00
Georgi GerganovandGitHub 589b48d41e contrib : add Resources section (#9675) 2024-09-29 14:38:18 +03:00
Georgi GerganovGitHubXuan Son NguyenXuan Son Nguyen
 		f4d2b8846a llama : add reranking support (#9510) 
* py : add XLMRobertaForSequenceClassification [no ci]

* py : fix scalar-tensor conversion [no ci]

* py : fix position embeddings chop [no ci]

* llama : read new cls tensors [no ci]

* llama : add classigication head (wip) [no ci]

* llama : add "rank" pooling type

ggml-ci

* server : add rerank endpoint

ggml-ci

* llama : aboud ggml_repeat during classification

* rerank : cleanup + comments

* server : accept /rerank endpoint in addition to /v1/rerank [no ci]

* embedding : parse special tokens

* jina : support v1 reranker

* vocab : minor style

ggml-ci

* server : initiate tests for later

ggml-ci

* server : add docs

* llama : add comment [no ci]

* llama : fix uninitialized tensors

* ci : add rerank tests

ggml-ci

* add reranking test

* change test data

* Update examples/server/server.cpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* add `--reranking` argument

* update server docs

* llama : fix comment [no ci]

ggml-ci

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-09-28 17:42:03 +03:00
Georgi Gerganov 739842703e llama : add comment about thread-safety [no ci] (#9449) 2024-09-28 15:13:42 +03:00
Georgi GerganovandGitHub b5de3b74a5 readme : update hot topics 2024-09-27 20:57:51 +03:00
Georgi Gerganov bb5f819975 sync : ggml 2024-09-24 11:01:18 +03:00
Georgi Gerganov c038931615 examples : adapt to ggml.h changes (ggml/0) 
ggml-ci
 2024-09-24 11:00:52 +03:00
Georgi GerganovandGitHub 31ac5834fe llama : keep track of all EOG tokens in the vocab (#9609) 
ggml-ci
 2024-09-24 10:16:06 +03:00
Georgi GerganovandGitHub cea1486ecf log : add CONT level for continuing previous log entry (#9610) 2024-09-24 10:15:35 +03:00
Georgi GerganovGitHubslaren
 		b0f27361f3 sampling : avoid expensive softmax during greedy sampling (#9605) 
* sampling : avoid expensive softmax during greedy sampling

ggml-ci

* speculative : fix default RNG seed + set sparams.n_probs

* Update tests/test-sampling.cpp

Co-authored-by: slaren <slarengh@gmail.com>

* sampling : add clarifying comment [no ci]

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-09-24 09:03:17 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		f3979df762 flake.lock: Update (#9586) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/4f807e8940284ad7925ebd0a0993d2a1791acb2f?narHash=sha256-IiA3jfbR7K/B5%2B9byVi9BZGWTD4VSbWe8VLpp9B/iYk%3D' (2024-09-11)
  → 'github:NixOS/nixpkgs/c04d5652cfa9742b1d519688f65d1bbccea9eb7e?narHash=sha256-PmUr/2GQGvFTIJ6/Tvsins7Q43KTMvMFhvG6oaYK%2BWk%3D' (2024-09-19)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-23 08:43:40 -07:00
Georgi GerganovandGitHub 37f8c7b4c9 perplexity : remove extra new lines after chunks (#9596) 2024-09-23 11:28:02 +03:00
Georgi GerganovandGitHub bf9c1013ac metal : use F32 prec for K*Q in vec FA (#9595) 
ggml-ci
 2024-09-23 11:27:47 +03:00
Georgi Gerganov d13edb17ed ggml : fix builds (#0) 
ggml-ci
 2024-09-20 21:15:05 +03:00
Georgi Gerganov 27609c49b9 ggml : fix trailing whitespace (#0) 
ggml-ci
 2024-09-20 21:15:05 +03:00
Georgi Gerganov 4301535326 sync : ggml 
ggml-ci
 2024-09-20 21:15:05 +03:00
Georgi Gerganov a6809c6a2e examples : add null threadpool args where needed (ggml/0) 
ggml-ci
 2024-09-20 21:15:05 +03:00
Georgi GerganovandGitHub d39e26741f examples : flush log upon ctrl+c (#9559) 2024-09-20 11:46:56 +03:00
Georgi GerganovandGitHub 6026da52d6 server : clean-up completed tasks from waiting list (#9531) 
ggml-ci
 2024-09-19 12:44:53 +03:00
Georgi GerganovandGitHub 0d2f22e45c scripts : verify py deps at the start of compare (#9520) 2024-09-18 18:34:32 +03:00
Georgi GerganovandGitHub 19514d632e cmake : do not hide GGML options + rename option (#9465) 
* cmake : do not hide GGML options

ggml-ci

* build : rename flag GGML_CUDA_USE_GRAPHS -> GGML_CUDA_GRAPHS

for consistency

ggml-ci
 2024-09-16 10:27:50 +03:00
Georgi GerganovandGitHub c4965a64f7 metal : handle zero-sized allocs (#9466) 2024-09-16 09:05:56 +03:00
Georgi GerganovandGitHub 90a2fff0e7 flake.lock: Update (#9488) 2024-09-15 19:14:23 -07:00
Georgi GerganovandGitHub 6262d13e0b common : reimplement logging (#9418) 
https://github.com/ggerganov/llama.cpp/pull/9418
 2024-09-15 20:46:12 +03:00
Georgi GerganovGitHubMichael Podvitskiy
 		1f4111e540 cmake : use list(APPEND ...) instead of set() + dedup linker (#9463) 
* cmake : use list(APPEND ...) instead of set() + dedup linker

ggml-ci

* cmake : try fix sycl

* cmake : try to fix sycl 2

* cmake : fix sycl build (#9469)

* try fix sycl build

* use CMAKE_CXX_FLAGS as a string variable

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* one more CMAKE_CXX_FLAGS fix (#9471)

---------

Co-authored-by: Michael Podvitskiy <podvitskiymichael@gmail.com>
 2024-09-14 10:55:05 +03:00
Georgi GerganovGitHubXuan Son Nguyen
 		0abc6a2c25 llama : llama_perf + option to disable timings during decode (#9355) 
* llama : llama_perf + option to disable timings during decode

ggml-ci

* common : add llama_arg

* Update src/llama.cpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

* perf : separate functions in the API

ggml-ci

* perf : safer pointer handling + naming update

ggml-ci

* minor : better local var name

* perf : abort on invalid sampler pointer

ggml-ci

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-09-13 09:53:38 +03:00
Georgi GerganovandGitHub d6a04f872d ggml : hide ggml_object, ggml_cgraph, ggml_hash_set (#9408) 
* ggml : hide ggml_object, ggml_cgraph, ggml_hash_set

ggml-ci

* ggml : add ggml-impl.h to backends

* ggml : fix compiler warnings

ggml-ci

* ggml : add assert upon adding nodes
 2024-09-12 14:23:49 +03:00
Georgi GerganovandGitHub d2b496bff4 batched-bench : remove unused code (#9305) 2024-09-11 10:03:54 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		cb9c933eb2 flake.lock: Update (#9360) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/af510d4a62d071ea13925ce41c95e3dec816c01d?narHash=sha256-ODYRm8zHfLTH3soTFWE452ydPYz2iTvr9T8ftDMUQ3E%3D' (2024-08-30)
  → 'github:hercules-ci/flake-parts/567b938d64d4b4112ee253b9274472dc3a346eb6?narHash=sha256-%2Bebgonl3NbiKD2UD0x4BszCZQ6sTfL4xioaM49o5B3Y%3D' (2024-09-01)
• Updated input 'flake-parts/nixpkgs-lib':
    'https://github.com/NixOS/nixpkgs/archive/a5d394176e64ab29c852d03346c1fc9b0b7d33eb.tar.gz?narHash=sha256-uFf2QeW7eAHlYXuDktm9c25OxOyCoUOQmh5SZ9amE5Q%3D' (2024-08-01)
  → 'https://github.com/NixOS/nixpkgs/archive/356624c12086a18f2ea2825fed34523d60ccc4e3.tar.gz?narHash=sha256-Ss8QWLXdr2JCBPcYChJhz4xJm%2Bh/xjl4G0c0XlP6a74%3D' (2024-09-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/71e91c409d1e654808b2621f28a327acfdad8dc2?narHash=sha256-GnR7/ibgIH1vhoy8cYdmXE6iyZqKqFxQSVkFgosBh6w%3D' (2024-08-28)
  → 'github:NixOS/nixpkgs/574d1eac1c200690e27b8eb4e24887f8df7ac27c?narHash=sha256-v3rIhsJBOMLR8e/RNWxr828tB%2BWywYIoajrZKFM%2B0Gg%3D' (2024-09-06)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-10 15:46:59 -07:00
Georgi Gerganov 00ba2ff781 metal : fix compile warning with GGML_METAL_NDEBUG (#0) 2024-09-10 10:17:43 +03:00
Georgi GerganovandGitHub 38ca6f644b readme : update hot topics 2024-09-09 15:51:37 +03:00
Georgi GerganovandGitHub e079bffb66 cuda : fix FA Q src index (1 -> 0) (#9374) 2024-09-08 22:01:02 +03:00
Georgi Gerganov a876861455 metal : update support condition for im2col + fix warning (#0) 2024-09-08 11:05:55 +03:00
Georgi Gerganov 385decbd63 sync : ggml 2024-09-08 11:05:55 +03:00
Georgi Gerganov 60a3107ccd scripts : option to increase git patch context 2024-09-08 11:05:55 +03:00
Georgi Gerganov ba1cf846ed cann : fix doxy (ggml/0) 2024-09-08 11:05:55 +03:00
Georgi Gerganov 51d964a4ef cuda : mark BF16 CONT as unsupported 2024-09-08 11:05:55 +03:00
Georgi GerganovandGitHub a5b5d9a101 llama.android : fix build (#9350) 2024-09-08 00:33:50 +03:00
Georgi GerganovandGitHub f12295b8a9 llama : fix empty ring buffer push (#9358) 2024-09-08 00:33:33 +03:00
Georgi GerganovandGitHub faf69d4237 llama : sanitize invalid tokens (#9357) 
* common : do not add null tokens during warmup

ggml-ci

* llama : check that the input tokens are valid

ggml-ci

* tests : fix batch size of bert model

ggml-ci
 2024-09-08 00:33:13 +03:00
Georgi GerganovandGitHub df270ef745 llama : refactor sampling v2 (#9294) 
- Add `struct llama_sampler` and `struct llama_sampler_i`
- Add `llama_sampler_` API
- Add `llama_sampler_chain_` API for chaining multiple samplers
- Remove `LLAMA_API_INTERNAL`
- Add `llama_perf_` API and remove old `llama_print_timings` and `llama_reset_timings`
 2024-09-07 15:16:19 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		7605ae7daf flake.lock: Update (#9261) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/8471fe90ad337a8074e957b69ca4d0089218391d?narHash=sha256-XOQkdLafnb/p9ij77byFQjDf5m5QYl9b2REiVClC%2Bx4%3D' (2024-08-01)
  → 'github:hercules-ci/flake-parts/af510d4a62d071ea13925ce41c95e3dec816c01d?narHash=sha256-ODYRm8zHfLTH3soTFWE452ydPYz2iTvr9T8ftDMUQ3E%3D' (2024-08-30)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/c374d94f1536013ca8e92341b540eba4c22f9c62?narHash=sha256-Z/ELQhrSd7bMzTO8r7NZgi9g5emh%2BaRKoCdaAv5fiO0%3D' (2024-08-21)
  → 'github:NixOS/nixpkgs/71e91c409d1e654808b2621f28a327acfdad8dc2?narHash=sha256-GnR7/ibgIH1vhoy8cYdmXE6iyZqKqFxQSVkFgosBh6w%3D' (2024-08-28)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-03 16:36:43 -07:00
Georgi GerganovandGitHub b69a480af4 readme : refactor API section + remove old hot topics 2024-09-03 10:00:36 +03:00
Georgi Gerganov c6d4cb4655 llama : minor style 2024-09-02 11:52:37 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		1d1ccce676 flake.lock: Update (#9162) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/c3aa7b8938b17aebd2deecf7be0636000d62a2b9?narHash=sha256-med8%2B5DSWa2UnOqtdICndjDAEjxr5D7zaIiK4pn0Q7c%3D' (2024-08-14)
  → 'github:NixOS/nixpkgs/c374d94f1536013ca8e92341b540eba4c22f9c62?narHash=sha256-Z/ELQhrSd7bMzTO8r7NZgi9g5emh%2BaRKoCdaAv5fiO0%3D' (2024-08-21)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-08-28 21:28:14 -07:00
Georgi Gerganov 20f1789dfb vulkan : fix build (#0) 
ggml-ci
 2024-08-27 22:41:27 +03:00
Georgi Gerganov 231cff5f6f sync : ggml 2024-08-27 22:41:27 +03:00
Georgi GerganovandGitHub 06658ad7c3 metal : separate scale and mask from QKT in FA kernel (#9189) 
* metal : separate scale and mask from QKT in FA kernel

* metal : ne01 check no longer necessary

* metal : keep data in local memory
 2024-08-26 18:31:02 +03:00
Georgi GerganovandGitHub fc18425b6a ggml : add SSM Metal kernels (#8546) 
* ggml : add ggml_ssm_conv metal impl

* ggml : add ssm_scan metal impl

ggml-ci
 2024-08-26 17:55:36 +03:00
Georgi GerganovandGitHub 879275ac98 tests : fix compile warnings for unreachable code (#9185) 
ggml-ci
 2024-08-26 16:30:25 +03:00
Georgi GerganovandGitHub 7a3df798fc ci : add VULKAN support to ggml-ci (#9055) 2024-08-26 12:19:39 +03:00
Georgi GerganovandGitHub e5edb210cd server : update deps (#9183) 2024-08-26 12:16:57 +03:00
Georgi GerganovandGitHub 554b049068 flake.lock: Update (#9068) 2024-08-18 07:43:32 -07:00
Georgi GerganovandGitHub d5492f0525 ci : disable bench workflow (#9010) 2024-08-15 10:11:11 +03:00
Georgi GerganovandGitHub 43bdd3ce18 cmake : remove unused option GGML_CURL (#9011) 2024-08-14 09:14:49 +03:00
Georgi GerganovandGitHub d3ae0ee8d7 py : fix requirements check '==' -> '~=' (#8982) 
* py : fix requirements check '==' -> '~='

* cont : fix the fix

* ci : run on all requirements.txt
 2024-08-12 11:02:01 +03:00
Georgi GerganovandGitHub 5ef07e25ac server : handle models with missing EOS token (#8997) 
ggml-ci
 2024-08-12 10:21:50 +03:00
Georgi GerganovandGitHub 8cd1bcfd3f flake.lock: Update (#8979) 2024-08-11 06:58:58 -07:00
Georgi GerganovandGitHub b72942fac9 Merge commit from fork 2024-08-09 23:03:21 +03:00
Georgi GerganovandGitHub 272e3bd95e make : fix llava obj file race (#8946) 
ggml-ci
 2024-08-09 18:24:30 +03:00
Georgi GerganovandGitHub 45a55b91aa llama : better replace_all (cont) (#8926) 
* llama : better replace_all (cont)

ggml-ci

* code : deduplicate replace_all

ggml-ci
 2024-08-09 18:23:52 +03:00
Georgi Gerganov 4305b57c80 sync : ggml 2024-08-09 10:03:48 +03:00
Georgi Gerganov afd27f01fe scripts : sync cann files (#0) 2024-08-08 14:56:52 +03:00
Georgi Gerganov 366d486c16 scripts : fix sync filenames (#0) 2024-08-08 14:40:12 +03:00
Georgi Gerganov e44a561ab0 sync : ggml 2024-08-08 13:19:47 +03:00
Georgi Gerganov 5b33ea1ee7 metal : fix struct name (ggml/912) 
ggml-ci
 2024-08-08 13:19:31 +03:00
Georgi GerganovandGitHub 0bf16de07b contributing : add note about write access 2024-08-06 11:48:01 +03:00
Georgi GerganovandGitHub f1ea5146d7 llama : better replace_all (#8852) 2024-08-05 08:53:39 +03:00
Georgi Gerganov 5587e57a76 sync : ggml 
ggml-ci
 2024-08-05 08:50:57 +03:00
Georgi GerganovandGitHub 4b77ea95f5 flake.lock: Update (#8847) 2024-08-03 19:53:20 -07:00
Georgi GerganovandGitHub 140074bb86 flake.lock: Update (#8729) 2024-07-30 05:58:57 -07:00
Georgi Gerganov 5e2727fe03 scripts : sync vulkan-shaders (#0) 2024-07-27 18:08:47 +03:00
Georgi Gerganov 56f20aa25d scripts : sync ggml-aarch64 sources 2024-07-27 18:07:33 +03:00
Georgi Gerganov 345c8c0c87 ggml : add missing semicolon (#0) 
ggml-ci
 2024-07-27 17:43:44 +03:00
Georgi Gerganov ae7985cd7b sync : ggml 
ggml-ci
 2024-07-27 17:43:44 +03:00
Georgi GerganovandGitHub 92090eca21 llama : add function for model-based max number of graph nodes (#8622) 
* llama : model-based max number of graph nodes

ggml-ci

* llama : disable 405B max_nodes path due to lack of complaints

ggml-ci
 2024-07-27 14:59:29 +03:00
Georgi GerganovandGitHub 4226a8d10e llama : fix build + fix fabs compile warnings (#8683) 
ggml-ci
 2024-07-25 19:57:31 +03:00
Georgi Gerganov 88954f7fbd tests : fix printfs (#8068) 2024-07-25 18:58:04 +03:00
Georgi GerganovandGitHub eddcb5238b ggml : add and use ggml_cpu_has_llamafile() (#8664) 2024-07-25 12:37:42 +03:00
Georgi GerganovandGitHub 938943cdbf llama : move vocab, grammar and sampling into separate files (#8508) 
* llama : move sampling code into llama-sampling

ggml-ci

* llama : move grammar code into llama-grammar

ggml-ci

* cont

ggml-ci

* cont : pre-fetch rules

* cont

ggml-ci

* llama : deprecate llama_sample_grammar

* llama : move tokenizers into llama-vocab

ggml-ci

* make : update llama.cpp deps [no ci]

* llama : redirect external API to internal APIs

ggml-ci

* llama : suffix the internal APIs with "_impl"

ggml-ci

* llama : clean-up
 2024-07-23 13:10:17 +03:00
Georgi GerganovandGitHub e7e6487ba0 contrib : clarify PR squashing + module names (#8630) 
* contrib : clarify PR squashing

* contrib : fix typo + add list of modules
 2024-07-23 11:28:38 +03:00
Georgi GerganovandGitHub 6f11a83e4e llama : allow overrides for tokenizer flags (#8614) 
ggml-ci
 2024-07-22 13:33:22 +03:00
Georgi GerganovandGitHub e093dd2382 tests : re-enable tokenizer tests (#8611) 
* models : remove duplicated gpt-2 vocab

* models : remove old stablelm vocab

* tests : re-enable MPT tokenizer tests

* tests : re-enable DeepSeek tokenizer tests

* cmake : sort

ggml-ci
 2024-07-22 13:32:49 +03:00
Georgi GerganovandGitHub 45f2c19cc5 flake.lock: Update (#8610) 2024-07-21 06:45:10 -07:00
Georgi GerganovandGitHub 07283b1a90 gguf : handle null name during init (#8587) 2024-07-20 17:15:42 +03:00
Georgi GerganovandGitHub d197545530 llama : bump max layers from 256 to 512 (#8530) 
* llama : bump max layers from 256 to 512

* llama : replace asserts with exceptions
 2024-07-19 16:50:47 +03:00
Georgi GerganovandGitHub be0cfb4175 readme : fix server badge 2024-07-19 14:34:55 +03:00
Georgi GerganovandGitHub d65a8361fe llama : disable context-shift for DeepSeek v2 (#8501) 2024-07-17 10:32:59 +03:00
Georgi GerganovandGitHub 0efec57787 llama : valign + remove unused ftype (#8502) 2024-07-16 10:00:30 +03:00
Georgi Gerganov 9104bc20ed common : add --no-cont-batching arg (#6358) 2024-07-15 14:54:58 +03:00
Georgi Gerganov 3dfda05956 llama : de-duplicate deepseek2 norm 2024-07-15 14:10:39 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		aaab2419ea flake.lock: Update (#8475) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/9f4128e00b0ae8ec65918efeba59db998750ead6?narHash=sha256-rwz8NJZV%2B387rnWpTYcXaRNvzUSnnF9aHONoJIYmiUQ%3D' (2024-07-03)
  → 'github:NixOS/nixpkgs/7e7c39ea35c5cdd002cd4588b03a3fb9ece6fad9?narHash=sha256-EYekUHJE2gxeo2pM/zM9Wlqw1Uw2XTJXOSAO79ksc4Y%3D' (2024-07-12)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-07-14 08:54:02 -07:00
Georgi GerganovGitHubDaniel Han
 		73cf442e7b llama : fix Gemma-2 Query scaling factors (#8473) 
* 9B - query_pre_attn_scalar = 256 not 224

See https://github.com/google/gemma_pytorch/commit/03e657582d17cb5a8617ebf333c1c16f3694670e

Gemma 9b should use 256 and not 224 (self.config.hidden_size // self.config.num_attention_heads)

* llama : fix Gemma-2 Query scaling factor

ggml-ci

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
 2024-07-14 14:05:09 +03:00