nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 19 Packages Projects Releases Wiki Activity
3,692 Commits 1 Branch 0 Tags
ba1cf846edeb94fc567a9ab0f7ef705282e6d7d3
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Georgi Gerganov ba1cf846ed cann : fix doxy (ggml/0) 2024-09-08 11:05:55 +03:00
Georgi Gerganov 51d964a4ef cuda : mark BF16 CONT as unsupported 2024-09-08 11:05:55 +03:00
Georgi GerganovandGitHub a5b5d9a101 llama.android : fix build (#9350) 2024-09-08 00:33:50 +03:00
Georgi GerganovandGitHub f12295b8a9 llama : fix empty ring buffer push (#9358) 2024-09-08 00:33:33 +03:00
Georgi GerganovandGitHub faf69d4237 llama : sanitize invalid tokens (#9357) 
* common : do not add null tokens during warmup

ggml-ci

* llama : check that the input tokens are valid

ggml-ci

* tests : fix batch size of bert model

ggml-ci
 2024-09-08 00:33:13 +03:00
Georgi GerganovandGitHub df270ef745 llama : refactor sampling v2 (#9294) 
- Add `struct llama_sampler` and `struct llama_sampler_i`
- Add `llama_sampler_` API
- Add `llama_sampler_chain_` API for chaining multiple samplers
- Remove `LLAMA_API_INTERNAL`
- Add `llama_perf_` API and remove old `llama_print_timings` and `llama_reset_timings`
 2024-09-07 15:16:19 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		7605ae7daf flake.lock: Update (#9261) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/8471fe90ad337a8074e957b69ca4d0089218391d?narHash=sha256-XOQkdLafnb/p9ij77byFQjDf5m5QYl9b2REiVClC%2Bx4%3D' (2024-08-01)
  → 'github:hercules-ci/flake-parts/af510d4a62d071ea13925ce41c95e3dec816c01d?narHash=sha256-ODYRm8zHfLTH3soTFWE452ydPYz2iTvr9T8ftDMUQ3E%3D' (2024-08-30)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/c374d94f1536013ca8e92341b540eba4c22f9c62?narHash=sha256-Z/ELQhrSd7bMzTO8r7NZgi9g5emh%2BaRKoCdaAv5fiO0%3D' (2024-08-21)
  → 'github:NixOS/nixpkgs/71e91c409d1e654808b2621f28a327acfdad8dc2?narHash=sha256-GnR7/ibgIH1vhoy8cYdmXE6iyZqKqFxQSVkFgosBh6w%3D' (2024-08-28)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-09-03 16:36:43 -07:00
Georgi GerganovandGitHub b69a480af4 readme : refactor API section + remove old hot topics 2024-09-03 10:00:36 +03:00
Georgi Gerganov c6d4cb4655 llama : minor style 2024-09-02 11:52:37 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		1d1ccce676 flake.lock: Update (#9162) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/c3aa7b8938b17aebd2deecf7be0636000d62a2b9?narHash=sha256-med8%2B5DSWa2UnOqtdICndjDAEjxr5D7zaIiK4pn0Q7c%3D' (2024-08-14)
  → 'github:NixOS/nixpkgs/c374d94f1536013ca8e92341b540eba4c22f9c62?narHash=sha256-Z/ELQhrSd7bMzTO8r7NZgi9g5emh%2BaRKoCdaAv5fiO0%3D' (2024-08-21)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-08-28 21:28:14 -07:00
Georgi Gerganov 20f1789dfb vulkan : fix build (#0) 
ggml-ci
 2024-08-27 22:41:27 +03:00
Georgi Gerganov 231cff5f6f sync : ggml 2024-08-27 22:41:27 +03:00
Georgi GerganovandGitHub 06658ad7c3 metal : separate scale and mask from QKT in FA kernel (#9189) 
* metal : separate scale and mask from QKT in FA kernel

* metal : ne01 check no longer necessary

* metal : keep data in local memory
 2024-08-26 18:31:02 +03:00
Georgi GerganovandGitHub fc18425b6a ggml : add SSM Metal kernels (#8546) 
* ggml : add ggml_ssm_conv metal impl

* ggml : add ssm_scan metal impl

ggml-ci
 2024-08-26 17:55:36 +03:00
Georgi GerganovandGitHub 879275ac98 tests : fix compile warnings for unreachable code (#9185) 
ggml-ci
 2024-08-26 16:30:25 +03:00
Georgi GerganovandGitHub 7a3df798fc ci : add VULKAN support to ggml-ci (#9055) 2024-08-26 12:19:39 +03:00
Georgi GerganovandGitHub e5edb210cd server : update deps (#9183) 2024-08-26 12:16:57 +03:00
Georgi GerganovandGitHub 554b049068 flake.lock: Update (#9068) 2024-08-18 07:43:32 -07:00
Georgi GerganovandGitHub d5492f0525 ci : disable bench workflow (#9010) 2024-08-15 10:11:11 +03:00
Georgi GerganovandGitHub 43bdd3ce18 cmake : remove unused option GGML_CURL (#9011) 2024-08-14 09:14:49 +03:00
Georgi GerganovandGitHub d3ae0ee8d7 py : fix requirements check '==' -> '~=' (#8982) 
* py : fix requirements check '==' -> '~='

* cont : fix the fix

* ci : run on all requirements.txt
 2024-08-12 11:02:01 +03:00
Georgi GerganovandGitHub 5ef07e25ac server : handle models with missing EOS token (#8997) 
ggml-ci
 2024-08-12 10:21:50 +03:00
Georgi GerganovandGitHub 8cd1bcfd3f flake.lock: Update (#8979) 2024-08-11 06:58:58 -07:00
Georgi GerganovandGitHub b72942fac9 Merge commit from fork 2024-08-09 23:03:21 +03:00
Georgi GerganovandGitHub 272e3bd95e make : fix llava obj file race (#8946) 
ggml-ci
 2024-08-09 18:24:30 +03:00
Georgi GerganovandGitHub 45a55b91aa llama : better replace_all (cont) (#8926) 
* llama : better replace_all (cont)

ggml-ci

* code : deduplicate replace_all

ggml-ci
 2024-08-09 18:23:52 +03:00
Georgi Gerganov 4305b57c80 sync : ggml 2024-08-09 10:03:48 +03:00
Georgi Gerganov afd27f01fe scripts : sync cann files (#0) 2024-08-08 14:56:52 +03:00
Georgi Gerganov 366d486c16 scripts : fix sync filenames (#0) 2024-08-08 14:40:12 +03:00
Georgi Gerganov e44a561ab0 sync : ggml 2024-08-08 13:19:47 +03:00
Georgi Gerganov 5b33ea1ee7 metal : fix struct name (ggml/912) 
ggml-ci
 2024-08-08 13:19:31 +03:00
Georgi GerganovandGitHub 0bf16de07b contributing : add note about write access 2024-08-06 11:48:01 +03:00
Georgi GerganovandGitHub f1ea5146d7 llama : better replace_all (#8852) 2024-08-05 08:53:39 +03:00
Georgi Gerganov 5587e57a76 sync : ggml 
ggml-ci
 2024-08-05 08:50:57 +03:00
Georgi GerganovandGitHub 4b77ea95f5 flake.lock: Update (#8847) 2024-08-03 19:53:20 -07:00
Georgi GerganovandGitHub 140074bb86 flake.lock: Update (#8729) 2024-07-30 05:58:57 -07:00
Georgi Gerganov 5e2727fe03 scripts : sync vulkan-shaders (#0) 2024-07-27 18:08:47 +03:00
Georgi Gerganov 56f20aa25d scripts : sync ggml-aarch64 sources 2024-07-27 18:07:33 +03:00
Georgi Gerganov 345c8c0c87 ggml : add missing semicolon (#0) 
ggml-ci
 2024-07-27 17:43:44 +03:00
Georgi Gerganov ae7985cd7b sync : ggml 
ggml-ci
 2024-07-27 17:43:44 +03:00
Georgi GerganovandGitHub 92090eca21 llama : add function for model-based max number of graph nodes (#8622) 
* llama : model-based max number of graph nodes

ggml-ci

* llama : disable 405B max_nodes path due to lack of complaints

ggml-ci
 2024-07-27 14:59:29 +03:00
Georgi GerganovandGitHub 4226a8d10e llama : fix build + fix fabs compile warnings (#8683) 
ggml-ci
 2024-07-25 19:57:31 +03:00
Georgi Gerganov 88954f7fbd tests : fix printfs (#8068) 2024-07-25 18:58:04 +03:00
Georgi GerganovandGitHub eddcb5238b ggml : add and use ggml_cpu_has_llamafile() (#8664) 2024-07-25 12:37:42 +03:00
Georgi GerganovandGitHub 938943cdbf llama : move vocab, grammar and sampling into separate files (#8508) 
* llama : move sampling code into llama-sampling

ggml-ci

* llama : move grammar code into llama-grammar

ggml-ci

* cont

ggml-ci

* cont : pre-fetch rules

* cont

ggml-ci

* llama : deprecate llama_sample_grammar

* llama : move tokenizers into llama-vocab

ggml-ci

* make : update llama.cpp deps [no ci]

* llama : redirect external API to internal APIs

ggml-ci

* llama : suffix the internal APIs with "_impl"

ggml-ci

* llama : clean-up
 2024-07-23 13:10:17 +03:00
Georgi GerganovandGitHub e7e6487ba0 contrib : clarify PR squashing + module names (#8630) 
* contrib : clarify PR squashing

* contrib : fix typo + add list of modules
 2024-07-23 11:28:38 +03:00
Georgi GerganovandGitHub 6f11a83e4e llama : allow overrides for tokenizer flags (#8614) 
ggml-ci
 2024-07-22 13:33:22 +03:00
Georgi GerganovandGitHub e093dd2382 tests : re-enable tokenizer tests (#8611) 
* models : remove duplicated gpt-2 vocab

* models : remove old stablelm vocab

* tests : re-enable MPT tokenizer tests

* tests : re-enable DeepSeek tokenizer tests

* cmake : sort

ggml-ci
 2024-07-22 13:32:49 +03:00
Georgi GerganovandGitHub 45f2c19cc5 flake.lock: Update (#8610) 2024-07-21 06:45:10 -07:00
Georgi GerganovandGitHub 07283b1a90 gguf : handle null name during init (#8587) 2024-07-20 17:15:42 +03:00
Georgi GerganovandGitHub d197545530 llama : bump max layers from 256 to 512 (#8530) 
* llama : bump max layers from 256 to 512

* llama : replace asserts with exceptions
 2024-07-19 16:50:47 +03:00
Georgi GerganovandGitHub be0cfb4175 readme : fix server badge 2024-07-19 14:34:55 +03:00
Georgi GerganovandGitHub d65a8361fe llama : disable context-shift for DeepSeek v2 (#8501) 2024-07-17 10:32:59 +03:00
Georgi GerganovandGitHub 0efec57787 llama : valign + remove unused ftype (#8502) 2024-07-16 10:00:30 +03:00
Georgi Gerganov 9104bc20ed common : add --no-cont-batching arg (#6358) 2024-07-15 14:54:58 +03:00
Georgi Gerganov 3dfda05956 llama : de-duplicate deepseek2 norm 2024-07-15 14:10:39 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		aaab2419ea flake.lock: Update (#8475) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/9f4128e00b0ae8ec65918efeba59db998750ead6?narHash=sha256-rwz8NJZV%2B387rnWpTYcXaRNvzUSnnF9aHONoJIYmiUQ%3D' (2024-07-03)
  → 'github:NixOS/nixpkgs/7e7c39ea35c5cdd002cd4588b03a3fb9ece6fad9?narHash=sha256-EYekUHJE2gxeo2pM/zM9Wlqw1Uw2XTJXOSAO79ksc4Y%3D' (2024-07-12)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-07-14 08:54:02 -07:00
Georgi GerganovGitHubDaniel Han
 		73cf442e7b llama : fix Gemma-2 Query scaling factors (#8473) 
* 9B - query_pre_attn_scalar = 256 not 224

See https://github.com/google/gemma_pytorch/commit/03e657582d17cb5a8617ebf333c1c16f3694670e

Gemma 9b should use 256 and not 224 (self.config.hidden_size // self.config.num_attention_heads)

* llama : fix Gemma-2 Query scaling factor

ggml-ci

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
 2024-07-14 14:05:09 +03:00
Georgi GerganovandGitHub c917b67f06 metal : template-ify some of the kernels (#8447) 
ggml-ci
 2024-07-13 18:32:33 +03:00
Georgi GerganovGitHubXuan Son Nguyen
 		4e24cffd8c server : handle content array in chat API (#8449) 
* server : handle content array in chat API

* Update examples/server/utils.hpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-07-12 14:48:15 +03:00
Georgi GerganovandGitHub 6af51c0d96 main : print error on empty input (#8456) 2024-07-12 14:48:04 +03:00
Georgi GerganovandGitHub 71c1121d11 examples : sprintf -> snprintf (#8434) 
* examples : sprintf -> snprintf

ggml-ci

* examples : use sizeof() instead of hardcoded constants
 2024-07-12 10:46:14 +03:00
Georgi GerganovandGitHub 370b1f7e7a ggml : minor naming changes (#8433) 
* ggml : minor naming changes

ggml-ci

* ggml : use PRId64 [no ci]

* ggml : revert FA K/Q names
 2024-07-12 10:46:02 +03:00
Georgi Gerganov a977c11544 gitignore : deprecated binaries 2024-07-11 11:20:40 +03:00
Georgi GerganovandGitHub 7a221b672e llama : use F32 precision in Qwen2 attention and no FA (#8412) 2024-07-11 10:21:30 +03:00
Georgi GerganovandGitHub 6b2a849d1f ggml : move sgemm sources to llamafile subfolder (#8394) 
ggml-ci
 2024-07-10 15:23:29 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		7fdb6f73e3 flake.lock: Update (#8342) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/2a55567fcf15b1b1c7ed712a2c6fadaec7412ea8?narHash=sha256-iKzJcpdXih14qYVcZ9QC9XuZYnPc6T8YImb6dX166kw%3D' (2024-06-01)
  → 'github:hercules-ci/flake-parts/9227223f6d922fee3c7b190b2cc238a99527bbb7?narHash=sha256-pQMhCCHyQGRzdfAkdJ4cIWiw%2BJNuWsTX7f0ZYSyz0VY%3D' (2024-07-03)
• Updated input 'flake-parts/nixpkgs-lib':
    'https://github.com/NixOS/nixpkgs/archive/eb9ceca17df2ea50a250b6b27f7bf6ab0186f198.tar.gz?narHash=sha256-lIbdfCsf8LMFloheeE6N31%2BBMIeixqyQWbSr2vk79EQ%3D' (2024-06-01)
  → 'https://github.com/NixOS/nixpkgs/archive/5daf0514482af3f97abaefc78a6606365c9108e2.tar.gz?narHash=sha256-Fm2rDDs86sHy0/1jxTOKB1118Q0O3Uc7EC0iXvXKpbI%3D' (2024-07-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/b2852eb9365c6de48ffb0dc2c9562591f652242a?narHash=sha256-C8e9S7RzshSdHB7L%2Bv9I51af1gDM5unhJ2xO1ywxNH8%3D' (2024-06-27)
  → 'github:NixOS/nixpkgs/9f4128e00b0ae8ec65918efeba59db998750ead6?narHash=sha256-rwz8NJZV%2B387rnWpTYcXaRNvzUSnnF9aHONoJIYmiUQ%3D' (2024-07-03)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-07-08 15:36:38 -07:00
Georgi Gerganov 3f2d538b81 scripts : fix sync for sycl 2024-07-08 13:51:31 +03:00
Georgi Gerganov 2ee44c9a18 sync : ggml 
ggml-ci
 2024-07-08 12:23:00 +03:00
Georgi Gerganov 6847d54c4f tests : fix whitespace (#0) 2024-07-08 12:23:00 +03:00
Georgi GerganovandGitHub 6f0dbf6ab0 infill : assert prefix/suffix tokens + remove old space logic (#8351) 2024-07-08 09:34:35 +03:00
Georgi GerganovandGitHub b5040086d4 llama : fix n_rot default (#8348) 
ggml-ci
 2024-07-07 14:59:02 +03:00
Georgi Gerganov 7ed03b8974 llama : fix compile warning (#8304) 2024-07-05 17:32:09 +03:00
Georgi GerganovandGitHub 148ec970b6 convert : remove AWQ remnants (#8320) 2024-07-05 10:15:36 +03:00
Georgi GerganovandGitHub 2cccbaa008 llama : minor indentation during tensor loading (#8304) 
* llama : minor indentation during tensor loading

ggml-ci

* llama : use int for layer iterators [no ci]
 2024-07-05 10:15:24 +03:00
Georgi GerganovandGitHub aa5898dc53 llama : prefer n_ over num_ prefix (#8308) 2024-07-05 09:10:03 +03:00
Georgi GerganovandGitHub 6c05752c50 contributing : update guidelines (#8316) 2024-07-05 09:09:47 +03:00
Georgi GerganovGitHubFrancis Couture-Harpin
 		e235b267a2 py : switch to snake_case (#8305) 
* py : switch to snake_case

ggml-ci

* cont

ggml-ci

* cont

ggml-ci

* cont : fix link

* gguf-py : use snake_case in scripts entrypoint export

* py : rename requirements for convert_legacy_llama.py

Needed for scripts/check-requirements.sh

---------

Co-authored-by: Francis Couture-Harpin <git@compilade.net>
 2024-07-05 07:53:33 +03:00
Georgi GerganovandGitHub 20fc3804bf convert : fix gemma v1 tokenizer convert (#8248) 
ggml-ci
 2024-07-04 10:41:03 +03:00
Georgi GerganovandGitHub 257f8e41e2 nix : remove OpenCL remnants (#8235) 
* nix : remove OpenCL remnants

* minor : remove parentheses
 2024-07-01 14:46:18 +03:00
Georgi GerganovandGitHub d0a7145ba9 flake.lock: Update (#8218) 2024-06-30 16:09:34 -07:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>Philip Taron
 		ab3679112d flake.lock: Update (#8071) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/e9ee548d90ff586a6471b4ae80ae9cfcbceb3420?narHash=sha256-4Zu0RYRcAY/VWuu6awwq4opuiD//ahpc2aFHg2CWqFY%3D' (2024-06-13)
  → 'github:NixOS/nixpkgs/d603719ec6e294f034936c0d0dc06f689d91b6c3?narHash=sha256-k3JqJrkdoYwE3fHE6xGDY676AYmyh4U2Zw%2B0Bwe5DLU%3D' (2024-06-20)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Philip Taron <philip.taron@gmail.com>
 2024-06-27 08:37:29 -07:00
Georgi Gerganov c70d117c37 scripts : fix filename sync 2024-06-26 23:25:22 +03:00
Georgi Gerganov f2d48fffde sync : ggml 2024-06-26 19:39:19 +03:00
Georgi Gerganov 4713bf3093 authors : regen 2024-06-26 19:36:44 +03:00
Georgi GerganovandGitHub 0e814dfc42 devops : remove clblast + LLAMA_CUDA -> GGML_CUDA (#8139) 
ggml-ci
 2024-06-26 19:32:07 +03:00
Georgi GerganovandGitHub a95631ee97 readme : update API notes 2024-06-26 19:26:13 +03:00
Georgi GerganovGitHubslaren
 		f3f65429c4 llama : reorganize source code + improve CMake (#8006) 
* scripts : update sync [no ci]

* files : relocate [no ci]

* ci : disable kompute build [no ci]

* cmake : fixes [no ci]

* server : fix mingw build

ggml-ci

* cmake : minor [no ci]

* cmake : link math library [no ci]

* cmake : build normal ggml library (not object library) [no ci]

* cmake : fix kompute build

ggml-ci

* make,cmake : fix LLAMA_CUDA + replace GGML_CDEF_PRIVATE

ggml-ci

* move public backend headers to the public include directory (#8122)

* move public backend headers to the public include directory

* nix test

* spm : fix metal header

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* scripts : fix sync paths [no ci]

* scripts : sync ggml-blas.h [no ci]

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-06-26 18:33:02 +03:00
Georgi GerganovandGitHub a927b0f3dd llama : optimize long word tokenization with WPM (#8034) 
ggml-ci
 2024-06-21 08:51:28 +03:00
Georgi GerganovandGitHub a04a953cab codecov : remove (#8004) 2024-06-19 13:04:36 +03:00
Georgi Gerganov 5326bcceeb ggml : sync 2024-06-18 09:50:45 +03:00
Georgi Gerganovandslaren e6ecc2be47 whisper : use ggml_backend_sched (whisper/2239) 
* whisper : use ggml_backend_sched (wip)

* use sched in whisper_allocr

* whisper : single backend in whisper_context

* whisper : remove whisper_state->backends_used

* whisper : remove whisper_context->backend

* whisper : reset scheduler after init

* whisper : fix external encoder (e.g. CoreML)

* whisper : cleanup

* whisper : handle null GPU buffer types + fix sycl

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-06-18 09:50:40 +03:00
Georgi GerganovandGitHub 7c26775adb llama : disable FA if KV head size do not match (#7982) 2024-06-17 19:40:01 +03:00
Georgi GerganovandGitHub 21be9cab94 rpc : fix load/store misaligned addresses (#7948) 2024-06-17 11:09:20 +03:00
Georgi GerganovandJohannes Gäßler 19b7a836f6 cuda : fix bounds check for src0 rows in MMVQ kernel (whisper/2231) 
* cuda : fix bounds check for src0 rows in MMVQ kernel

* Update ggml-cuda/mmvq.cu

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2024-06-16 20:32:49 +03:00
Georgi GerganovandGitHub bc6c457fa3 flake.lock: Update (#7951) 2024-06-16 09:16:21 -07:00
Georgi GerganovandGitHub 52399254b3 unicode : avoid char32_t (#7957) 
ggml-ci
 2024-06-16 14:51:40 +03:00
Georgi GerganovandGitHub cddaf028ad ggml : fix handling of zero blocks in IQ quants (#7955) 
ggml-ci
 2024-06-16 14:50:12 +03:00
Georgi Gerganov c8a82194a8 github : update pr template 2024-06-16 10:46:51 +03:00
Georgi GerganovandGitHub 66ef1ceedf metal : utilize max shared memory for mul_mat_id (#7935) 2024-06-14 17:14:09 +03:00