nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 27 Packages Projects Releases Wiki Activity
3,443 Commits 1 Branch 0 Tags
e7e6487ba06634edf58dfdf9673bad9df41b445a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Georgi GerganovandGitHub e7e6487ba0 contrib : clarify PR squashing + module names (#8630) 
* contrib : clarify PR squashing

* contrib : fix typo + add list of modules
 2024-07-23 11:28:38 +03:00
Georgi GerganovandGitHub 6f11a83e4e llama : allow overrides for tokenizer flags (#8614) 
ggml-ci
 2024-07-22 13:33:22 +03:00
Georgi GerganovandGitHub e093dd2382 tests : re-enable tokenizer tests (#8611) 
* models : remove duplicated gpt-2 vocab

* models : remove old stablelm vocab

* tests : re-enable MPT tokenizer tests

* tests : re-enable DeepSeek tokenizer tests

* cmake : sort

ggml-ci
 2024-07-22 13:32:49 +03:00
Georgi GerganovandGitHub 45f2c19cc5 flake.lock: Update (#8610) 2024-07-21 06:45:10 -07:00
Georgi GerganovandGitHub 07283b1a90 gguf : handle null name during init (#8587) 2024-07-20 17:15:42 +03:00
Georgi GerganovandGitHub d197545530 llama : bump max layers from 256 to 512 (#8530) 
* llama : bump max layers from 256 to 512

* llama : replace asserts with exceptions
 2024-07-19 16:50:47 +03:00
Georgi GerganovandGitHub be0cfb4175 readme : fix server badge 2024-07-19 14:34:55 +03:00
Georgi GerganovandGitHub d65a8361fe llama : disable context-shift for DeepSeek v2 (#8501) 2024-07-17 10:32:59 +03:00
Georgi GerganovandGitHub 0efec57787 llama : valign + remove unused ftype (#8502) 2024-07-16 10:00:30 +03:00
Georgi Gerganov 9104bc20ed common : add --no-cont-batching arg (#6358) 2024-07-15 14:54:58 +03:00
Georgi Gerganov 3dfda05956 llama : de-duplicate deepseek2 norm 2024-07-15 14:10:39 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		aaab2419ea flake.lock: Update (#8475) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/9f4128e00b0ae8ec65918efeba59db998750ead6?narHash=sha256-rwz8NJZV%2B387rnWpTYcXaRNvzUSnnF9aHONoJIYmiUQ%3D' (2024-07-03)
  → 'github:NixOS/nixpkgs/7e7c39ea35c5cdd002cd4588b03a3fb9ece6fad9?narHash=sha256-EYekUHJE2gxeo2pM/zM9Wlqw1Uw2XTJXOSAO79ksc4Y%3D' (2024-07-12)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-07-14 08:54:02 -07:00
Georgi GerganovGitHubDaniel Han
 		73cf442e7b llama : fix Gemma-2 Query scaling factors (#8473) 
* 9B - query_pre_attn_scalar = 256 not 224

See https://github.com/google/gemma_pytorch/commit/03e657582d17cb5a8617ebf333c1c16f3694670e

Gemma 9b should use 256 and not 224 (self.config.hidden_size // self.config.num_attention_heads)

* llama : fix Gemma-2 Query scaling factor

ggml-ci

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
 2024-07-14 14:05:09 +03:00
Georgi GerganovandGitHub c917b67f06 metal : template-ify some of the kernels (#8447) 
ggml-ci
 2024-07-13 18:32:33 +03:00
Georgi GerganovGitHubXuan Son Nguyen
 		4e24cffd8c server : handle content array in chat API (#8449) 
* server : handle content array in chat API

* Update examples/server/utils.hpp

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>

---------

Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
 2024-07-12 14:48:15 +03:00
Georgi GerganovandGitHub 6af51c0d96 main : print error on empty input (#8456) 2024-07-12 14:48:04 +03:00
Georgi GerganovandGitHub 71c1121d11 examples : sprintf -> snprintf (#8434) 
* examples : sprintf -> snprintf

ggml-ci

* examples : use sizeof() instead of hardcoded constants
 2024-07-12 10:46:14 +03:00
Georgi GerganovandGitHub 370b1f7e7a ggml : minor naming changes (#8433) 
* ggml : minor naming changes

ggml-ci

* ggml : use PRId64 [no ci]

* ggml : revert FA K/Q names
 2024-07-12 10:46:02 +03:00
Georgi Gerganov a977c11544 gitignore : deprecated binaries 2024-07-11 11:20:40 +03:00
Georgi GerganovandGitHub 7a221b672e llama : use F32 precision in Qwen2 attention and no FA (#8412) 2024-07-11 10:21:30 +03:00
Georgi GerganovandGitHub 6b2a849d1f ggml : move sgemm sources to llamafile subfolder (#8394) 
ggml-ci
 2024-07-10 15:23:29 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		7fdb6f73e3 flake.lock: Update (#8342) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/2a55567fcf15b1b1c7ed712a2c6fadaec7412ea8?narHash=sha256-iKzJcpdXih14qYVcZ9QC9XuZYnPc6T8YImb6dX166kw%3D' (2024-06-01)
  → 'github:hercules-ci/flake-parts/9227223f6d922fee3c7b190b2cc238a99527bbb7?narHash=sha256-pQMhCCHyQGRzdfAkdJ4cIWiw%2BJNuWsTX7f0ZYSyz0VY%3D' (2024-07-03)
• Updated input 'flake-parts/nixpkgs-lib':
    'https://github.com/NixOS/nixpkgs/archive/eb9ceca17df2ea50a250b6b27f7bf6ab0186f198.tar.gz?narHash=sha256-lIbdfCsf8LMFloheeE6N31%2BBMIeixqyQWbSr2vk79EQ%3D' (2024-06-01)
  → 'https://github.com/NixOS/nixpkgs/archive/5daf0514482af3f97abaefc78a6606365c9108e2.tar.gz?narHash=sha256-Fm2rDDs86sHy0/1jxTOKB1118Q0O3Uc7EC0iXvXKpbI%3D' (2024-07-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/b2852eb9365c6de48ffb0dc2c9562591f652242a?narHash=sha256-C8e9S7RzshSdHB7L%2Bv9I51af1gDM5unhJ2xO1ywxNH8%3D' (2024-06-27)
  → 'github:NixOS/nixpkgs/9f4128e00b0ae8ec65918efeba59db998750ead6?narHash=sha256-rwz8NJZV%2B387rnWpTYcXaRNvzUSnnF9aHONoJIYmiUQ%3D' (2024-07-03)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-07-08 15:36:38 -07:00
Georgi Gerganov 3f2d538b81 scripts : fix sync for sycl 2024-07-08 13:51:31 +03:00
Georgi Gerganov 2ee44c9a18 sync : ggml 
ggml-ci
 2024-07-08 12:23:00 +03:00
Georgi Gerganov 6847d54c4f tests : fix whitespace (#0) 2024-07-08 12:23:00 +03:00
Georgi GerganovandGitHub 6f0dbf6ab0 infill : assert prefix/suffix tokens + remove old space logic (#8351) 2024-07-08 09:34:35 +03:00
Georgi GerganovandGitHub b5040086d4 llama : fix n_rot default (#8348) 
ggml-ci
 2024-07-07 14:59:02 +03:00
Georgi Gerganov 7ed03b8974 llama : fix compile warning (#8304) 2024-07-05 17:32:09 +03:00
Georgi GerganovandGitHub 148ec970b6 convert : remove AWQ remnants (#8320) 2024-07-05 10:15:36 +03:00
Georgi GerganovandGitHub 2cccbaa008 llama : minor indentation during tensor loading (#8304) 
* llama : minor indentation during tensor loading

ggml-ci

* llama : use int for layer iterators [no ci]
 2024-07-05 10:15:24 +03:00
Georgi GerganovandGitHub aa5898dc53 llama : prefer n_ over num_ prefix (#8308) 2024-07-05 09:10:03 +03:00
Georgi GerganovandGitHub 6c05752c50 contributing : update guidelines (#8316) 2024-07-05 09:09:47 +03:00
Georgi GerganovGitHubFrancis Couture-Harpin
 		e235b267a2 py : switch to snake_case (#8305) 
* py : switch to snake_case

ggml-ci

* cont

ggml-ci

* cont

ggml-ci

* cont : fix link

* gguf-py : use snake_case in scripts entrypoint export

* py : rename requirements for convert_legacy_llama.py

Needed for scripts/check-requirements.sh

---------

Co-authored-by: Francis Couture-Harpin <git@compilade.net>
 2024-07-05 07:53:33 +03:00
Georgi GerganovandGitHub 20fc3804bf convert : fix gemma v1 tokenizer convert (#8248) 
ggml-ci
 2024-07-04 10:41:03 +03:00
Georgi GerganovandGitHub 257f8e41e2 nix : remove OpenCL remnants (#8235) 
* nix : remove OpenCL remnants

* minor : remove parentheses
 2024-07-01 14:46:18 +03:00
Georgi GerganovandGitHub d0a7145ba9 flake.lock: Update (#8218) 2024-06-30 16:09:34 -07:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>Philip Taron
 		ab3679112d flake.lock: Update (#8071) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/e9ee548d90ff586a6471b4ae80ae9cfcbceb3420?narHash=sha256-4Zu0RYRcAY/VWuu6awwq4opuiD//ahpc2aFHg2CWqFY%3D' (2024-06-13)
  → 'github:NixOS/nixpkgs/d603719ec6e294f034936c0d0dc06f689d91b6c3?narHash=sha256-k3JqJrkdoYwE3fHE6xGDY676AYmyh4U2Zw%2B0Bwe5DLU%3D' (2024-06-20)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Philip Taron <philip.taron@gmail.com>
 2024-06-27 08:37:29 -07:00
Georgi Gerganov c70d117c37 scripts : fix filename sync 2024-06-26 23:25:22 +03:00
Georgi Gerganov f2d48fffde sync : ggml 2024-06-26 19:39:19 +03:00
Georgi Gerganov 4713bf3093 authors : regen 2024-06-26 19:36:44 +03:00
Georgi GerganovandGitHub 0e814dfc42 devops : remove clblast + LLAMA_CUDA -> GGML_CUDA (#8139) 
ggml-ci
 2024-06-26 19:32:07 +03:00
Georgi GerganovandGitHub a95631ee97 readme : update API notes 2024-06-26 19:26:13 +03:00
Georgi GerganovGitHubslaren
 		f3f65429c4 llama : reorganize source code + improve CMake (#8006) 
* scripts : update sync [no ci]

* files : relocate [no ci]

* ci : disable kompute build [no ci]

* cmake : fixes [no ci]

* server : fix mingw build

ggml-ci

* cmake : minor [no ci]

* cmake : link math library [no ci]

* cmake : build normal ggml library (not object library) [no ci]

* cmake : fix kompute build

ggml-ci

* make,cmake : fix LLAMA_CUDA + replace GGML_CDEF_PRIVATE

ggml-ci

* move public backend headers to the public include directory (#8122)

* move public backend headers to the public include directory

* nix test

* spm : fix metal header

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* scripts : fix sync paths [no ci]

* scripts : sync ggml-blas.h [no ci]

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-06-26 18:33:02 +03:00
Georgi GerganovandGitHub a927b0f3dd llama : optimize long word tokenization with WPM (#8034) 
ggml-ci
 2024-06-21 08:51:28 +03:00
Georgi GerganovandGitHub a04a953cab codecov : remove (#8004) 2024-06-19 13:04:36 +03:00
Georgi Gerganov 5326bcceeb ggml : sync 2024-06-18 09:50:45 +03:00
Georgi Gerganovandslaren e6ecc2be47 whisper : use ggml_backend_sched (whisper/2239) 
* whisper : use ggml_backend_sched (wip)

* use sched in whisper_allocr

* whisper : single backend in whisper_context

* whisper : remove whisper_state->backends_used

* whisper : remove whisper_context->backend

* whisper : reset scheduler after init

* whisper : fix external encoder (e.g. CoreML)

* whisper : cleanup

* whisper : handle null GPU buffer types + fix sycl

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-06-18 09:50:40 +03:00
Georgi GerganovandGitHub 7c26775adb llama : disable FA if KV head size do not match (#7982) 2024-06-17 19:40:01 +03:00
Georgi GerganovandGitHub 21be9cab94 rpc : fix load/store misaligned addresses (#7948) 2024-06-17 11:09:20 +03:00
Georgi GerganovandJohannes Gäßler 19b7a836f6 cuda : fix bounds check for src0 rows in MMVQ kernel (whisper/2231) 
* cuda : fix bounds check for src0 rows in MMVQ kernel

* Update ggml-cuda/mmvq.cu

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
 2024-06-16 20:32:49 +03:00
Georgi GerganovandGitHub bc6c457fa3 flake.lock: Update (#7951) 2024-06-16 09:16:21 -07:00
Georgi GerganovandGitHub 52399254b3 unicode : avoid char32_t (#7957) 
ggml-ci
 2024-06-16 14:51:40 +03:00
Georgi GerganovandGitHub cddaf028ad ggml : fix handling of zero blocks in IQ quants (#7955) 
ggml-ci
 2024-06-16 14:50:12 +03:00
Georgi Gerganov c8a82194a8 github : update pr template 2024-06-16 10:46:51 +03:00
Georgi GerganovandGitHub 66ef1ceedf metal : utilize max shared memory for mul_mat_id (#7935) 2024-06-14 17:14:09 +03:00
Georgi GerganovandGitHub a9cae48003 tests : add non-cont unary tests (#7857) 
* tests : add non-cont unary tests

* ggml : update unary asserts and "supports_op"

ggml-ci
 2024-06-12 16:00:22 +03:00
Georgi GerganovandGitHub bfaa676b08 ggml : improve ggml_is_contiguous logic (#7856) 
* ggml : improve ggml_is_contiguous logic

ggml-ci

* ggml : support more contiguous cases

ggml-ci
 2024-06-12 15:24:20 +03:00
Georgi GerganovandGitHub 704a35b183 server : restore numeric prompts (#7883) 2024-06-12 14:42:29 +03:00
Georgi GerganovandGitHub 4bfe50f741 tests : check the Python version (#7872) 
ggml-ci
 2024-06-11 10:10:20 +03:00
Georgi GerganovandGitHub c28a83902c examples : remove --instruct remnants (#7846) 2024-06-10 15:00:15 +03:00
Georgi GerganovandGitHub d9da0e4986 server : improve "prompt" handling (#7847) 2024-06-10 14:59:55 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		10ceba354a flake.lock: Update (#7838) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/ad57eef4ef0659193044870c731987a6df5cf56b?narHash=sha256-SzDKxseEcHR5KzPXLwsemyTR/kaM9whxeiJohbL04rs%3D' (2024-05-29)
  → 'github:NixOS/nixpkgs/051f920625ab5aabe37c920346e3e69d7d34400e?narHash=sha256-4q0s6m0GUcN7q%2BY2DqD27iLvbcd1G50T2lv08kKxkSI%3D' (2024-06-07)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-06-09 16:04:50 -07:00
Georgi GerganovandGitHub e95beeb1fc imatrix : handle partial entries (#7833) 2024-06-09 20:19:35 +03:00
Georgi GerganovandGitHub ee459f40f6 server : fix --threads-http arg (#7801) 2024-06-06 19:19:59 +03:00
Georgi GerganovandGitHub f83351f9a6 imatrix : migrate to gpt_params (#7771) 
* imatrix : migrate to gpt_params

ggml-ci

* imatrix : add --save-frequency cli arg

* common : fix --no-ppl
 2024-06-06 16:30:58 +03:00
Georgi GerganovandGitHub 2b3389677a ggml : refactor rope norm/neox (#7634) 
* ggml : unify rope norm/neox (CPU)

* ggml : fix compile warning

* ggml : remove GLM rope mode

ggml-ci

* metal : better rope implementation

ggml-ci

* cuda : better rope implementation

ggml-ci

* naming : n_orig_ctx -> n_ctx_orig

ggml-ci

* dev : add reminders to update backends

ggml-ci

* vulkan : fix ggml_rope_ext() usage

* cuda : fix array size + indents

ggml-ci
 2024-06-05 11:29:20 +03:00
Georgi GerganovandGitHub 1442677f92 common : refactor cli arg parsing (#7675) 
* common : gpt_params_parse do not print usage

* common : rework usage print (wip)

* common : valign

* common : rework print_usage

* infill : remove cfg support

* common : reorder args

* server : deduplicate parameters

ggml-ci

* common : add missing header

ggml-ci

* common : remote --random-prompt usages

ggml-ci

* examples : migrate to gpt_params

ggml-ci

* batched-bench : migrate to gpt_params

* retrieval : migrate to gpt_params

* common : change defaults for escape and n_ctx

* common : remove chatml and instruct params

ggml-ci

* common : passkey use gpt_params
 2024-06-04 21:23:39 +03:00
Georgi GerganovandGitHub 554c247caf ggml : remove OpenCL (#7735) 
ggml-ci
 2024-06-04 21:23:20 +03:00
Georgi GerganovandGitHub 0cd6bd3483 llama : remove beam search (#7736) 2024-06-04 21:23:05 +03:00
Georgi Gerganov 5ca0944a15 readme : remove obsolete Zig instructions (#7471) 2024-06-04 19:43:01 +03:00
Georgi GerganovandGitHub 6d1616944d ggml : prevent builds with -ffinite-math-only (#7726) 
This enforces a check that -fno-finite-math-only was set and that the operating
compiling mode is not in finite maths mode. This is because during rewriting of
silu and softmax for cpu #7154 there emerged an issue where the result that was
observed when >1 slot was nondeterministic as found by @JohannesGaessler.

@LostRuins narrowed the problem down to -ffinite-math-only which was theorised
to be due to SiLU, instead of flushing small values to 0, returns NaN or some 
other garbage. @jart proposed a fix that @ggerganov then implemented in this fix

ref https://github.com/ggerganov/llama.cpp/pull/7154#issuecomment-2145661825
 2024-06-04 17:01:09 +10:00
Georgi GerganovandGitHub 549279d804 llama : avoid double token-to-piece cache (#7654) 
ggml-ci
 2024-06-03 08:34:43 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		1669810d7c flake.lock: Update (#7686) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/8dc45382d5206bd292f9c2768b8058a8fd8311d9?narHash=sha256-/GJvTdTpuDjNn84j82cU6bXztE0MSkdnTWClUCRub78%3D' (2024-05-16)
  → 'github:hercules-ci/flake-parts/2a55567fcf15b1b1c7ed712a2c6fadaec7412ea8?narHash=sha256-iKzJcpdXih14qYVcZ9QC9XuZYnPc6T8YImb6dX166kw%3D' (2024-06-01)
• Updated input 'flake-parts/nixpkgs-lib':
    'https://github.com/NixOS/nixpkgs/archive/50eb7ecf4cd0a5756d7275c8ba36790e5bd53e33.tar.gz?narHash=sha256-QBx10%2Bk6JWz6u7VsohfSw8g8hjdBZEf8CFzXH1/1Z94%3D' (2024-05-02)
  → 'https://github.com/NixOS/nixpkgs/archive/eb9ceca17df2ea50a250b6b27f7bf6ab0186f198.tar.gz?narHash=sha256-lIbdfCsf8LMFloheeE6N31%2BBMIeixqyQWbSr2vk79EQ%3D' (2024-06-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/bfb7a882678e518398ce9a31a881538679f6f092?narHash=sha256-4zSIhSRRIoEBwjbPm3YiGtbd8HDWzFxJjw5DYSDy1n8%3D' (2024-05-24)
  → 'github:NixOS/nixpkgs/ad57eef4ef0659193044870c731987a6df5cf56b?narHash=sha256-SzDKxseEcHR5KzPXLwsemyTR/kaM9whxeiJohbL04rs%3D' (2024-05-29)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-06-02 14:13:12 -07:00
Georgi GerganovandGitHub a323ec60af server : update js (#7670) 2024-05-31 22:23:04 +03:00
Georgi GerganovandGitHub 16926dff92 readme : link homebrew discussion 2024-05-31 15:04:58 +03:00
Georgi GerganovGitHubjunchao-loongson
 		0c27e6f62e ggml : fix loongson compile warnings (#7537) 
* ggml : fix loongson compile warnings

ggml-ci

* Fix loongarch quantize test fail.

Fix unexpected error introduced during rebase code.

* tests : disable json test due to lack of python on the CI node

ggml-ci

---------

Co-authored-by: junchao-loongson <zhaojunchao@loongson.cn>
 2024-05-31 14:17:10 +03:00
Georgi GerganovandGitHub 5921b8f089 llama : cache llama_token_to_piece (#7587) 
* llama : cache llama_token_to_piece

ggml-ci

* llama : use vectors and avoid has_cache

ggml-ci

* llama : throw on unknown tokenizer types

ggml-ci

* llama : print a log of the total cache size
 2024-05-31 02:01:41 +10:00
Georgi Gerganov 55d62262a9 metal : remove invalid asserts (#7617) 2024-05-29 22:21:20 +03:00
Georgi Gerganov 975ec63ff2 metal : add missing asserts (#7617) 2024-05-29 20:45:25 +03:00
Georgi GerganovandGitHub fb76ec31a9 ggml : fix YARN + add tests + add asserts (#7617) 
* tests : add rope tests

ggml-ci

* ggml : fixes (hopefully)

ggml-ci

* tests : add non-cont tests

ggml-ci

* cuda : add asserts for rope/norm + fix DS2

ggml-ci

* ggml : assert contiguousness

* tests : reduce RoPE tests

ggml-ci
 2024-05-29 20:17:31 +03:00
Georgi GerganovandGitHub cce3dcffc5 cuda : non-cont concat support (#7610) 
* tests : add non-cont concat tests

* cuda : non-cont concat support

ggml-ci
 2024-05-29 15:38:26 +03:00
Georgi Gerganov 00281b7be3 scripts : remove mpi remnants 2024-05-29 14:31:18 +03:00
Georgi Gerganov 2ab977282b sync : ggml 2024-05-29 14:29:52 +03:00
Georgi Gerganov 72de268bec ggml : restore ggml_rope_xpos_inplace (ggml/0) 
ggml-ci
 2024-05-29 14:29:33 +03:00
Georgi Gerganov 6bd12ce409 sycl : fix assert (#7563) 2024-05-28 22:22:50 +03:00
Georgi Gerganov edc29433fa tests : fix test-tokenizer-0.sh 2024-05-28 15:04:09 +03:00
Georgi GerganovandGitHub 8b99e2aa66 llama : handle unknown utf8 bytes (#7588) 2024-05-28 13:55:35 +03:00
Georgi GerganovandGitHub 0548a4187f ggml : generalize GGML_OP_CONCAT (#7563) 
* ggml : generalize GGML_OP_CONCAT (WIP)

ggml-ci

* tests : add dim != 2 tests

* metal : generalize concat kernel

* tests : naming

* cuda : generalize concat kernel

ggml-ci

* sycl : add warning and assert

* ggml : fix op params handling

* metal : bugfix kernel

ggml-ci

* ggml : reimplement CPU and Metal

* cuda : add asserts

ggml-ci

* ggml : fix ptrs

ggml-ci
 2024-05-28 11:04:19 +03:00
Georgi GerganovandGitHub 1d8fca72ae metal : add GGML_OP_REPEAT kernels (#7557) 
ggml-ci
 2024-05-27 12:10:19 +03:00
Georgi GerganovandGitHub 62bfef5194 metal : disable FA kernel for HS=256 (#7556) 
ggml-ci
 2024-05-27 10:38:39 +03:00
Georgi GerganovandGitHub eaf6e03174 llama : add comments about experimental flags (#7544) 2024-05-27 09:24:13 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		dff451cfa1 flake.lock: Update (#7540) 
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/4a6b83b05df1a8bd7d99095ec4b4d271f2956b64?narHash=sha256-%2BNpbZRCRisUHKQJZF3CT%2Bxn14ZZQO%2BKjxIIanH3Pvn4%3D' (2024-05-17)
  → 'github:NixOS/nixpkgs/bfb7a882678e518398ce9a31a881538679f6f092?narHash=sha256-4zSIhSRRIoEBwjbPm3YiGtbd8HDWzFxJjw5DYSDy1n8%3D' (2024-05-24)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-05-26 08:54:56 -07:00
Georgi Gerganov 9588f196b1 train : change default FA argument (#7528) 2024-05-25 15:22:35 +03:00
Georgi GerganovGitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
 		d041d2ceaa flake.lock: Update (#7232) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/e5d10a24b66c3ea8f150e47dfdb0416ab7c3390e?narHash=sha256-yzcRNDoyVP7%2BSCNX0wmuDju1NUCt8Dz9%2BlyUXEI0dbI%3D' (2024-05-02)
  → 'github:hercules-ci/flake-parts/8dc45382d5206bd292f9c2768b8058a8fd8311d9?narHash=sha256-/GJvTdTpuDjNn84j82cU6bXztE0MSkdnTWClUCRub78%3D' (2024-05-16)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/63c3a29ca82437c87573e4c6919b09a24ea61b0f?narHash=sha256-4cPymbty65RvF1DWQfc%2BBc8B233A1BWxJnNULJKQ1EY%3D' (2024-05-02)
  → 'github:NixOS/nixpkgs/4a6b83b05df1a8bd7d99095ec4b4d271f2956b64?narHash=sha256-%2BNpbZRCRisUHKQJZF3CT%2Bxn14ZZQO%2BKjxIIanH3Pvn4%3D' (2024-05-17)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-05-24 08:59:06 -07:00
Georgi Gerganov 74f33adf5f readme : remove trailing space (#7469) 2024-05-23 17:43:18 +03:00
Georgi Gerganov 1debe72737 ggml : silence UB sanitizer error during iq2_xxs quantization (#0) 2024-05-23 17:25:38 +03:00
Georgi GerganovandGitHub 55ac3b7aea ci : use Pythia models instead of OpenLlama (#7470) 
* ci : start using Pythia models over OpenLlama

ggml-ci

* ci : disable q2_k ppl tests

* ci : use convert-hf-to-gguf.py

* ci : update gg_get_model

* ci : fix convert outfile name

ggml-ci

* llama : gptneox arch use F32 attn prec

ggml-ci
 2024-05-23 15:28:14 +03:00
Georgi Gerganov a61a94e543 llama : rename n_ctx -> cache.size, less confusing (#0) 2024-05-23 12:38:18 +03:00
Georgi GerganovandGitHub d48c88cbd5 ggml : remove ggml_flash_attn and ggml_flash_ff (#7463) 
ggml-ci
 2024-05-23 10:00:44 +03:00
Georgi GerganovandGitHub e84b71c2c6 ggml : drop support for QK_K=64 (#7473) 
* ggml : drop support for QK_K=64

ggml-ci

* opencl : restore QK_K=256 define
 2024-05-23 10:00:21 +03:00