nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity
6,170 Commits 1 Branch 0 Tags
4227c9be4268ac844921b90f31595f81236bd317
Commit Graph
25 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Aman GuptaandGitHub efe3a90996 CUDA cmake: add -lineinfo for easier debug (#15260) 2025-08-12 17:21:45 +08:00
Aman GuptaandGitHub 34c9d765bf CUDA: add attention sinks for tile and wmma (#15178) 
* CUDA: add attention sinks for tile and wmma

* Review: formatting changes + remove syncthreads from tile + remove warp_reduce_max from wmma
 2025-08-09 20:00:24 +08:00
Aman GuptaandGitHub 784524053d Fix params bug in diffusion example (#14993) 2025-08-01 01:22:58 +08:00
Aman GuptaandGitHub 8a4a856277 Add LLaDA 8b Diffusion model (#14771) 
* Add support for Llada-8b: diffusion model

* Add README

* Fix README and convert_hf_to_gguf

* convert_hf_to_gguf.py: address review comments

* Make everything in a single example

* Remove model-specific sampling

* Remove unused argmax

* Remove braced initializers, improve README.md a bit

* Add diffusion specific gguf params in set_vocab, remove setting rope_theta and rms_norm_eps

* Remove adding the mask token

* Move add_add_bos_token to set_vocab

* use add_bool in gguf_writer.py
 2025-07-31 19:49:09 +08:00
Aman GuptaandGitHub 0a5036bee9 CUDA: add roll (#14919) 
* CUDA: add roll

* Make everything const, use __restrict__
 2025-07-29 14:45:18 +08:00
Aman GuptaandGitHub 446595b9b3 Docs: add instructions for adding backends (#14889) 2025-07-27 09:36:43 +08:00
Aman GuptaandGitHub 8c988fa41d CUDA: add fused rms norm (#14800) 2025-07-23 09:25:42 +08:00
Aman GuptaandGitHub 2be60cbc27 docs : fix link for tools/perplexity in README.md (#14780) 2025-07-20 20:13:47 +02:00
Aman GuptaandGitHub 938b785764 Clang-format: local files first + fix BinPacking (#14779) 2025-07-20 19:42:34 +08:00
Aman GuptaandGitHub f9a31eea06 CUDA: set_rows + cpy.cu refactor (#14712) 2025-07-18 14:54:18 +08:00
Aman GuptaandGitHub ab14019821 Support diffusion models: Add Dream 7B (#14644) 
* Support diffusion models: Add Dream 7B

* Move diffusion to examples

* Move stuff to examples. Add patch to not use kv-cache

* Address review comments

* Make sampling fast

* llama: remove diffusion functions

* Add basic timings + cleanup

* More cleanup

* Review comments: better formating, use LOG instead std::cerr, re-use batch, use ubatch instead of max_length

* fixup!

* Review: move everything to diffusion-cli for now
 2025-07-16 20:03:51 +08:00
Aman GuptaandGitHub 9c9e4fc635 llama-context: add ability to get logits (#14672) 2025-07-14 21:01:41 +08:00
Aman GuptaandGitHub 7de5c7cab6 CUDA: add set rows for f32 and f16 (#14551) 
* CUDA: add set rows for f32 and f16

* Review: change kernel params, use strides from host

* Use 1-d kernel

* Review: use int64_t for blockDim.x, rename nb->s for clarity
 2025-07-12 16:31:38 +03:00
Aman GuptaGitHubGeorgi Gerganov
 		11ee0fea2a Docs: script to auto-generate ggml operations docs (#14598) 
* Docs: script to auto-generate ggml operations docs

* Review: formatting changes + change github action

* Use built-in types instead of typing

* docs : add BLAS and Metal ops

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2025-07-10 23:29:01 +08:00
Aman GuptaandGitHub 75c91de6e9 CUDA: add bilinear interpolation for upscale (#14563) 2025-07-08 10:11:18 +08:00
Aman GuptaandGitHub b9c3eefde1 CUDA: add bf16 and i32 to getrows (#14529) 2025-07-07 21:45:43 +08:00
Aman GuptaandGitHub 55c2646b45 CUDA: add dynamic shared mem to softmax, refactor general usage (#14497) 2025-07-03 07:45:11 +08:00
Aman GuptaandGeorgi Gerganov 55a1c5a5fd CUDA: add softmax broadcast (#14475) 
* CUDA: add softmax broadcast

* Pass by const ref

* Review: Use blockDims for indexing, remove designated initializers

* Add TODO for noncontigous input/output
 2025-07-02 15:48:33 +03:00
Aman GuptaandGitHub 0a5a3b5cdf Add Conv2d for CPU (#14388) 
* Conv2D: Add CPU version

* Half decent

* Tiled approach for F32

* remove file

* Fix tests

* Support F16 operations

* add assert about size

* Review: further formatting fixes, add assert and use CPU version of fp32->fp16
 2025-06-30 23:57:04 +08:00
Aman GuptaandGitHub 27208bf657 CUDA: add bf16 and f32 support to cublas_mul_mat_batched (#14361) 
* CUDA: add bf16 and f32 support to cublas_mul_mat_batched

* Review: add type traits and make function more generic

* Review: make check more explicit, add back comments, and fix formatting

* Review: fix formatting, remove useless type conversion, fix naming for bools
 2025-06-29 01:30:53 +08:00
Aman GuptaandGitHub aa064b2eb7 CUDA: add mean operation (#14313) 
* CUDA: add mean operation

* add back sum_rows_f32_cuda

* Review: early exit if col!=0
 2025-06-22 12:39:54 +08:00
Aman GuptaandGitHub c959f462a0 CUDA: add conv_2d_transpose (#14287) 
* CUDA: add conv_2d_transpose

* remove direct include of cuda_fp16

* Review: add brackets for readability, remove ggml_set_param and add asserts
 2025-06-20 22:48:24 +08:00
Aman GuptaandGitHub 9eaa51e7f0 CUDA: add conv_2d_dw (#14265) 
* CUDA: add conv_2d_dw

* better naming

* simplify using template

* Review: fix operation ordering in ggml-cuda, use __forceinline__, use more const
 2025-06-20 09:50:24 +08:00
Aman GuptaandGitHub 2e42be42bd compare-llama-bench: add option to plot (#14169) 
* compare llama-bench: add option to plot

* Address review comments: convert case + add type hints

* Add matplotlib to requirements

* fix tests

* Improve comment and fix assert condition for test

* Add back default test_name, add --plot_log_scale

* use log_scale regardless of x_values
 2025-06-14 10:34:20 +02:00
AmanandGitHub 7781e5fe99 webui: Wrap long numbers instead of infinite horizontal scroll (#14062) 
* webui: Wrap long numbers instead of infinite horizontal scroll

* Use tailwind class

* update index.html.gz
 2025-06-11 16:42:25 +02:00