10 Commits
Author SHA1 Message Date
96b6912103 metal : single allocation of encode_async block (#9747)
* Single allocation of encode_async block with non-ARC capture in ggml-metal.m

* Moving Block_release to the deallocation code

* Release encode block when re-setting encoding buffer count if needed

* Update ggml/src/ggml-metal.m

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2024-10-07 15:26:31 +03:00
Paul TsochantarisandGitHub e5ca3937c6 llama : do not cap thread count when MoE on CPU (#5419)
* Not capping thread count when MoE inference is running on CPU

* Whitespace
2024-02-09 12:48:06 +02:00
Paul TsochantarisandGitHub ceebbb5b21 ggml alloc: Fix for null dereference on alloc failure (#5200)
* Fix for a null pointer dereference if a metal GGML buffer fails to be allocated

* Freeing the allocated buffers rather than the pointer in ggml-alloc.c

* Fixed the fix of the fix
2024-01-29 23:19:29 +01:00
Paul TsochantarisandGitHub d2f650cb5b metal : free metal objects (#5161)
* Releasing MTLFunction references after Metal pipeline construction

* Keeping the `ggml_metal_kernel` structure

* Spacing fix

* Whitespace fix
2024-01-28 21:50:16 +02:00
Paul TsochantarisandGitHub 6dd3c28c9c metal : remove unused n_buffers and buffers (#5129) 2024-01-26 14:16:07 +02:00
Paul TsochantarisandGitHub 1e605f4102 metal : fix memory leak, dangling pointer and unused autorel (#5007)
* Metal memory: Small memory leak on init, dangling pointer, and unused autorelease pool in graph compute

* SPM header potential fix

* Reverting symlinks
2024-01-18 10:47:24 +02:00
Paul TsochantarisandGitHub 7563293665 metal : remove unnecessary nil check (#4986) 2024-01-17 10:07:24 +02:00
Paul TsochantarisandGitHub 158f8c9e21 metal : localized logic in ggml_metal_graph_compute (#4924)
* Metal: Localized logic in `ggml_metal_graph_compute`, minor performance improvement

* Whitespace

* Collecting command buffer completions on single thread

* Whitespace

* Reduce diff noise
2024-01-16 19:05:19 +02:00
Paul TsochantarisandGitHub 2a7c94db5f metal : put encoder debug group behind a define (#4873) 2024-01-11 16:31:52 +02:00
Paul TsochantarisandGitHub a206137f92 Adding Emeltal reference to UI list (#4629) 2023-12-25 18:09:53 +02:00