shibe2 and GitHub
948ff137ec
server : fix handling of characters that span multiple tokens when streaming ( #4446 )
2023-12-13 21:57:15 +02:00
shibe2
465219b914
CLBlast: Add outer loops over src0 for broadcasting in mulmat
...
Reduce repeated dequantization of the same data.
2023-10-20 22:30:52 +04:00
shibe2 and GitHub
1117d06607
opencl : fix element-wise multiplication ( #3656 )
2023-10-18 15:09:22 +03:00
shibe2
40e5ce054f
CLBlast: Fix temporary buffer size for f16 conversion (wsize)
...
Fix buffer overflow.
Reduce the size to fit just one 2D slice.
Assert sufficient size.
2023-10-17 21:02:30 +04:00
shibe2 and GitHub
1e0e873c37
CLBlast: Fix matrix-vector multiplication ( #3544 )
2023-10-12 21:59:47 +02:00
shibe2
e2583cbc29
CLBlast: Fix handling of on-device tensor data
...
Fix uploading tensor data to device, including 3D, 4D, and non-contiguous tensors.
Use correct offsets into data that is already in VRAM.
Correct handling of OpenCL events when multiple commands are queued.
2023-10-05 18:25:23 +04:00
shibe2 and GitHub
665018c749
CLBlast: Add broadcast support for matrix multiplication ( #3402 )
...
Broadcast src0 into src1 across dimensions 2 and 3 when needed.
This is required for models that use GQA.
2023-10-02 21:26:15 +02:00
shibe2 and GitHub
36b904e200
ggml-opencl.cpp: Make private functions static ( #3300 )
2023-09-21 14:10:26 -04:00
