llama_cpp

6,528 Commits 1 Branch 0 Tags

2 Commits

Author	SHA1	Message	Date
Oleksandr KuvshynovandGitHub	e5155e6986	server : export max observed n_past value (#15361 ) Add tracking for high watermark cache usage and make it available in /metrics endpoint. Use-case: Tracking largest needed cache usage under realistic workload to better understand memory requirements and be able to adjust cache size/quantization for model/cache accordingly.	2025-08-18 00:28:58 +02:00
Oleksandr KuvshynovandGitHub	e4376270d9	llama.cpp: fix warning message (#11839 ) There was a typo-like error, which would print the same number twice if request is received with n_predict > server-side config. Before the fix: ``` slot launch_slot_: id 0 \| task 0 \| n_predict = 4096 exceeds server configuration, setting to 4096 ``` After the fix: ``` slot launch_slot_: id 0 \| task 0 \| n_predict = 8192 exceeds server configuration, setting to 4096 ```	2025-02-13 08:25:34 +02:00

Author

SHA1

Message

Date

Oleksandr KuvshynovandGitHub

e5155e6986

server : export max observed n_past value (#15361 )

Add tracking for high watermark cache usage and make it available in /metrics endpoint.

Use-case: Tracking largest needed cache usage under realistic workload
to better understand memory requirements and be able to adjust
cache size/quantization for model/cache accordingly.

2025-08-18 00:28:58 +02:00

Oleksandr KuvshynovandGitHub

e4376270d9

llama.cpp: fix warning message (#11839 )

There was a typo-like error, which would print the same number twice if
request is received with n_predict > server-side config.

Before the fix:
```
slot launch_slot_: id  0 | task 0 | n_predict = 4096 exceeds server configuration, setting to 4096
```

After the fix:
```
slot launch_slot_: id  0 | task 0 | n_predict = 8192 exceeds server configuration, setting to 4096
```

2025-02-13 08:25:34 +02:00