llama_cpp

7,201 Commits 1 Branch 0 Tags

3 Commits

Author	SHA1	Message	Date
Oleksandr KuvshynovandGitHub	c5fef0fcea	server: update readme to mention n_past_max metric (#16436 ) https://github.com/ggml-org/llama.cpp/pull/15361 added new metric exported, but I've missed this doc.	2025-10-06 10:53:31 +03:00
Oleksandr KuvshynovandGitHub	e5155e6986	server : export max observed n_past value (#15361 ) Add tracking for high watermark cache usage and make it available in /metrics endpoint. Use-case: Tracking largest needed cache usage under realistic workload to better understand memory requirements and be able to adjust cache size/quantization for model/cache accordingly.	2025-08-18 00:28:58 +02:00
Oleksandr KuvshynovandGitHub	e4376270d9	llama.cpp: fix warning message (#11839 ) There was a typo-like error, which would print the same number twice if request is received with n_predict > server-side config. Before the fix: ``` slot launch_slot_: id 0 \| task 0 \| n_predict = 4096 exceeds server configuration, setting to 4096 ``` After the fix: ``` slot launch_slot_: id 0 \| task 0 \| n_predict = 8192 exceeds server configuration, setting to 4096 ```	2025-02-13 08:25:34 +02:00

Author

SHA1

Message

Date

Oleksandr KuvshynovandGitHub

c5fef0fcea

server: update readme to mention n_past_max metric (#16436 )

https://github.com/ggml-org/llama.cpp/pull/15361 added new metric
exported, but I've missed this doc.

2025-10-06 10:53:31 +03:00

Oleksandr KuvshynovandGitHub

e5155e6986

server : export max observed n_past value (#15361 )

Add tracking for high watermark cache usage and make it available in /metrics endpoint.

Use-case: Tracking largest needed cache usage under realistic workload
to better understand memory requirements and be able to adjust
cache size/quantization for model/cache accordingly.

2025-08-18 00:28:58 +02:00

Oleksandr KuvshynovandGitHub

e4376270d9

llama.cpp: fix warning message (#11839 )

There was a typo-like error, which would print the same number twice if
request is received with n_predict > server-side config.

Before the fix:
```
slot launch_slot_: id  0 | task 0 | n_predict = 4096 exceeds server configuration, setting to 4096
```

After the fix:
```
slot launch_slot_: id  0 | task 0 | n_predict = 8192 exceeds server configuration, setting to 4096
```

2025-02-13 08:25:34 +02:00