nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

download: add option to skip_download (#23059)

Browse Source 
* download: add option to skip_download

* fix

* fix 2

* if file doesn't exist, respect skip_download flag
This commit is contained in:
Xuan-Son Nguyen
2026-05-29 16:30:55 +02:00
committed by GitHub
parent da3f990a47
commit 06d26dfdff
8 changed files with 126 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
tools/server/README.md
View File

@@ -1661,23 +1661,30 @@ Listing all models in cache. The model metadata will also include a field to ind
{
"data": [{
"id": "ggml-org/gemma-3-4b-it-GGUF:Q4_K_M",
"in_cache": true,
"path": "/Users/REDACTED/Library/Caches/llama.cpp/ggml-org_gemma-3-4b-it-GGUF_gemma-3-4b-it-Q4_K_M.gguf",
"status": {
"value": "loaded",
"args": ["llama-server", "-ctx", "4096"]
},
"architecture": {
"input_modalities": [
"text",
"image"
],
"output_modalities": [
"text"
]
},
...
}]
}
```
Note:
1. For a local GGUF (stored offline in a custom directory), the model object will have `"in_cache": false`.
2. Adding `?reload=1` to the query params will refresh the list of models. The behavior is as follow:
1. Adding `?reload=1` to the query params will refresh the list of models. The behavior is as follow:
- If a model is running but updated or removed from the source, it will be unloaded
- If a model is not running, it will be added or updated according to the source
3. When the model is loaded, the info from `/v1/models` is forwarded to router's `/v1/models`. This includes metadata about the model and the runtime instance.
2. When the model is loaded, the info from `/v1/models` is forwarded to router's `/v1/models`. This includes metadata about the model and the runtime instance.
The `status` object can be: