nikola/llama_cpp
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

server: (router) add model management API (#23976)

Browse Source 
* wip

* server: (router) add SSE realtime updates API

* nits

* wip

* add download API

* add download api

* update docs

* add delete endpoint

* fix std::terminate

* fix crash

* fix 2

* add tests

* nits
This commit is contained in:
Xuan-Son Nguyen
2026-06-17 18:04:58 +02:00
committed by GitHub
parent b4024af6c2
commit 4b4d13ae72
16 changed files with 855 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/server/server-queue.h
+5 -1
View File
@@ -154,11 +154,15 @@ public:
// Send a new result to a waiting id_task
void send(server_task_result_ptr && result);
// broadcast a new result to all waiting tasks
// (used by router mode)
void broadcast(server_task_result_ptr && result);
// terminate the waiting loop
void terminate();
};
// utility class to make working with server_queue and server_response easier
// RAII wrapper to make working with server_queue and server_response easier
// it provides a generator-like API for server responses
// support pooling connection state and aggregating multiple results
struct server_response_reader {