server + ui: ping silent SSE streams every 1s and kick only after 3s so slow prefill never drops healthy connections (#25241)
* server + ui: ping silent SSE streams every 1s and kick only after 3s so slow prefill never drops healthy connections * server + ui: sse_ping_interval becomes a per-request body field Address review from ngxson: the global default returns to 30 so API clients see no behavior change, and the WebUI sends sse_ping_interval: 1 in the request body since it owns the 3s visibility-kick contract and declares the cadence it needs. Positive values keep the existing > 0 gate, -1 keeps its disabled semantics. * server: move sse_ping_interval into the request schema Address review from ngxson: the field is now a typed field_num with hard limits (-1, INT32_MAX) bound to task_params, seeded from the CLI default alongside the other inherited parameters. The raw json_value read and its redundant comment are gone, and schema evaluation brings type and range validation for free.
This commit is contained in:
@@ -1,3 +1,3 @@
|
||||
// grace window after a visibilitychange before we kick a reader whose socket likely died
|
||||
// while the tab was hidden. covers brief background pauses without thrashing live streams
|
||||
export const STREAM_VISIBILITY_KICK_MS = 1000;
|
||||
export const STREAM_VISIBILITY_KICK_MS = 3000;
|
||||
|
||||
@@ -255,6 +255,7 @@ export class ChatService {
|
||||
}),
|
||||
stream,
|
||||
return_progress: stream ? true : undefined,
|
||||
sse_ping_interval: stream ? 1 : undefined,
|
||||
tools: tools && tools.length > 0 ? tools : undefined
|
||||
};
|
||||
|
||||
|
||||
Vendored
+1
@@ -265,6 +265,7 @@ export interface ApiChatCompletionRequest {
|
||||
stream?: boolean;
|
||||
model?: string;
|
||||
return_progress?: boolean;
|
||||
sse_ping_interval?: number;
|
||||
tools?: ApiChatCompletionTool[];
|
||||
// Reasoning parameters
|
||||
reasoning_format?: string;
|
||||
|
||||
Reference in New Issue
Block a user