Kontext-Limit sichtbar machen: Gateway-Warnung bei finish_reason=length + Etappen-Regel
User-Wunsch 15.07. ('es sollte eine Warnung geben - Kontext ist nicht
unendlich'): 4 P3-Worker starben still an finish_reason=length.
- Gateway erkennt abgerissene Antworten (Stream-Chunk + Non-Stream) und
meldet je Modell max. alle 10 min in den Briefkasten (silent -> Chronik),
Zustellung per MC_ANNOUNCE_HTTP ans Steuerpult (Store bleibt Ein-Schreiber).
- Steckbrief-Hook: KONTEXT-BUDGET & ETAPPEN-REGEL - gezielt lesen, grosse
Aufgaben in Etappen schneiden (kanban_create/complete mit Plan) statt
am Limit zu sterben.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -4,7 +4,7 @@ from fastapi import APIRouter, Request
|
||||
from fastapi.responses import JSONResponse, StreamingResponse
|
||||
|
||||
from config import LLAMA_SWAP_URL
|
||||
from services.gateway_stream import record_stream_chunk, record_usage
|
||||
from services.gateway_stream import record_stream_chunk, record_usage, warn_truncation
|
||||
from services.router_logic import IMAGE_PART_TYPES, VISION_CAPABLE, choose_for_lane, has_image
|
||||
from services.routing_policy import load_policy
|
||||
|
||||
@@ -201,7 +201,11 @@ async def _proxy(path: str, request: Request):
|
||||
|
||||
r = await client.post(url, json=body, timeout=600.0)
|
||||
resp_json = r.json()
|
||||
record_usage(resp_json.get("usage") if isinstance(resp_json, dict) else None, alias)
|
||||
if isinstance(resp_json, dict):
|
||||
record_usage(resp_json.get("usage"), alias)
|
||||
if any(isinstance(c, dict) and c.get("finish_reason") == "length"
|
||||
for c in resp_json.get("choices") or []):
|
||||
warn_truncation(alias)
|
||||
return JSONResponse(resp_json, status_code=r.status_code, headers=routed)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user