Kontext-Limit sichtbar machen: Gateway-Warnung bei finish_reason=length + Etappen-Regel

User-Wunsch 15.07. ('es sollte eine Warnung geben - Kontext ist nicht
unendlich'): 4 P3-Worker starben still an finish_reason=length.
- Gateway erkennt abgerissene Antworten (Stream-Chunk + Non-Stream) und
  meldet je Modell max. alle 10 min in den Briefkasten (silent -> Chronik),
  Zustellung per MC_ANNOUNCE_HTTP ans Steuerpult (Store bleibt Ein-Schreiber).
- Steckbrief-Hook: KONTEXT-BUDGET & ETAPPEN-REGEL - gezielt lesen, grosse
  Aufgaben in Etappen schneiden (kanban_create/complete mit Plan) statt
  am Limit zu sterben.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Hitonabi
2026-07-15 14:43:36 +02:00
parent 3468bc9c80
commit 3cf111d973
4 changed files with 39 additions and 2 deletions
+6 -2
View File
@@ -4,7 +4,7 @@ from fastapi import APIRouter, Request
from fastapi.responses import JSONResponse, StreamingResponse
from config import LLAMA_SWAP_URL
from services.gateway_stream import record_stream_chunk, record_usage
from services.gateway_stream import record_stream_chunk, record_usage, warn_truncation
from services.router_logic import IMAGE_PART_TYPES, VISION_CAPABLE, choose_for_lane, has_image
from services.routing_policy import load_policy
@@ -201,7 +201,11 @@ async def _proxy(path: str, request: Request):
r = await client.post(url, json=body, timeout=600.0)
resp_json = r.json()
record_usage(resp_json.get("usage") if isinstance(resp_json, dict) else None, alias)
if isinstance(resp_json, dict):
record_usage(resp_json.get("usage"), alias)
if any(isinstance(c, dict) and c.get("finish_reason") == "length"
for c in resp_json.get("choices") or []):
warn_truncation(alias)
return JSONResponse(resp_json, status_code=r.status_code, headers=routed)