Gateway: Kontext-Limit-Warnung ignoriert Warm-Pings (max_tokens<=16)
Lucys Augen-Waermer (alle 10 min, max_tokens=1 an "vision"), warmup.sh und models/load enden konstruktionsbedingt mit finish_reason=length — die Warnung vom 15.07. hielt jeden Ping fuer einen abgerissenen Worker und spammte den Briefkasten (~alle 20 min, Serie 16.07. abends). Wer freiwillig auf <=16 Tokens deckelt, will keine echte Antwort -> keine Warnung; echte Abrisse melden weiter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -239,7 +239,7 @@ async def _proxy(path: str, request: Request):
|
||||
async def gen():
|
||||
try:
|
||||
async for chunk in r.aiter_raw():
|
||||
record_stream_chunk(chunk, alias)
|
||||
record_stream_chunk(chunk, alias, body.get("max_tokens"))
|
||||
yield chunk
|
||||
finally:
|
||||
await r.aclose()
|
||||
@@ -251,7 +251,7 @@ async def _proxy(path: str, request: Request):
|
||||
record_usage(resp_json.get("usage"), alias)
|
||||
if any(isinstance(c, dict) and c.get("finish_reason") == "length"
|
||||
for c in resp_json.get("choices") or []):
|
||||
warn_truncation(alias)
|
||||
warn_truncation(alias, body.get("max_tokens"))
|
||||
return JSONResponse(resp_json, status_code=r.status_code, headers=routed)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user