Brain-Bench + Latenz-Metrik (Review P1-9/P1-10)

- Bench-Matrix Qwen3.6 (5 Configs, separater Port): MTP n-max 3 bestaetigt (+26% tg),
  n-max 4 lohnt nicht, KV Q8_0 gratis (78,6=78,6 t/s) bei halbem KV-Speicher
- deploy-Config: -ctk/-ctv q8_0 am hermes-Eintrag (Live-Schaltung: User-Freigabe noetig)
- voice.py: chat_first_content-Metrik (echte Hirn-Latenz bis erster Inhalts-Token)
- Report um Umsetzungs-Nachtrag ergaenzt

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Hitonabi
2026-07-02 10:56:23 +02:00
parent 45fb61000a
commit e08d812598
3 changed files with 34 additions and 2 deletions
+7 -1
View File
@@ -223,6 +223,7 @@ async def voice_chat(body: ChatIn) -> StreamingResponse:
async def gen():
t0 = time.perf_counter()
first = True
first_content = True
try:
async with httpx.AsyncClient(timeout=httpx.Timeout(None, connect=5.0)) as client:
async with client.stream(
@@ -233,9 +234,14 @@ async def voice_chat(body: ChatIn) -> StreamingResponse:
yield f"data: {{\"error\": \"Hermes {r.status_code}: {detail}\"}}\n\n".encode()
return
async for chunk in r.aiter_raw():
if first: # Time-To-First-Byte des Hermes-Streams (gefühlte Lucy-Latenz)
if first: # Time-To-First-Byte des Hermes-Streams (Verbindungs-Overhead)
record_stage("chat_ttfb", (time.perf_counter() - t0) * 1000.0)
first = False
# Erster CONTENT-Delta = echte Hirn-Latenz (Agent-Overhead + LLM-TTFT) —
# chat_ttfb misst nur den SSE-Start (~5 ms) und ist dafür blind.
if first_content and b'"content"' in chunk:
record_stage("chat_first_content", (time.perf_counter() - t0) * 1000.0)
first_content = False
yield chunk
except httpx.HTTPError as exc:
yield f"data: {{\"error\": \"Verbindung zu Hermes fehlgeschlagen: {exc}\"}}\n\n".encode()