Feat: virtuelle Gateway-Lanes 'coding' + 'chat' (Router macht die Arbeit)

Der :9001/v1-Gateway bietet zwei virtuelle Modelle an, die der Router auf echte
Modelle abbildet:
- coding → coder (Standard) · heavy (riesiger/architektonischer Kontext) ·
  fast (triviale Nicht-Code-Kurzfrage)
- chat   → fast/heavy (= bisheriges model:auto, weiter als Alias unterstützt)

router_logic.choose_for_lane() kapselt die Lane-Logik (Code-Indikatoren DE+EN,
damit echte Coding-Anfragen nie auf fast abrutschen). gateway_proxy routet die
Lane-Namen und listet sie in /v1/models, sodass IDEs einfach "coding" wählen.
Lucy/Hermes (:8642) bleibt unberührt — andere Ebene.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Hitonabi
2026-06-29 17:43:33 +02:00
parent 8a802241f7
commit bb922b2a21
2 changed files with 87 additions and 17 deletions
+15 -5
View File
@@ -4,25 +4,35 @@ from fastapi.responses import JSONResponse, StreamingResponse
from config import LLAMA_SWAP_URL
from services.gateway_stream import record_stream_chunk, record_usage
from services.router_logic import FAST, FAST_NO_THINK, choose_model
from services.router_logic import FAST, FAST_NO_THINK, LANES, choose_for_lane
router = APIRouter(prefix="/v1")
# Virtuelle Lanes, die der Gateway zusätzlich zu den echten Modellen als „Modell" anbietet.
_LANE_LABELS = {"coding": "Coding (Router → coder/heavy/fast)", "chat": "Chat (Router → fast/heavy)"}
@router.get("/models")
async def models():
async with httpx.AsyncClient(timeout=10) as c:
r = await c.get(f"{LLAMA_SWAP_URL}/v1/models")
return JSONResponse(r.json(), status_code=r.status_code)
data = r.json()
# Lanes ganz oben einblenden, damit IDEs einfach „coding"/„chat" wählen können.
lanes = [{"id": lane, "object": "model", "owned_by": "mc2-router",
"description": _LANE_LABELS.get(lane, lane)} for lane in LANES]
if isinstance(data, dict) and isinstance(data.get("data"), list):
data["data"] = lanes + data["data"]
return JSONResponse(data, status_code=r.status_code)
async def _proxy(path: str, request: Request):
body = await request.json()
requested = str(body.get("model") or "auto")
if requested == "auto":
alias, reason = choose_model(body)
if requested.lower() in ("auto", "chat", "coding"):
lane = requested.lower()
alias, reason = choose_for_lane(lane, body)
body["model"] = alias
routed = {"x-mc-routed-to": alias, "x-mc-route-reason": reason}
routed = {"x-mc-routed-to": alias, "x-mc-route-reason": reason, "x-mc-lane": lane}
else:
alias = requested
routed = {"x-mc-routed-to": requested}