Feat: virtuelle Gateway-Lanes 'coding' + 'chat' (Router macht die Arbeit)
Der :9001/v1-Gateway bietet zwei virtuelle Modelle an, die der Router auf echte Modelle abbildet: - coding → coder (Standard) · heavy (riesiger/architektonischer Kontext) · fast (triviale Nicht-Code-Kurzfrage) - chat → fast/heavy (= bisheriges model:auto, weiter als Alias unterstützt) router_logic.choose_for_lane() kapselt die Lane-Logik (Code-Indikatoren DE+EN, damit echte Coding-Anfragen nie auf fast abrutschen). gateway_proxy routet die Lane-Namen und listet sie in /v1/models, sodass IDEs einfach "coding" wählen. Lucy/Hermes (:8642) bleibt unberührt — andere Ebene. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -4,25 +4,35 @@ from fastapi.responses import JSONResponse, StreamingResponse
|
||||
|
||||
from config import LLAMA_SWAP_URL
|
||||
from services.gateway_stream import record_stream_chunk, record_usage
|
||||
from services.router_logic import FAST, FAST_NO_THINK, choose_model
|
||||
from services.router_logic import FAST, FAST_NO_THINK, LANES, choose_for_lane
|
||||
|
||||
router = APIRouter(prefix="/v1")
|
||||
|
||||
# Virtuelle Lanes, die der Gateway zusätzlich zu den echten Modellen als „Modell" anbietet.
|
||||
_LANE_LABELS = {"coding": "Coding (Router → coder/heavy/fast)", "chat": "Chat (Router → fast/heavy)"}
|
||||
|
||||
|
||||
@router.get("/models")
|
||||
async def models():
|
||||
async with httpx.AsyncClient(timeout=10) as c:
|
||||
r = await c.get(f"{LLAMA_SWAP_URL}/v1/models")
|
||||
return JSONResponse(r.json(), status_code=r.status_code)
|
||||
data = r.json()
|
||||
# Lanes ganz oben einblenden, damit IDEs einfach „coding"/„chat" wählen können.
|
||||
lanes = [{"id": lane, "object": "model", "owned_by": "mc2-router",
|
||||
"description": _LANE_LABELS.get(lane, lane)} for lane in LANES]
|
||||
if isinstance(data, dict) and isinstance(data.get("data"), list):
|
||||
data["data"] = lanes + data["data"]
|
||||
return JSONResponse(data, status_code=r.status_code)
|
||||
|
||||
|
||||
async def _proxy(path: str, request: Request):
|
||||
body = await request.json()
|
||||
requested = str(body.get("model") or "auto")
|
||||
if requested == "auto":
|
||||
alias, reason = choose_model(body)
|
||||
if requested.lower() in ("auto", "chat", "coding"):
|
||||
lane = requested.lower()
|
||||
alias, reason = choose_for_lane(lane, body)
|
||||
body["model"] = alias
|
||||
routed = {"x-mc-routed-to": alias, "x-mc-route-reason": reason}
|
||||
routed = {"x-mc-routed-to": alias, "x-mc-route-reason": reason, "x-mc-lane": lane}
|
||||
else:
|
||||
alias = requested
|
||||
routed = {"x-mc-routed-to": requested}
|
||||
|
||||
Reference in New Issue
Block a user