Feat: idiotensichere Erweiterte Ansicht - Fit-Ampel (OOM-Gate) + Rollen-Uebernahme-Warnung
Manueller HF-Install hatte zwei scharfe Kanten: - kein Fit/OOM-Schutz: zu grosses Modell stuerzte erst beim Laden ab - stiller Rollen-Diebstahl: exklusiver Alias wanderte kommentarlos weg Backend: /api/fit leitet params_b jetzt aus KATALOG (echte Metadaten, MoE-bewusst) oder Namens-Schaetzung ab (params_b<=0) -> Fit-Vorschau fuer beliebige HF-Repos. Frontend (AddModel): Hardware-Fit-Ampel (perfect/marginal/OOM) mit ~params/req_gb/tps nach 'Quants laden'; OOM-Gate (zweiter, roter Klick noetig); Warnung welches Modell die gewaehlte Rolle aktuell haelt und sie verliert. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -5,7 +5,7 @@ from fastapi import APIRouter, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
from config import HF_DOWNLOAD_ENV, MODELS_DIR
|
||||
from services import discover, hf, jobengine, llamaswap
|
||||
from services import catalog, discover, hf, jobengine, llamaswap
|
||||
from services.fit import evaluate_fit, extract_params_b, max_ctx_for
|
||||
|
||||
router = APIRouter(prefix="/api")
|
||||
@@ -31,11 +31,19 @@ def discover_models(force: bool = False) -> dict:
|
||||
|
||||
|
||||
@router.get("/fit")
|
||||
def fit(params_b: float, quant: str = "Q4_K_M", ctx: int = 8192, name: str = "") -> dict:
|
||||
def fit(params_b: float = 0, quant: str = "Q4_K_M", ctx: int = 8192, name: str = "") -> dict:
|
||||
"""Hardware-Fit-Vorschau. params_b<=0 → aus KATALOG (echte Metadaten, MoE-bewusst)
|
||||
oder sonst aus dem Namen geschätzt. So liefert die 'Erweiterte Ansicht' eine
|
||||
Ampel/t-s-Schätzung für beliebige HF-Repos, bevor heruntergeladen wird."""
|
||||
ram = _ram_gb()
|
||||
pb = params_b
|
||||
if pb <= 0:
|
||||
meta = catalog.meta_for_name(name) if name else None
|
||||
pb = float(meta["total_params_b"]) if (meta and meta.get("total_params_b")) else extract_params_b(name)
|
||||
return {
|
||||
"fit": evaluate_fit(params_b, quant, ctx, ram, name=name),
|
||||
"optimal_ctx": max_ctx_for(params_b, quant, ram),
|
||||
"params_b": round(pb, 1),
|
||||
"fit": evaluate_fit(pb, quant, ctx, ram, name=name),
|
||||
"optimal_ctx": max_ctx_for(pb, quant, ram),
|
||||
"sys_ram_gb": round(ram, 1),
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user