Feat: Vulkan/RADV-Engine + vocab-gepruefte Spec-Drafts + Provisioning/Sync

Engine-Cutover ROCm/HIP -> Vulkan/RADV (gfx1151): +12-22% tg auf MoE (llama-bench
verifiziert, fast 53->65 t/s). ROCm-Build bleibt als Rollback unter /opt/llamacpp.

- Backend: vocab-aware Speculative Decoding. services/gguf_meta.py liest den
  Tokenizer-Fingerprint (model/pre/n_vocab) direkt aus dem GGUF-Header (ohne Modell-Load);
  register_model + migrate_config haengen nur VOCAB-KOMPATIBLE Drafts an (inkl. --spec-type,
  das in dieser llama.cpp-Generation noetig ist). Neue Endpoints /api/models/drafts + /{id}/draft.
- Frontend: idiotensichere Spec-Draft-UI (SpecDraftModal) - nur kompatible Drafts waehlbar,
  inkompatible gesperrt mit Begruendung; SPEC/SPEC?-Badge nach echtem Aktiv-Status; Rolle in AddModel.
- maintenance.py: Engine-Update-Quelle -> ggml-org/llama.cpp (Build-Nummer-Vergleich),
  ENGINE_PATH=/opt/llamacpp-vulkan.
- Startup-Warmup der brains (deploy/warmup.sh, self-detaching ExecStartPost) + deploy/provision-engine.sh.
- Cleanup: tote LiteLLM gateway/config.yaml + alle Referenzen (config.py/backup.py/backup.sh) entfernt;
  README + docs/memory aktualisiert.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Hitonabi
2026-06-27 01:54:29 +02:00
parent 530d77ff1b
commit c48e583790
23 changed files with 1128 additions and 508 deletions
+24
View File
@@ -88,10 +88,34 @@ export interface ModelInfo {
incomplete: boolean
prompt_cache: boolean
spec_draft_model: string | null
spec_type: string | null
spec_active: boolean
parallel_slots: number
capabilities: Capabilities
}
export interface VocabFingerprint {
model: string | null
pre: string | null
n_vocab: number | null
arch: string | null
}
export interface DraftInfo {
path: string
filename: string
size_bytes: number | null
vocab: VocabFingerprint | null
compatible: boolean | null // null = nicht prüfbar (Ziel-GGUF fehlt)
}
export interface DraftsResp {
target_path: string
target_exists: boolean
target_vocab: VocabFingerprint | null
drafts: DraftInfo[]
}
export interface DiscoverModel {
name: string
author: string
+10
View File
@@ -8,6 +8,7 @@ import {
type AgentStatus,
type ConnectResp,
type DiscoverResp,
type DraftsResp,
type Health,
type Job,
type Memory,
@@ -31,6 +32,7 @@ export const qk = {
agentStatus: ["agent-status"] as const,
updates: ["updates"] as const,
discover: ["discover"] as const,
drafts: (target?: string) => ["drafts", target ?? ""] as const,
connect: (params?: string) => ["connect", params ?? ""] as const,
memory: (q?: string, category?: string) => ["memory", q ?? "", category ?? ""] as const,
}
@@ -70,6 +72,14 @@ export const useUpdates = (refetchInterval?: number) =>
export const useDiscover = () =>
useQuery({ queryKey: qk.discover, queryFn: () => api<DiscoverResp>("/api/discover") })
// Verfügbare Spec-Draft-Modelle + Vocab-Kompatibilität zum Ziel-Modell (GGUF-Pfad).
export const useDrafts = (target?: string) =>
useQuery({
queryKey: qk.drafts(target),
queryFn: () => api<DraftsResp>(`/api/models/drafts?target=${encodeURIComponent(target ?? "")}`),
enabled: !!target,
})
export const useConnect = (params?: string) =>
useQuery({
queryKey: qk.connect(params),