Kandidaten live testbar: gpt-oss-120b (2,3x schneller als heavy) + VL-30B in Config (Review P3-17 abgeschlossen)
Bench (Box, Vulkan, 32k ctx): gpt-oss-120b tg 54-55 t/s vs. Qwen3.5-122B 23,5 t/s bei 60 statt 73 GB. Beide OHNE Alias-Wechsel deployt — Qualitaets-Entscheid beim User. brains-Gruppe nach Reload wieder angewaermt (hermes/embed/vision ready). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -68,6 +68,27 @@ models:
|
||||
out: [text]
|
||||
tools: true
|
||||
context: 131072
|
||||
gpt-oss-120b:
|
||||
# heavy-Kandidat (Bench 02.07.: tg 54-55 t/s vs. Qwen3.5-122B 23,5 t/s = 2,3x, 60 statt
|
||||
# 73 GB). Noch OHNE heavy-Alias — erst Qualitaets-Check (deutsch/Reasoning), dann Entscheid.
|
||||
cmd: |
|
||||
llama-server -m /srv/models/gpt-oss-120b-GGUF/gpt-oss-120b-mxfp4-00001-of-00003.gguf --host 127.0.0.1 --port ${PORT} -c 32768 -ngl 999 -fa on --no-mmap --jinja --cache-reuse 256 -cram 16384
|
||||
ttl: 600
|
||||
capabilities:
|
||||
in: [text]
|
||||
out: [text]
|
||||
tools: true
|
||||
context: 32768
|
||||
Qwen3-VL-30B-A3B-Instruct:
|
||||
# vision-Upgrade-Kandidat (Bench 02.07.: tg 90-92 t/s, MoE 3B aktiv, stark auf GUI-Benchmarks).
|
||||
# Noch OHNE vision-Alias — erst Screenshot-Qualitaets-Check, dann Entscheid (brains-Budget!).
|
||||
cmd: |
|
||||
llama-server -m /srv/models/Qwen3-VL-30B-A3B-Instruct-GGUF/Qwen3-VL-30B-A3B-Instruct-Q4_K_M.gguf --host 127.0.0.1 --port ${PORT} -c 32768 -ngl 999 -fa on --no-mmap --mmproj /srv/models/Qwen3-VL-30B-A3B-Instruct-GGUF/mmproj-F16.gguf --jinja
|
||||
ttl: 300
|
||||
capabilities:
|
||||
in: [text, image]
|
||||
out: [text]
|
||||
context: 32768
|
||||
GLM-4.6V-Flash:
|
||||
cmd: |
|
||||
llama-server -m /srv/models/GLM-4.6V-Flash-GGUF/GLM-4.6V-Flash-Q4_K_M.gguf --host 127.0.0.1 --port ${PORT} -c 131072 -ngl 999 -fa on --no-mmap --mmproj /srv/models/GLM-4.6V-Flash-GGUF/mmproj-BF16.gguf --jinja
|
||||
|
||||
Reference in New Issue
Block a user