You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
llama_model_load: error loading model: unknown model architecture: 'gemma4-assistant'
llama_model_load_from_file_impl: failed to load model
srv load_model: failed to load draft model, 'C:\Users\User\.lmstudio\models\Janvitos\gemma-4-26B-A4B-it-qat-assistant-MTP-Q8_0-GGUF\gemma-4-26B-A4B-it-qat-assistant-MTP-Q8_0.gguf'
srv operator (): operator (): cleaning up before exit...
main: exiting due to model loading error
I understand that in Qwen the model came with MTP, but in Gemma 4 the QAT model comes alone and with a separate draft. Am I doing something wrong or is this build not yet prepared for this?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
I am trying to use your compiled version on Windows with CUDA to run
.\llama-server.exe -m "C:\Users\User\.lmstudio\models\mradermacher\gemma-4-26B-A4B-it-qat-q4_0-unquantized-heretic-GGUF\gemma-4-26B-A4B-it-qat-q4_0-unquantized-heretic.Q4_K_S.gguf" --model-draft "C:\Users\User\.lmstudio\models\Janvitos\gemma-4-26B-A4B-it-qat-assistant-MTP-Q8_0-GGUF\gemma-4-26B-A4B-it-qat-assistant-MTP-Q8_0.gguf" --host 127.0.0.1 --port 10000 -ngl 99 --n-cpu-moe 30 -c 262144 -t 10 -tb 16 -b 2048 -ub 1024 -np 1 -fa on -ctk turbo4 -ctv turbo3 --top-k 20 --repeat-penalty 1.0 --top-p 0.95 --min-p 0.0 --mlock --no-mmap -kvu --spec-type mtp --spec-draft-n-max 2 -fit offbut it doesn't work for me, I get this error.
I understand that in Qwen the model came with MTP, but in Gemma 4 the QAT model comes alone and with a separate draft. Am I doing something wrong or is this build not yet prepared for this?
What should I do, guys? Thanks!!
Beta Was this translation helpful? Give feedback.
All reactions