chiprook

← AI

AI23 hOctober 16, 2026, 21:33

GLM-5.3-Flash (321B) runs locally on a single RTX 5090

The 321B-parameter GLM-5.3-Flash model with a 3.05 bpw EXL3 quant (125 GB) was run locally on one RTX 5090 with 128 GB of RAM using Project Maya. It decodes at about 25 tokens per second with a 512K-token context.

DEV Community

Open source

GLM-5.3-Flash (321B) runs locally on a single RTX 5090

Model: 321B parameters, ~18B active per token, up to 1M-token context
The 3.05 bpw EXL3 quant is 125 GB and does not fit in 128 GB of RAM
Project Maya tiers experts across GPU, RAM and NVMe, moving them live
About 25 tokens/s at 512K context on an RTX 5090

#GLM-5.3-Flash#ProjectMaya#Nvidia#RTX5090

More stories in swipe mode🔥 Trending now: An AI model couldn't find test tokens, so it brute-forced a wallet to fund its own task.

Read next

AI

Cantina releases apex-flash-1, an open-weights vulnerability research model

AI

Open-source coding LLMs compared: GLM-5.3-Flash, Qwen3.8-Flash-Next, DeepSeek V4 Flash

AI

GLM-5.3-Flash: 320B Open-Weight Model With 18B Active and 1M-Token Context

AI

Z.ai Details GLM-5.3-Flash Inference Build on 100,000 Chinese Chips