fix(deepseek-v4): correct V4-Flash attention and MoE math, and make the 4N recipe reproducible - #930

Merged

wenxie-amd merged 18 commits into

dev/tas/deepseek-v4-phase2from

dev/lhz/dsv4_fix

Aug 5, 2026

Commits on Jul 28, 2026

fix(deepseek-v4): drop YaRN m_scale from the attention softmax scale
lhzhang333
committed
10524f8
fix(deepseek-v4): rotate compressed KV at original-sequence positions
lhzhang333
committed
70a0fc0
fix(deepseek-v4): de-rotate the attention output before the O projection
lhzhang333
committed
2255b56
fix(deepseek-v4): keep the indexer QK in high precision by default
lhzhang333
committed
4dbfb57
feat(deepseek-v4): add opt-in indexer distillation loss
lhzhang333
committed
1ae566c

Commits on Jul 29, 2026

Commits on Jul 31, 2026

merge dev/tas/deepseek-v4-phase2 and resolve conflicts
lhzhang333
committed
ee80eb0

Commits on Aug 3, 2026

update scripts
lhzhang333
committed
1458f9e
style(deepseek-v4): satisfy isort and black on the V4 changes
lhzhang333
committed
e26f662