DeepGEMM
A clean, performant FP8 GEMM library built specifically for Hopper Tensor Cores. Supports both standard continuous GEMM and fine-grained grouped MoE GEMMs with fine-grained per-tile scaling, achieving over 95% of theoretical peak FP8 tensor core throughput.
https://github.com/deepseek-ai/DeepGEMM
- Implements fine-grained 128x128 tile scaling to prevent FP8 underflow and dynamic range clipping.
- Supports grouped GEMMs to execute all MoE expert calculations in a single unified kernel launch.
- Under 300 lines of core CUDA kernel code, emphasizing readability and extreme performance.
Plain-language overview
DeepGEMM implements FP8 general matrix multiply on Hopper tensor cores, including grouped GEMMs used in MoE layers. The DeepGEMM repository emphasizes readable CUDA while chasing a large fraction of theoretical FP8 peak. Fine-grained per-tile scaling (128×128 blocks in README examples) mitigates FP8 dynamic range issues during training and inference.
DeepSeek trained V3/R1 with FP8 pipelines documented in the V3 paper/README; DeepGEMM is the modular GEMM piece engineers can reuse outside the monolithic training codebase. Build instructions, supported matrix shapes, and micro-benchmarks are in the repo — verify on your GPU SKU before planning cluster upgrades.
For API users, FP8 inside DeepSeek's cloud is opaque; this library matters when you self-host MoE weights or research quantization. See also FlashMLA for attention decoding and the DualPipe schedule for training-scale pipeline parallelism.