FlashMLA

Checked on 15 Nov 2026 against FlashMLA.

An ultra-fast decoding kernel tailored specifically for Multi-Head Latent Attention (MLA) on NVIDIA Hopper architectures. Employs Tensor Memory Accelerator (TMA) asynchronous memory copies to maximize memory bandwidth utilization during variable-length generation.

https://github.com/deepseek-ai/FlashMLA

  • Optimized for variable sequence length decoding on H800/H100/H200 GPUs.
  • Takes full advantage of Hopper TMA hardware features for zero-copy global-to-shared memory transfers.
  • The repository describes optimized MLA kernels for Hopper GPUs and variable-length MLA decoding.

Plain-language overview

FlashMLA targets decoding with Multi-Head Latent Attention on NVIDIA Hopper GPUs (H100/H800/H200 class). The FlashMLA repository documents kernels that use Tensor Memory Accelerator (TMA) loads to feed tensor cores during variable-length generation — the regime where standard attention becomes memory-bandwidth bound.

MLA, described in the DeepSeek-V2/V3 technical materials, compresses KV cache footprint compared with standard multi-head attention. FlashMLA is the production-oriented implementation DeepSeek open-sourced so other teams can reproduce efficient MLA decoding on Hopper hardware. README sections cover build requirements, supported head dimensions, and benchmark scripts; run those on your cluster rather than trusting third-party throughput claims.

FlashMLA sits beside DeepGEMM (FP8 matmul) and DeepEP (MoE dispatch) in the training-and-inference stack DeepSeek used for V3/R1/V4 clusters. None of these repos replace the Hugging Face model weights; they accelerate operators once you already run DeepSeek architectures.

All libraries