Engram
Engram explained
Engram is DeepSeek's conditional memory module: a scalable lookup table that stores factual patterns in hashed N-gram memory and retrieves them in constant time, complementing MoE "compute" sparsity with a separate memory sparsity axis. The Engram repository implements the paper "Conditional Memory via Scalable Lookup" and notes that V4.1-Flash integrates Engram conditional memory (196B parameters on the model card). This is unofficial summary text — read the Engram paper (PDF) for the full design.
The breakthrough memory architecture integrated into DeepSeek-V4.1-Flash. Engram stores massive factual world knowledge and long-range repository symbols in a 196B parameter hashed associative memory table, retrieved in constant O(1) time without triggering heavy transformer attention layers.
https://github.com/deepseek-ai/Engram
- Decouples factual knowledge storage from transformer attention layers, keeping active parameter count ultra-lean.
- Constant O(1) retrieval latency regardless of knowledge base size.
- The V4.1-Flash model card lists Engram conditional memory at 196B parameters.
Plain-language overview
Engram implements conditional memory: a large lookup table of hashed n-gram memory that retrieves facts without running full self-attention over the entire vocabulary of stored patterns. The Engram repository hosts code and the "Conditional Memory via Scalable Lookup" paper PDF linked from the README.
DeepSeek-V4.1-Flash's model card lists Engram conditional memory at 196B parameters — separate from the 8B/16B active compute path — so the model can store encyclopedic knowledge while keeping per-token FLOPs low. This directory's V4.1-Flash page summarizes API pricing; Engram explains why the backbone can stay smaller while context-heavy tasks still work.
Researchers comparing Engram to standard retrieval-augmented generation should read the paper for hash routing, sparsity axes (compute vs memory), and ablations. We do not reproduce training recipes here; use official artifacts only.