FROM MODEL STRUCTURE TO SYSTEM BEHAVIOR
Models, systems, and the data between them.
Start with the foundations, explore a model architecture, or follow the hardware and communication that support it.
Modeling Foundations
Modeling Foundations
Build intuition with DeepSeek V2/V3 and Qwen3: attention, experts, retained state, training, and serving.
Model architectures
Qwen3.8
Explore the 27B and Flash-Next variants, recurrent attention, sparse reads, and n-gram memory.
DeepSeek V4.1 Flash
Follow the causal encoder/decoder, compressed attention, and residual-stream ownership.
Kimi K3
Trace depth recurrence, Kimi Delta Attention, latent experts, and layer-by-layer state.
Systems
Data
Data Representation
Follow text into token IDs, packed sequences, attention and loss masks, and stored records.
Inspectable schematics and explicit accounting assumptions. Scientific publication remains blocked pending evidence and review; runtime experiments are not certified. Every page works offline. No telemetry or external runtime assets.