Explore the full GPT algorithm in 243 lines of pure Python. Train tiny models, visualize attention, generate names — all interactively.
Run microGPT entirely in your browser. No Python installation, no dependencies — just open and experiment.
Watch attention maps in real-time. See exactly how tokens attend to each other in the transformer.
Based on Andrej Karpathy's 243-line pure Python implementation. Learn GPT from its most atomic form.
The viral open-source project that's helping thousands understand how LLMs actually work under the hood.
Convert text into token IDs using BPE tokenizer. Each word becomes a number the model can process.
Tokens flow through embeddings and multi-head attention. The model learns contextual relationships.
Final layer outputs probability distributions. Greedy or sampling picks the next token — autoregressive generation.
Demystifies LLMs: See exactly how gradients flow, why causal masking matters, in the most atomic form possible. Perfect for students, researchers, and self-learners who want to understand the heart of modern AI.