Andrej Karpathy
Andrej Karpathy's most useful AI lesson is not a paper; it is a repo where a GPT becomes legible in roughly 600 lines split across train.py and model.py. nanoGPT, released in 2022, has about 59,000 GitHub stars as of May 2026 and still tells you more about transformers than most vendor decks. His career keeps returning to one move: shrink the machine until a working engineer can hold it in their head.
Born in Bratislava in 1986 and raised in Toronto, Karpathy earned his Stanford PhD in 2015 under Fei-Fei Li. He helped build OpenAI in 2015, led AI at Tesla from 2017 to 2022, returned to OpenAI in 2023, announced Eureka Labs in July 2024, and joined Anthropic's pre-training work on May 19, 2026.
The build-to-understand method
Karpathy teaches by removing everything that is not the mechanism. micrograd makes reverse-mode autodiff small enough to read in one sitting. minGPT strips GPT training into a minimal PyTorch shape. nanoGPT moves from toy clarity toward a real training loop that can reproduce GPT-2 124M behavior on OpenWebText with an 8xA100 node in about 4 days, according to the repo README.
That matters because LLMs often arrive as cloud objects: endpoint, bill, latency, mystery. Karpathy's artifacts push in the opposite direction. The reader sees tokens, embeddings, attention heads, residual streams, loss curves, checkpoints, and generation. The spell breaks.
The public artifacts
| Artifact | Year | Concrete thing it changed |
|---|---|---|
| CS231n | 2015 | Made convolutional neural networks teachable through assignments, notes, and lecture videos |
micrograd |
2020 | Shows autodiff and a tiny neural net library in a few hundred lines |
nanoGPT |
2022 | Gives builders a readable GPT training repo with about 59,000 GitHub stars as of May 2026 |
llm.c |
2024 | Moves LLM training toward plain C/CUDA, away from hiding every detail behind Python frameworks |
| Eureka Labs | 2024 | Treats AI education as a product, with LLM101n as the first named course direction |
| Anthropic | 2026 | Puts him back inside frontier pre-training research after two years of public education work |
The pattern is not "explain AI to beginners." The pattern is executable pedagogy. A Karpathy lesson usually leaves behind an object: a notebook, a repo, a lecture, a training script, a phrase like concept software 2 0 that engineers can argue with.
What's contested
The open argument is whether small reconstructions teach the real thing or only its toy shadow. A 300-line training loop can show why attention works; it cannot show the procurement, data filtering, eval design, failure analysis, and cluster politics behind a frontier model. The danger is false fluency: reading nanoGPT can make a builder feel closer to GPT-4 than the distance warrants.
There is also a Tesla-shaped caution. Karpathy's public AI story includes a bet on vision-heavy autonomy, but real roads punish rare cases: sun glare, construction signs, ambiguous human intent, stale maps, sensor disagreement. The lesson may be less "neural nets eat software" and more "learned systems move the hard part into data, measurement, and the edge-case ledger."
Why this has to do with other realms
Karpathy sits close to person richard feynman because both make prestige machinery feel rebuildable. Feynman takes quantum electrodynamics down to arrows and amplitudes. Karpathy takes transformers down to tokens and matrix multiplies. In both cases, the teacher does not merely simplify; he chooses the smallest artifact that still behaves like the real thing.
There is a second bridge to mission voyager 1. Voyager works because its engineering became inspectable: trajectory, power limits, antenna geometry, time delay, the Golden Record. Karpathy's best repos do the same for AI. They turn a remote machine into a thing with parts.
An open question
If the next generation learns AI by rebuilding small versions before touching paid APIs, what myths about intelligence, software, and expertise will fail to survive first contact with the code?
Key sources
- Karpathy, Andrej. "Software 2.0" (2017), https://karpathy.medium.com/software-2-0-a64152b37c35 — the essay that named neural networks as a new kind of code artifact.
- Karpathy, Andrej, Justin Johnson, and Fei-Fei Li. "Visualizing and Understanding Recurrent Networks" (2015), arXiv:1506.02078 — early work using character-level models as an inspectable testbed.
karpathy/nanoGPTGitHub repository (2022), https://github.com/karpathy/nanoGPT — the load-bearing public artifact for his GPT pedagogy.karpathy/microgradGitHub repository (2020), https://github.com/karpathy/micrograd — a minimal autodiff engine for teaching backpropagation.- TechCrunch, "OpenAI co-founder Andrej Karpathy joins Anthropic's pre-training team" (May 19, 2026) — source for the Anthropic move.
- TIME100 AI profile of Andrej Karpathy (2024) — source for the education-company framing around Eureka Labs.
Further Reading
- tech llm — the machine Karpathy keeps making small enough to inspect.
- concept rag — the place where many builders meet LLM systems after the model itself.
- Stanford CS231n course materials — the pre-LLM teaching artifact that shaped a generation of computer-vision engineers.
- Richard Hamming, The Art of Doing Science and Engineering (1997) — a clean companion text on craft, taste, and technical judgment.
- "Let's build GPT: from scratch, in code, spelled out" by Andrej Karpathy (YouTube, 2023) — the video version of the reconstruction method.
See Also
- concept software 2 0
- tech llm
- concept rag
- person richard feynman
- mission voyager 1
- concept transformer
Abhishek's take
Karpathy matters to me because he refuses to let AI stay theatrical. I trust a tool more after I can trace one input, one loss, one failure case, and one fix through the small version. The point is not purity; the point is knowing where the magic stops and the engineering begins.
Tags: #karpathy #openai #tesla #llm-wiki #education #ai-craft