Welcome! Type "help" for available commands.
$
Educational Tutorial
Summary
A hands-on workshop designed to teach the fundamentals of Large Language Models by building a GPT training pipeline from scratch. Users will create a ~10M parameter model capable of generating text, covering everything from tokenization to transformer architecture.
Highlights
- Builds a functional GPT model from the ground up
- Covers essential components: tokenization, transformer architecture, and training loops
- Optimized to train a 10M parameter model on a laptop in under an hour
- Supports local execution (Mac, Linux, Windows) and Google Colab
auto-generated
Context
Audience
Developers and learners interested in AI who have basic Python proficiency
DomainMachine Learning
Formatopen source tutorial
Accessopen source
Topics
Related
PyTorchnanoGPTTransformersPythonLarge Language Models