LLM Chronicles
A fast-paced whiteboard animation series unraveling Deep Learning and Large Language Models. Dive into core concepts, from Neural Networks basics to t...
Educational Tutorial
Summary
A hands-on workshop designed to teach the fundamentals of Large Language Models by building a GPT training pipeline from scratch. Users will create a ~10M parameter model capable of generating text, covering everything from tokenization to transformer architecture.
Highlights
- Builds a functional GPT model from the ground up
- Covers essential components: tokenization, transformer architecture, and training loops
- Optimized to train a 10M parameter model on a laptop in under an hour
- Supports local execution (Mac, Linux, Windows) and Google Colab
auto-generated
Context
Audience
Developers and learners interested in AI who have basic Python proficiency
DomainMachine Learning
Formatopen source tutorial
Accessopen source
Source Overview
This is a Python workshop for writing a GPT training pipeline from scratch, covering tokenizer, transformer architecture, training loop, and text generation.
It trains a ~10M parameter model on Shakespeare data on a laptop in under an hour using character-level tokenization. It requires Python 3.12+ and runs on Apple Silicon, NVIDIA GPU, CPU, or Google Colab.
Topics
Related
PyTorchnanoGPTTransformersPythonLarge Language Models
