assignment1-basics/CLAUDE.md at main · stanford-cs336/assignment1-basics
Student version of Assignment 1 for Stanford CS336 - Language Modeling From Scratch - stanford-cs336/assignment1-basics
Online Course
Summary
Stanford CS336 is an implementation-heavy course guiding students through building language models from scratch, covering data collection, transformer construction, training, and deployment. It emphasizes deep Python proficiency and systems optimization, requiring significant coding with minimal scaffolding.
Highlights
- Comprehensive walkthrough of the entire language model lifecycle, from data cleaning to deployment.
- Heavy emphasis on systems optimization, including profiling, benchmarking, and multi-GPU training.
- Requires strong prerequisites in Python, deep learning, linear algebra, and probability.
- Five implementation-focused assignments covering basics, systems, scaling, data, and alignment/RL.
auto-generated
Context
Audience
Computer Science Students and AI Engineers
DomainMachine Learning
Formatonline course
Accessfree online
Source Overview
Stanford CS336 is a 5-unit course on building language models from scratch, covering data collection, transformer construction, training, evaluation, and deployment.
It requires strong Python, deep learning, systems, calculus, probability, and machine learning background. The course includes five implementation-heavy assignments on basics, systems, scaling, data, and alignment/RL, plus GPU compute guidance, late-day rules, and an honor code.
Topics
Related
Transformer Neural NetworksPyTorchDeep LearningDistributed GPU TrainingReinforcement Learning from Human Feedback
