angelos-p/llm-from-scratch
Contribute to angelos-p/llm-from-scratch development by creating an account on GitHub.
Welcome! Type "help" for available commands.
$
Open Source AI Project
Summary
TimeCapsuleLLM is an experimental project training language models from scratch on historical datasets to eliminate modern bias. By avoiding fine-tuning, it aims to simulate the worldview and language of specific eras, currently focusing on 1800–1850 London.
Highlights
- Trains from scratch to prevent modern concept leakage
- Uses nanoGPT architecture
- Focuses on 1800–1850 London data
- Scaling dataset from 50 to 600 texts for better coherence
auto-generated
Context
Audience
Machine Learning Researchers and AI Developers
DomainMachine Learning
FormatOpen source repository
AccessOpen source
Topics
Related
nanoGPTHistorical LinguisticsLLM Bias ReductionAndrej Karpathy
