Hacker News vector search dataset using ClickHouse
Dataset containing 28+ million Hacker News postings & their vector embeddings
Open Source Archive Tool
Summary
HackerBook is a static, offline archive of 20 years of Hacker News data that runs entirely in the browser using SQLite WebAssembly. It packages the dataset into immutable, content-hashed shards, allowing users to query and browse the full history without a server.
Highlights
- Runs entirely client-side in the browser via SQLite WASM, requiring no backend server.
- Uses content-hashed filenames for shards to ensure perfect cache correctness and immutability.
- Provides a toolkit for building daily snapshots from BigQuery raw data using Node.js.
- Designed for durability and offline access, ensuring the community data is never lost.
- Optimized for performance by fetching only necessary gzipped SQLite shards on demand.
auto-generated
Context
Audience
Developers and data enthusiasts interested in archiving community discussions and client-side database technologies
DomainWeb Development
Formatopen source repository
Accessopen source
Topics
Related
SQLite WebAssemblyHacker NewsStatic WebsiteBigQueryNode.jsKiwix