LLM Memory
Some thoughts on implementations
LLM Daydreaming
gwern.netSaved July 17, 2025
Summary
Large language models lack continual learning and a background process for spontaneous idea generation, making them unable to achieve genuine breakthroughs despite extensive training.
The proposal introduces a "day-dreaming loop," where models continually combine and assess concepts in the background, feeding valuable insights back into memory. This process is computationally expensive but could generate novel, proprietary data essential for future AI improvements.
