Bitter Lesson

General-purpose methods (search and learning) that scale with computation consistently outperform human-engineered, domain-specific knowledge in AI.

The Bitter Lesson, articulated by Richard Sutton in 2019, observes a 70-year trend in AI: leveraging massive computation with simple, general methods ultimately yields greater progress than embedding complex human knowledge or hand-crafted features. This is the 'bitter' truth: our clever, domain-specific ideas plateau, while scalable approaches win over the long run. Key examples include Deep Blue's deep search beating human-knowledge-based chess programs in 1997, and AlphaGo Zero's self-play (pure learning/search) surpassing the original AlphaGo (which used human expert data). The lesson directs research toward general algorithms—like deep learning and search—that can fully exploit Moore's Law and its exponential computational growth.

https://www.incompleteideas.net/IncIdeas/BitterLesson.html