The Bitter Lesson
from Richard Sutton
The Baader–Meinhof phenomenon is a cognitive bias that takes place after the introduction of a new concept wherein suddenly this concept appears everywhere you turn. I’ve been experiencing this with this essay as of late - seeing it crop up in pieces like Dario Amodei’s Machines of Loving Grace.
I’ve been re-reading it; trying to distill the core elements that capture my attention. The opening is a summation of the key point:
The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.