The 'attention' mechanism in large language models was first introduced by Google's 'Attention Is All You Need' paper.

Transformers fixed the amnesia issue by introducing an attention mechanism, allowing the model to look back at any previous word directly and selectively get exactly the information it needed.

Every transformer-based AI model since 2017 uses an attention mechanism where compute scales quadratically with context length.