The Kimi team's recent breakthrough fixes the amnesia problem of current AI models.

Transformers fixed the amnesia issue by introducing an attention mechanism, allowing the model to look back at any previous word directly and selectively get exactly the information it needed.

The Kimi team's breakthrough created a new AI architecture design that can reconfigure itself and learn on the fly.

The Kimi team designed attention residuals where every layer can reach back and look at any previous layer's output, preventing the AI from getting confused or forgetting its earlier thoughts.