If AI models are built too deeply, the learning signal flowing backwards through the model would vanish before reaching the beginning, a problem called the vanishing gradient problem.

Residual connections allowed AI models to scale from only a few dozen layers to hundreds or even thousands of layers deep.

In models with residual connections, the final result is a massive cumulative pile of data, where the importance of any single layer's contribution shrinks, burying early information.

For years, AI models could not be built very deep because they would be hard to train.