Jensen Huang states that NVIDIA had NVLink 72 instead of NVLink 8 when mixtures of experts came out, allowing a 4-10 trillion parameter model to run in one computing domain.

Jensen Huang states that NVIDIA started as an accelerator company, but the problem with accelerators is that the application domain is too narrow.

Jensen Huang states that people initially thought the amount of high-quality data would limit AI intelligence, but this pre-training scaling law showed larger models with more data result in smarter AI.

Jensen Huang states that in distributed computing at NVIDIA's scale, the CPU, GPU, networking, switching, and distributing the workload across computers are all problems.