AI Training Compute Growth Rate Revealed Through New York Times Visualization

The Math Behind Modern AI Models

The New York Times has turned Epoch AI's research into a visualization showing just how much math goes into training a modern AI model. The piece draws on Epoch AI's estimates of training compute and capability scores.

Training Compute Growth Rate

Epoch AI measures training compute in FLOP, short for floating-point operations. Each FLOP is a single arithmetic calculation involving decimal numbers. According to Epoch AI, training compute for frontier language models has grown at roughly 5x per year since 2020. That works out to a doubling roughly every 5.2 months.

The Drivers of Growth

Epoch AI has also broken down where that growth comes from for frontier models after 2018. It points to three main levers: Hardware quantity: a median of roughly 1.69x per year, meaning labs are simply buying and wiring together more chips.

Training duration: roughly 1.53x per year, as runs last longer. Hardware performance: roughly 1.41x per year, as each chip gets faster. Epoch AI puts the combined effect at roughly 4.27x annual growth.

The Chip Count Behind the Curve

As of mid-2026, Epoch AI estimates there are approximately 20 million AI chips in use globally, measured in NVIDIA H100 equivalents. Projections suggest that count could double approximately every nine months, potentially reaching 200 million by the end of 2028. Epoch AI's AI chip users explorer launched in September 2026. The underlying database covers 86 sites, which together account for approximately 44% of global AI compute as of that date. A report dated October 2, 2026, assessed that high-bandwidth memory (HBM) shipped between 2025 and 2027 may support between 30 million and 170 million concurrent frontier-model AI agents. HBM is the specialized, ultra-fast memory stacked alongside AI chips so they are not left waiting on data. The same report found that the figure could scale to roughly 1.9 billion agents running on cheaper models.