This became obvious with the launch of Nvidia’s Blackwell accelerators a year ago. The chips boasted 5x the performance uplift over Hopper, which sounded great until you realized it needed twice the die count, a new 4-bit datatype, and 500 watts more power to do it.

The reality was, normalized to FP16, Nvidia’s top-specced Blackwell dies are only about 1.25x faster than a GH100 at 1,250 dense teraFLOPS versus 989 — there just happened to be two of them.

Sounds like Nvidia is trying to pull a Multi Frame Generation style misleading promo tactic with enterprise GPUs too.

Jesus Christ 600kW of power needed per rack. Most normal datacenters are still at something like 20 - 40 kW.