[…]the authors of the report determined that the energy used for a single query to the open-source Llama 3.1 8B engine used around 57 joules of energy to generate a response.[…]

A larger model, like Llama 3.1 405B, needs around 6,706 joules per response – eight seconds of microwave usage.

In other words, the size of a particular model plays a huge role in how much energy it uses.
Although its true size is a mystery, OpenAI’s GPT-4 is estimated to have well over a trillion parameters, meaning its per-query energy footprint is likely far higher than the Llama queries tested.[…]

AI video generation, on the other hand, is an energy sinkhole.

In order to generate a five-second long video at 16 frames per second, the CogVideoX AI video generation model consumes a whopping 3.4 million joules of energy – equivalent to running a microwave for an hour or riding 38 miles on an e-bike, Hugging Face researchers told the Tech Review.