Qualcomm's CEO on AI performance in laptops: 'People talk about TOPS but they should be talking watts'

pcgamer.com:

In a keynote speech at Computex 2024, Cristiano Amon, the CEO of Qualcomm, talked about how its new Snapdragon X chips have plenty of AI performance, up to 45 TOPs from its NPU. But he also pointed out that while talking about «industry-leading performance» is all well and good, people should also be talking about how much power such chips are using.

«People want to talk about TOPS, but they also need to talk about watts, because that's actually very, very important.» In case you're wondering what on Earth the CEO of Qualcomm is referring to, TOPS is the now-standard metric for the performance of NPUs—neural processing units, that handle basic AI routines, leaving the CPU and GPU free to do other tasks.

One TOP equates to a trillion operations per second, which sounds like a huge number. Well, it is, but compared to even a budget-level graphics card, it's not very much at all. However, NPUs are designed to do AI tasks in the background, so that means they need to fulfil their performance requirements using as little energy as possible. And since all electrical energy used by a chip gets converted into heat, power-efficient NPUs help to keep the whole processor cooler.

Amon then compared the NPU in Qualcomm's Snapdragon X to those made by AMD and Intel, claiming a performance-per-watt 2.6 times better than AMD and 5.4 better than Intel's Core Ultra 7 chips. Those are some pretty bold claims but as there are no independent reviews of Snapdragon processors yet, they cannot be verified.

But even if the actual performance per watt of power consumed is only 50% of what Qualcomm claim it is, that's a sizeable improvement over the competition.

And it's not just about heat, even though Amon went on to point out a Snapdragon-powered laptop running 20 degrees Celsius cooler in an AI task than an equivalent laptop using its GPU. Getting more performance per unit of energy means you could turn things down a notch, to use even less power, and get a handy boost to a laptop's