Google's AI made up a fake cheese fact that wound up in an ad for Google's AI, perfectly highlighting why relying on AI is a bad idea

pcgamer.com:

First reported by Ars Technica, the «big cheese» of online search appears to have really curdled things, and in the heart of America's dairyland, no less. An ad for Google's Gemini AI that played in Wisconsin during the Super Bowl was first shown online last week with a «hallucination»—that is to say, a bald-faced lie—visible during a demonstration of the «writing aid.» The ad was quietly edited to remove the hallucination before it went live during the big game.

Apparently everybody just soft-launches their Super Bowl ads on YouTube a week in advance now, and that goes for Google's gaggle of 50 spots showcasing how Gemini could help small business owners, with a unique 30-second showcase for each state of the Union. In Wisconsin, the ad focused on a cheesemonger writing online store copy for his assorted victuals.

The weirdness came partway through, when the ad actually showed Google Gemini in action. It told the cheese vendor that Gouda accounts for «50 to 60 percent of the world's cheese consumption.» Now, Gouda's hardly a hardcore real head pick like Roquefort or BellaVitano, but there's also no way it's pulling in cheddar or mozzarella numbers. Travel blogger Nate Hake and Google-focused Twitter account Goog Enough documented the erroneous initial version of the ad, but Google responded by quietly swapping in a more accurate Gemini-suggested blurb in all live versions of the ad, including the one that aired during the Super Bowl.

Adding another wrinkle to the story, the erroneous statistic can seemingly be sourced to the Gouda page on cheese.com, an SEO-focused website with a love for cheese and a loose cannon, devil may care approach to the facts. Google Gemini will supposedly provide links to the sources it pulls from, much like AI Overview in search, and you could charitably argue that this feature just wasn't on display during the ad.

But that raises other concerns: Is the ad then just pure "bullshot" like all those pre-rendered E3 game trailers of yore or