Coder creates an 'infinite maze' to snare AI bots in an act of 'sheer unadulterated rage at how things are going' on the content-scraped web
I'm getting a little tired of our deep-learning future, folks. While generative AI and deep learning technology isn't inherently bad—it's being used for folding proteins and advancing medical science, for example—the routine doling-out of slop has everyone paranoid, content trawlers are scraping through the web sucking up everything in sight, and an endless parade of techbros are insisting their software can think, feel, and make important decisions, despite evidence to the contrary.
An anonymous coder, writing to 404media under the fake name Aaron B, has also been feeling the malaise, and he's doing something about it. Aaron B has created a digital flytrap to keep web-crawling bots in a thus-far undetectable doom loop, scanning the same pages over and over until someone human comes over and fixes it.
The program, dubbed Nepenthes (after the tropical pitcher plant), exploits a weakness in these crawlers—the fact that they're apparently really dumb, as Aaron B explains:
«It's less like flypaper and more an infinite maze holding a minotaur, except the crawler is the minotaur that cannot get out. The typical web crawler doesn't appear to have a lot of logic. It downloads a URL, and if it sees links to other URLs, it downloads those too. Nepenthes generates random links that always point back to itself—the crawler downloads those new links. Nepenthes happily just returns more and more lists of links pointing back to itself.»
The site itself lays it out in full, as well as the usage cases—allowing you act defensively to «flood out valid URLs within your site's domain name, making it unlikely the crawler will access real content.» You can also, as the site suggests, play offense by ignoring the code's existing list of known trawler IPs and allowing bots to «suck down as much bullsh*t as they have diskspace for, and choke on it.»
There's also a link to a demo of the software in action, as well as an all-caps warning that «THIS IS DELIBERATELY MALICIOUS SOFTWARE INTENDED TO
Read more on pcgamer.com
