What If AI Could Tame Itself?

tech.hindustantimes.com:

Technology companies are falling over themselves to promote expertise in generative AI, the hot new technology that churns out text and images as well as humans can. But few are clamoring for the title of “safest AI firm.”

That is where Anthropic comes in. The San Francisco-based startup was founded by former researchers at OpenAI who rankled at its increasingly commercial focus and split away to create their own firm. Anthropic calls itself an “AI safety” company that's building “steerable” systems, including a large language model similar to the one underpinning OpenAI's ChatGPT.

Anthropic's approach to building safer AI might seem unusual. It involves creating a set of moral principles — which the company hasn't yet divulged — for its own chatbot to follow. This works by having the AI model continuously critique the chatbot on its answers to various questions and asking whether those responses are in line with its principles. This kind of self-evaluation means Anthropic's chatbot, known as Claude, has much less human oversight than ChatGPT.

Can that really work?

I recently spoke to Anthropic's co-founder and chief scientist, Jared Kaplan. In our edited Q&A, he admits that more powerful AI systems will inevitably lead to greater risks, and he says his company, which bills itself as a “public benefit corporation,” won't see its safety principles compromised by a $400 million investment from Alphabet Inc.'s Google.

Parmy Olson: Anthropic talks a lot about making “steerable AI.” Can you explain what that means?

Jared Kaplan: By steerable, what we mean is that systems are helpful and you can control their behavior to a certain extent. [OpenAI's] first GPT models, like GPT-1, GPT-2 and GPT-3, as they became more powerful, there