Modder wires ChatGPT into Skyrim VR so NPCs can roleplay and remember past conversations

pcgamer.com:

Spend enough time in any RPG and you'll eventually run out of things to talk about with its characters. But what if they had a never-ending supply of dynamically generated anecdotes? What if you could ask them questions that weren't listed on a menu in front of you? What if they could even remember the experiences they've had with you in the game, and could talk about them at length? Those are the questions modder Art From The Machine is trying to answer in Skyrim VR.

In the video below you can see some scenes from the work-in-progress mod, which uses OpenAI's large language model ChatGPT to generate responses, xVASynth for text-to-speech so the NPCs can be fully voiced, and Whisper for speech-to-text, so players can speak into their mic and the NPCs can understand them. 

That's a lot of different systems being put into play, and the early results are some pretty stilted responses. But at the same time you see a glimmer of the potential of NPCs wired up with AI to create dynamic conversations:

There are a bunch of different conversations and snippets shown in the video, but the most interesting is a chat with Ulfberth War-Bear, star employee of Warmaiden's in Whiterun. Thanks to the mod, Ulfberth knows the shop's hours of operation and can tell them to the player (though awkwardly) and can even calculate how soon the store will close based on the in-game time. 

Yeah, Ulf's pronunciation is pretty awful—though it does sound like his voice, and he's lip synched, which is impressive—but I like the idea of playing a game where a simple question like «How long until you close?» pops into my head and I can just ask a shopkeeper and get a straightforward answer. That'd help make an NPC feel more like a real person.

The modder