Unreal's new MetaHuman Animator can turn an iPhone video into a scarily accurate game animation in three minutes

pcgamer.com:

 Yesterday's State of Unreal 2023 presentation was rather overshadowed by a certain surprise game announcement(opens in new tab), but Epic showed off some genuinely amazing tech coming to Unreal Engine 5 in the near future. We got to see impressive new procedural environment tools(opens in new tab), Lords of the Fallen showing off the engine's enhancements to character creation and realistic armour and clothing(opens in new tab), and more. But what really blew my socks off was the MetaHuman Animator stage demo.

We've seen MetaHuman(opens in new tab) before—it's a tool that allows developers to generate highly realistic human faces, fully rigged for animation. Actually animating those faces in a way that matches their realistic look—avoiding the uncanny valley—has been a difficult and time-consuming process thus far, however. Motion capture—that is, getting actors to perform the movements and then turning that footage into animation—has produced the best results, but requires specialised equipment and months of expert work.  

Well, until now. On stage, Epic showed off a new feature called MetaHuman Animator, designed to completely streamline the process. During the stage demo(opens in new tab), you can see actor and developer Melina Juergens—star of the Hellblade games—recording a video of her face on an iPhone, before a technician uploads it, processes it, and turns it automatically into a full animated sequence in the space of literally three minutes. 

The final video isn't completely finished—an animator would ideally go in and touch up elements of it, and obviously work it into the in-game scene—but it looks 90% of the way there, and that's astonishing. You don't even have to map it onto a MetaHuman modelled on that