Sony Patent Could Lead to Improved Facial Animations in Games

gamerant.com:

News has surfaced of Sony patenting a method of AI-generated facial performance capture to ease the workload on animators. It’s no secret that Sony Interactive Entertainment studios are industry leaders when it comes to capturing true-to-life performances in the video game medium. With a huge increase in fidelity across the past generation of consoles, in the space of a decade developers like Naughty Dog and Sony Santa Monica have gone from hand drawing facial and movement animation with painstaking effort to building games and characters entirely through performance capture. It’s no wonder, then, that Sony has filed a patent for more game-changing performance capture technology.

Filed on April 1, 2022, by Sony Interactive Entertainment LLC, the patented system takes mesh data from a human actor captured by a 3D camera to record the depth of the face. This polygonal mesh is used in game development to build accurate human models and is typically handcrafted by animators using the vertices dots on a mocap actor’s face as reference. This method is currently the standard across most triple-A videogame development, including the Insomniac's upcoming Spider-Man 2. This system would automatically generate mesh data, however, using a mix of 3D and 2D cameras to train an Artificial Intelligence program that can accurately capture facial expressions and build mesh data with little need for human intervention.

Sony Hoping to Minimize Frame Rate Dips with New Patent

This patent follows an industry trend in its development of AI to produce work that previously took a great deal of human handiwork. The video game industry has seen a huge rise in the use of AI to do many things, from upscaling the resolution of textures to interpolating