Nvidia has offered up more details on the new features coming to its GPUs thanks to the release of DLSS 4, as well as how some of its technologies, such as its new transformer models and enhanced frame generation work.
In an interview with Digital Foundry, vice president of applied deep learning research at Nvidia Bryan Catanzaro spoke about the technology.
Speaking about frame generation, Catanzaro explained how Nvidia’s Optical Flow accelerator was used. The technology, originally used for Nvidia’s automotive division’s research into self-driving cars, is an evolution of the company’s video encoding algorithms.
However, Optical Flow was difficult to improve, so Nvidia has decided to switch things up with a fully AI-based solution for DLSS 4’s take on frame generation. “When we built Nvidia DLSS 3 Frame Generation, we absolutely needed hardware acceleration to compute Optical Flow,” explained Catanzaro. “We didn’t have enough Tensor Cores and we didn’t have an Optical Flow algorithm that was good enough.
We hadn’t developed a real-time Optical Flow algorithm that ran on Tensor Cores that could fit our compute budget. We had the Optical Flow accelerator, which Nvidia had been building for years as an evolution of our video encoder technology, and it’s also been a part of our automotive computer vision acceleration for self-driving cars.” “It made sense for us to use that for Nvidia DLSS 3 Frame Generation.