Nvidia announces DLSS 3.5 with ray reconstruction, boosting RT quality with an AI-trained denoiser

eurogamer.net:

Gamescom 2023 has kicked off — as you might have noticed — and Nvidia has unveiled DLSS 3.5, a new version of DLSS that adds 'ray reconstruction' — essentially AI-tuned denoising that boosts the quality of ray tracing effects on RTX graphics cards. The firm also announced a new community remaster of the legendary Half-Life 2, new DLSS 3 and Reflex titles and more.

Let's cover the DLSS 3.5 news first, as this is fascinating. The critical information here is that real-time ray tracing relies on a relatively small number of rays that are cast — certainly far fewer than one ray per pixel — so we don't have a complete picture of the 'ground truth', we have an image with both signal and noise. That means a denoising step is required to fill in the missing pixels, which uses techniques like temporal acculuation and spatial interpolation to generate a higher-quality final image.

So far, so standard. Normally these denoisers are hand-tuned, but the innovation is that Nvidia has trained an AI network (on five times as much data as they used to train DLSS 3) that can achieve better results. It can use different methods with different ray-traced effects, recognise good pixels and discard bad pixels, and retain the high frequency detail that would normally be smoothed away by upscaling.

As well as improving fidelity, Nvidia says that replacing multiple denoising steps with a single AI-trained model can also lead to a slight performance advantage. They showed an example (in the video above) where Cyberpunk 2077 jumped from ~104 to ~112fps with ray reconstruction enabled. However, in general we should expect performance that's «about the same» with the feature enabled.

Notably, this new denoiser requires additional information from