'Cyberpunk 2077' Finally Shows What DLSS Is Good For

More recent Nvidia graphics cards have a proprietary feature called Deep Learning Super Sampling (DLSS), and while it's often been touted as a powerful new rendering tool, the results have sometimes been underwhelming. Some of this is down to the oddly mixed-message around how DLSS was rolled-out: it only works on more recent Nvidia cards that are still near the cutting edge of PC graphics hardware… but DLSS is designed to render images at lower resolutions but display them as if they were rendered natively at a higher resolution. 

This is a companion discussion topic for the original entry at https://www.vice.com/en_us/article/88a5ax/cyberpunk-2077-finally-shows-what-dlss-is-good-for

I don’t claim to have any special knowledge of how Nvidia’s algorithm works (and the nature of deep learning suggests that even they can never be completely sure how it works) but I suspect that it’s doing some clever tricks with hiding noise where you can’t see it. The basic gist is that, at a given fixed total quantity of noise (due to hardware limitations, for example), where do you allocate it in order to maximize image quality and minimize subjective noticeability? Vision researchers have discovered that certain types of noise are more noticeable than others so if you allocate more processing power to eliminate those (and save power by ignoring the less noticeable types), you can deliver a more subjectively higher quality image at the same level of hardware constraint. That’s my guess for what DLSS might be doing under the hood, and how it is possible for it to deliver results sometimes (subjectively) superior to native 4K.

I would think one of the biggest reasons DLSS can sometimes “be better” than native 4K is because the neural network algorithm is trained on images that are at 16K resolution. And I think DLSS will likely always be blurrier than native rendering it’s just that in some cases it can recreate fine details (like text) that are absent in native 4K. The use case I’m most interested in (since I don’t have an RTX card) is if Nintendo ever embraces it with a Switch upgrade so that the games wouldn’t look so terrible on my TV.