Nvidia DLSS 5: The Generative AI Redefining Video Games

Nvidia made waves at its GTC conference by announcing DLSS 5, a major update to its AI-based scaling technology. This new tool, described by CEO Jensen Huang as the "GPT moment for graphics," promises to combine traditional rendering with generative AI to deliver a spectacular advancement in visual realism. However, this announcement has also sparked mixed reactions, with some critics labeling the technology as a "mess" that could undermine the artistic intent of video games.

DLSS 5 stands out from previous versions by utilizing generative AI to rework lighting and materials in video games, adding details that appear newly created. Games that support this technology, such as Resident Evil Requiem, Starfield, Hogwarts Legacy, and EA Sports FC, have been showcased with notable improvements in lighting and shadows. Unlike earlier versions that used machine learning to bridge the gap between high and low graphical settings, DLSS 5 aims to provide a more realistic visual experience.

According to Nvidia, the AI model of DLSS 5 is trained end-to-end to understand the complex semantics of scenes, including characters, hair, fabrics, and translucent skin, as well as environmental lighting conditions like front lighting, back lighting, or overcast weather. This deep understanding allows DLSS 5 to generate visually accurate images that manage complex elements such as subsurface scattering on skin, the delicate sheen of fabrics, and light-material interactions on hair, all while preserving the structure and semantics of the original scene.

The impact of DLSS 5 is particularly evident in the changes made to the appearance of character models. For instance, in Resident Evil Requiem, the application of generative AI to the original assets transformed the appearance of protagonist Grace Ashcroft, giving her a radically different look, as if she had used an Instagram filter that made her lips fuller and applied intense eyeshadow. Similarly, in Starfield, the application of DLSS 5 produced results where sharpness is pushed to the extreme, with bright reflections giving characters' features and hair a stage-lit appearance, despite an environment that lacks such lighting.

Todd Howard, head of Bethesda Game Studios, expressed his enthusiasm for DLSS 5, stating, "When NVIDIA showed us DLSS 5 and we got it running in Starfield, it was amazing to see how it brought the game to life. We played it. We can't wait for all of you to do the same." However, not everyone shares this enthusiasm. Game developer Mike Bithell criticized the technology, saying, "For when you absolutely, positively want no artistic direction in your gaming experience. Disappointing to see someone take this nonsense seriously. What a mess."

Despite the criticism, Nvidia asserts that DLSS 5 provides game developers with controls that allow them to decide how this technology impacts a game's appearance. Therefore, it is possible that at the time of its release, the in-game effects will differ from what we see currently. Nvidia clarifies that DLSS 5 respects artistic intent in two ways: by integrating the game's color and motion vectors for each frame into the model, anchoring the output to the source 3D content, and by providing developers with detailed controls such as intensity and color calibration. Artists can use these controls to adjust blending, contrast, saturation, and gamma, and determine where and how enhancements are applied to maintain the game's unique aesthetic. Developers can also mask specific objects or areas to exclude them from enhancement.

DLSS 5 will be available this fall, and so far, it is only confirmed to be compatible with a limited number of games, which, in addition to those mentioned earlier, include major titles like the remake of The Elder Scrolls VI: Oblivion and Assassin's Creed Shadows.