#gpu stuff

Clicking into the article it is part of a greater project to perform other tasks i dont personally thing AI should be getting used for, but in a vacuum as its own stage in the pipeline i believe it fits in with the original vision i had of machine learning.

Train massive expensive models to perform small dedicated tasks on low end consumer hardware to gain performance on stuff that otherwise might not have even been possible with a traditionally designed algorithm, let alone at that speed.

Pose estimation for extracting motion data from a video via computer vision, for example was very anticipated in the vr community for motion capture, would allow animation data for artists to be recorded with a $20 webcam instead of an expensive mocap rig, to start out at least.

Now what i will say is that they seem to be building it into a neural rendering engine for lighting and whatnot which i'm not fully on board because i think that crosses outside the bounds of technical and into artistic.

But i dont know how its implemented. I will say, if i were to use AI there are a few specific tasks i would like performed that otherwise wouldnt be viable on a low end machine using a rasterizer.

The first one Nvidia actually mentions, which is global illumination. Solutions for this are possible traditionally it just takes a lot of effort getting everything set up and is expensive for more complicated scenes. Taking in the static lightmap and object data in the form of a separately rendered layer, positional information, extra color parameters, etc.) generates a low resolution real time global illumination texture or light probes or a function that gets lightly blended in as a texture in the projects traditional rasterizer.

The other task i'd want to see is reflections and subsurface scattering. Basically, this would involve training a depth estimator. Feed the depth map, color scene rendering, and camera information to generate the screen space position to sample for the reflection.