#deferred rendering

While I was thinking more about voxels I decided to push ahead with writing a deferred renderer - something I've been keen to learn more about for a while.

If you're not familiar with the concept, a deferred renderer works by 'deferring' calculations such as lighting until after the geometry has been drawn. During the geometry rendering pass, all the geometry data such as the normals, diffuse colour and world space position are written to separate frame buffers. These are then combined with lighting information in a second pass.

Advantages of this approach is that you don't waste GPU cycles lighting a pixel only to have it overwritten by a later draw call, and perhaps more importantly, you only need to render lights over the area they affect, allowing dozens or even hundreds of lights in a scene where the traditional forward renderer would be lucky to manage ten.

The downside, however, is that you have to maintain all these buffers, which can incur a serious memory cost. My naive implementation currently uses three RGBA16 buffers, one for the diffuse colours, one for normals, and one for 'misc' data (one channel for depth, and two for different lighting settings - which I'll talk about separately),  largely because that was the easiest way to set it up with Cinder. 

There are various optimisations that can be made however. For example for the normals you only need to store the X and Y components - you know that the normal is a unit vector, and that the Z must be positive (since the pixel is facing the camera) so it's possible to regenerate the Z value in the lighting shader. Similarly, you only need to store depth for the position, because you can use the screen-space X/Y along with depth to reconstruct a full 3D position. It's also possible to store information in the alpha channel (although for some reason when I tried that the value is treated as actual alpha information - there must be a flag I've missed!).

Hopefully with a bit more work, I can reduce memory usage to around 1/3 of what I'm currently using. But in my next update I'll be talking about post processing.

In the mean time, here's a great set of slides about the deferred renderer in Killzone 2 which I recommend viewing:

Why

We can do a lot of lights

Scene complexity is completely decoupled form lighting

Post processing is easier because we already have the normal, position and depth of every pixel

We can do even more lights

Problems

Alpha blending is not 'out of the box' possible

Heavy on memory

Heavy on fill rate (#pixels a graphics card can render per second)

No MSAA

How

First we render all geometry to +-three buffers. Then we use these buffers to draw the final lit image. A quick way to implement this is to draw a full-screen quad for every light, but this is not very efficient: it kills the fillrate. A better approach would be by using light volumes, or screen aligned quads (early depth-test !)

My first implementation used 3 g-buffers filled in like this:

color/Illumination buffer: GL_RGBA8

position/spec buffer: GL_RGBA32F

normal/gloss buffer: GL_RGBA16F

Which is a total buffer size of 32 + 128 + 64 = 224 bytes! And that is too big. It goes sloooooow. (high memory usage)

My current g-buffer layout:

color/Illumination buffer: GL_RGBA8

normal buffer: GL_RG16F

spec/gloss buffer: GL_RGBA8

32 + 32 + 32 = only 96 bytes! As you can see no position buffer and no z-component for the normal buffer. We even have 16bytes of unused space, but this could be useful later on. (motion vector, extra material properties, ...)

The position will be reconstructed from the depth buffer and is explained in detail here: http://bassser.tumblr.com/post/11626074256/reconstructing-position-from-depth-buffer

Although you could think that, with the amount of game engines already available, most people would avoid the effort of trying to implement a new game engine from ground 0.

But the truth is quite the opposite, as for all the tech people around the world, the challenge of developing a game engine from scratch is something so important as a climber to try to climb to the top of the Everest, even if 50 other guies beat it to the mark.

With this principle in mind, there is a new game engine, developed in XNA, in the block, and that from the overall descriptions and media  materials, seems to be quite interesting (although I have not tested it yet.)

The engine is called Ploobs Game Engine an it is a full Game Engine developed in C# .Net 3.0 and XNA 3.1 using Deferred Rendering. The principal Features are: 

Forward Rendering e Deferred Rendering

Artificial Inteligence (PathFinding, NN, GA, Agents and World Abstractions, Steering Behaviors ...)

3D and 2D Sound

Physics Integrated (Bepu API and JigLibX API)

Animation Integrated  (XnaAnimation API, heavily altered to fit on the Deferred Render)

Dynamic Water and Ocean with waves

Dynamic Reflection and Refraction

Dynamic Lights (Spot, Point and Directional)

BumpMaps, SpecularMaps,Glow Maps and Paralax Mapping

Vegetation (Real Modeled Trees and Procedurally Generated Billboards)

Terrain with HeightMaps and Multitexture

Dynamic Shadow (Shadow Mapping and Cascade Shadow Mapping) with filtering

Deferred with Antialiasing (PostEffect)

Post Effects: ToonShading, Blur, Noise, Wiggle, Circular Glow, DOF, Bloom, HDR, Radial Blur, Negative, Black and White, SSAO, Color Correction, Gama Correction, Saturation, Contrast, Fog, Ambient Scattering and much more ...

Extensible Particle System (soft particles also)

Transparency (with Deferred Shading, extra Forward pass)

EnvironmentMapping

Skybox and Dynamic SkyDome

Billboards (Gpu Spherical and Cylindrical)

Animated Billboards and Textures

Video Player Embedded

Hardware Instancing (static and dynamic)

Trigger System

Message System

Picking System

Input Control System

Resource Management System

Screen Manager System

Culling System

Profilling System (Real Time Graphs, Performance Counters, CSV Result Exporter ...)

Works Well with Nvidia PerfHud and Microsoft Pix

Graphics, Physics and Render Targets easily Debuggable 

Cameras (First Person, Quake like, Third Person, Static, Follow Path, Interpolators ...)

Integrated With .NET Windows Forms

Integrated With 3DS Max using an Exporter Plugin 

Scene Loader supporting Lights, Cameras, Models and Dummies

Procedural Textures and Models Facilities

Has Lots of Math and Physicw Helpers

Extensible Design and Easy to Use

A couple of months ago, I have posted here an article about a implementation of a deferred rendering function developed in XNA by Catalin Zima .

Now some some has taken on the code, and ported it to the XNA 4.0 Framework, and released again the code in open source, which represents a very good opportunity to understand how deferred rendering works, and all the post-processes functions that can be implemented through its usage.

The new code has been developed by Roy Triesscheijn , a computer science student, which has released it in the open source model.

  Based on words from Roy: "The code is a bit shorter now, (especially setting effects is so much easier in XNA4). And thus should be slightly easier to understand. The code includes everything that Catalin’s original source code examples included. The camera can be moved using the keyboard (arrows, or w-a-s-d to move, z and x to zoom) or using the Xbox controller.

As a little benchmarking tool, I’ve already turned of VSync and FixedTimeStep for you. On my modest machine (AMD Athlon GP9400 Quad Core, 4GB ram, and an AMD HD4850) I could easily render 600 lights at 60fps+. (However this looks ugly). The default settings (103 lights) runs at 380~400 fps."

  The source code can be downloaded here: http://www.roy-t.nl/files/DeferredLightingXNA4.rar