#i2350

We had an in-class activity where the group which came up with the best algorithm for water would receive bonus points. I did not participate in the activity myself, though I did listen in on the discussion of my group, and took notes as people battled for board space to prove their algorithm was the best. The results were as follows:

The first group stated the most obvious (and simple) route to creating water.

You start with a plane.

Seems simple enough.

Then, apply everything you would any object within your game/simulation/whatever; a displacement map to change the surface of the plane and a normal map to make the lighting (which is then applied) look good. To simulate the reflection on the water, they pitched the idea to take the view from the camera and project it to the texture of the plane.

The reflection would require the use of post-processing effects, but everything else is as simple as anything that would be applied to almost every other object in the game. Is the water opaque, so as not to be able to see what’s below the surface? What kind of displacement map is used, and how does it move? How big is the plane, and would this algorithm work for massive bodies of water?

The next idea was added to the board.

Instead of using a normal map they pitched using a cloud map for distortion, with cloud map noise, as well as using a reflection map for the lights on the water.

For the displacement map, instead of applying a single 2D map, they decided to use two passes or perpendicular maps. Essentially they’d be displacing the water twice, in two different directions, to get a better ripple effect. Another idea they expanded on was the reflection on the water. They took the view from the camera, same as before, but before they set it as the texture of the plane they flipped it upside-down, clipped it to the plane and applied a distortion.

Then to highlight the crest and trough of the waves they used selective colouring, and used the vertex shader to find where it hit the land and made a particle emitter to emit particles along the new-found “shoreline” to make it look good. Of course, don’t forget to add transparency to the plane to make it somewhat see-through, and apply god rays beneath the water. (I won’t go into the detail of god rays here…)

The next group to take a stab at creating an algorithm decided to use quad trees for displacement. This is more memory efficient and allows for more detail to be added close up. They also pitched using physics in collision; each object has a different weight and will hit the plane at different speeds. The faster and heavier the object, the larger the “splash”, or the greater the sin waves produced in the displacement map.

And then of course, climate controls pop up. A splendid idea, if you ask me, as long as you know how to use it.

Someone came up with a very particular idea for moonlight; about 4 pixels above the surface of the plane, he made another (transparent) plane whose sole purpose would be to reflect the moonlit shimmer off the water. It’s not a bad idea, but did not earn any points in the grand scheme of Dr. Hogue’s class.

Finally, the last presented algorithm mentioned the fluid simulation. Essentially, for shallow water, you would use the fluid sim to modify the texture of the plane using shaders. He also added a pressure wave and which simultaneously contained the height of the plane’s surface to aid in collision detection. He then proposed to use a cube map for the reflection off the water’s surface.

The only one to mention other forms of water was waterfalls. Using particles that flow from one source of water, and fall until they collide with another source of water (then emit a different particle to represent spray) could make a nice waterfall. In addition, anything to hit the surface of the water creates a ripple displacement effect which continues to build as it gets applied.

Underneath the water is supposed to be distorted, so apply another displacement to all the geometry seen beneath the plane. This effect would need to be a post-processed effect, and function similar to shadow mapping or god rays. The result is distorted geometry beneath a slightly transparent surface of water.

Of course, there’s still a lot missing from all of these, though no one deigned to bring it up during class. For instance, no one mentioned the fact that those objects which interact with the water GET WET!!! Simple water interactivity was completely missed! Though when you only have 10 minutes to come up with an algorithm, I suppose it’s alright to miss a few things.

Here’s an in-depth tutorial on how to implement bloom (with pictures!)

What is bloom?

Bloom makes lights seem more realistic. It makes it seem like the light bends around objects as well as brightening the lights further. It is probably the most-used shader effect in video games today.

How does bloom work?

There are a few steps in implementing bloom, which involves creating and saving to FBOs. You need to single out the bright areas of your image and blur them before adding them back into the original image. It’s as simple as it sounds, when it comes to implementing shader effects using FBOs.

I’ll start with the blur:

There are several methods you can use to blur your screen. The first (and simplest), is the box blur. It involves using a convolution kernel filter to sample from the nearby pixels and find the final colour. The box blur kernel is a 3x3 matrix of all 1’s. Essentially what that means is all the surrounding pixels are weighted equally to factor into the final colour of the fragment (or center pixel).

Each pixel’s colour value is multiplied by its overlaying offset from the kernel, and the sum is then averaged out to determine the final colour value. In the case of this 3x3 matrix, the total sample would be divided by 9.

Here’s what a box blur convolution kernel matrix looks like:

The shader code for a convolution kernel looks something like this:

The box blur has its limits, however. It can appear “boxy” (hence the name), and doesn’t always give the desired effect. A slightly more complex filter you can use is the Gaussian blur. It is meant to blur in a more radial manner, making it more natural, and nicer to look upon. The Gaussian kernel is larger than the box blur, which means it will take more samples from the original image and make the blur more profound. The only thing about this method is that it is extremely inefficient. Each offset of the kernel must be individually coded in, which makes 25 statements of changing the offset and setting the kernel value.

Here’s what the Gaussian blur 2D convolution kernel matrix looks like:

A comparison between the box blur and the Gaussian blur (box blur is the top row and Gaussian is the bottom row):

The Gaussian blur does produce a nicer effect than the box blur (though the difference is subtle). However, the 2D pass is extremely inefficient. The kernel needed to be large in order to achieve the desired effect, and this takes up time. A way to make this method more efficient is by making it in two passes instead of one.

The Gaussian 1D convolution kernel matrix:

Nothing about how it’s done is changed; it’s just done in 2 separate passes for the horizontal and vertical blurs. I started with the horizontal blur, using the matrix above for the kernel, and saved the image to an FBO. Then I ran a second pass on that FBO, to blur the (already blurred) image vertically.

See here what each the horizontal and vertical passes look like. If you apply any one of these blurs to the other, you’ll achieve the exact same effect as if you had done the passes together (if anything better, as the 1D matrix has a length of 7 instead of 5). The horizontal blur is the top left and the vertical blur is the one in the bottom right:

When creating the blur, one small step you could also take to make the blur even more profound is to write the result to an FBO with a smaller resolution. This will, in fact do half of the work for you.

You can really see the difference in this image here; the first row is rendered to an FBO a ¼ of the size of the window. The second row is rendered to an 1/8th of the size, and the last row is rendered to a 1/16th:

The next thing I’ll go over (and what makes bloom affect just the bright areas) is the bright pass. This is a step performed before the blur to target the bright areas of the scene, and there are many ways to do it.

One way to do it is by using a constant value to determine the middle luminance value between what you want to brighten and darken. This value can be anywhere between 0 and 1, and defines what the colour of the fragment will be. Anything with a luminance value less than the middling value will get darker and anything greater will get brighter. Using this middle value and the luminance, the “brightness” of the fragment can be found: brightness = luminance ^ (middle / luminance).

When the middling value is set to 0.5, the graph representing the brightness value would look like this:

After you have the brightness, simply multiply the colour by the brightness value you just got and there’s the final colour for the fragment. Here’s the code for the luminance and bright pass functions:

  Another way to make the bright pass, and the way that I chose to use, is by simply subtracting 1 from the colour and clamping the value between 0 and 1. When you subtract 1 from the colour value, anything that goes below 0 gets clamped to 0, and becomes black. This only works when used in addition to HDR, where colour values are allowed to go above 1. Then, the only places where colour will remain above zero are those that are above 1 to start.

The value you subtract doesn’t have to be 1, either. It can be anything you choose; the lower the value, the more colour values you allow to show up after your bright pass. Whichever value you choose will be the lowest colour value you allow into your bright pass. For example, I subtracted 0.8 from my colour value to allow more colour through, then multiplied it by its luminance to magnify its brightness further.

There are still many other ways to separate the brighter and darker areas from the scene; this was only 2 of them. No one way is right, and they’re all able to be modified to your purpose, as I did with the method I chose. But let’s go on.

The last thing I’ll cover for creating bloom is putting it all together:

Order is rather important for any effect you decide to implement. The first thing you need to do is render the entire scene to an off-screen FBO. You can apply whatever you like to this first step; I applied Phong lighting using 3 different coloured lights. Having a few bright areas on the screen to work with makes for a nice bloom effect, and Phong lighting provides a nice specular highlight perfect for bloom. The next step is to apply the bright pass mentioned above to the rendered scene and save the result to another FBO. Then you apply a blur to that FBO and add it back to the original image.

After creating the proper bloom effects and saving the scenes to off-screen FBOs, you should have 2 textures (FBOs) that hold the original scene and the processed scene. You will need a way to add these together to create the final effect. This is called a composite shader. Additive blending will simply take the two textures, locate the correct placement using UVs then add the colour values together. Black has a colour value of 0, so it won’t do anything where white has a colour value of 1, which will brighten the scene.

Here’s what the post-processed scene should look like; after the bright pass and again after the blur:

  All in all, implementing bloom properly requires the use of 3 FBOs (or 4 if you’re doing two 1D Gaussian passes) and 4 shader programs:

FBOs->scenePass, brightPass, blurPass, blurPass2

Shaders->Phong, Bright Pass, Gaussian Blur, Composite

Render the scene to the scenePass FBO using the Phong shader program. This is also the pass in which you render the geometry to the screen.

Write the scene (from scenePass) to the brightPass FBO at a resolution a ¼ of the size of the window (or another size) using the Bright Pass shader program. Do not render to the screen, but to a quad scaled to the same size as the brightPass FBO.

Write the scene (from brightPass) to blurPass using the Horizontal component of the Gaussian Blur shader program.

Write the scene (from blurPass) to blurPass2 using the Vertical component of the Gaussian Blur shader program. Both blurPass and blurPass2 should be sized the same way as brightPass.

Run the Composite shader program which renders to a full-screen quad.

Here’s an image of before(top) and after(bottom) bloom is applied:

  You can really tell that there’s bloom in the bottom row in the image, not only because the spheres are brighter than they are in the top row, but because of the white glow overlapping the surface. You can really see it on the left-most sphere; it has a white glow along the top and an orange glow along the bottom.

Alright here's an other quick one, this is one of the game I feel is a work of art as a whole. I am talking about Trine 2, there is no way I can describe this game other than art.

The shaders in this game are amazing, the story is great, gameplay is fantastic, and it's a really cheap game. If you don't own this game buy it! 

It has... too much good stuff in any screenshot for me to explain completely what is going on. Though I can say that they don't ever go easy on the bloom shader. Yet you still are able to see clear edges and objects you can interact with still stand out as any respectable 2.5D platformer should do.

There's dynamic light source linked to objects in the scene such as chandelier inside and weapons, like the fire arrow from the thief, with dynamic shadows.

There's fog to render things in the distance, particles with more bloom! Distortion effect around stuff like fire. Beautiful light rendition, from specular  diffuse, and emisive.

This is going to be a quick one, toon shading also known as cell shading. It's a really simple process all it requires is knowing light.

If you'r able to calculate light level, you can easily stagger it so instead of being smooth there's different light stages. Something like:

if(intensity > 0.95) intensity = 1.0;

else if(intensity > 0.85) intensity = 0.85;

else if(intensity > 0.75) intensity = 0.75;

else if(intensity > 0.65) intensity = 0.65;

else if(intensity > 0.5) intensity = 0.5;

else if(intensity > 0.45) intensity = 0.45;

else if(intensity > 0.35) intensity = 0.35;

else if(intensity > 0.25) intensity = 0.25;

else if(intensity > 0.15) intensity = 0.15;

else intensity = 0.1;

It gives the object a nice look.

And that's it for toon shading, only need 3 lines of code.

Well, the semester is coming to a close, and I thought I'd wrap up my blogs with an overview of our Game Development Workshop game - Vase. Of course, I have to relate this to our shaders course, so of course, I will be talking about the graphics techniques we used.

Vase is an isometric action game. It features a female heroine who has been cast in to a world where all the vases and pots have come to life to wreak havok upon the world. It is her job to smash these pots with her spells, and restore peace and order to the land.

We wanted the graphics in Vase to accompany the comedic nature of our game, and still be visually appealing. In Vase, we use per fragment lighting, cel shading + sobel edge detection to create toon shading, motion blur using the accumulation buffer, and baked ambient occlusion textures.

So here is what Vase looks like without any shaders. Keep in mind that some of the textures were incorrect at this point.

After we apply all of our graphics algorithms to the game, here is what it looks like.

Without the motion blur.

And now with the motion blur. Motion blur is triggered when the player collects a time stone powerup (the purple object in the picture).

As you can see, we wanted to make our game as colourful and vibrant as possible. We also wanted it to look like a hand drawn cartoon.

The first shading techniques we applied to the game were per fragment lighting and cel shading. We used the phong model for our lighting calculations. The diffuse, specular, and ambient values are calculated in the fragment shader, then added together to create the lighting intensity. From here, we take the intensity and limit it to certain values to create the cel shading. For example... if (intensity > 0.75){ intensity = 0.75; } We use 10 levels of intensity in our shading. On top of this, we threw in our sobel edge detection shader to get those black outlines.

Creating motion blur using the accumulation buffer was relatively easy. We have a boolean variable called accumBlur, which gets set to true if the player collects a powerup. After a few seconds, accumBlur is set to false and the motion blur stops. Here is what our display looks like. if(accumBlur){     static int i = 0;     const int n = 2;     if(i==0){glAccum(GL_LOAD, 1.0/n);}         else{glAccum(GL_ACCUM, 1.0/n);}               i++;     if (i>=n){          i=0;          glAccum(GL_RETURN, 1.0);          App->Display();       }   }  else {      App->Display(); } The 'n' variable represents how many frames are to be blurred. Since we didn't want the motion blur to make the game seemingly unplayable, we only blur two frames.

The ambient occlusion textures are baked on to all of our models. Here is an example of the gate model texture with no AO.

And here is an example of the gate, when AO has been baked on.

The ambient occlusion textures are meant to represent how light reflects internally on an object.

The year is winding down and tommorow is the last day to hand in homework questions to get XP. Everyone who doesnt have 40 xp is scrambling trying to get that amount so they can write the final exam. Luckly for me I am not one of those people how are scrambling. I currently have 55 XP and I'm happy with that. I'm going to try my best to get 20 more XP to get full marks for homework but we will see what happens because our game is also due in a couple of days. I would be extremely happy if i could hit that but if not that okay. 

So you may ask how I got to 55 XP well... i will explain.

I first thing that was done was toon shading using sobel edge detection and its upgrade to support hatching which i think is really cool looking.

So above is the toon shading with sobel edge applied to an Ambient occlusion texture which gave me 10 XP.

BELOW IS SOME CODE ON HOW TO DO TOON SHADING

#version 120

// 2 varyings

varying vec2 texcoord;

varying vec3 normal;

// 2 uniforms

uniform sampler2D inputImage;

uniform sampler2D qmap;

// 1 constant

const vec3 lightEyeDir = vec3(0,0,1);

void main()

{

vec3 n =  normalize(normal);

//this is our q-map input

float diffuse = max(0.0,dot(n, lightEyeDir));

//alternative lighting coefficient

float blocky = texture2D(qmap, vec2(diffuse, 0.5)).r;

vec3 c = texture2D(inputImage, texcoord).rgb;

//gl_FragColor.rgb = c * diffuse * blocky;

gl_FragData[0].rgb = c * blocky;

gl_FragData[1].rgb = n*0.5+0.5;

}

The next thing i did was object and tangent space normal mapping. This was one of the most interesting homework question for me. I am more of an artist than a programmer and to get a better understanding of how normals work in video game was awesome. Object space is cool with normals maps the are very colorful.

For those who do not know how object space normal mapping is done you must:

sample from a texture (an object space normal map)

uncompress the normal's(this because normals are from 0-1 when we need them to be from -1 to 1

calculate displacement of light from object

normalize that displacement calculation

calculate diffuse

output color + diffuse

Object space is a good technique for static objects but when you have animating objects it is not good. This is where tangent space normal maps come in. Tangent space is similar to object space but the light must be brought into tangent space and uses a TBN matrix(tangent, bitangent and normals). To bring the light into tangent space requires math which is very important and i will explain how to do it. So your normal is already given to you by your map so that is covered but now we need to find your tangent and bitangent. This may seems complicated but its really not too difficult. To find them you get triangle from an object. Each triangle has a U and a V and we can use them to find e1 and e2 which are you tangent and bitangent. 

So you take your U1 - U0 + your V1 - V0 and your U2 - U0 + your V2-V0 and you can get your e1 and e2 which is what you are looking for.

Once you have the information you need the you apply a transpose and your light is in tangent space. This allows you to use maps like this:

and you can get your model to look all bumpy.

So that was another 10 xp and alot with the art question it put me to 55 XP. 

Alright so last I've 100%, this is about saints row 3. Well one level in particular as it changes the game aesthetics completely. I'm talking about http://decker.die where you get send off into a virtual reality and shenanigans happen.

First I'll talk a bit about this picture. As you can see you are now a blow up doll... well there's been worse but yeah frequent model changes throughout the level happens without having a loading screen which leads me to believe that the toilet, your character model, and the sex doll all have the same amount of vertices and are being rearranged into different form with some kind of blending.

Next there the lines around everything which makes me think of crosshatching of some kind but instead of being applied to shadows like toon shading, it's being applied to the edges of models with a bit of RGB distortion too.

The bullet trail behind the projectiles is neat, it changes the location of displayed pixels on the Y axis to be higher. Also lightsources galore... look at all of them in there glory most of this is baked into the textures if I remember this game properly but still sploosh. 

Here we have the level exposition which is a great looking shot but not much more to talk about in this one, just thought it looked impressive.

And the boss where there's as much Gaussian blur as... probably shouldn't finish this joke slightly inappropriate  Also your avatar now has a new texture which the light on it is animated, and texture is the mesh of the model though it might not be accurate, haven't seen the model itself in maya or equivalent.

*WARNING* no images in this post, sorry *WARNING*

Alright, it's six in the morning and I can't sleep because I was thinking of shaders and computer graphics so I decided to get those blogs done. I don't know what's up with me but the hardware part of computer graphics seems to catch my eye all the time I've done a vlog on NVIDIA roadmap; and now I want to talk about another NVIDIA GPU feature SLI, ATI got their own version of this called crossfire but I'm unclearabout how it works and as an NVIDIA person after having been burned on several occasions by ATI I'm not too interested. 

After this runon sentance let's get to it. 

What is SLI?

SLI is a method NVIDIA uses to utilize more than one GPU to render what is being displayed.

How does it work?

There are multiple ways SLI does that process, one is by utilizing a different GPU to render to next frame, so if both render 60 frames per second put together they'll get up to 120 frames per seconds.

What else can it do?

Well having more processing power let's the user experience more immersive computer graphics. Such as realtime ambient occlusion (SSAO) in games such as ours SSAO might be able to be used both for games which require so much more information to be calculated it takes a really big toll on the GPU.

Anisotropic Filtering, it's a way the GPU can calculate the distance between the camera and any texture being applied to objects and mipmap it accordingly. The farter the texture the blurryer it get while staying crisp at close range to the camera.

Antialiasing, blurring out lines to make it seem like they aren't being displayed on a pixelated screen is a major challenge for programmers. There are many ways to do so but there isn't a perfect one so the formulas keep getting refined. NVIDIA has it's own formula called FXAA which has many neat feature and isn't to taxing to the GPU, such as adding transparency by multisampling edge of displayed objects and adjusting alphas accordingly.

PhysX, see previous blog about planetside where I ramble about this feature!