Dhruv Govil

So today, I'd like to talk about one of the tools I made while working on Cloudy with a Chance of Meatballs 2. It's a tool that has seen a surprisingly quick and widespread adoption at work, both on this show and other shows.

The tool is essentially a deep compositing front end for animators.

Deep compositing has been a hot topic in the industry lately. While not a new concept, it's been seeing a lot of adoption at major studios from CG films like Animal Logic's Legend of the Guardians to Academy Award winner Life of Pi.

While it's been mostly limited to compositors, I'd like to write about how we're using it for animators.

(I can't share any images from work, but I'm using what images and videos I can find online to illustrate this. Credit due to whoever owns these)

What is deep compositing?

I guess I should start of by explaining what deep compositing is.

It's essentially a system where each pixel stores not just its final data from the camera's point of view, but also information for slices along the depth. 

So for example, one pixel could contain the information for every side of a glass prism, allowing you to place an object in between.

Disclaimer, we're not really using deep pixels, but rather just using the deep compositing nodes in Nuke to merge sequences of rgb and depth together to make a deep image.

A good talk on this is:

Why use it for animation?

So for animation, this may seem like a not so interesting technology. After all, we're working in a 3d space anyway.

Well the real fun parts of this comes out when you need to split up a shot. You can take any number of splits in the shot, with their depth, and quickly get back a comped together sequence with every split respecting their depth from the camera.

You don't need to worry about sorting the splits, because the depth takes care of that for you. Things in one split can go behind AND in front of objects in another split with no issues.

Essentially, it sorts the pixels in each split according to its depth value and you something like the above if you visualize it.

The beauty of this is, the animators don't even need to know what's going on. All we do is the following:

Enable depth output for all their playblasts (disk space is cheap)

Ask them to run our GUI tool when they want to see the results

The GUI

So obviously, we can't expect animation to comp things themselves. So I built a little UI for them.

It looks for any playblasts in a shot with depth outputs. It sorts it by names and then presents the animator with a selection of all their sorted playblasts to choose from.

They can set a few parameters if they want, and there are different intuitive filters. The tool then constructs a python file that we run Nuke with.

The nuke file builds the nuke script locally and then sends it to our queue to render. It comes back in a few minutes and they can see a massive number of files combined together instantly.

So what does this give us?

This gives us a lot of new possibilities! Shots can be way larger without having to think about it. We can push crowd shots and split up the work between several animators, and they can see their updates quickly.

It's allowing for a lot of massive shots to go through with very few headaches.

Even with a single animator, it allows them to focus on different parts of the scene, and offload heavy rigs to another split.

The tool was initially written for one show:

but since then has been adopted by most shows as an easy way to work.

I've had a lot of animators and leads/sups tell me that they love using this tool because it has removed so many headaches.

But this isn't interactive...

So using this tool at work means animators have to continuously playblast their changes to see it with other peoples work.

Now here's where it gets really exciting!

Maya allows for image planes to use depth maps. This looks like a regular image plane in every view, but with one cool thing:

If you have a character in the image plane, I can move my 3d characters in scene around these characters in the image plane. It's like the objects in the image plane exist in your scene, without any of the overhead.

You no longer need to worry about alphas, or using multiplanes to simulate things. And the beautiful part? We don't need to enable anything else to get it to work.

I could just reuse the bits from my deep compositing tool and change it to setup the image planes.

What could this mean?

Well I'm releasing the tool at work tomorrow, but it means:

animators can work more interactively with each other without overhead.

We can have massive environments loaded in without slowing down the shot

You can load effects in the scene without needing to worry about generating geometry etc. Just render it out with depth.

Autodesk are awesome for enabling that, and I can't believe I haven't known about it till now.

  What are the downsides?

So now, if you're like me, you're thinking: "Okay this sounds awesome but it can't be all peaches and cream"

Well that's true. There are two significant downsides, depending on your needs:

1) It eats disk space. You're writing double the data for each playblast and each comp takes up a similar amount of space.

2)It can't handle transparency. Or rather, maya can't give you depth information for transparent objects. It'll give you the depth of the nearest opaque object behind it, or infinity if it doesn't see anything.

  Neither of these have been big issues in practice. It's just something to be mindful of.

    Anyway, to wrap things up, I'm just psyched that something I've written is helping out so much. 

Little python snippet that will create maya key pairs.

I recently enrolled in iAnimate  and find myself doing a lot of repetetive stuff as I animate. So I decided to use my TD skills for the better, and to share any snippets etc that I make.

First little snippet is one I use to key all my selected objects on every 4 frames and delete everything in between.