RW CREATIONS

I just finished an iteration of my greenscreen workflow. I’m documenting the steps so I can see how to streamline the work.

1. Actors shot on greenscreen with side by side GoPro Hero4 rig.

2. Pre-Comp1 Footage was placed in an 8000x8000 After Effects composition. Footage was rotated 90 degrees, denoise with Reduce Noise v4, and manually placed in an over-under configuration. (this was probably a mistake as I don’t have any tools to deal with over-under)

3. Pre-Comp2 Pre-comp1 was run through Keylight process producing an over-under cutout of the actors.

4. Background plate was captured from Unreal Engine using Kite and Lightning plugin.

5. Pre-Comp2 and background plate were placed in After Effects composition. Here’s where the problem with over-under came in. I need to adjust the position of the actor plate in 360 space, but also 3D space. I have good tool (Dashwood Stereo) to adjust stereo, and decent tool to adjust location in sphere (Mettle SkyBox) but not to do both. Skybox has only one stereo tool and it is to take a 2D image, create a stereo copy and move the two images along the 360 sphere. I need a tool that knows stereo over-under and can move the 2 images in the 3D sphere while keeping the stereo.

6. Color using Lumetri.

7. Output as MP4, h264 for upload to Youtube.

8. Metadata tag using Youtube app.

I also took a frame of the 360 and created a tight crop and created an anaglyph using Stereo Photo Maker. I think this is important to have a “gateway” image to draw people to the VR on Youtube. I’ll write about this more, the VR ecosystems seem so closed to me right now.

Improvement Areas

Working in stereo is tough. After Effects is really not setup to do this, and after spending several months with Mettle Skybox it doesn’t give me the tools I need. Dashwood Stereo tools are good, but Mac only and I’m moving off that boat! I’ve ordered BlackMagic Fusion Studio in the hopes that it will address some of these issue.

This  article ( https://www.fxguide.com/featured/art-of-stereo-conversion-2d-to-3d-2012/ ) points out just some of the things to think about with 3D. I think they should be mostly applicable to 360 stereo. 

I’m currently working towards a complete stereographic 360 workflow. I want to capture as much as possible in the field. However, maybe I should take a clue from cinema, seems most “3D” movies are probably shot in mono and converted after.

I’m working on another rig based on last weekend’s shooting. So I was searching for inspiration and ran across this page of ridiculous rigs! 

Keylight is magic, but trying to drop out green screens in one pass is not the correct way to do things.

But, I keep forgetting the layer order and losing this note, so here: http://www.stevenolver.com/triple-pass-keying-in-after-effects/ is a correct runthrough.

Now there are some buzzwords any VC company is going to be very interested in, but it looks like Avegant has a headstart in actually getting this technology to consumers. I attended a talk by Ed Tang at SIGGRAPH where he discussed Avegant’s history, and then talked about their latest product. 

https://youtu.be/LrNwP_eTC9c

The basic idea is to have multiple screen layers at different focus distances all simultaneously viewable. This let’s your eye do what it does naturally which is constantly change focal point. Objects projected on the screen are at the correct point in space so when focusing on close objects, distant objects are naturally out of focus.

I attended a great talk by the folks at Unity regarding photogrammetry. Photogrammetry plays an important part in realistic 3D environments. That combined with PBR texture workflow is the next step for the game engines. This is old hat for VFX offline renderers, but new Nvidia GPUs make this level of detail possible in realtime renderers.

Information about the process and even an ebook is available. To summarize their main tool was Reality Capture, but they also used some tools that were new to me. Knald was used for textures, and Artomatix for tiling.

At SIGGRAPH 2017 I attended a talk by John Knoll, VFX supervisor at ILM, where he outlined his work on Rogue One. One bit of technology that I found very interesting was his team’s use of a handheld virtual camera. This was an iPad like device that allowed Gareth Edwards, the director, to fly through a scene to capture unique camera angles of a CG scene. The example used was the space battle scene at the planet portal where the camera flies alongside a group of X-wings in a very natural path as they dive towards the portal. In the past this would have gone through extensive storyboarding, and a group of artists would have keyframed the camera, reviewing the sequence for several iterations before final approval. With this technology Gareth was basically able to iterate instantly over a couple camera flight paths, choose his favorite, and pass actual motion track information to the production crew.

Here’s a camera prototype for 360 still photography using two GoPros. It is powered by an Arduino that remotely triggers the cameras using a hacked GoPro wireless remote. This proto has a stepper also to rotate the camera through 360 degrees. The resulting images can be assemble using PTGui and a “twisted pano” workflow to get stereo 360. 

I just got back from SIGGRAPH 2017 and my mind is full of exciting new technologies and techniques. I’m a big ILM fanboy so their talks were quickly tagged in my scheduling app. John Knoll’s talk for Autodesk gave a quick rundown of the effects used for Rogue One. This article touches on some of the similar topics John discussed live.

Virtual Set

https://blenderartists.org/forum/showthread.php?343637

Looking at my renders a couple things bother me. One, when looking out the window the eye adjusts for the glare and the window sill is basically a shadow. When taking a picture the glare makes it so the sill is often not visible, but certainly not well defined. I need something that acts more like a camera. Enter Real Camera

I’m working on some architectural type renders for my VR project. I have been constantly frustrated by Blender’s lack of CAD type tools. Ever since taking a 2D CAD class in college, typing in exact measurements for objects is how I naturally work. I had resorted to using Sketchup, then importing Collada files in to Blender, but having to deal with the import and cleanup process is far from ideal.

Enter Archimesh! It’s an addon that is perfect for this. Can’t believe I’ve been working the last couple weeks on a workaround when the ideal solution was right there. I can type in the wall sizes, ceiling height add windows and doors. 

I continue to be impressed by Blender. So far any problem I have come up with there’s someone that has run in to the same issue and has a working solution. This is a big change from a couple years ago when most things I tried were very experimental in nature.

I love vintage analog synths! I’ve built a couple modules from the late great Ray Wilson and have dabbled with others. A classic happened across my lab bench recently, a classic from the great Tom Oberheim, a TVS-1. It is in good physical shape, but not functioning at all. Thanks to the schematics at Ersatz Planet I’m pretty sure I can get this baby working, and maybe expand with some homebuilt modules!

Problem List:

1. No -15V power

2. No Sound

3. Lower E on keyboard not triggering

After working on my multiple GoPro setups I imagined “what would this look like if it were easy?” That led me to purchase an MFT Stereo lens from Loreo. Having a stereo camera removes the need to worry about shutter sync, syncing individual clips, color/exposure difference between cameras etc.

First off, I was a little surprised by the narrow field of view. It is a 40mm lens, but man I’ve been shooting so many wide angles that seems LOONG! A little research and I ordered two of these wide angle conversion lenses  Still pretty wide for what I’m shooting, but I’m not sure I have a choice except to find a fisheye converter.

Unfortunately this lens is very slow. There are three choices F11,F16 and F22. Pretty slow when shooting under studio lights. The other adjustment is distance. At first I thought this was focus distance, but then I noticed the two images moved as I moved the slider. The Loreo uses focus coupled Parallax Compensation system, basically slide this slider so the two images are framed the same and image pitch as well as focus are adjusted, pretty neat.   

Below is a test, image was taken with a Lumix G7, single image was split in half in Photoshop, each piece was imported to Premiere where I applied my Dashwood 3D workflow. I made an anaglyph for quick viewing as well.

Just saw that the Facebook 360 camera specs are on Github. The cameras from PtGrey, however, don’t appear to be in stock. Who wants to make a $30K camera!!

https://github.com/facebook/Surround360

https://www.ptgrey.com/surround-360-camera-bundle

VRography?

I recently got to thinking what’s next in photography?