Chris R. Hackett

Working with Swanson Studio I created some 360 shoe rotations for the launch of the Streak, Structure, Elite, and Pegasus.

http://www.nike.com/us/en_us/c/innovation/zoom.html

It looks like the guys at Cineform are really into this idea of the 5D shooting raw.  They've released an unofficial executable that can process the ML5D *.raw into CineformRAW giving it 10:1 compression.

Unfortunately it's a Windows only solution.  But after the conversion, the file can be played in both Windows and Mac.

An interesting note on this, Cineform appears to be shifting their licensing policy from the RAW codec only being available in the $299 version to allowing RAW processing using the free version of Cineform..  The catch - you only get 4:2:2.  Not a big deal to me.

Now that USB 3.0 is found on (almost) every new Mac, the Macworld Lab decided to put its real-world speed to the test, especially compared with FireWire 800 and Thunderbolt.

Judging by the talk on set, you would think there was only once successor to Firewire and eSATA – Thunderbolt.  With a 20Gbps througput it's hard to not be impressed by the daisy chaining connection bus, but there seems to be a lot of confusion as to where it's strengths are, and where its weaknesses lie.

First off, let's break down the numbers.  Thunderbolt has a current bandwidth cap of 20Gbps in the newest Macbook Pros, and the rumor of an optical cable in the future to bump that number up to 50Gbps.  If that will be backwards compatible remains to be seen, but I wouldn't get my hopes up.  USB 3.0 on the other hand clocks in at a paltry 5Gbps in comparison, 1/4 the speed on a modern Macbook.  In terms of bandwidth, Thunderbolt is a clear winner.

But let's look at the drawbacks of Thunderbolt.  For starters, there's the daisy chaining issue (or lack thereof) and the omission of any type of hub.  I own two thunderbolt devices, a Sonnetech Tempo 3/4 Adapter and a Blackmagic Design Ultrastudio Express.  Neither of these devices daisy chain, rendering my TB ports useless past those devices.  Sure, I could add them at the end of a daisy chain, but I could never use them on the same chain.  On top of that there are some weird issues with the Tempo, where it can be the only drive bus in a chain, meaning I can't use Belkin's esata adapter in the same chain as the Tempo.  Major disadvantage.

Another annoying issue with Thunderbolt is the lack of a hub.  You cannot take a single TB port and turn it into 2x TB ports.  With my previous Macbook Pro, which had only 1x TB port, this was infuriating.  I could only hook up one of the devices previous listed, and on top of that Apple is now using the TB port for external monitors (and network cards on the retina).  Both of these things are non daisy chain-able   As you move into the world of Thunderbolt, you quickly find your are piling up a collection of dead end devices.  There is a solution to the display issue, either buy a Macbook Pro Retina ($2200) which has an HDMI port, or get an Apple monitor ($1000).  A monitor that lacks basic VESA mounts and has no standard signal hook ups.  My, Thunderbolt is looking very expensive.

Now there's USB 3.0, which has the ability to hub.  With the device pictured above, I can take one USB 3.0 port and branch it into 10 ports, no need to worry about daisy chaining.  With my USB 3.0 RAID 5 box, I can get speeds up to 190MB/s, blowing Firewire 800 out of the water, and clocking in at twice the speed of eSATA.  Now we are running up on a limitation of USB 3.0 when connecting multiple high speed devices to one hub.  The speed 5Gb/s applies to the port itself meaning I am now splitting 5Gb/s over 10 connections.  Still, that's not too bad.  It translates into about 640MB/s.  If my RAID 5 is taking 190MB/s that leaves 450MB/s for any other drives.  Assuming I'm pluggin in single, non-raided drives, I could get 4x drives at 100MB/s and and split the remaining 50MB/s over miscellaneous peripherals (keyboard, mouse, control surface).  That's highly economical, and high performance (in comparison to FireWire 800, eSATA, and single drive Thunderbolt enclosures).

Another issue with Thunderbolt is the cost vs performance when it comes to external hard drives.  Regardless of connection type the hard drive in an enclosure is connected via SATA, unless you are using the far more expensive and higher performing SAS connection.  What this means, is you are getting standard SATA III speeds which is 6Gb/s.  Another issue is that if you're using a spinning 7200rpm drive (almost everyone is) your transfer speed comes nowhere near close to consuming that 6Gb/s bandwidth, and usually averages around 75MB/s (0.546875 Gb/s).  So when you buy a single drive 7200rpm hard drive, you are really paying and average of $50 more for the same hardware performance.  This issue does not apply to the really high end enclosures, such as Promise's Pegasus R6 which can attain speed up to 800MB/s.  But you do pay a pretty penny for such high speeds.

Less than three weeks ago the folks at Magic Lantern released a hack for the 5d MkIII to enable 14bit RAW capture.  To put that into perspective, Redcode RAW is 12bit, ProRes 4444 is 10bit and the standard 5d capture is 8bit 4:2:0.

I was able to get my hands on some RAW footage from the 5D, and the question came up as to what the best workflow should be to preserve as much color data as possible.  In a perfect world you would use Magic Lantern’s utility, raw2dng and convert the RAW file from the camera to a DNG sequence.  Unfortunately, in it’s current condition, the utility does not create a DNG sequence that is readable in Davinci Resolve, my color suite of choice.  The best explanation I could find for this is a post on Creative Cow stating that the thumbnail information is read as the DNG sequence, resulting in footage that is interpreted as incredibly small.

The main issue here is how to preserve the color work flow if Resolve cannot read the raw data.  It makes it impossible to create edit proxies that are then onlined for color.  So I need transcode this DNG sequence into a container that is readable by the editor and by Resolve.

A few of the candidates I tested are ProRes 4444, DNxHD 444, DPX Sequence, and BMC RGB 10bit.  Initially I loaded the DNG sequence into After Effects and started out by just doing a flat export with default settings (after changing the project settings to 32 bit).  I took the results and put them into Resolve without much luck.  Latitude was poor, and the highlights were not very recoverable.

My next attempt was to flatten the image using the Adobe RAW utility inside of After Effects and then export to the codecs I previously listed.  The results were fantastic.  The latitude was great, excellent highlight recover, and fantastic color.

To recap, my workflow ended up as follows:

Convert RAW data from the 5D to a DNG sequence.

Open the DNG Sequence in After Effects.

Change the project settings to 32bit.

Flatten the DNG sequence using AE’s RAW plugin.

Export to any 10 bit codec (ProRes 444, DNxHD 444, BMC RGB, DPX Sequence).

Edit

Send to Resolve

An interesting side not were the sizes of the converted files:

Goodsync is not a DIT tool, but it's better then most out there.  I've used it for a few weeks now, playing with it's ability to sync from sftp's and networked folders, and I felt pretty confident with it's reliability.  But I never really thought about using it as my main DIT tool, I always thought it would be supplementary.  Well, until today.

I started the day attempting to launch Elklight Velarium to copy the data coming off of a PIX 240 SSD.  Velarium is a super cool DIT centered software.  It uses a lot of custom scripting to autocreate folders, transcode, and copy to a second drive.  Unfortunately after setting it up it crashed every time I started a copy.  Emailed the developer, no response, making me feel great that I was only using and evaluation.  I think there's a lot of promise in Velarium, but not for the PIX 240 on my Macbook Pro.

I moved on to a copy of software I own, Shotput Pro.  My biggest gripe about Shotput Pro is their draconian license management system and lack of discount for purchasing a second license.  So I launch the copy of the software I paid real money for, and I'm blocked by an "activation" screen.  I would copy my activation code into the dialog box, but my computer that was previously activated (my tower) is powered of at my house.  So shotput's out of the question.

DoubleData can't read anything that's not Red or Alexxa.  I would go for Pomfort's Silverstack, but that costs $599.  Red Giant's DIT tool isn't out yet.  So that leaves rsync and Goodsync.

By using DoubleData's method of copying first to a RAID5 and then out to secondary and tertiary drives, I was able to provide a simple, secure, and verified backup.  The initial copy goes to the RAID5.  This ensures that a quick transfer moving at ~200MB/s is also protected with drive redundancy.  From there, I used Goodsync to create Backup jobs that propagated my changes from the central RAID5.  You could think of the RAID5 as the server, and the satellite drives as clients.  The source copy is never removed until it exists on the RAID5 plus a sattelite drive, providing parity and a copy.

This enabled me to make changes quickly and effectively in ways that would be impossible with the aforementioned tools.  At one point the Director came over and asked me to rename the cards to match the scene and take numbers.  With ease, I renamed the MOVs from the RAID5 and synchronized the name changes out to the satellite drives

And Goodsync even has the ability to perform MD5 checksums.

When it comes down to each of these suites available in the DIT world are fantastic in specific situations.  Shotput and Velarium for mainting complex naming and folder structures over long shoots, DoubleData for any Alexa and Red shoot, and Silverstack...which looks really cool if I had $599 to drop on copy software.

I picked up an Ultrastudio Express for a recent shoot.  I wanted to get a an SDI/HDMI input card for a while, but finding the right piece of equipment that is inter-operable between my tower and my laptop, and at the same time planning for the future so the device uses won't be outdated...well it can be difficult.

I have to say, minor annoyances aside, the Ultrastudio Express really is worth it's price tag (I picked it up for $475).  Not only does it take in SDI and HDMI and cross converts between the two, it also comes with some free VTR software which is fantastic for things like the Canon 5D MKIII's new clean HDMI out.

Some of the minor annoyances and work arounds I've run across:

The device is very particular to what hz is coming in.  The hz in the control panel needs to match the signal that the camera or computer is outputting.  There is no auto-detection.

The Expresss unit does not do an input of 60hz progressive via HDMI or SDI, which can make some complications when attaching more monitors further down the line as detailed below.

If you are taking an input from an SDI connection over 23.98hz and output via HDMI, your HDMI signal will default to 23.98hz which most monitors will not read.  There are two work arounds to this issue:

You can output SDI at 60hz at 720p then output 60hz 720p to HDMI, or upscale to 1080i.  Not the most desirable, but definitely a work around.  The problem with this is that any monitor down the line is only getting 720 signal, and if the camera op is using the SDI output cable they also get a 720 signal.

As far as I know this is a Windows only solution.  If you output 23.98 1080p from the Epic and then open up the Blackmagic system preferences, go to the "Processing" tab you'll see a box you can check that says "Output to HD 23.98p as HD 59.94i".  Checking that box will make the signal work with all HDTVs and most high end monitors but not all monitors.  As far as I know this is not on Mac.

Both of these issues can be easily resolved by getting the Ultrastudio 4k, but it's a bit to big for me an another $600.

Besides that, when you get the unit dialed in, it performance like a dream (as long as you have a good read/write on your drive).  I haven't used it for an external monitor for grading yet, but from everything I've read it works great.

One of the things I've noticed as an IT professional moving into the media production world is the large amount of resources to use trendy or stylish external drives such as the Lacie rugged drives or G-Tech drives.  While yes, these are fantastic drive options that are durable, easy to use, and fast, they are also expensive.  There is a cheaper alternative.

Instead of purchasing a drive with an enclosure, you can purchase the drive by itself, and you can even do it locally at computer specialty places such as ENU Inc.  Take a look at the difference. 

A Lacie 500gb rugged drive with Firewire 800 goes for $139.99 on Lacie's website.  In comparison, a drive without an enclosure at ENU (same size and speed) goes for $69.00.  That's a sizeable difference.  If you DIT has the correct hardware to read bare metal drives, you're saving a lot of money on drives.  If you buy two drives per production, you're saving $141.98 by switching to bare metal drives.

There is another component to think of when using these drive – can the editor plug the drive into his machine?  There is a simple solution for that, purchase a reusable hard drive enclosure.  Here is an example of a full enclosure for $44.99 and here is a more basic open enclosure for 24.99, or another full enclosure for $29.99.  You only need to purchase one of these, and still have two drives.  

If one of the two drives is used for archival purposes, having a dedicated enclosure seems to be a waste.  And if your editor has multiple drive readers, you only have to purchase the enclosure once.  When you buy a Lacie or G-Tech you are essentially purchasing the drive and the enclosure over and over again.

You need one more thing in this scenerio to have complete data security and come out cost effective, which is a way to store the drives when they're not in an enclosure.  Using a hard drive box is a great way to keep your data save, and it can easily be labeled and put on a shelf.  Not to mention they are relatively cheap, with the fanciest hard drive box coming in under $10.

Installing

It seems like a sea change for Linux is coming, from Steam to Lightworks (I'm even using an official Spotify client right now).  Seven years ago when I installed my first copy of Mandriva on a Pentium IV, this was unimaginable (although sorely hoped for).  And now I have in front of me a fresh install of Lightworks!  Perfect timing for creating a Linux only DIT station.

To start out with, Lightworks only comes in Debian 64bit package designed to work with Ubuntu 12.04 64bit.  Since I was running 12.10 32bit (I chose this due to 64bits often buggy handling of flash) I decided to do a full wipe and install the recommended operating system.

After a fresh install of 12.04 I had to install the restricted nVidia drivers through System Settings -> Additional Drivers.

You can see that I'm actually using an experimental driver, this was reccomended by Steam.  If I were to be building a Linux DIT station I would stick with the stable driver unless told to otherwise by Lightworks.

I downloaded the DEB from Lightworks and installed it with the Ubuntu Software Center, and the icon popped up in my dock bar.  Now when I first tried Lightworks on Windows I had a hell of a time getting it to just launch.  The first few versions just didn't launch at all.  I was not expecting how easy it was to install the NLE on Linux (which is known for being complicated), so when I clicked on the icon and it ran perfectly I was a bit blown away.

First Start

Upon first launch you'll see an error similar to this:

What it really wants you to do is to create a new project space for your Local or Remote projects.  To do this just click on the gears icon, click "Manage Project Spaces" and add a local space.

Once you do that, just select your space as active by clicking to the left of the gears on the initial project creation screen.

Now there's one thing here I already find annoying.  I can't pick a custom frames per second for my project, instead it's determined by system presets that seem to be hard coded into the system.  Oh well, it is a "professional" editing system, which means you should always be working in supported project settings.  No 12fps art film for you Mr. Student FIlm.

Interface

After creating my first project I am presented with a blank canvas.  As and FCP7, CS6, and Media Composer 5 user, I am horrified.  There are no presets, there's no window layout, just a blank blue screen.  This will definitely be the biggest turn off to seasoned editors on Apple and Adobe systems.  But I will not be deterred!  Onwards!

Here you can see how the workspace works.  Each function of the workspace gets its own window.  Some of this can be farely confusing.  For instance, importing footage starts by clicking the import button and then adding your footage, after you do this a bin will pop open.  If you're paying attention to the little shark in the left corner, that bin is a transient bin.  This means that once you close that window that organization of clips disappears.  They're still in your project, but there's not collection of them saved anywhere.  So the first thing I do is give that bin a name other than import, and change its status topermanent.

To create the timeline I had to click on the edit button which created a new timeline and window combo.  I then placed them haphazardly and clicked on the pin buttons.  This attaches it in space so they can't be moved.  Oddly enough, not all boxes have these pins, only certain special boxes.

It should be noted here that Timeline = Edit = Sequence.  Each company has it's own name for a timeline (Apple now calls it a Story with FCPX).  Here, Lightworks has chosen to call it an "Edit".

This seems extremely confusing to me already.  Windows floating everywhere?  What's going on here?  And what happens when I hit X on these windows, where do they go?  Does my edit disappear?  Where is the icon for my edit in the bin?  All of these questions are answered in the search windows.

Search plays a powerful role in Lightworks.  When create or import an item it does not automagically create a bin or icon to correspond with it as Premier would do.  Instead it puts it in its database, hidden from the user.  The way to access this is through specific search commands, and then either giving the resulting bins permanence or dragging these items to permanent bins.

Fortunately, in order to handle this maze of an interface, Lightworks has incorporated something called "Rooms".  It's very similar to Linux work spaces.  I think when I create a project in Lightworks I will have an edit room, a footage room, and a search room.  I imagine you could make a room for each edit you work, giving each edit it's own layout and specific workspace.

Conclusion

After playing with Lightworks for a bit, this is the best piece of video editing software I've ever used on Linux.  It could play back professional codecs quickly without delay, and handed most everything I threw at it (we had issues with h264, but it is in alpha).

I can already see this as an essential DIT tool if I were to use a Linux rig.  It can playback and verify files, it can even play REDCODE.  You can also pull codec information from imported videos, which was lacking in almost all of the NLE competitors on Ubuntu.

The one thing that irks me about Lightworks is their slogan, "Made by editors, not programmers."  Implying that if you're an editor, you're going to agree with the way they run their ship.  Well I can tell you right now I don't know a single editor in my life that edits like this.  No interfaces?  Heavy emphasis on search?  All the editors I know would be a bit confused if put in front of this software and told, "Hey, make a movie kid."  Actually, when playing around with Lightworks, I couldn't figure out how to make a cut or even delete a clip from a bin without reading the manual.  And I didn't read the manual, so I'm still wondering.

Being available for dirt cheap or free depending on your needs, makes this an amazing opportunity for college students and non-profits edit and create professional videos.  I said that previously with kdenlive, but said its future was bleak without XML export.  Lightworks has the ability to export AAF, EDL, or OMF.  All professional formats that interchangeable with a number of post audio and video applications.

I haven't even touched the color correction in the software yet, but I did play with the scopes, and they are awesome.  Real time playback is always a gift in an NLE (listen up Adobe).

I keep being impressed at how awesome Ubuntu is. Last night I came across a little article on encoding video files to ProRes using ffmpeg. I had to try it, since the closest thing to encoding ProRes on any system that isn't Mac costs $10,000 and runs on Window Server 2008.

I'm running 12.10, which out of the box does not have the correct ffmpeg version to support ProRes. Actually, out of the box it tells you that ffmpeg is depreciated and to use avconv. But I'm not doing that! I updated ffmpeg to the latest using these instructions.

After that I recorded a video of me playing with kdenlive using "RecordMyDesktop" another free tool in Ubuntu's arsenal. I then took the video I created and put it into kdenlive and exported via "transcode clips".

This window is just a front end for ffmpeg (by default there's a bunch of settings for Avid codecs). I copied from the recipe here and inserted it into on of the presets, and encoded! The final export is a ProRes files!

Surprisingly kdenlive had issues reading this file without artifacting, but VLC and QuickTime on Mac played the file just fine. Now I'm so closed to be able to my job as a DIT using only Linux! But we need a couple things as of yet... One would be Davinci Light for Linux (or even this thousand dollar license on Linux, I think right now only the $20k version runs on Ubuntu) and a copy of RedCineX-Pro. With those two tools I could handle just about any camera, create bullet proof backups, and create one lights!

Recently I've been on a big Ubuntu kick.  I haven't used the operating system in years and now I find myself looking it over again, seeing what's changed in the video department.  Turns out a lot!  The best example was the NLE KdenLive.  Check out this video, look at how repsonive the waveform is.

That's much better than FCP7, CS6, or Avid, all through using FFMPEG.  It was able to decode ProRes, albeit poorly, and it has support for Avid codecs.  You might notice the single window layout, after monkeying with it I was able to get what looks like a standard two up display.