Teh Interwebs of Joe

So I am in the process of relocating to JoeKelly.co for my new blog.   I’ve imported all the posts from Tumblr and am posting new items there. 

Now most of the old posts seem somewhat broken and need to be fixed, so I am going to work on that over time. But in general I am done with Tumblr.   It served its purpose but, I wanted more control over my content. 

So find me @ http://joekelly.co/    At some point this domain name will redirect there.  Also another reason I am done with this platform is Tumblr has gone out of their way to block a bunch of posts for “adult content” that have nothing more than backpacking and map photos like attached:   Fuck this site. 

From the post that went up today about the camera setup I put on the roof, now let's talk about what drives it. 

The Old Way

The way I used to do this was very complicated.   It required a minimum of 2 VMs on a server, a long and unstable USB cable, a really expensive, and temperamental USB to Network driver, and a lot of (figurative) duct tape to hold the whole thing together. 

Because the camera was a desktop grade USB device, it needed special drivers to run it.  Those drivers would not run on a server platform.  And they could not run more than one camera on a single machine.   So I came up with the way to have seperate “Driver VMs” that would handle running the cameras.   They were inefficient and and always overloaded due to how bad the drivers were.  (MS Lifecam drivers)

I then had them deposit the frames onto a server that would run the scheduled scripts to encode and upload the video.  This process broke down most often. 

Because I was running all in dos shell and how temperamental ffmpeg is (and how crappy WebcamXp was in writing files), I had to rename every file before it would encode.   Every hour the processing server would run a few scripts that would do the following:

Rename all the files in a sequential and simple order

FFMpeg would convert the frames to video

An NcFTP or SFTP session would be sent to upload the completed video to the cloud. 

Every step in there was highly resource hungry both in disk IO and CPU. I had 6 cores allocated to the process VM and 2 to the Camera Drivers.  And if anything went wrong the entire system (each camera drivers VM and then the processing server) would have to be rebooted in order.   The USB to Network drivers sent close to 200Mb/sec between the endpoints per camera as well.  Which is FAR more than the cameras needed... but is an example of how inefficient this whole thing was.

So that all sucked, but it worked. 

The New Way

When I setup our CCTV/Home Security camera system I wanted it to be streamlined, efficient.  I setup my old HTPC (replaced with a Nuc), upgraded the CPU to a IvyBridge Core i5 3475S CPU, and 8GB of DDR3-1600 mem.   This is a low power but highly capable Quick Sync enabled CPU.

I offload all the NVR duties to QuickSync, meaning at normal load with 7 cameras being processed the CPU idles at 18%.    And the NVR is processing close to 75Mb/sec of video data.  At normal use it uses under 45w of power.

Blue Iris as the NVR (http://blueirissoftware.com/) to handle all the image handling and writing.   I set the sky view camera up to post images every 30 seconds:

It writes the files sequentially with file 0 always being the latest:

This is great as it makes a consistent set of files to read from.   Only issue is that if you feed these in order of lower to higher, the time-lapse will run backwards. 

This is where having a Linux bash shell makes all the difference.   You can read a directory backwards with some powershell or dos scripts, but its complex and doesn't work efficiently.    Under a bash shell you can “cat” the directory and feed it into ffmpeg via pipe.

The encoder string I came up with is: 

cat $(ls -tr *jpg) | ffmpeg -f image2pipe -framerate 40 -vcodec mjpeg -i - -vcodec libx264 -preset veryfast -vf scale=1024x768 -b:v 2200k -maxrate 3800K -bufsize 3M -profile:v high -level 4.2 -threads 2  -g 250 -y -pix_fmt yuv420p -r 30 /mnt/d/SkyViewCam/Timelapse/skyviewX.mp4

The “ cat $(ls -tr *jpg) “ reads out the contents of the directory backwards in order and pipes the data to ffmpeg as an mjpeg stream. 

FFMpeg is installed into the Ubuntu environment directly, it is not using any binaries installed on the windows side.

Under Windows 10 when you enable the linux subsystem, the shell can access the windows file system “ /mnt/d/ “ is the D:\ drive as seen in windows. 

I limited the encode to 2 threads to keep headroom open for Blue Iris.

I also have the script set to do the ftp upload (using FTP for now as its easy and a completely isolated environment I am hosting just the uploaded file in)  With the variables there being populated from a configuration at the start of the script. 

ftp -v -p -n $HOST <<End-Of-Session user "$USER" "$PASSWD" binary cd $PATH put $FILE del $FILE2 ren $FILE $FILE2 bye End-Of-Session exit 0

I have the FTP command upload a new file under a different name, then delete the old file, and then rename the new file to the old one.   This avoids having write conflicts while people are viewing the video. 

Run the schedule off the windows Task Scheduler:

Under the “Actions” you simply fire up “Bash” and then feed it the linux path for where the script file resides.   It's very simple.  I have one for the video encode and one to upload the stills every 5 minutes. 

Here is where the scripts are in Windows:

 Lessons Learned

A few problems I ran into regarding using bash to run commands on a schedule under Windows 10:

When you run a task using Bash shell under Windows 10, currently in the current build of Windows 10 1709, you are unable to use the “Hidden” Feature... it doesn't work, the bash shell will open in view every time. And worse, you cant even fire up the script if you enable “Run whether user is logged on or not” .  If you enable that option the scripts simply won't run.  It is a known issue that is supposed to get resolved in the coming builds.   It took me 2 days to figure this one out.

The work around for the user login issue is to just have the NVR box auto-login at boot. 

The hidden aspect is a pain just because now I see pop-ups of command window every 5 minutes and 30 minutes as events happen.   Not a big deal since I am not on the console often, but definitely an issue.  Also... as someone reminded me - Do not write the bash scripts in notepad... they don't work.  I ended up writing mine in VIM.

Over the last 7 years I’ve had a webcam of some sort looking over the areas of Denver we’ve lived.     For teh last 1.5 years though the camera has been down as we moved to our house and worked to figure out how to get a camera in a position that works, and also work on the technology I used for this. 

First camera I put up looked out over Denver from our 29th floor apartment in the heart of downtown.  Below is the video from the 431 days it was up:

Below is a saved time-lapse of a large storm that rolled through in 12/2011 in Denver:

Then we moved a little bit away and for 3 years we had a camera looking out over the ballpark.  The below video is 841 days (one image a day) while we lived there:

I also recorded the construction of an apartment building next door at that time:

In both cases, these cameras were USB webcams secured to the inside of a window in our apartment.   They had a long 20ft USB cable and some serious hacks (multiple virtual machines, USB to Network converters, etc...) to make them work for this purpose.  They were temperamental, prone to crashing, and overall a real PITA to keep working.   But since these were apartments there was no reason to invest time or energy into permanent solutions. 

The New Camera

We moved a in Oct 2016 to our new house and finally got around to getting a new camera up.   We don't have any downtown views from where we are now (3 miles out of town down)  but we do have some good sky and mountain views. 

First goals with the new camera was for it to be simple, and using much more efficient technology.   I built and installed a home security camera system around the outside of our  house.  All PoE cameras, hard wired, and using Blue Iris as an NVR on a dedicated PC.   This is  FAR cry from where I was the last few years.

We now have 6 (sometimes 7 if I am using my 3D printer and monitoring it) cameras running full time.   One of them is dedicated for this blog (the Sky View camera on the upper left).  This is a view from the Blue Iris system that runs all the cameras.

The Camera

I bought a cheap $49 3Mpix PoE camera that has a Sony Exmor/Starvis sensor in it to give some solid range in day and night.  I bought 2 of these from HDView on Amazon... since I figure for $49 they will die over time.  The rest of our cameras are higher end and 2 -3x more expensive 4Mpix PoE cams.  This gives a native resolution of  2048x1536.  For this site, the resolutions are cut in half to 1024x768 to save on file size.  A note about security... all of these Chinese cameras are prone to exploits and vulnerabilities.  All of mine are firewalled off from the internet and only talk to the NVR server.   Exposing most consumer grade webcams to the internet directly is a recipe for disaster.  Practice safe computing. The Mount

To keep the camera in place, I had to fab a mount for it.  My goal was a no-puncture mount that can go on our roof to avoid me drilling holes or doing any permanent damage to our roof.   It had to be weighted, solid, and handle weather. 

So I went to Home Depot, bought some pressure treated wood and went to work:

I designed it to be low and solid, using minimum of wood, and holding 2 12x12 pavers for weight.    Then I hit it with some black paint to match our roof:

I mounted the camera next.  Even though the connections are waterproof, I wrapped them in electrical tape to give a bit more resilience to water and weather.  

Then the next step was putting it on the roof and setting the field of view.

I ran the CAT6 cable out through the wall and secured the cable.

Total time from starting to cut the wood to getting the camera up and running on the roof was 4 hours.   It took a few days to get the scripts and stuff done to get it published to the blog, but it was a fast project. 

The View

   The resulting view is:

With the camera in its final location, it is now ~ 45ft above the street, and with a clear shot of the sky and the mountains.   It is also framed to capture the construction site across the street as they are building some new town houses. 

The very wide angle 2.8mm lens makes the mountains seem small, but from here you can see Longs Peak up in Rocky Mountain National park.  It’s snow covered so it's hard to see on the camera right now:

We had crazy winds and gusts of up to 60Mph yesterday and the camera handled it better than most of our patio furniture.   Not as much as even a shake seen on any frame of the timelapse from yesterday. Check one of the first 24 hour time-lapse views below and watch the winds whipping the trees around:

I will post another post about the scripts and processes I am using with Blue Iris and other tools to post and compile the time-lapse. 

The time-lapse  and stills being uploaded to the site are done every 30 minute for a 24hr time-lapse loop, and every 5 minutes the still image is updated. 

After being a Canon user for SLR bodies and lenses for the better part of 12 years now, I made the jump last week to a new camera/brand.  After testing, reading, researching, and watching the maturity of Sony Alpha line of cameras, it seems late 2017 the line really matured to the point it needed to be for me. I sold all my Canon goodies and upgraded to a truly high end Full Frame setup with a Sony A7RIII body, and a Sony 12-24mm f/4 G lens, and a 24-105 f/4 G Lens (on order still).  In the future I am looking to order the 100-400 G Master Lens as well.    

This replaces my Canon 7D Body, Tokina 11-16 Lens,a Tamron 28-300, and a Canon SL1 body with a few other assorted Canon lenses.  This new setup is full of potential and is a completely new learning curve for photography.  I sold all of my RC racing hardware in the last few weeks in order to invest in properly high end photography gear to focus on this as a hobby coming up.   The Sony is lighter, smaller, and more capable than anything Nikon or Canon have in their line up right now.  The small size and weight will make it awesome for backpacking and hiking. 

Some of the key reasons for the move to Sony:

In Body Image Stabilization - No matter the lens, the camera does Image Stabilization on the sensor itself. 

Small size and weight (it is 1lb lighter with the wide angle lens than my Canon 7D with a similar lens, thats huge)

A sensor that rivals even medium format 30k$ camera backs currently for dynamic range, and image quality

Electronic Viewfinder (this I didnt think I would like... but its AWESOME) - the ability to fine tune/manual focus on the view finder to a very accurate level.

Full Frame sensor - I wanted to make the move to full frame. 

Very high end AF system using phase and contrast detection across 400+ points

Great selection of 1st party lenses that are very high quality (A recent change for Sony)

Good memory buffer and AF tracking for sports photography

Canon, Nikon, Pentax, etc... all could do some of that, but no camera had it all.  Now I think the Nikon and Canon may be better at AF for high speed sports, only he Sony A9 rivals them, but the A7R is still a leap better than all buy the best Nikon and Sony. The Sony sensor blows the Canon sensors even in their top end cameras out of the water.   The Nikon D850 sensor is splitting hairs with the Sony sensor. The sony glass they have released in the last 2 years is toping most charts for sharpness, distortion, and CA.  With a 42Mpix sensor in the Sony, weak glass will be VERY evident, so this is forcing me to concentrate on buying higher end lenses going forward.   

Some downsides to the Sony:

Short battery life (roughly 700 pictures per charge, but it can be powered off a USB battery pack if needed and charged via that) 

MASSIVE RAW files, this impacts all new big sensor cameras - the RAW file is close to 90MB in size per picture, meaning a 64GB UHSII SD card will hold 700 pics only. 

The Small size does make it a learning curve on where to keep your hand to access everything

The menu system has a ton of options, and it is very important to save custom shot setups in the custom modes on the mode wheel... if not you could be hunting a bit to set stuff up. 

Cost... this isnt exclusive to the Sony... all the top end cameras and glass cost a lot, just something to suck up if you want to play in that space. 

I took some test shots on my patio to get a feel for the lens and camera and to see really how sharp this 12-24 lens can be and then to see how the images come out of the camera.   These are raw exports from the camera with no modification being done to them.   Just set the camera in Aperture priority and set a reasonable f/5.6, and left the ISO on Auto and used the auto focus of the camera.

The clarity and definition of the bottle is amazing at both extremes of the zoom and the focus area is not even centered in the sweet spot of the lens. This should be a fun adventure learning to use this new and very complicated camera, and seeing what I can get out of it.  I pushed my 7D pretty hard in some conditions and it never failed me, let's hope this A7RIII can do the same. 

A video from our groups graduation hike on Segment 9 of the CT last Sept. On the 8 mile hike in, I was chatting with some of the students about knowing your gear inside and out, and being able to set camp up in sub optimal conditions.   I mentioned I’ve set my tent up enough that I could do it blind folded.   Then one of the students made a bet with me... and the above video is the result.

A few more pics from the CMC Backpacking School trip down Segment 9 of the Colorado Trail on Sept 18th 2016. 

I did a quick solo hike on July 27th 2016 up a trail I have been wanting to check out for some time.   This is the  “Hells Hole” hike in the Mount Evans Wilderness. 

This was an awesome early morning hike!   I saw no other person on the entire 8.2 miles until the very end.     I had a quiet lunch under an ancient twisted pine in a glacial cirque near Mt Evans. 

Catching up on posting some pictures from other backpacking trips from 2016.  This is a trip we did to the Comanche-Venable loop down in the Sangre de Cristo mountains.   This was a nice overnight of ~ 13 miles 4000ft of vert. 

I did the hike with a good friend and a co-instructor I teach with in the CMC The hike in was warm then at camp a storm rolled in for a short bit.

Ski season is all over for me!  All the gear is packed for storage, and time to reflect on all the fun over this last season!   The Ski Tracks app shows I did 18 days, but I forgot to use it one day in there so its actually 19. My phone also shut off a number of times on ski days so the total vertical I would guess is close to 20K lower than what it should be.   Still getting over 200K vertical is OK in my book :)