Little Bit More

Advancing Real Time Graphics

ARKit for visual effects

This neat experiment shows another use-case for AR: real-time CGI characters. 

Just imagine a next-gen version of [this app](https://itunes.apple.com/us/app/action-movie-fx/id489321253?mt=8).

Like Invision, for VR. 

Brilliant. Apple’s ARKit strikes again.

Steve Yedlin, DOP for the next Star Wars movie, makes a great case for why chasing after ever greater amounts of resolution is a bit of a fools’ errand, with vastly diminishing returns beyond 4K.

Video games as brain-training 

Can a video game make you less distracted? Neuroscientists at UCSF think so. Dr. Adam Gazzaley and his team at the Neuroscape project have been incubating games that could possibly improve a person’s brain function, and a company called Akili is trying to push these games through the FDA approval process to turn them into "prescriptions." In this episode of Next Level, Lauren Goode visits the UCSF labs and tries the games herself.

Playful Palette: Say goodbye to swatches

Adobe Research and the University of Toronto have designed a digital palette interface with features similar to real-world physical paint mixing.

Ready Player One - First trailer

Real-time Mixed Reality with ARKit

Mixed-Reality capture (example) is becoming increasingly popular (and well-supported by the various VR platforms), but still requires a lot of extra hardware (additional tracker(s) for the camera and a green screen, for starters). 

Experiments like this by Normal show how consumer-grade tech like Apple’s ARKit might soon democratise this process. 

People have been trying to recreate the feel of the holographic chess board from the first Star Wars for a long time now. This version, which requires a pair of ~$3000 Hololenses to play is even beyond the realm of crazy of this classic 2007 attempt, but bear in mind these are still prototypes not meant for mainstream consumers, and cheaper AR is right around the corner. 

Mobile AR is coming sooner than you thought

The Mira Prism seems to be a wireless holographic display powered by your iPhone (6 and up), and they’re aiming to release a consumer version by the end of this year (with devkits available in the fall), with a remarkably reasonable price-tag of $149 ($99 if you pre-order).

From the tech-specs it looks like the resolution will be a bit on the low side (a bit lower than the Oculus DK1—which came out in 2012), but the viewing angle will be 60°—which isn’t that great, but still double what the Hololens offers today (which everyone seems to describe as the biggest let-down upon first using it).

What’s also nice is they seem to be building a complete AR ecosystem, where people with a smartphone can still consume AR content, albeit on a flat-screen. No doubt efforts like Apple’s ARKit are a big part of their initial focus on iOS.

ARKit: Inside Out VR Tracking

The ARKit is able to do the tracking and get the camera throughout to Unity very quickly and therefore it’s possible to do both Pass-Through AR (where the single camera feed is displayed to both eyes), as well as Inside Out positional tracking for VR at around 60 frames per second on an iPhone 7. That’s not enough for quick head movements, but as long as you are walking around and observing at a “gallery pace”, the tracking does a pretty good job of keeping you in the right spot. For safety, we display the tracking marker points as a point cloud while you’re moving around in VR to minimise the chances of bumping into things. So far we’ve not had any accidents.

https://nexusinteractivearts.com/egg/insideout/

Currently, high-end VR HMD’s like the Rift or the Vive use Outside-In tracking, which means that they require some form of external sensors (or beacons) with which the HMD can figure out its relative orientation in 3D space. Mobile VR implementations (like Gear VR, or Cardboard) don’t have any tracking capabilities other than the built-in gyroscope, which means the device can know where it’s pointing, but it cannot know how far it has moved, restricting most games or apps to more of a 360° panoramic experience.

Inside-Out tracking will add the same positional tracking capabilities to mobile devices, and can upgrade existing room-scale VR to world-scale VR. There’s already a number of mobile VR/AR prototypes, which require either completely new hardware (like Google’s Project Tango which comes with advanced 3D sensors) or additional hardware that plugs into your phone (like the Bridge).

Dance Reality: Learn How To Cut A Rug with ARKit

Dance Reality uses Apple's latest augmented reality technology to place footprints onto the floor in front of you.  You look through your phone to step on the footprints, then you follow the animation to practice common dance patterns (see video below).  The app is not intended to be a substitute for proper dance instruction, but rather a tool to practice steps and rhythm anywhere, anytime. Practice makes perfect!

It’s obviously not going to replace actual dancing-lessons (also no real way to look cool on the dance floor while staring at your phone), but I like the implementation. I hope examples like these will inspire more practical, useful implementations of Augmented Reality and help the technique transcend gimmickry. 

babylon.js

A complete JavaScript framework for building 3D games with HTML5, WebGL, WebVR and Web Audio

Microsoft’s also no slouch when it comes to Open Source & the web these days. It looks like 3D is quickly going to become another discipline webdesigners are going to have to master. 

So how many frameworks and tools can we choose from to make 3D web-content right now?