Hammer & Tusk

I was using the iPhone 6 Plus and a Google Cardboard headset. My first step was literally to pull up Chrome and type in “vr porn.” The first three sites that came up were Pornhub, VR Smash, and Virtual Real Porn. I’d heard of Pornhub (we’ve covered the launch of their free VR section), so that was where I decided to start.

I clicked on the site, and then selected the first video I came across. When I hit play, the double images were stacked one on top of the other instead of being side to side. I rotated my phone a few times trying to figure out if I was doing something wrong, with no luck. Investigation revealed a small note: “must have VR viewer.” The note was a clickable link to the app store. I wasn’t thrilled that I would have to download a prioritary app just to view the video (I don’t usually trust developers I don’t know), but journalistic curiosity had to be satisfied. I downloaded the app that was recommended (Free VR Viewer by Ginger Monkey) and I was on my way.

Except I wasn’t.

It still didn’t work. Nothing was happening. After I downloaded the app it didn’t direct me back to the website, and when I navigated back to it and clicked play on the video, it was identical - stacked instead of side to side, and with the same note about needing a VR viewer. I was starting to think a casual user would have given up; but I was determined. I tried downloading the video, thinking maybe I needed to watch it from my phone. The download link didn’t seem to work. I tried the next one, which read “Download iOS app.” Weirdly, that was the link I needed to play it (as far as I could tell it didn’t download anything to my phone, despite the clearly labelled misnomer.) If this hadn’t been a work experiment on my work phone, nothing would have made me click a suspicious download link on a porn site.

Okay, so I was past the unintuitive hurdle which seemed designed as some culling ritual to remove the unworthy. Now that I had accessed the video that was expressly designed for Cardboard - it still didn’t work! At least here the interface was intuitive for anyone who’s used Cardboard on Youtube. I clicked the little viewer symbol and finally, I was cooking with gas.

Or at least I was watching VR porn.

Bad, bad VR porn.

The video was super low resolution, and the guy had his face blanked out like some kind of defendant on Cops. Everything was hard to see, and all of the action was taking place to my right so I had to turn awkwardly to the side to watch it. (I later realized there was a button that I could use to recentre the view, but again, it was hard to find and I couldn’t access it while in Cardboard view, so I kept being tossed out, adjusting it, and then firing it back up.)

Okay, one bad video does not a bad platform make. I tried a second one. This one was a point-of-view video, which I later discovered is basically all you can find, despite my initial offering. The resolution was a lot better and things weren’t quite as fuzzy, but instead of a person there was just a large black fuzzy hole (pun not intended). The actress leaned down and in a sultry voice murmured, “Feel my toes wrapped around your ****.” I looked down at said toes - but they had vanished into the darkness. Any sense of immersion was gone, and her interacting with my invisible body was off putting. I clicked away.

The third video was as fuzzy and indistinct as the first. This POV video had a male body laying flat on a bed like a creepy bearskin rug. Something about the angle made the body look eerily flat, and the background had been stitched together to remove the head, so I was sitting with my head on top of some kind of weird supernatural headless horseman bearskin rug. Needless to say, it was a little distracting. The terrible acting going on in front of me didn’t help matters. I moved on.

I clicked through video after video. A bunch of them were doubled - there was no way to calibrate for the viewer I was using, so the display was off for most of them - and the ones that weren’t doubled were painfully low resolution. I had heard that the female POV videos were even more disconcerting than the male ones, but I couldn’t find any. I gave up and headed over to VR Smash to see if their offerings were any better.

VR Smash didn’t require me to sign up for a third party viewer, which was good. I still couldn’t calibrate for my viewer, but I did realize I could change the size of the image between two options, which helped a lot. So I found a clip that wasn’t POV (I have firmly decided that those are creepy and I didn’t want to see one more bearskin rug man) and clicked the cardboard symbol. I turned my head - nothing happened. I dipped and turned again. Nope. It was fake VR. I tried two more clips. They were the same. Sort of 3D, but not 360. I headed back to Pornhub - at least their bad VR porn was actually VR porn.

I had pretty much given up on the notion that there was ANY good VR porn when I stumbled across a video that managed to do it well. The picture quality was a lot higher than other videos, with no blurring or obvious pixels. The conceit was that the viewer was on the other side of a webcam, and the girl was talking to her computer. You could look around 180 degrees, about what a webcam would show, and the girl was very close to the camera. The sense of presence was incredible - I felt like she was really talking to me. My only issue was that I wasn’t sure what the clip gained from being in VR. Since she was talking to a webcam, it would have made more sense to watch on a computer. Then again, I’m not sure I would have been as transported inside the scene if it hadn’t been in VR. There were moments when I forgot she wasn’t sitting on the bed across from me.

Key Takeaways

1. Journalistic Integrity Faces New Challenges

Integrity is the cornerstone of great journalism. Facts cannot be tarnished with opinion, the truth must be delivered directly to the audience, and the journalist cannot interfere with their subjects. Some of these are harder to adhere to than others, and a commitment to each needs to be reaffirmed in the face of this new medium.

RYOT’s Tyson Sadler explained that not affecting the subject matter is a difficult one with 360 content. Because directors and camera operators can’t be near the equipment when filming (or they would be visible in the shots), there’s a lot less control than there was with traditional film. That can lead to a temptation to reclaim that control by setting up shots, or taking shots over again if they don’t suit. There’s also a lot of re-creation that goes on with 360; getting interactive footage is challenging and expensive, and there often isn’t enough footage to build an entire story around. So many directors fill in 360 videos with 160 video, still images, and picture-in-picture interviews. Emblematic, for instance, uses crime scene footage and real 911 audio files to create digital representations of past events. That can be a problem if documentary makers aren’t differentiating for the audience, something Emblematic is always careful to do.

There was a lively debate on the differences between documentary and journalism, with AP’s Paul Cheung arguing that documentary is always more staged than on-the-ground reporting, while Emblematic’s Cassandra Herrman felt that documentary’s responsibility to portray truth is just as far-reaching as that of more traditional coverage. Both agreed that not everyone (on both sides of the journalism spectrum) are as conscientious as they need to be, especially with VR’s more immersive properties on the table.

2. The Eye Can Wander

Refah Mahmoud of Dryft Digital started the conversation about directing. Famous directors like Spielberg have come out against virtual reality (though Spielberg himself has done an about-face, signing up for a VR scripted series), afraid that without the frame their role is meaningless. Directors decide where you should look; their entire purpose is to frame the story, and that relationship is so fundamental that one of the first things we think of visually is a director holding their hands up in a rectangle to simulate the camera’s gaze.

With virtual reality, that relationship changes completely. The audience’s interaction is better likened to theatre, where the director must consider every part of the landscape where the viewer’s eye might stray, and use tricks and techniques to draw the gaze where they want it. In theatre that can mean placing the action closer to the front, changing the lighting, and having the speakers as focus. In VR that means visual movement that will draw the eye to the next moment (animals work really well), audio cues (for devices that support spatialized audio), and leaving off a scene on the same angle you start the next one on.

If a theatre director fails to keep the audience’s eye where it’s supposed to be, the consequence might be that they miss some of the plot. But because VR is still such a nascent technology, the consequences of losing a viewer might mean they never try another experience, not realizing the problem wasn’t with the platform itself. Because of that, putting out quality content now has to be at the forefront of every creator’s mind, and making sure the audience doesn’t feel like they’re “missing” the show is integral to that success.

3. The Empathy Machine

“I asked my staff to go film some fireworks,” Paul Cheung said, referring to his complaint that right now, virtual reality seems to be the sole domain of depressing stories. VR is often described as an empathy engine, and filmmakers have seen huge success in bringing light to important stories of hardship around the world. As Taylor Owen of UBC’s digital journalism program explained, virtual reality “presents a new means for representation: evoking emotion & empathy.” Immersive experiences have the ability to connect with the viewer more deeply than traditional documentaries - and truly interactive VR brings an element that even 360 cannot. It digs into emotions, the sense of presence making the viewer feel like part of the experience.

From the Ebola outbreak to Syrian refugees, all of the docs demoed at So/VR were about very weighty subject matter. The use of the empathy engine in this way has positive effects - after watching Clouds over Sidra (a film about Syrian refugees) doubled the donations the charity received. And while those tear-jerking experiences are contrasted with nature stories and sporting events, there’s still a lot of room for positive human interest stories in the VR world.

One of the highlights of So/VR was the chance for journalists to get hands-on with the technology. Many had never experienced the HTC Vive, HoloLens, or Samsung Gear, and one or two had no familiarity with even Cardboard. Understanding it from the user’s perspective is critically important to creating content, so the chance to get hands-on had many attendees singing the technology’s praises. If there had been an HTC Vive booth, several would have walked away $1000 poorer.

If there was one lesson learned over the course of the day, it was that the appetite for virtual reality is real, powerful, and growing. It will only be a matter of time before this nascent medium takes its place on the stage, and reporters will be there - talking about the story, and making it.

Hammer & Tusk would like to thank our amazing sponsors for helping make So/VR possible.

The UX is slick (it highly resembles Instagram), but possibly too slick. There are very few buttons or menus, and no built-in tutorial to help you figure out the different screens. The first time I used it I tried to take videos of my entire environment, lining each shot up perfectly, which took forever. The second time around I realized I could take a video of my subject matter and then “fill” the environment by quickly snapping all around me, which was way faster (and way more fun). Once I got that down, I splashed out a bunch of videos with no problem at all.

Speaking of Instagram, Splash users have profiles and can follow or be followed by both friends and strangers. The app doesn’t offer a native search function to add your friends, which is an oversight, but you can “share your profile” through text or email. Once Splashes are made they can be shared through pretty much every social media channel you can think of, and the process was dead easy. My only complaint is that when I first tried to share it on Youtube I thought the video was flat; but it just took awhile to load, and looks great now. Facebook also supports the full 360 experience.

This offering is a step up from Android’s native ability to take 360 photos, adding both movement and sound. I could see a great partnership between Splash and Snapchat, or Splash and Vine, both of which have similar concepts for a 2 dimensional world.

The next time I go hiking, there won’t be a Facebook album full of still pictures of mountains and trees - there will be Splashes.

----

One of the most important factors for providing spatialized sound is the Head Related Transfer Function (HRTF). HRTF is a mathematical function describing the frequency response of a person’s ear. By creating a dummy head and inserting microphones into the ear, researchers (mostly working out of universities) could see how sound was distorted by the ear canal. They took measurements of sound coming from every conceivable angle, and from that data created the HRTF. Spatializer software checks which angle sound should come from and compares it to HRTF measurements to design 3D sound.

The challenge is that every person’s ear is different, and often in significant ways. The frequency response of your ear can be radically different from that of your parents, and even things like how you wear your hair or what clothing you have on can have a huge impact on how you perceive 3D audio. (Because your brain quickly adapts to “normal” changes you don’t perceive a difference if you wear your hair differently, but it’s there.) Despite this, all of the measurements on which the HRTF was built were taken either using artificial heads, or with a very small number of real human ears. The result is that spatialized audio has been unintentionally built for silicon heads, which works well for some, but provides a subpar experience for others.

The second most important factor in 3D audio spatialization is whether the company uses object based audio or cinematic based audio.

Imagine a virtual reality game where every object has a sound source attached. If there are twenty birds flying around your head, each one has a separate audio source, emitting sound from a different direction and distance. That is object based spatialization. If you have twenty birds, it requires much more CPU processing power than if you have five birds. This is the best approach for the highest audio quality, but it can be extremely problematic for mobile VR, where your CPU capabilities are quite limited.

Cinematic audio is completely different. If you have a recording in the real world, a spatialized recording, or if you have a movie with 3D sound, then you don’t have all the audio sources sound separately, because the microphone picks up everything all at once - all the ambience, the speech of the actors, car noises, everything. You may have four or six microphones, but there’s no ability to parse those audio streams for greater clarity. This is how live scene are recorded, and movies with 3D audio.

Once you have that audio, you have to translate it to a three dimensional world, and this is done with a metaphorical ‘sound wall.’ Audio designers create what is essentially a sphere of speakers around the user. The position of those speakers can’t change or move; they’re set.  If a bird flies across the scene the speakers don’t change position; rather, the sound tracks through to be emitted by different speakers based on its location. So if there is one speaker to the left and one to the right and the bird flies between, the sound fades out of one and comes up in the other.

Generally speaking, game designers build with object based VR, but when designing for mobile they’re forced to resort to tricks to get the soundscape to work without throttling the CPU. For instance, a designer might choose two birds that interact closely with the player, and then the other eighteen birds will be processed as a single sound source. That can be a serious detriment to audio immersion, but it’s what Oculus uses for its spatializer software.

The other tactic for reducing CPU consumption is commonly used for Google Cardboard apps, and it called a fixed cinematic setup. Rather than being object based, it maps each bird to a specific speaker. Essentially, this allows them to convert a game created with object-based audio into a game that’s cinematic audio, so it can run comfortably on mobile. But this results in the same audio quality as every other cinematic-based system.

Gabor Szanto, CTO of Superpowered, explained their unique solution to this problem. Advanced audio algorithms for processing on resource constrained devices let them provide the benefit of object-based audio without the drawbacks. By being incredibly efficient with regards to CPU, they allow more spatialized sound for the same output, outperforming Google and Oculus by as much as twenty times. That’s a significant difference on a mobile device - Oculus allows 10 distinct spatialized sounds, whereas Superpowered allows 60.

Rendering on mobile devices is computationally very expensive, and unlike desktop, mobile has resource-limited processing. Most people believe the way to solve this is to invent better hardware - bigger, faster chips and more memory. But that takes a long time to develop, and meanwhile users are left with a non-ideal experience. Solutions like this provide immediate relief for developers who need a solution today - not in Q4 of 2020.

Many people believe that mobile will be the future of VR. Google is certainly putting their money behind it, and others aren’t far behind. Facebook is creating 360 content for their news feed, Samsung is betting big on the Gear system, and even Apple is rumoured to be getting ready to dip their toes into the virtual reality market. But if we’re going to bet on the horse, we have to make sure we’re putting the best product forward. Experiences have to be created with mobile in mind, and that means finding solutions to make 3D audio work - and make it as immersive as it can be.

--

In the press, VR and AR are often mentioned separately, and this conference really unified the two types of mixed realities via enterprise use cases. 

Below are some CB Insights data I pulled using their Trends feature on how often the two terms are mentioned in the press:

Now, look at CB Insights Trends on how VR and AR are mentioned in relation to gaming and enterprise:

When AR and VR are mentioned in tandem with gaming, VR dominates the conversation; however, when AR and VR are combined with the term enterprise, they are almost on par. This mirrored what I saw at AWE.

I saw representatives from various aerospace, defense, manufacturing, marketing, and logistics companies walking around the demo floor. At one of the talks, the CEO of APX Labs (makers of an enterprise smart glass platform) talked about the business impacts of smart glasses for work.  The company’s survey shows that 93% of companies are exploring wearables (wearables include heads up displays, smart watches, smart glasses etc.), 83% use multiple wearable devices, and 79% of started using wearables in the last 18 months. A representative from Boeing discussed how using smart glasses with voice command reduces the amount of time between intent and action, thus speeding up its assembly of a wire harness. As a former engineer at an aerospace company, having spent time as both as a manufacturing engineer on the production floor and within a supply chain capacity, I could see AR’s potential impacts on a production line.

At AWE 2015, another Boeing presentation talked about the results of an internal study on how trainees performed when assembling a mock wing and found that a group using an AR enabled tablet assembled the wing 30% faster and 90% more accurately on their first try than the groups using traditional methods.

While many of the AR devices are considered “immature” for consumer adoption, manufacturers are already experimenting, and in Boeing’s case, already using AR devices in production hardware. Osterhout Design Group (ODG) had a large booth showcasing how companies are already using its smart glasses. I had seen a demo by EON Reality where using ODG’s glasses, I could see pipes and troubleshoot them by moving the gauges just by looking up or down left or right.

Marketing, sales, and commerce use cases were also shown at the conference. It was evident how mixed reality can be used as a new purchase path and how better visualization can reduce returns. Augment, an AR company focused on sales, e-commerce, and communication, demonstrated how sales teams can display products for customers in the store. It showed how Coca-Cola uses the platform and how it is linked to Salesforce’s CRM to help drive sales. Virtuality uses VR to create a shopping experience by essentially placing products in VR where users can interact with them. A virtual shopping experience could reduce costs by reducing the need for physical store space and increase sales by analyzing consumers’ interaction with virtual products. We’re already seeing brands considering this - a recent Wall Street Journal article reported how Cadillac is replacing a portion of its physical stores with virtual showrooms.

As the hardware devices and software for mixed reality get more advanced, enterprise applications will get better and companies will find more interesting and creative ways to incorporate mixed reality into their business operations. The applications I saw at AWE included marketing, entertainment, medical, manufacturing and more. I can’t wait to see what’s coming next.

----

The article mentioned above made the rounds at our office, and someone asked if we could try it out. “Sure!”I cheerfully told everyone who asked. How hard can it be? All I need is a long thin board and a stuffed animal. We’ve got room to set this up in our VR room! I thought. I jumped online and started scanning through the articles, trying to figure out where to download the experience. “Take my money!” I screamed at the screen. But no one would.

The experience is only available to try live at the VR Zone Project i Can, in Tokyo. If you happen to be in the area, feel free to try it out and post yet another video to Youtube in order to mock us with all of the cool stuff that we’ll never be able to do!

2. Breathe Deep

Image this: using only your breath, you can control your journey through a fantastical landscape, playing a simple game while learning the tricks and techniques of deep breathing meditation. Sounds great, right?

NO VR FOR YOU!

DEEP is a meditative VR game controlled by breathing. Players don a headset and a custom controller (basically a fancy belt that goes across your ribcage) to explore an underwater world. The game teaches players yogic breathing techniques that can alleviate stress, anxiety, and mild depression. If you think that sounds cool, you would not be alone. The first two pages on a Google search for “vr breathing meditation” are nothing but articles about this amazing game. GAME. Not one of the articles I read called it a ‘demo,’ and none mentioned that it wasn’t available to consumers. Even the website was confusingly blank of purchasing details; it took me ten minutes of concerted searching to realize I couldn’t find a purchase link because it wasn’t available to purchase.

That sure didn’t help my breathing.

3. Geographic Content

It’s not exactly uncommon for the internet to be American-centric, but it IS uncommon for that to be such a problem (unless your American friends constantly post Daily Show clips that you can’t watch because the content is locked for their region). Pages and pages of internet ‘ink’ have been spilled lauding The Walk, a terrorizing 360 video of walking on a tightrope between New York’s Twin Towers. Sounds great, right? Apparently it is. If you live in the US, feel free to tell me all about it.

Then there’s the lack of non-English (or non-language based) content. One developer in South Korea bemoaned that even when a game doesn’t require language to play, all of the instructions for how to play are in English, and right now very few games have the capital to translate into multiple languages.

For consumers outside of the US, figuring out what they can actually play can be a huge headache.

4. Cat Petting Simulator

There’s only one problem with the adorable cat petting simulator that has everyone talking - it isn’t real. Someone made it as a joke, but unlike joke games like Desert Bus, which went on to raise more than $2.5 million for charity, the creator of this internet meme has no plans (thus far) to turn the single gif into an actual VR experience.

Most people agree that VR is more about what it will become than what it currently is. The potential is there, so close we can almost touch it, but while current experiences can be incredibly immersive, most of them feel more like demos than polished consumer experiences. Take a game like Job Simulator. It captures a player’s attention because it let us toy with the interface of touch controllers; but at the end of the day, you spend more time seeing what different things you can set on fire in the frying pan than trying to “win” the game. Job Simulator was one of the best reviewed launch games for the Vive, not because people think we’ll still be playing it ten years from now, but because it shows you what VR could become.

In this environment, the role of a reporter and reviewer is complicated. Games can’t be judged on the promise of what VR might one day be - but that means that any flaws are considered, perhaps rightly so, as flaws in what VR is currently capable of, not as flaws in the game. Take Call of the Starseed. It hasn’t gotten a single bad review - people are engrossed by its gorgeous graphics, its engaging story, and the smart way it uses VR to create an immersive mystery. Not a single review mentions the very long periods of blackness as you “blink” from one spot to the next. Motion is an ongoing problem in VR, as it tends to be a trigger for simulator sickness. One solution to eliminate movement is to make the world disappear and then have it reform a second later - “blinking.” The practice is unfortunately cruel to immersion, and becomes a challenge as gameplay progresses. But reviews don’t mention it because everyone knows that moving around in VR makes users sick. I’d much rather blink than vomit.

But the decision to leave the kid gloves on can do a disservice to the consumer. Take as another example Hover Junkers, also one of the highest-rated Vive games. Instead of employing a “blink” to stop motion from making anyone sick, Hover Junkers has decided not to cater to users who can’t handle motion. I played the game for roughly a minute and a half before I declared it too horrifyingly nauseating to continue; I passed the controls over to a colleague, who gleefully took over. Reviews haven’t mentioned Hover Junkers nausea index because getting sick is such a common problem in VR, but that isn’t fair. Games like Call of the Starseed have ensured that I can still play them; I deserve to know that Hover Junkers hasn’t done the same. Similarly, if you were the kind of person who got really claustrophobic, those periods of blackness might be a hard no, whereas Hover Junkers’ zooming motion might not pose a problem.

It seems clear that it isn’t fair to developers to criticize something that no one else can do better. On the other hand, artificially inflated reviews can bring consumer expectations too high.  I don’t want to be told that each new game is the Second Coming of VR; I want to know what it will really be like to play, warts and all, so I can appreciate the game for what it is.

So is it time to take the kid gloves off? No. The technology is still new - things don’t work very well. A lot of games are still functioning mostly as demos - I know for my part, the things I get the most excited to show people are Tilt Brush and Irrational Exuberance. But do we need to inject a little more grounded commentary into our boundless enthusiasm?