Eric Azares

tl;dr I am a tech consultant & UX Designer but in my spare time experimented with 360 panos, learned Unity game dev, won a vr hackathon & stumbled upon a 360/VR project.

Started Making 360 Pano Images

In 2011 I ordered a photography kit from TourWrist (now known as TheSphere) to shoot and stitch 360 photos. At the time it felt like a magic teleportation window-- especially when viewed through an iPad. Here’s my old 360 Pano Gallery: https://www.thesphere.com/profile/5568

Tried Oculus at a Museum in Cambridge

My first encounter with the Oculus Rift was in autumn of 2014 at the Computing History Museum in Cambridge . It was part of an exhibition that traced the history of video games from 1970′s to the modern times - the Oculus the future gaming. Although basic by today’s standards, I enjoyed the demo. 

Learned What Tools Professional Use

Flash forward to Autumn 2015- I was attending UX and UI conferences and learned that a small minority of product designers began using Unity for prototyping. Specifically, they used it to design automobile dashboards, touch screen kiosks and other interfaces beyond the traditional computer screen. 

Also, Unity seemed to be growing in popularity among indie game developers. The seed for learning Unity was planted in my head.

Inspired by Apps POWERED BY UNITY

I don’t recall how I discovered Google Cardboard, but around November 2015 I ordered a cheap knock off from Amazon.com.  I started with some free apps that claimed to be an IMMERSIVE, VIRTUAL REALITY, 360 EXPERIENCE and... “MADE WITH UNITY”

Learned New Skills

I was convinced Unity development would be a necessary skill set for designing future products. I signed up for some online Coursera classes & eventually a short class in NYC. 

I tinkered with the Google Cardboard SDK and learned new tricks through YouTube and local tech meet ups. I continued experimenting and even won a local hackathon event.

Good Timing

The objective was to create a functioning VR App over the weekend. Here are some lessons learned from this experience.

tl;dr

Keep app simple - make it do ONE cool thing

Emphasize learning - step out of comfort zone and learn a new skill

Meet people - you might meet future collaborators or future even employers

Backup frequently - just in case you can’t figure out why your game is broken and you want to roll back to most recent functioning version

What we did

Got semi-prepared - I took the time to get familiar with the tools we would be using. We didn’t know what the theme would be, but we knew we wanted to use the Google Cardboard SDK.

Assembled team of complementary skills - If we wanted any chance of finish a functioning app in 48 hours, we needed skills in different disciplines. Our team consisted of a coder, 3D modeler and audio engineer. Also, enthusiasm and willingness to learn about the other skills was important. For example, I was the coder, but I picked up some knowledge about mixing audio levels and applying materials to 3D models.

Focused on making a simple app - We had limited time, so we wanted to make a simple, polished game. We preferred a simple finished game over a complicated unfinished game.

Saved time by working parallel - after establishing our vision, we split the work and worked simultaneously. While i worked on the coding, the designer worked on 3D models in Blender and the audio engineer worked on sound effects and music. For testing purposes I used temporary sounds and basic cubes as placeholders for the characters.

Built and Ran Immediately; continuously added features - We constantly test our functioning app on an Android phone to ensure that our ideas worked the way we wanted.

What we would have done differently

Read and confirm the upload instructions we almost missed the deadline because we didn’t follow all the necessary submittal steps. We uploaded, but didn’t submit it to the right page.

Back up all functioning builds -  There a couple times when the new code ‘broke’ the game and we wanted to roll back to the most recent working version of the game.

Budget more time for playtesting- Our app would have come out better if had more feedback, earlier on. 

This past weekend I had the rare opportunity to experiment with Google’s  Project Tango mobile device at a mini hackathon (hosted by Staten Island MakerSpace).

 In just few hours, I developed  two simple apps that explore the device’s motion tracking & depth sensing capabilities. 

What is Project Tango?

Project Tango is equipped with special motion tracking sensors, depth sensors and special cameras. This allows the device to sense where it is and how it moves through a space.

For example, if you are holding the device as you walk around the room, it will continuously detect your position in relation to the objects in the room-- walls, tables, floors, objects. The device also uses visual cues to recognize the world around it.

Making Apps

Adding Objects to a surface

In my first app you can add 3D lion statues anywhere in your room. This interaction can be useful for interior design and architectural planning.

Below you can see the depth perception and spatial awareness in action. The two identical lions sit on two different planes: the top stairs and the bottom floor.

Now take a look at the same two statues from the reverse angle. As you can see, the lions stay in place, just as they would in real life. Also, notice the size difference of a third lion in the foreground.

Real Time Motion Tracking

Next, I experimented with Project Tango’s motion tracking feature. My second app tracks your movement in real time and renders a trail behind you.  Below you can see a blue line visualizing my path as I walk around a room.

For fun, I used a fish to represent the user’s location and created an ocean environment on the screen. As the user walks around in real space, the fish moves around in the ocean.

Here’s a video of the two apps in action!

The Idea

Virtual reality (VR) can be a fun way to explore far away places. We can even use this technology to present useful information in the 3D environment.

For example, a (virtual) tourist may want to know the historical significance of a monument. Rather than reading a placard or guidebook, we can digitally display the information directly on the (3D) object itself.

In this mini-project I’m creating a basic VR tour for the historic runestones in Jelling, Denmark.  This blog posting is meant to be a general overview of the process.

Equipment: Unity3D , Google Cardboard and Android Phone

Planning the Design

Here’s a photo reference of the site:

The plan was to present historical information adjacent to the stone monuments. Floating cubes serve as interactive point of interest markers.

The panels would remain hidden until the user triggers the markers. This could be achieved through either a gaze input (focusing on object for a prolonged amount of seconds) or by pushing the button on the Google Cardboard unit.

Building the Environment & Interaction

Using the photo reference from Denmark  I built a very basic 3D environment in Unity.

I used the Google Cardboard SDK to create the VR camera. Next, I created the point of interest marker (green cube) and information panel.

Thee panel would remain hidden until activated by the user. To achieve this, I used a raycasting technique to detect the collision of the user’s gaze. This is illustrated with the blue line below:

Testing on VR Device

Now for the test. With Google Cardboard, the phone serves as the virtual reality screen and is inserted into the cardboard housing.  Below you can see the stereoscopic display.

When everything is set up, the user can explore and interact with the VR world through head rotation/tilt, and gaze/button inputs.

I plan to continue expanding this VR project and further refine the interactions.

That’s it for now!

As I worked on my game’s UI, many questions popped up. During this design process I actually borrowed some techniques common to informational dashboard design in software. 

Here are some of my thoughts:

Wiring up the data

First I had to make sure my code was actually working. Was health subtracting when attacked by enemies? Was my mine count increasing when I picked up a power up?

I displayed the raw numbers with text labels during this phase of testing.

Icons vs Words

Once the power meters and inventory counters were hooked up properly, I began designing the presentation of the HUD (heads up display).

Now, in the context of a game environment the player’s gaze is focused on the center of the screen. Icons can be a quick way to transmit information. Words can slow down communication, because the player must divert their attention to read and translate the text. With icons, the player can discern the general size and shape of a graphic with just a quick glance.  

Here’s how screen (sorta) appears when we are focused on the game:

Notice how our peripheral vision is blurry, but we can still identify the icons.

However, icons can backfire if they are ambiguous or too similar to each other. Therefore, it’s important to choose a meaningful graphic. In this case, the icons bare the likeness of the game objects (mine, shield dome, skull).

Learnability is also another concern, because these graphics were made especially for this game. These icons are not universally understood outside the game ( red octagons  for ‘stop’, men/women restroom symbols, etc). So again, it’s important to choose icons carefully.

Placement of the most important information

“How many mines do I have left?”

The more I tested the game, the more I realized that the land mine was the player’s primary weapon. I placed the mine counter in upper left corner, where important information is typically displayed ( in Western languages).

Displaying a finite number of weapons

For the sake of gameplay, the player can only hold 4 shields at a time.  So instead of using a numeric counter, I represented the shield count with a set number of icons. With this visualization technique, the player can quickly glance at the number of shapes to see how many are on reserve.

The shield counter is placed at in the upper left corner right underneath the mine counter. This was done for consistency and to prevent extra eye movement across the screen. 

Visualizing the player’s health

The health bar’s length is a ‘preattentive’ attribute-- we can clearly and immediately perceive it without conscious effort. This is a good  visualization choice because it quickly communicates data without demanding attention from the player. 

Line length is just one of the many preattentive attributes. Take a look at the groups depicted below. Notice how one member of the group immediately stands out. These visualization techniques can be helpful when you need to draw attention to certain information.

Image credit: Colin Ware, Information Visualization: Perception for Design

From Simple Squares to Flaming Skulls

Earlier this week , I prototyped my gaming with simple shapes without the distraction of special effects ( until now!).

Here’s  the main character destroying the enemies with a force field. You’ll handy little  ‘camera shake’ upon collision.

But in this iteration, the enemies just evaporate just seem to evaporate. So about some particle explosions upon contact?

Now let’s kick it up a notch by changing the environment, applying textures and replacing the cubes with skulls and barrels :

I recently came across Drivemode, an app that provides drivers a safe way to use their phone. Users can access music, navigation, and messaging apps using simple swipes and audio commands.

One feature I found useful was the calendar navigation. The app accesses the appointments from your Google Calendar and automatically adds the location to the navigation options. I found this feature very handy, especially if you have several business meetings in different locations.  

But what happens if there is a traffic delay or change of plans? How could the system adapt to these changes and deliver timely information to the driver? 

I decided to examine this scenario and created a video prototype of possible solution:

Below you’ll find notes and sketches from my design process.

Storyboard

I sketched out a rough 9 panel storyboard of this scenario. In this context, a business man will be attending several meetings in different locations throughout the day.

 The meeting time, location, and attendees are all contained in his Google Calendar.

Next, he connects his mobile phone to the infotainment system

He then chooses the navigation option 

In this system, the driver interacts with the app, using the  car’s physical touchpad and rotary button (instead of the touch screen).

The driver’s meeting locations are pulled from the calendar app and presented in the navigation app. The system will even suggest when the driver leave, based on his distance from the destination.

Driver begin journey and encounters traffic. The driver will be late.

 The navigation system detects the delay, and calculates the new estimated time of arrival or ETA.

The driver can update the attendees with the new ETA by sending a precomposed text message.

“Message sent” confirmation

User Workflow & UI Sketches

Next, I examined how the  user would flow from screen-to-screen to complete the tasks. I also began exploring designs for the screens. For example, what information and actions should appear on the alert screen?

UI Design

The UI was meant to be minimal, delivering information that is important to the driver at that moment in time. To help the user plan his journey, I included the “Leave in xx min” on the calendar screen.

A second addition is a smart precomposed text. The system can anticipate the user’s needs and provide a text containing the new ETA, ready to send to the participants.The features utilizes data from the navigation app and the contact info from the calendar.

UI Animations  & Video

Following the patterns of the Drivemode app, I created smooth transitions from one screen to the next.

These animations were created using Adobe After Effects.

Finally, I composited the screens, sync’ed the audio and edited the footage.

And the final video again:

Some Design Take Aways

Designing for a car is very from a desktop or mobile device. Here are some guidelines and lessons learned while designing this prototype:

Glanceable interface - Minimize interface to keep eyes on the road and hands on the wheel. This can be achieved using large fonts, high contrast between background and foreground, and most importantly, studying and understanding the user’s needs.

Utilize other modalities - To minimize glances away from the road, modalities such as audio and haptic feedback can be used. For example, the Drivemode provides audio feedback when making selections, and allows for voice input. The device also vibrates when navigating from screen to screen.

Make use of control physical controls - The app provides vibration feedback when swiping the touch screen. However, it may be ideal to take advantage of the physical affordances of tactile buttons on steering wheels or the the rotary knob in the center. This would allow the driver to keep their eyes on the road while they ‘feel’ their way through the interface.

Anticipate user needs - What information do they seek at this given point in time. For example, the user will want to know what time they need to leave to reach their next destination in time. An on-screen “leave in 15 minutes” can aid the user.

Use contextual awareness to provide timely information - The driver needed to know his new ETA, based on his current location and traffic patterns.

My plan is to make a simple driving game with Unity in the next 10 days. I’ve seen veteran game developers pump out amazing games in only 24 hours.

But I’m still a beginner, so I’m not expecting to make a masterpiece. Rather, my goal is to continue learning Unity. Below are my initial thoughts about the process. I’ll blog the process and we’ll see how the game evolves over the next 10 days.

Planning

Here are some rough sketches of my game idea:

Writing things out will help break down what actually needs to be programmed. How does player move the character? When you press ‘up’ does the vehicle move forward relative to the camera or relative to the way it’s facing?

Making the game simple & playable; not complex and unplayable

“Don’t try to make Assassin’s Creed in 2 weeks” 

Or in my case, “don’t try to make Need for Speed XII in 10 days”.  Just like everyone, I have a lot of big ideas of giant worlds and complex gameplay. But I need to consider what is feasible in 10 days for a one man team (who is a beginner).

My initial idea was a  racing game with lots of power-ups, cameras, game mechanics,etc, but I’ve had to scale it back. This means working on the core gameplay (i.g. moving around) and worrying about the extra features later.

Based on what I know about software development process in general, I’m 100% sure that, even w/ my scaled back version, my plan will end up being ‘too much’ to do in 10 days.

For inspiration, I looked at an old NES game, R.C. Pro-Am :

Make it immediately ‘fun’

I’m going to immediately thrust the player into action and give them that fist pumping moment when they win. Or at least a big exhale.

Testing and repeat

Only a few hours into coding and how testing is essential to achieving ‘fun’. The beauty of Unity is the ability to test along the way to make sure you’re on the right track. 

Flexible Plans(?)

“Wouldn’t it be cool if...?”

At the very beginning I outlined my plan for a driving game. But after only a few minutes of experimentation, I found that ‘bashing into things and making explosions’ could be fun too. So I’m adjusting the  driving game to be more of a demolition derby. 

Here’s a screenshot from the game Full Throttle, for reference:

Focus on learning

Most of all, I want to learn new techniques and practice problem solving. I’ll avoid just downloading stuff from the asset store and blindly slapping them together. However, if I do end up using free scripts, I’d like to understand why the code works (at a basic level).

I’m going to keep the game aesthetics in mind, but I want focus my attention on the gameplay. After 10 days, I’m totally ok with just basic 3D shapes bashing into each other. Here’s a screenshot of what I have so far:

“Video games are not real” -anonymous

That phrase sounds obvious, but it’s a good reminder to game designers who are creating scenes. The lighting, colors, shadows are all stylized version of the real world. Even ‘realistic’ games employ special effects.  

Pictured above: not real life - a video game

Now, some of these concepts are not unique to modern gaming. Films have been using color filters and lighting techniques to inspire specific moods. Painters have exaggerated lines, shadows and perspective for centuries. 

Pictured above: nope, not real life either- a painting

Even the photos on your phone are not exactly ‘real’:

Pictured above: a real photo.. with sharpening, increased contrast and 2 other Instagram filters, ok ok this isn’t 100% real either. Still looks nice though.

In the context of modern gaming we’re talking about Rendering and Post Effects. Below are some of the popular techniques:

Screen Space Ambient Occlusion (SSAO)

Source: http://forum.unity3d.com/threads/sessao-high-quality-ssao-w-color-bleed.323511/

As you can see, the simulated shadow allows you to see more details, especially along edges and cracks. You can read more about SSAO here.

Depth of Field (DOF)

Source: http://forum.unity3d.com/threads/dof-on-ios-how-we-did-it-mini-post-mortem.143887/

See how the character is in focus while the background is blurred? If you are into photography, you may already be familiar with this concept. Read more about DOF in Unity here.

Bloom

This adds an HDR glow especially to reflective surfaces and highlights. Here’s some information about Bloom in Unity.

VIDEO: Unity 2D scroller (TURN AUDIO ON).

In this gaming experiment I combine the elements of Flappy Bird (infinite runner) with Asteroids (2D shooter). For the graphics, I modified free vector art in Illustrator

Game Interface

Here’s a breakdown of the game interface:

We keep track of the player’s health in the upper left corner. Points are deducted when the players collides with an enemy.  In the upper right we track the number of enemies destroyed by bullets.

Parallax Environment

There are four 2D layers at work- the ground, hills, mountains, and sky. The hills and mountains both move from right to left,  at different rates.  Throw in some moving ramps and fast paced music and voila-- the illusion movement.

Ramp ‘Icebergs’ produce a varied terrain

There are actually only 2 varieties of ramps that are spawned at random heights in the ground-- think of it like an ‘iceberg’ sticking out of the water.

End Game Menu

When health reaches 0 the game is over. The player can try again or quit.

So there we have it- a very simple game!  That’s it for now, but I may revisit this game later on

“Unity is stupid.  You must treat it like a small child and give it specific instructions or it won’t work” And you know what, that is actually a good thing.  In fact that’s how all programming works. 

Let’s say we’re planning a top down shooter. A couple weeks ago, I would have said something like “Move the guy around and shoot stuff. Also make the enemies explodes.” It’s good start, but not enough information for Unity to make the game work. 

Programming games from scratch requires us to think critically about inputs and outputs, down to every detail.

For example let’s take the top down shooter. As a player, a top down shooter is very simple. You aim and you fire at enemies. However, as a programmer I have to think about every single object and interaction on the screen. 

How long should the explosion last when the laser collides with an enemy? What sound effect should play?  Those are some questions to ask. But even before you get to that level of detail you have to consider even more basic questions: What key does the user press to fire the laser? How fast should it move? How do I even make a laser?

Here are some of my thoughts from today’s class

Unlimited(?) game potential with simple mechanics

Everyone’s Flappy Bird homework looked different-- one was a mermaid swimming through the ocean and another was a Pokemon character floating through clouds. 

They all had the same basic mechanics but felt very different because of their themes. One project was a floating cup cake avoiding hungry mouths. When it hit the ground someone shouted ‘5 second rule!’ Hey that could part of a future game mechanic-- if you let the object sit on the ground fore more than 5 seconds, the game is over!

Learn to read documentation

Another great skill is learning how to reference documentation. There’s a vast library of feature available, and it’s impossible (and impractical) to memorize it all. It seems like we learn features ‘as we go’. That is, we a learn stuff as we need it.

Talking code out loud

 1) How many component communications are happening in your flappy bird game. Describe each one.

Sprite renderer talks to transform to know where to position/render

RigidBody talks to transform based on the physics engine

The Bird Controller Script talk to the RigidBody to tell it to apply a force

Collider talks to the rigid body to tell it where phyics should occur

 2) What are some ways you can identify variables and functions just by looking at them or using MonoDevelop?

You can look at the casing and icons in the intensense context menu. For example:

Property: ‘P’ icon and lowercase letters

Method: ‘M’ icon and camelcase letters

 3) What is a kinematic rigidbody? Give an example of when you should use it

When a rigid body is kinematic, the author moves the object, not physics. An example would be the main player in top down 2D game.

 4) What is a prefab? How can we use it in a game?

A prefab is like a template. It can be used to create various instances of the same game object.

 5) When do you type a semi colon in c#?

At the end of a line.

 6) In Unity, what do you call it when there is a collection of components that work together to achieve a purpose or multiple purposes, all contained on a single object?

   Components are in the inspector (?)

 7) In a Unity scene, what component is mandatory for a Game Object to have? Why?

Today we began our first class project, a  Flappy Bird clone. Here’s a screenshot of the official game:

We started with an introduction to  MonoDevelop, the IDE where we will be authoring our C# scripts.

During the design process, we had to consider some aspects of our game:

 User inputs - What does user press to make the bird move?

 Physics - How will the bird fly up and fall?

 Movement - How will the obstacles move from right to left?

We created a simple 2D sprite that is affected by the physics of gravity. For now, the sprite is a simple white ellipse, which we will replace with a bird graphic later in the process. That takes care of one game mechanic.  

Next, we made the 2D sprite respond to various inputs: mouse clicks, arrow keys, space bar and any key.

In Unit you add your script the way you would add a component. For example, we created a BirdController script and then dragged it onto the sprite. 

We created a 2nd script, ObstacleController, that is responsible for the obstacle’s movement. By the end of class we established basic mechanics, using basic shapes as placeholders. 

As you can see,  the square represents the obstacle which moves right to left. The circle represents the bird, which is controlled by the press of a keyboard button.

In this scene I experiment with physics and lighting. The falling spheres are objects with rigid bodies that respond to gravity. They collide with the objects in the center of the room, knocking over the pedestal.

In the background are baked colored lights that illuminate the back wall.

Unity is installed and we started becoming acquainted with the interface.  Panels, shortcuts, menus-- a familiar sight if you’ve used 3D software or Video editing programs in the past. I do like how there usually multiple way to access certain options-- top menu, keyboard shortcut or context menu. 

Next, we then experimented with 3D objects and physics.

Meshes, colliders and of course, Rigidbody ( tick the Use Gravity box)

Here are some thoughts from class:

A mouse is more useful than a touchpad when manipulating objects and navigating through 3D space.

Shortcuts will be learned over time. Through repetition the key combinations will become muscle memory

Baked lights are good for non-moving objects

We started by learning basic concepts of coding such as variables and loops.  We did some exercises creating shapes and patterns in an online Java editor-- I found this more satisfying than  producing a  line of “Hello World” text (which is typical Day 1 programming stuff).  

We also did some exercises on https://studio.code.org/s/course2.