#game engineering

If you want to sit back in your chair and ruminate on what you would have done better in the new Final Fantasy game, more power to you. But don’t mistake your intellectual exercise for making games, and if you truly want to make games for a living, you will need to do much, much more.

You will start not knowing anything about anything. But everybody else starts there too.

Once you have decided to make the jump from wanting to make games to making games, you will soon face the unpleasant truth that you have no idea what you are doing. You don’t know how to create and tune tight gameplay systems. You don’t know how to implement a camera and render a scene. You don’t know how to model a character, bone a rig, and make a reload animation.

I taught myself how to program in high school, had an internship at a huge Seattle tech company, and got my bachelor's degree in Computer Science at university. I thought moving to programming games would be a snap. But it wasn’t. Because making games is not like making anything else. I knew how to implement a linked list, but I didn’t know how to pool objects to prevent memory allocation. I knew how to calculate the trajectory of a cannonball, but not how to distill that motion into an update loop so the game could show the ball travelling across a scene.

These are not things that get taught in standard high school and college courses. Making video games requires a lot of specialized knowledge about how to merge the axes of art, technology, and design to create a fun, engaging experience. Don’t expect to be able to jump in straight away and make your dream game. I wish it were that easy, but part of making games is learning that you are constrained by what is possible and by what you know how to do.

Luckily, there are a lot of people who will help you figure out what you’re doing

Making games is hard, but if you’re dedicated, it’s easy to find resources to help. Full, professional video game engines are available to you, right now, for free. Unity, Unreal, GameMaker, and a plethora of smaller tools can be found with a simple web search, and most won’t charge you one slim dime to start developing.

Along with these tools, there is a vast trove of tutorials to help you get started using them. YouTube tutorials, quick start guides, and blog posts about individual problems are all at your fingertips. There are also in-depth guides like the Handmade Hero series to help an aspiring game creator get their bearings and start creating simple content. If you are dedicated, you can find a description of how to do just about anything you’ve ever seen in a game you’ve played.

Beyond these static resources, I’ve found a number of game creators are very open to discussing their work with an enthusiastic amateur. I am almost certain I wouldn’t have a job in the industry today if I hadn’t repeatedly reached out over Steam to a AAA dev I met playing Dota, or if I hadn’t sent an email with my portfolio to a game designer I follow on Twitter asking for feedback. Your mileage may vary, but it is exponentially easier to build up your skill-set if you can talk to somebody who has already made the journey.

You will be judged on what you have done, so do something

This may seem like a catch-22: to get a job in the video game industry, you must have already made video games. It’s hard, but the truth is that anybody looking to hire a new employee wants to know what they are capable of. If you can’t show anything off, the message is that you aren’t capable of doing anything.

But there are ways around this barrier. Many game companies look at mods to judge a junior candidate, so if you make Halo maps or Skyrim mods or Civilization scenarios, take screenshots, preserve source files for download, and slap them on a portfolio. If you’re an artist, make unique, imaginative scenes and characters, and build up a demo reel. Even if your work is garbage, put it together to show you can do something and keep working to replace your early trash with newer, better content.

If you can find a team of people to start making simple games, even better. Very few people are capable of executing every part of a game concept as well as they like, so if you’re a programmer and you know an artist, team up and start making bubble bobble clones until you have better ideas.

I cannot stress this enough: having nothing to show for yourself except a dream and a degree is much, much worse than having a portfolio full of small, simple mods/art/games that you’ve seen through to completion.

Games are made by teams, so learn how to be a team player

Almost nothing you’ve ever played was made completely by a single person. World-class programmers, artists, and designers know where their strengths lie and know how to find people whose strengths complement their own. If you want to make a game you are proud of, you are going to need to work with other people, and they’re going to need to want to work with you.

Be humble. Be kind. Listen when others bring up their own ideas and concerns, and be considerate of them. Contribute your own ideas, but don’t become so attached to them you tear a rift in your team. Not every game you make is going to be your dream game, and some of them might not be games you would even play. But if you’re constantly negative about the work, people will notice and they won’t want to work with you.

And always give your best work. Every game you work on is an opportunity to hone your craft, to learn new techniques and methodologies, and to create something you can be proud to say you worked on. There will be times where you won’t be able to make something as good as you would like it to be, but there is never an excuse to deliver lazy or sloppy work, even if you can get away with it.

Transitioning to Hands

In IEYTD, the core idea is that the player interacts with objects in the environment to solve puzzles. With the mouse, you are constantly using telekinesis, during which you can aim at objects using a reticle and interact with them. This mode allows players to interact with objects that are farther away from them. As we transitioned to hands, we did not want to lose this ability, but at the same time, we wanted players to experience the beauty of holding an object in VR.  As a result, we came up with two modes for hands: Local Grab Mode and Telekinesis (TK) Mode.

Local Grab Mode

The ability to grab objects using virtual hands is a powerful experience and supporting this mechanic was crucial to IEYTD’s gameplay. In the game, the player’s virtual hands are represented as a pair of spy gloves along with animations to show different hand poses. In this mode, when the player grabs an object, the hands fade out completely. This is done for a couple of reasons. The first, and more obvious reason, is scope. There are a large variety of objects that can be picked up and manipulated in IEYTD. Therefore, creating a unique animation and grab pose for each of them was not feasible. Secondly, the way objects are held varied based on the person and the context in which they were being used.

As the player’s hands explore the environment, the objects that are “grabbable” become highlighted. We predict this with the help of a trigger volume and a raycast from the player’s head to their hand. The trigger volume helps maintain a list of grabbable candidates and the raycast helps us select the object that is closest to the player’s head. The assumption we make here is that the grabbable candidate that is closest to the player’s head, is the one that is most likely to be interacted with.

Telekinesis (TK) Mode

The TK mode is our solution to allow players to interact with objects that are not within their reach,  allowing our puzzles to be more flexible and making it possible to create more diverse environments. It also solidifies the game’s super spy theme because it adds an additional “spy-like” element to the players’ arsenal of tools.

For TK mode, our goal was to translate the reticle based on interaction from the mouse controls to the hands. To achieve this, we parent a reticle to each hand and assign it a forward vector. The reticle is then positioned in 3D space along this vector using a raycast against the level’s collision geometry. This implementation worked in most cases; however, there were a couple of undesirable consequences.

The first one was that the reticle moved considerably. Since it was parented to your hand, the slightest hand movement would cause it to move. At greater distances, this movement was magnified, and ironically, made it difficult to accurately target far away objects. To address this issue, we add a “smoothed reticle” that follows the original reticle using a damping curve. The amount of smoothing applied is inversely proportional to the displacement of the original reticle. As a result, small displacements of the reticle are smoothed out and large displacements are immediate.

The second issue was a consequence of how we render the reticle in the game. In order to avoid clipping issues and to give a 2D feel to the reticle, we render it on top of all world geometry. While this solution works well when the reticle is parented to your head, it wasn’t the case with hands. The player’s perception of the reticle’s position was severely impacted. The problem was most noticeable when there were objects at different depths along the player’s line of sight. To the player, the reticle would appear to be on top of the object that was closest in their line of sight, when it was actually on top of an object at a different depth. This discrepancy caused confusion amongst players as to why they couldn’t interact with the object they were targeting. To solve this problem, we do a raycast from the head to the reticle and detect if there is an interactable object along its path. If an object is detected, we snap the reticle in front of it.

Hands Collision Model

A huge challenge with hands in VR is how they interact with the world from a physics standpoint. Game physics is an approximation and when you combine it with virtual hands, whose motion is not constrained by world geometry, terrible and often funny things happen. The hand collision model is made up of two aspects: Hand Physics and Held Object Physics.

Hand Physics

In IEYTD, the hands do not have colliders; they only have trigger volumes to grab objects. Players can push their virtual hands into geometry and it will clip right through. At one point, we considered giving visual feedback when this happened, but eventually didn’t do it because we felt that it would alert the player even more to the fact that their hands were inside geometry. Lastly, we did not want players to be able to push objects forcefully out of closed containers, especially ones that were locked away and crucial to the puzzle.

Held Object Physics

Determining the physics behavior of objects held by the player is an interesting problem. Our first iteration was a model where held objects collided with the environment, and when the collision amount was above a certain threshold, the object dropped from the player’s hand. At first, this solution seemed to work well, but as the levels and interactions became more complex, it made object interaction frustrating. We had stacks of cash, cigars and grenades placed inside tight spaces. When the player grabbed these objects, especially during speed runs, the tight spaces would cause the physics system to constantly try to resolve penetration, resulting in severe physics glitches. And often, such penetration caused the collision amount to exceed the object drop threshold, resulting in the player dropping objects unintentionally. It was frustrating and became a hindrance to puzzle solving.

Our second approach was to turn off collision when the object was picked by the hand. Early tests were promising because you no longer saw physics glitches when grabbing objects from tight spaces. But, it did come with the caveat that players could now stick objects into geometry, and upon release, they would bounce into unpredictable locations as physics tries to resolve the penetration. However, in our playtests, it was fairly uncommon and most of the times when it happened, people wouldn’t notice it.

There was still one issue we had to solve with this approach. Since we turned off colliders when you picked up an object, it no longer detected collisions. As a result, we lost the ability to break objects. This change affected our puzzle solving since breaking objects is a core puzzle mechanic and secondly, it is immersion breaking to not acknowledge such an impulsive action. In order to address this issue, we turn on colliders on the held object based on the hand’s acceleration. Therefore, if the player moves the held object quickly, we turn on its colliders for a brief moment, thereby allowing it to send and receive collision events. This solution worked well since it was an intuitive action to perform when breaking objects.

Lastly, the colliders are never turned off on the held object when the hand is in Telekinesis Mode. We experimented with this, however it felt very unnatural to TK objects in and out through geometry without any form of resistance. It also had a higher chance of losing objects inside geometry.

Ears of a Super Spy

Audio plays a crucial role in delivering a good VR experience. At its core, IEYTD is a puzzle game, however it is also a physics sandbox. Players are able to stack objects, throw them around, break them and even shoot them. As a result, these objects collide with the environment a lot and it was important that they make believable collision sounds when it happens.

In order to achieve this audio interaction, we designed a system that allowed us to tag environment colliders as Soft, Hard, Glass and Metal surfaces. The image to the right shows a paint over of the different surfaces in the Car Level. The sound designer created two collision sounds (normal and heavy) for each type of surface, and for every object that can be picked up by the player. The idea is then to adjust the volume levels of these two sounds on impact, based on their collision velocity, and then play them simultaneously. Our initial tests were positive, and with some additional tweaking, we were able to get the system to produce a good approximation of surface-based collision sounds. The following diagram is an overview of how the system works.

Mistakes of a Super Spy

Hidden Volumes

As mentioned earlier, IEYTD is also a physics sandbox. It allows players to have fun with physics while they try to solve a puzzle. We love this aspect of our game; however, it comes at a cost. Players can easily lose an object that is crucial to the puzzle and put the game in a stale state.

We decided to solve this problem using a technique that games have done before: have an x-ray shader on objects that are hidden. At first, we had concerns about it breaking the player’s immersion, but then again you are a super spy with telekinetic powers, so x-ray vision didn’t seem too farfetched. We did not want all occluded objects to use the x-ray shader, since it would compromise objects that are hidden from a puzzle standpoint. Instead, we built a system that allowed us to define approximate areas that are not reachable by the player. We did this using trigger volumes and named them Hidden Volumes. When an object falls into this volume, an x-ray shader is applied to it, allowing the player to see it through geometry.

Next, we needed a way for the player to be able to interact with objects inside the Hidden Volume. In order to do this, we put the objects into a separate physics layer, thereby allowing us to create a layer-specific raycast to detect and prioritize their targeting. This solution allowed the players to pick up objects through geometry.

Blind Volumes

As the name suggests, the sole purpose of this volume is to reduce the visibility of the player when they peek into places that are either a crucial part of the puzzle or simply lack any geometry. Under the hood, they are just trigger volumes that are placed in the affected regions of the level. These trigger volumes, by default, cause the screen to fade to black when the player’s head enters it. In other scenarios, it was more convenient to set up a single Blind Volume that caused the screen to fade to black when the player’s head exited it.

Compartment Lids

In IEYTD, we have a number of compartments containing objects that the players can pick up. In order to prevent them from grabbing these objects without opening the compartment door (or lid), we put the lid collider on a special physics layer. Using a raycast against this layer, from the player’s head to their hand, we detect if this collider is hit. If it is, then we deduce that the player is trying to access an object inside a compartment without opening it. When this happens, we simply disable the player’s ability to grab objects.

Power of VR

The development of IEYTD has been a challenging and rewarding experience. The techniques described here are the result of countless playtests and iterations, and knowledge shared by the VR community. VR is a powerful medium that allows us developers to entice players in new and exciting ways. And, it does a pretty darn good job of transporting them to a world we’ve created. Here’s hoping that IEYTD gets people closer to their dreams of being a sophisticated and responsible Super Spy.

I think the argument could be made about building a kind of camaraderie with the people you are working with. Especially since that job wasn’t exactly fun. We showed up, we knew we had a job to do, and we powered through it.

Interesting. So what brought you into the video game industry?

Well, I have always liked video games. I originally had an Atari, and later I talked my parents into getting me a Commodore 64. One of the caveats from my parents was- since it was relatively expensive- that I needed to learn and do more stuff with it than just play games. And that meant learning programming.

Oh cool.

They were excited at how I was able to manipulate data and code, and get objects to appear on the screen. Then in college, I knew that I wanted to do programming of some sort. I graduated college with a double major in computer science and math from Assumption College. But then it was ’how do I get into games?’ Keep in mind, this was the early 90’s, and there weren’t the type of game programs that there are now. I mean, I took zero classes specific to game programming. I did a lot of the learning on my own, taking information from different books and magazines. The career of a ‘game programmer’ was really abstract. I knew that people could do it, but I didn’t know how. I ended up answering an ad in a newspaper for an entertainment company called Humongous Entertainment.

No way?

Yes. They called me back and I got the job. I had other job applications out there, but this was the offer that came back. I was doing tax software at the time, so I was able to make the jump from programming tax software to making video games.

It was a nice change.

Yes, I bet! Game design and engineering sounds a little more interesting. From there, what brought you to Schell Games?

We were thinking about moving back to the east coast from Seattle to be closer to family. During that time, I found out that my brother-in-law in Pittsburgh was working at Carnegie Mellon. He got me into contact with Jesse [Schell]. We exchanged a couple of emails, had a call, and later, when I was in Pittsburgh on a family visit [my wife is from Pittsburgh], I stopped by to see the studio, had an interview and it went well. Everything lined up, so we packed up from Seattle, and moved out.

Good deal. With your experience being in the video game industry for a while, you see that it is changing pretty rapidly. From where the video game industry is now, at this moment, where do you see it going in the next five or so years?

Now that is a super hard question, and I wish I had a smart answer for it. In reality, I don’t know. It is all changing so fast. If you would have asked me five years ago if I thought Schell Games would be so focused on VR (virtual reality), I would have thought you were crazy. To me, it is less about where I think the industry is going, and more about staying open to opportunities as the industry moves along. 

Okay, that makes a lot of sense.

That’s the great thing about Schell Games. We don’t get married to one single technology, one single style of game or platform, and we’re willing to try a lot of different things. We’ve built a reputation that people would say, “Well, that’s a weird idea” or “That’s a weird piece of technology. Let’s see what Schell Games can do with it.” It has served us well, because we are able to jump on whatever new exciting technologies crop up.

Since I started at Schell Games, we’ve transition ourselves three or four times. One of my first projects was Pixie Hollow, a flash-based, MMO for kids. At the time, flash-based and MMOs were super hot. We probably did three or four of those projects. Then as flash development cooled down, we started using the Unity platform, went hard into tablet games like Lexica, and we did a few more mobile games. Now, though we still dabble in mobile games, we’re very focused on VR. I’m not even talking about the work we do with theme parks and other location-based games.

As for our VR development, we worked on I Expect You To Die for about two years. And even before that, we had some experiments on the DK1 VR headset, when it was on Kickstarter. So our ability to experiment with things like VR and other technologies is pretty cool.

Cool. Have any experiments helped Schell Games recently?

Absolutely. We’ve been playing with the HoloLens, and have been discussing what type of experiences could be done well for that technology. Tango is also a good example. We sent a couple of devs to Google’s campus for a tech/game jam-type of workshop. On Tango, we experimented with creating a Jenga game, and Google loved it. They had us go back, clean it up a little bit for CES, and that experiment led directly to our Domino World game.

What do you like about working for an indie studio in general, and Schell Games specifically?

We have a great group of people here and it means a lot. I like the fact that we do a lot of different things. We’re not on an assembly line, making the same game over and over again. Although, don’t get me wrong, I’d love to have a hit franchise.

Of course.

There is something very compelling about the variety of tasks and challenges here. As far as an indie studio, it is kind of the same thing. We’re a ‘big’ small company. It’s a tough industry though. You see other studios laying off staff, and studios going out of business and closing down. For Schell Games, we’ve been able to keep this thing going for nearly 15 years.

That’s a real feat.

The fact that we’ve been able to make it this long, without any major layoffs or big ‘scale-downs,’ is atypical for our industry. And it goes back to Jesse, because he really cares about the people. He’ll make choices that will defend the people of Schell Games, not just defending the business of Schell Games. And sometimes it’s a hard balance, because there are cases where those goals don’t align. Sometimes it would be easier to lay people off, but we try to get the best group of people, grow them, and keep them.

You mentioned that with the diversity of projects and things that you are working on, there are challenges. How do you stay on top of your craft and your discipline while managing your team?

Yes, it is really hard to maintain excellence in your craft when you are a full-time manager. I would like to say that I program all the time on my own. In reality, I do a little bit, but it’s not enough to keep up with the people who are doing it full-time. For me, it’s about reading, going to conferences, and learning and absorbing as much as I can. I need to be aware of what’s out there; to understand concepts rather than implementations; to know what things are trending; and to learn what people have tried, succeeded and failed at. The most important thing for me now is to be able to recognize talent. It’s much more important that I can identify a good programmer than be a good programmer. I have to put people in the right spots. That’s my job. If I can assemble teams that work well together and build great content, I’ve succeeded.

Right. Makes sense.

You want to be able to hire people smarter than you. Hire smart people and put them in the right place to succeed. If you do that, everything gets easier.

Excellent. When you got onto the management track, what concepts or lessons did you have to learn on the job? Or were there things about leadership you didn’t know before you got into a leadership position?

Yes, there are a lot of lessons. I have been in leadership positions for quite a while, from project direction to studio leadership. Going back to one of the studios I was at before Schell Games, I had a very good engineering manager. I thought she was interesting because she was not a programmer. I had her as a manager and she was incredibly transparent, saying, “look, programming is not my thing, but I’m here to help manage the department.” She would focus on team interactions, professional development and career paths, and she would solicit feedback from others to understand if a person was a good engineer, or not.  

Very interesting.

It is something that I have always taken with me. The person managing you on a project should not be the one managing your career. Project goals and career goals often don’t align. If someone said “hey, I want to learn this new thing” or “I’m a little tired of database stuff, I want to do something different,” career-wise, it makes sense. If your project manager had to make that choice, it is a much harder sell because that manager is looking at the health of the project, not the health of your career.

That’s a good point.

Normally that tension is going to be there. Being able to manage it is something that I picked up.

Early on, when I was starting as a Technical Lead, I was often the best programmer on the team. The first thing I had to learn was, it doesn’t matter how good a programmer you are, you’re not going to be doing more programming work than the five people underneath you. It’s not important that I’m busy programming, it is important that *they* are busy programming, and then I fill in where I can.

You have to set them up for success, and let them leverage your knowledge to make them more efficient. It is more important than me pounding out code.

(Matt describing the engineering department during the 2017 Spring Open House)

Yes, I see that.

As responsibilities increase, being open and honest with people becomes very important. You learn that hiding from a difficult conversation isn’t going to help. Have those conversations. Be very forthright. You get your credibility from how you follow through on things. If I say “okay, I’ll take that up with the VP group,” close the loop. Even if the answer is no, I could close the loop by saying, “Hey, we tried to get the extra staff, but we can’t do it for ‘x,y,z reason.’ I realize the project is difficult, here’s why we’re doing it this way and why this supports the studio. I will work with you to get through this issue.” 

If you don’t follow through, close those loops, and do what you said you were going to do, you lose that credibility.

All very good points. Where do you think Schell Games is going to be in five years?

It’s hard to say. As a manager, I go through all the scenarios, right? The worst case, the best case, nothing is out of the range of possibilities. So, for the sake of argument, the worst case scenario is that we stay ‘as is,’ which wouldn’t be a very bad case for us. We keep some level of work, a level of internal projects, and level out. But I can easily see us doubling in size in five years. We put ourselves in the position where we have one of the top titles in VR in I Expect You To Die. If somebody mentions VR games, our game comes up. It’s a good place to be.

It’s definitely in the upper-echelon, for sure.

Right. Thinking back to when CD-Rom games were new, sales for these games started very slowly, and then took off. Myst was a huge beneficiary of that trend. The place where we want to be is the “Myst of VR.” 7th Guest was a step down from Myst, but still a hugely successful game. We can be at that same level with I Expect You To Die. It would really do a lot for us.

When you are thinking about the future, what do you think is unique being a part of a leadership team like the one at Schell Games?

I’m not sure if it’s unique, but I feel we spend a lot of time thinking proactively about company processes, and how they relate to the games we’re trying to build. We’ve grown from 12 employees to over 100 since I’ve been here. While not perfect, I feel we’ve avoided a lot of pitfalls along the way. As a result, the teams have been able to focus on making great games.

(Happy Atoms, a digital and physical chemistry modeling set, is one of the many projects the studio developed in 2016)

What’s your favorite video game? (that you’ve played, or admire for its design, art, etc.)

I’m going to go with The World Ends With You. It was a DS title, and was a weird, little game about collecting and battling with pins. It had a cool mechanic and an interesting story. I played it for a looooong time trying to get 100% complete. I think I still have one pin to get.

What advice do you have for people trying to get into the video game industry?

First, make stuff. Either with friends or solo. Your early stuff will be bad, and that’s okay. Focus on the learning. Try and build something you can show. Secondly, learn some code. Even if you don’t intend to be an engineer, knowing a little code will give you a lot of freedom.

Last question: What, in your opinion, makes the leadership team at Schell Games, click?