#dealing with data

About a week ago, I posted an anonymous salary survey to try gathering compensation data from various industry contacts. I got 168 responses total and I’ve collated the results. I wasn’t too surprised by them. I’ll be posting more about them and breaking them down in other ways since I have a good amount of data to break down across different dimensions (seniority, discipline, location, salary statistics, etc.). That said, let’s get to the data.

First, we have the average salary by discipline broken down by seniority worldwide. The survey had only $150,000+ as the final bracket, so any bar with an average salary above $150,000 could be much higher. Artists saw the biggest jump in compensation to senior, while QA and Customer Support are the least compensated disciplines overall. Producers don’t really start pulling in the big money until they reach lead or executive status, while Programmers show a pretty regular and strong growth as they gain experience. This is just worldwide, however, so let’s take a look as we break it down by region.

I received the most responses from California by far, probably because the survey was circulated in a particularly large Californian game company’s internal slack. This encompasses both Southern California and Northern California. California is an expensive place to live! Pay across the board is significantly higher in California than worldwide averages. The only reported discipline in California that isn’t paid at least $50,000 per year on average is junior QA.

The salary for the rest of the USA is pretty low in comparison. These areas were broken down into the Midwest (mostly Chicago area), Seattle, Texas, East Coast, Utah, and Canada. Programmers continue to see the most reliable growth. Overall, you can definitely see that the general salary bands are lower if you aren’t in California, which corresponds to the cost of living. I do feel a little sorry for the folks from the Seattle area though, they have a cost of living approaching California’s and don’t get paid as much as Californians do.

Finally, we can take a look at Europe which is the aggregate of the responses I got from the UK, Ireland, Germany, Finland, France, and Sweden. The biggest thing you probably notice is significantly lower compensation across the board in Europe. Few disciplines pass the $50,000 annual mark. Even programmers, who tend to be the most all-around solid earners, barely break the $50,000 mark.

There were a few other responses from across the world - a handful from South America (Brazil), two from Southeast Asia (Singapore and Malaysia), one from Russia, and one from Iran, but I didn’t get enough responses to work them in. I would love it if I could get more data, perhaps from bigger Asian players like China and Japan, but this is still a pretty good first go. I think I knew the generalities, but never realized how low pay in Europe was across the board, or how much more California pays than the rest of the US.

[Join us on Discord] and/or [Support us on Patreon]

The FANTa Project is being rebooted. [What is the FANTa project?]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Long questions: Ask a Game Dev on Tumblr

Just because feedback is detailed and well-written doesn’t mean it is representative of the overall population’s concerns. This is because the things that hardcore players care about aren’t necessarily the same things everybody else cares about. Fans, hardcore or otherwise, all tend to focus on the certain specific elements of the game that they enjoy and dismiss the others that they don’t care for. However they also fall into the trap of thinking that the things they care about are the things everybody cares about. 

This isn’t to say detailed feedback isn’t valuable - it very much is. The more accurate information we have, the better equipped we are to make future development decisions. Well-written feedback, especially feedback about specific things you like and dislike about the game, is always read and internalized. However, just because we read it and internalize it doesn’t mean we’ll do what you want. We have our own development priorities, and that includes the budgetary realities of the game, things other players care about, things the developers themselves want to do, and so on.

Imagine asking hardcore football fans what they would fix about the football game experience and being told that the seats in the stadiums aren’t comfortable enough. That wouldn’t be applicable to address issues of the millions of fans watching broadcasts at home no matter how detailed the feedback about the seats is. There hundreds of fans who watch football games on TV for each one that goes to football games in person. This is the point of the 80, 20, 5 post - you have to realize that the things one group of hardcore fans complains about is not necessarily representative of the player base at large. It’s incredibly difficult to get accurate information about the entire player base as a fan. The only reason I know anything about it at all is because I’ve seen the numbers and the data for the various games I’ve worked on and there are common recurring behaviors among the audiences for them.

In closing, I’ll leave you with this little nugget of wisdom from our Discord server:

[Join us on Discord]

The FANTa Project is currently on hiatus while I am crunching at work too busy.

[What is the FANTa project?] [Git the FANTa Project]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Long questions: Ask a Game Dev on Tumblr

Whenever a program loads a file, it expects a specific file format for the type of file it is loading. For example, a bitmap (BMP) is literally a map of bits. It starts with a header section that declares file info like how big the image is, and then it goes into a list of RGB values for each pixel in the bitmap. If you have the size of the file and the pixel color data, you can reconstruct the entire image in any app that can read the file and understands how the data are arranged in the file. If the file data don’t match the expected format, the result may not work. This is why many other programs can read multiple file formats - Microsoft Word can read DOC files, but can also read XML files, HTML files, TXT files, RTF files, PDFs, and more. It’s because each of these file types has an established file format that the programmers decided to make Microsoft Word able to read and parse. 

Similarly, game engines also read asset data files. A 3D model from Maya is really just a collection of vertex and polygon information. A texture map is an image with some additional mapping data of pixels to vertices on a 3D mesh. An animation rig is mostly a collection of polygon-to-bone mapping data combined with bone movement constraints. An animation is mostly a series of bone positions at specific times. As long as our programmers know how those data are laid out in those specific files, we can make the game engine understand what those data mean. And... if we know how to make the game engine read files in that file format, we can also build other more lightweight tools like animation previewers to read those same file formats and manipulate them. 

I’ve worked with tools that allowed me to do more than just previewing animations - they allowed me to attach collision spheres on specific bones in specific animations on specific frames for purposes like the player attacking. We can then create a new file that stores which animation bones that collision spheres attach to on which specific animation frames, and then the game engine can read that data to know when to make attacks active based on animation data. We also made that tool mark which animations can be interrupted during which frames to create attack combo strings - Attack Animation A can branch to combo extender B between frame 35 and frame 60, which can then branch to combo finisher C or combo extender D between frame 24 and frame 56. This means that this tool can be used by combat designers like me to take the animation data from the animators and create and tune cool attacks and combos with the tool for use in game. This is just one example of things programmers can enable when they know how asset data get laid out in a file.

[Join us on Discord] and/or [Support us on Patreon]

The FANTa Project is being rebooted. [What is the FANTa project?]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Long questions: Ask a Game Dev on Tumblr

It’s not really technical resources so much as trying to keep track of (and verify) a whole bunch of game data at once. Checkpoint saves simplify things a lot for the programmer tasked with writing the save system, the designers choosing checkpoints, and can be helpful in guarding against broken game states. In order to understand why they’re helpful, it’s important to understand how restoring a saved game actually works.

Imagine you have a bunch of boxes of Lego bricks and a clean play space. Inspiration strikes and you decide to build a castle out of Lego bricks. You spend some time, carefully choosing which piece goes where and assemble your castle. However, you run out of time before you finish the castle - the west tower isn’t done yet and you have plans to build out the dungeon and courtyard, but you’ve got an appointment or something. After you leave, the automated Lego-cleaner robot takes the castle apart and puts all of the used Lego pieces back into their respective storage boxes for future use.

At this point, your in-progress castle is physically gone, completely disassembled into its component parts. You’ve got a clean slate if you wish to start building something new. But if you want to continue working on the same castle from before, you need to remember where each piece of the castle was before it was disassembled and place that same piece there. How difficult can this be to keep track of? Well, that depends on the number of pieces. The more pieces you have to remember to reconstruct the castle state, the more complicated the restoration process has to be.

This is basically how it works for saving game state. We have to figure out what pieces have been moved where, write them down, and then (later) completely reconstruct the entire game state from the bits of information we’ve written down.

If we’re allowing manual saves, it depends on the scope of what we’re saving. Do the location of enemy corpses matter? If so, we have to remember them. Do the visual variations of the enemy corpses matter? If so, we have to remember them. Do the items you’ve picked up matter? Do the items the enemies have dropped matter? What about enemy AI state - if you’ve broken stealth and the enemy knows where you are, does that get saved? And if so, do we save the AI state of all of the enemies? This also makes testing these save restorations potentially very difficult because we have to test that everything the players care about is the same as when they saved. The more data there is, the more complicated the testing process becomes. Imagine saving everything’s state in Breath of the Wild. How much would you have to keep track of?

As you might have guessed, this can be a real rabbit hole of potential edge cases. The worst case scenario is saving a state that is actually broken - imagine saving a fraction of a second before you get killed by a rocket or something. If we save the rocket information, restoring the save would get you killed 100% of the time, rendering the save data effectively useless to the player. 

“What does this have to do with checkpoints?” you might ask.

Checkpoints present a lot of known quantities. We can guarantee you’re at a certain location. We can guarantee you’re in good health. We might even require you not to be in combat to save, so that we avoid the inescapable death scenario above. Since we choose the location, we can guarantee that it’s safe. And, since we can lock off past areas as the player finishes them, we can discard any and all of the old data from those areas. A checkpoint save is basically a way to simplify and standardize a large number of game variables leading up to that point. When we mark a checkpoint, we need to keep track of significantly fewer elements. That makes it easier for designers to figure out what must be saved and what can be inferred, for programmers to write the code needed to save the relevant stuff, and for testers to make sure that everything comes back after restoration the way a player would expect. 

Checkpoints are a lot like using pre-built standardized sections for the Lego castle. It might not be exactly what you built before, but it’s close enough that the differences are functionally moot and makes things a lot simpler to keep track of. With checkpoint saves, we’re trading complexity and granularity for reliability and simplicity. Sometimes we prefer to keep that complexity and granularity of details (e.g. a game like Civilization), but other times the little details really don’t matter so much (e.g. a game like Uncharted or Spider-Man).

Related reading:

[How does a save file get corrupted?]

The FANTa Project is currently on hiatus while I am crunching at work too busy.

[What is the FANTa project?] [Git the FANTa Project]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Long questions: Ask a Game Dev on Tumblr

Sure thing. First the how: 

[here’s a link to the basics on what kind of telemetry data we gather]. 

If you want a deeper dive, try the [#telemetry-data tag] on this blog. 

Now the why.

Since games have existed, game developers have needed feedback in order to improve. The game can’t get better, more engaging, or more fun unless we get feedback on what we did right and what we did wrong. So we’ve been gathering information about what kind of games you like, what features in games you like, what sort of things you’d be willing to spend money on, what you dislike, what causes the most frustration, what you want more of, what you want less of, and so on and so forth. We’ve done this through many different means - surveys, focus groups, playtests, interviews, trade shows, word of mouth, reading feedback posts from social media and web forums, etc. 

Getting feedback is hard. Getting honest feedback is even harder. Getting specific feedback is really hard. Non-dev playtesters often lack the right vocabulary or context to describe their feelings adequately about the game after a playtest. They just don’t have the right context to put it into words, especially if they aren’t core gamers, and AAA+ games always aim for the broadest audience possible. Playtesters also tend to try to answer the way they believe they should feel rather than how they actually feel, especially in groups where they feel they’ll be judged by their peers. And it’s also super difficult to get them to remember specifics about the play session unless we really keep the play session laser-focused on just one thing. If a playtester plays a new game for twenty minutes, chances are good that they will have to pick up at least 3 or 4 different game systems, often many more than that. If we ask about something in specific, we often either get vague answers or nothing at all unless it was one of the major game systems (like combat) that they engaged with. And, of course, there’s no way that we can pull in millions of players for playtests without automation. We don’t have that kind of time or budget.

What does this have to do with monitoring players?

Telemetry data allows us to track what players do during play sessions, and that information honestly tells us what they were doing in the game. The data doesn’t tell us what we want to hear, it doesn’t try to make itself seem like a more hard core player, and it doesn’t ever forget the specifics that we look for.  That information is invaluable in telling us where players had the most trouble and the things they spend the most time doing. We don’t base all decisions solely on telemetry data, but it provides valuable missing perspective when we get other forms of player feedback that we can corroborate. It provides us a more accurate picture of the player experience so we can identify the problems that need fixing and the things that players enjoy most. We need that information in order to improve games. 

When you see things like quality of life improvements, more intuitive interfaces, a better tutorial process, more engaging character arcs, smoother quest flow, better dialogue, and overall hard-to-quantify gameplay improvements, you’re actually seeing the results of professional developers taking feedback from players at scale and incorporating it into their work. Much of that feedback and knowledge comes from gathering that telemetry data, because honest, accurate, and specific feedback is super duper hard to get otherwise. So… no, we probably aren’t going back to the age of unwatched players. That would make the gameplay experience much, much worse for everybody involved.

The FANTa Project is currently on hiatus while I am crunching at work.

[What is the FANTa project?] [Git the FANTa Project]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Long questions: Ask a Game Dev on Tumblr

Let’s take a higher level look at this. First off, what exactly is a save file? When you’re playing a game, you’re making decisions, you’re finding items, you’re getting new abilities, and so on and so forth. It’s like building a castle out of legos - this piece went here, that piece went there. Then, when you finish playing the game and exit, it’s like taking apart your castle and putting all of the lego blocks back into lego storage. There’s no castle there anymore. If we wanted to be able to restore that castle later, we have to know where all of the pieces you used were so that we can put everything back to where you last left it.

So let’s imagine that we’re building some kind of RPG and there’s a character that we want to save. Well, what data do we care about?

Character’s name, probably 

Character level, probably a number from 1-99

Character class, probably a number to represent which class it is (e.g. 1 = warrior, 2 = rogue, 3 = wizard, etc.)

Character’s total experience points, let’s say a number from 1-9,999,999

So, this would be an example of our theoretical save file in data - a chunk of information:

When the game reads the file, it would read the name - AAGD, then the level - 15, then the class - 3 (Wizard), then the next seven digits for experience points. Then we can reconstruct this character - a Wizard named AAGD that’s level 15 and has 90,801 experience points.

But imagine that the game accidentally write the wrong number into there someplace. Let’s say that we kill a monster and earn 200 experience points, so now we want to save this new data to the save file. But… something goes wrong and the memory address we’re writing to is off when we save it. So, instead of writing 0,091,001 experience to the right place in the save file, we accidentally write it 3 spaces too early, like so:

Suddenly, when the game reads our save file, it now thinks that our character is level 0, has an unknown class (9), and has 1,001,801 experience points. This save file has been corrupted, because the data in doesn’t make any sense anymore. If this save file is all we have to look at, there is no way for us to figure out what is correct because we have no record of what came before it. It’s important to note that some of this data is still usable - the name is still correct, for example. But because not all of it is usable, we can’t reconstruct the save file.

Some of you might be canny enough to realize that we can also use this information to cheat. If you know the format for this save file, for example, we can do something like this:

Instantly, this save file now represents a “legitimate” level 99 character with maximum experience.

This is basically what happens when a save file gets corrupted - the data that was in it no longer makes any sense, and there’s no way to figure out what should be there because we have no record of what came before. Some sort of bad data got inserted into it in such a way that one or more elements of the save file don’t work anymore, so you have to throw the entire thing out.

This week we continue the Design Phase of the FANTa Project!

[What is the FANTa project?] [Git the FANTa Project]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Long questions: Ask a Game Dev on Tumblr

It’s FANTa Friday! This week, I played around a bit (when I could, the project I’m being paid for is nearing a pretty big milestone) with the scripting language and spent a few hours familiarizing myself with some of the elements of the Skyrim Creation Kit. If you plan on following along, you should get yourself set up too. All of the files that will be necessary for the FANTa project should now be available [here on github]. I hope that today’s post will help you understand how the Creation Kit organizes its data which, should you decide to use it, allow you to learn how to organize and construct your game elements.

First, what I did - I built a quick and dirty spell that throws a magic energy ball that force pushes NPCs away from you, knocking them on their asses. It’s named “Force Push” and you can grant it to yourself after installing FANTa.esp that you should be able to download from [here]. If you have Skyrim and want to give it a try, install the mod (installation instructions [here], start the game, open up the console, and run “player.addspell 05000D62″. That ought to get you started. It doesn’t work on regular objects. Yet.

Think of Skyrim’s architecture as a set of building blocks with a hierarchy. Our Force Push prototype is a Spell. A Skyrim Spell consists of data like an ID, a name, a type, a means of casting, which hand you can equip it on, casting animations, etc. However, the Spell itself doesn’t inherently do anything by itself. Instead, it functions like a bag that holds one or more magic effects.

A Magic Effect is a collection of even more data. Here, we can set up different elements to a particular effect that can be used as part of a spell (or even reused as part of a family of spells). The effect has its visual and auditory components like what they look like when activated, when they hit, what kind of sounds to play at various times, what flags to set on nearby NPCs (like “turn hostile”), etc. The effects can dispel other effects, interact with particular schools of magic, or even require specific conditions be met before working. All of Skyrim’s current spells are built from multiple Magic Effects firing when the spell is cast. Beyond this, each effect can also run one or more scripts. 

Scripts are where we can start really flexing the creative juices. This is where we can make the game do explicit things with the code that don’t explicitly exist. In the Force Push script I wrote [here], you should be able to see what I did - I calculated the direction from the caster to target (plus moved it slightly upward) and used that as the direction of the havok force to apply to the target with a magnitude of 500. I tried 5000 at first, but that actually ended up shoving the NPC so hard she went through the ground. If you want to start playing with things, you could start by messing with some of those values. It should be easy to figure out how to edit the Force Push script to shove people upward or in a specific direction.

Remember, we’re still just prototyping stuff. None of this is set in stone yet. Our goal with any prototype is to get something together that’s close enough to what we want, even if it’s quick and dirty. It isn’t balanced, it isn’t pretty, and it doesn’t do everything I want... but it does provide me something that’s close to our concept that we can play around with it to find the fun. And... sending enemies or even regular NPCs flying is pretty amusing so far. We can use this, for example, to create a puzzle that involves shoving an NPC into a specific trigger zone. Next week, we’ll continue with the prototyping to try getting some form of High Jump working.

P.S. Feedback is important for a project like FANTa. I know that FANTa is supposed to be not quite a tutorial, but I also want to make sure that I’m providing enough in these posts for people to follow along without getting lost along the way. Please let me know if I’m skipping over/assuming too much, if I’m not moving fast enough, or just let me know I’m going at a good level of detail to get to what we need to do. The last thing I want is to leave behind the people that FANTa is for.

This week we continue the Design Phase of the FANTa Project!

[What is the FANTa project?] [Git the FANTa Project]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Long questions: Ask a Game Dev on Tumblr

Quick Announcement - the vote for the Follow-Along Project ends TONIGHT AT MIDNIGHT PACIFIC TIME, so [get your vote in if you haven’t yet]. It’s been an extremely close race so far, and I’m looking forward to seeing what we end up doing.

Good question! Things vary quite a bit from game to game, but there are certain asset types that tend to be much bigger than others.

The smallest files on disc tend to be code - libraries and executables are always compiled down to very small sizes. All of the code that runs a game like Call of Duty often gets compiled down to only a few megabytes after optimization. 

Animations tend to be more middle-of-the-road. The size of animation data is a factor of how many bones that are being animated. The complexity of the animation skeleton is the determining factor of how big animation data gets. The more bones you have to animate, the more data you need to track where those bones move relative to each other from frame to frame.

Audio and pre-rendered video compression have improved significantly recently, allowing us to compress reasonably high quality audio and video into more reasonable file sizes. We tend to reuse many different sound clips and music in games, so audio usually doesn’t take up a lot of disc space unless it’s a game with a lot of dialogue. Using pre-rendered video has its share of issues (as evidenced from yesterday’s post), and it can also cause performance hitches as we load the video player software, so we tend to minimize the amount of pre-rendered video as much as we can.

The biggest data hogs tend to be models and textures. For performance reasons, we often build several different [LODs (Levels of Detail) of models], loading and unloading each one when necessary. This sort of model duplication means every different model has several different versions we have to store on the disc. Similarly, these models need texture maps - flat 2D images that are mapped in 3D onto specific polygons on the displayed model. These texture maps can also come of varying resolution for more or less detail. 

Originally, the texture map was just a direct view of the what was on the model but advancing 3D technology has added additional types of texture maps to provide more information - [normal maps, specular maps, ambient occlusion maps], etc. While these are not directly seen by the player, the engine processes this data to provide more context to what is being displayed. 

As you may have guessed, [a single character model might have 3 different LODs, each with a texture map, a specular map, a normal map, and an ambient occlusion map]. That’s 12 (or more) sets of textures and 3 models for just the one character. This also includes other in-game objects like tables, chairs, cups, swords, bottles, doors,  guns, windows, trash cans, computers, train cars, rocks, and food. This also includes the environment, which needs its own set of data for the shape, textures, and level of detail (from a distance). 

I’m purposely not getting into the whole nature of data compression, because that is a whole different can of worms. But this should give you some idea of just how much work is needed to create the assets for a game. Just modeling the character isn’t enough, you need four different models for levels of detail. Just texturing the character isn’t enough, you need to make the other maps too. And so on and so forth. Each of these assets that we create has to be on the disc somewhere for the game to read in and use to display on screen.

We’re going to do a follow-along project, either designing a Game System or Designing a Level! [Click here to vote for which]. Voting ends TONIGHT, so get your votes in while you can!

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Long questions: Ask a Game Dev on Tumblr