#programming architecture

Code systems are code systems. Good code architecture tends to follow pretty universal principles, regardless of whether the system is governing loot tables or lighting systems. Constructing software systems is about seeing the general rules at work and using those to write code that adheres to those rules.

It really helps to take a larger view of what a game is from a software engineering perspective. A game (or any major piece of software) is a bunch of systems comprised of smaller subsystems, and how those individual systems interact with each other. All code systems all need to do three things:

Determine when the system needs to do its work

Return the processed results from that internal work to external systems that need those results to do their own work

Do their own internal work to process and handle requests correctly

When planning out what a system will do, it helps to divvy up the work into these three buckets. Once you know what the system needs to do, the engineer can break down the individual functions and data members she'll need in order to actually do that work.

Let's move forward with an example - say that I wanted to build a stealth takedown system in my action adventure game. The design document says that I want the player to be behind an unaware enemy, press a button, and then play a paired animation that kills the enemy. Using the three buckets mentioned previously, let's break it down.

When does the system need to do its work?

Player and enemy position

Player and enemy facing direction

Enemy awareness state

Game controller input state

What results do I need to provide?

I need to know when the player meets the conditions of being behind and facing an unaware enemy (call the UI system to show the button prompt)

I need to call the animation system to play the animations on the player and the enemy (call the animation system to play the animation)

I need to kill the enemy (call the damage system to kill the enemy)

What do I need to do the internal work to provide those results?

I need to calculate whether the player is behind the enemy

I need to calculate whether the player is facing the enemy

I need to determine whether the enemy is aware of the player

I need to know when the player presses the attack button

As each of these elements is built and works, we can use them to interact with each other. Logical checks like whether the player is behind the enemy will determine whether the action can be taken. Actions like performing the takedown animations are then attached to the conditions. These combine to form the rules from the designer and a system is born.

[Join us on Discord] and/or [Support us on Patreon]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Short questions: Ask a Game Dev on BlueSky

Long questions: Ask a Game Dev on Tumblr

These discussions boil down to assigning tasks to the processor that is best suited to handle them. GPUs are very good at a smaller set of tasks, but cannot do others at all. CPUs can do all of the tasks, but aren’t as good as the GPU at the tasks the GPU is good at. This means that all of the calculations that the GPU doesn’t want to do (i.e. is bad at doing) should be handled by the CPU. With this in mind, if there are more calculations that need to be done than the GPU can handle (and thus requiring the CPU to pick up the extra calculations), the game is GPU-limited. If the GPU is idling because there aren’t enough calculations for it to process while the CPU is still running at full load, the game is CPU-limited.

GPUs are specifically optimized for processing 3D graphics. This usually means they are very good at performing floating point math operations - usually a lot of addition and multiplication of floating point numbers really really fast. You may have heard of the term "FLOPS” (megaflops, teraflops) before. FLOPS are FLoating point Operations Per Second. Floating points are how we store precision decimal numbers in data. A FLOP is a mathematical calculation run on some floating point numbers. FLOPS is how we measure the amount of work a GPU can do over a unit of time (like a second). GPUs are very very good at performing a lot of small mathematical calculations in parallel (i.e. at the same time), which is also makes them so sought after for things like bitcoin mining in addition to processing computer graphics.

Everything else that isn’t 3D graphics or mathematical operations usually gets handled by the CPU. This usually means handling things like creating and destroying objects, determining which data to process, and handling more abstract concept tasks - pathfinding and AI decisionmaking, netcode, procedural content generation, saving and loading, that sort of thing. A general rule of thumb is that the GPU does all the math, while the CPU has to decide what math needs to be done. The CPU can also do the math if/when needed, but not as fast as the GPU can.

Are you in Professional Game Dev? Please fill out the [Anonymous Salary Survey]!

[Join us on Discord] and/or [Support us on Patreon]

The FANTa Project is being rebooted. [What is the FANTa project?]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Long questions: Ask a Game Dev on Tumblr

I posted something about server tickrate a little bit ago. Here’s a couple of Blizzard’s network engineers from about two months ago talking about their networking systems and how they handle latency and server tick rate. Note that they call it “client update rate”, which is probably more accurate. The server tick rate is the rate at which the server does its internal game calculations. The “client update rate” is the rate at which the server sends out all of the current world state to all of the connected clients, which isn’t necessarily tightly coupled with the server tickrate.

Tickrate, for those who don’t know, is the speed at which the server of a multiplayer game updates itself. This has much to do with engineering and optimization because a high tick rate runs into potential performance issues. The higher the tick rate, the smoother the experience will feel because it means the server and game logic will react faster to the events that transpire during play.

When running a multiplayer game, the server’s job is to tabulate and process all of the events and things that happen during a given frame (or tick). For example, let’s say that during this frame of gameplay, Slagathor fires at Elliot while Molly crouches and primes a grenade. The server must:

Process all movement updates (including animations) for all connected players and any NPCs to update orientation, position, and state

Update the gun data (rounds in magazine, total rounds of ammunition) and any tabulation involving tracking needed statistics - kills, shots fired, accuracy, etc.

Play Slagathor’s firing animation to update Slagathor’s hit boxes in case somebody decides to shoot at Slagathor

Check if Slagathor hit Elliot, which means doing the following calculations:

Check if Slagathor’s aim vector intersects Elliot’s current hit box. If the answer is yes, then do the following:

Handle damage calculations to Elliot

Queue Elliot’s damage and/or death animation for being shot

Track the grenade data for Molly (how long it has been held)

Prep and send data packets to send to all of the players so that their clients can play the appropriate sound/visual effects and update their view of what the other characters are doing (animation, position, etc.).

As you can see, this can get really complicated really fast. The server has to handle all of the game’s logic for all of the connected players and all of the AI and everything else. The higher the tick rate, the less time it has to do all of these calculations. Worse still, if the server isn’t dedicated (i.e. the server is also a player), the game also has to handle all of the client calculations for all the visual stuff too. In addition to all of this, the server also needs to package all of the data and send it to every player each tick so that the players can see what is going on. Doubling the tick rate doubles the bandwidth the server needs to run.

If you’ve got a server tickrate of 20Hz, you’re looking at 50ms of processing time between each update.  Doubling this to 40 means that you’re halving the amount of time to process - down to 25ms per frame to update. Having to update the entire game state in half the time can be a potentially difficult optimization problem, especially for a potential difference of only 25ms in reaction time for the player. Most humans can react in around 130-170ms to visual or auditory stimuli, and even then you need to factor in network latency as well, since that is time in addition to your human response time that you must spend waiting for the server to send you information. Thus, the gain from an increase in tickrate is generally minimal unless you’ve nearly eliminated network latency to begin with (<30ms). If you can do that, then a tickrate increase will have a much more visible effect. However, this is likely to be a very small fraction of the player base who can actually take advantage of this, and only those with very low latency will see the benefit. Since the vast majority of players will be playing online, the increased tick rate will only be beneficial for two main situations - competitions on a closed network, and those players who have really good connections that result in very low latency to the game servers.

The main benefit of added experience is the increased understanding of how systems interact with each other. Typically, this means that a more senior engineer would be given more responsibility with how a given system interacts with other systems. For today, we’ll talk about implementing a game system that you could reasonably expect to give to someone of (practically) any level of experience - an achievement system.

We can start by looking at the sort of planning and breakdown - how would engineers of varying skill levels attempt to approach a problem like this?

A junior engineer would probably begin just by looking at what it’s supposed to do and work his way backward. You need to notify the user via a popup. You need to track which ones in specific are complete, and which are partially complete. You need to work with the platform’s existing code to handle achievements (Steam, XBL, PSN, etc.). And then you’ll also need to develop some debugging tools at the very least to reset your achievements in order to test them for functionality. Typically, each of these tasks would take a couple of days to a week - that’s time spent researching the technology, learning how the code systems interact with each other, and then building it out. The junior engineer would also most likely be the person who handles the creation of the achievements himself, since it is unlikely others would be able to pick up the system and understand it very easily. I expect that the code would probably not be very portable - written to handle a lot of special cases as they crop up, rather than architected to handle a variety of cases from the get-go, simply because the junior programmer didn’t realize such things would be a concern when he began. I would expect a junior engineer or an intern to finish out the system over the summer - around two months.

A mid-level engineer would probably approach it from a more data-driven perspective. He would already be cognizant of the sort of constraints of other platforms, and understand how they already work. He’d know that such a system would need debugging tools to start with, and roughly what sort. He’d understand that similar systems would probably already exist to handle similar tasks (localization, for example), so he could model her system to work like those, and possibly cannibalize or reuse aspects of those systems to architect her own system. The data-driven design would start showing up here - it would make it easy to add new types of achievements, and the system would be taught to the designers so they could handle the creation of the actual achievements themselves with the interface and tools the mid-level engineer creates. This way, he can spend her time on other things, and only needs to return to this code if there are bugs that appear. I would expect this to take significantly less time overall than the junior engineer since I would expect the mid-level engineer to prepare for most of the pitfalls along the way, and he would need to spend a lot less time researching the code. I could see this system being finished in a few (3-4) weeks - designing the architecture, writing the systems, and then documenting it and training the designers to use it before handing it off to them.

A senior engineer would probably look at it from the perspective of how it interacts with other systems. She would most likely embrace the data-driven aspects of it and take it a step further - how does this system work with others? What similarities does it share? A senior engineer is generally able to take advantage of a broader knowledge base as well - she can figure out how the other systems work together, so rather than simply using them as a jumping-off point, the senior engineer would simply go in and rewrite an existing one to handle what she needs. The main difference between the senior engineer and the mid-level is just that the senior engineer has had more experience, so she knows exactly how to build such a system - there’s almost nothing that is unfamiliar to her about how the code works. All you have to tell her is “a system to handle X” and she’ll already know all of the relevant issues that may spring up, as well as what questions to ask. A senior engineer would be able to get this system up and running in about a week, with a second week spent wrangling designers for specifications and then training those designers how to use it. In addition, the system would probably be portable enough to be used by other teams on other projects without too much extra effort.

Because of the mastery involved with the code, a senior engineer’s time is usually better spent directing others and assigning tasks. That way, you get the most out of everyone - the senior engineer can use her knowledge to direct the mid-level engineers into areas of their expertise where they can shine, and can parcel out tasks small enough for the junior engineers to tackle that (most likely) won’t lead to the mistakes of assumption they would tend to make otherwise. There’s an ongoing joke in software development - the more senior you are, the less code you actually write. And it’s true - you spend more time reading and understanding the way things work than you do actually writing new things. The code that you write, however, is much more critical than the code your subordinates will be writing. At the senior level, you’ve already seen the solution that the more junior folks may not see. All that’s left is to direct them on how to write it. 

~~~~~~~~~~~~~~~~~~~~~~