#sound programming

Wow, what a difficult week. Here’s a list by day of what I did, and then I’ll explain in a bit more detail:

Implemented Basic File I/O

Added Keyboard handling and moved the message handling to its own function; set up my compiler options to handle win32 compatibility, force warnings to errors and eliminate warnings that I didn’t need.

Split up compiling for x64 and x86 into different options in the build batch; unified keyboard and xinput controller handling

Forced frame rate to 30 fps

Improved audio synchronization

Started debugging the audio sync and drawing out the debug info to the screen

Split up the game code from the platform code in the compiler; compiled the game code into a DLL for on the fly code update changes

Day 1

The important thing here was to create some basic file loading code so that we can, next week, start to load some bitmap files to the screen and start to manipulate them. We added the ability to read, write, and handle the file memory correctly.

A lot of this is still debug code, but it’ll work for my purposes in the short term. One interesting takeaway: “__FILE__” returns the path to the current file, which for testing purposes allowed a quick way to test a file loading, reading, and writing.

Day 2

Today was a lot of explanation of compiler options and what they do. We treated warnings from level 4 and above as errors to make sure we’ve fixed things like possible data loss issues from implicit data casts. At the same time, we eliminated some warnings that we don’t care about, such as warnings that a struct/union doesn’t have a name and others.

The rest of the day was spent implementing handling keyboard messages so that we can properly utilize the keyboard to do the same work as the controller.

Day 3

I set up a VM running WinXP and split out the compile options for x86 and x64 so that we can attempt to run on XP. Luckily I had VirtualBox already installed for a Linux distro, so setting up WinXP was a cinch.

Some of our work for getting the game to run on WinXP was dropping the usage code from 64-bit in some places where it was unnecessary down to 32-bit. Getting it working with as minimal effort as we took was very cool.

We also spent some time with the controller code so that we handle a keyboard as the first controller, and also to set the rest of the controllers to act as digital input devices like a keyboard.

Day 4

We set up the code to limit the game to 30 fps. Not my personal preference, but I may end up setting it to 60 later on in development. In doing so, we utilize QueryPerformanceCounter() to get the wall clock, or highly-accurate time code for determining how long a thing takes to run in the game.

Next, we determine our intended frame time (33ms) and then set the process to Sleep for whatever amount of time is remaining until we reach that point. To do that, we have to release the process to the Windows Scheduler, which can be very coarse for our purposes (multiple milliseconds). Well that’s not going to work for us to have consistent frame rate, so we had to call timeBeginPeriod(1) to force the scheduler to update our process at a 1ms rate.

Day 5

We implemented debug visualization of our sound timing, both with the play cursor and with the write cursor for our given sound loop. We went through a few iterations of this between now and tomorrow, but the intent here was to see what our sound latency was like, as well as find out how consistent we were being with our sound writing.

Day 6

So we spent a long time debugging the audio sync and updating our debug draw to match so that we could find out how to make sure that our audio synchs well with slow and fast sound cards. A long time with very dry coding, a lot of which I still don’t quite grasp.

We pulled our debug drawing info out into a struct and function to handle it better. We also hooked up a debug pause command so we could stop the animation and see what was going on.

Here’s what it looks like when it’s running.

Here’s a clearer picture of what we’re tracking when we pause it.

Okay, so here’s where it gets complex, and I may not even be explaining it 100% correctly, so I apologize ahead of time if that’s the case. Our buffer acts as a circular buffer, so when we reach the end of it, we start writing to the beginning of the buffer, over and over, so the cycle is just getting the information from the current positions inside that buffer.

Looking at that second image, we’re drawing the Play Cursor (white), which is where our audio is currently playing. Our Write Cursor (red) is where we are writing the next bit of data to be output in the next frame update. The Play Window (fuchsia) is a variable range where our play cursor could be at any given time; in our case, it’s plus or minus 5 milliseconds (I’m only drawing it at 5ms ahead because as I understand it, we don’t really care if it’s 5ms behind).

So the first line is our total output, where we display the last 30 frames of data. The second line is our output location for the current play and write data. The third line shows the byte we expect to lock (for writing) and the total amount of data we intend to write (between the white and red). However, because of the latency issues of the sound output due to the sound card, it’s difficult to always be 100% certain where we’re going to be writing to. That’s where the Play Window comes in as we might be farther ahead or behind based on our expected graphics flip (for which we can be more certain).

So where the Play Cursor actually ended and where we are writing to may be different (fourth line). It’s hard to tell from the animation, but sometimes the expected and actual line up more closely than they are in the sample frame. Here’s Casey’s note on it:

We define a safety value that is the number of samples we think our game update loop may vary by (let's say up to 2ms).

When we wake up to write audio, we will look and see what the play cursor position is, and we will forecast ahead where we think the play cursor will be on the next frame boundary.

We will then look to see if the write cursor is before that by at least our safety value. If it is, the target fill position is that frame boundary plus one frame. This gives us perfect audio sync in the case of a card that has low enough latency.

If the write cursor is *after* that safety margin, then we assume we can never sync the audio perfectly, so we will write one frame's worth of audio plus the safety margin's worth of guard samples.

(For more accurate explanation, definitely watch the video for this day. It’s long but Casey does a much better job explaining it than I’m sure I did.)

So far, sound programming seems to be the most difficult thing in the world. I do not envy those programmers out there who work on this stuff for a living, because I still don’t 100% understand what’s going on with it.

Day 7

Lastly this week, we set the game code to be compiled into a DLL so that we could load and update it dynamically while running the game. This way we can have the code function like a script engine, where we can make a change, recompile the game code, and have it auto-update. That’s pretty friggin’ sweet, and even though this is the first day’s worth of implementation, I’m excited by the direction that’ll lead us with the rest of the game (particularly for iteration and testing; holy shit the rest of you designers-turned-programmers, this is where it’s at!).

Summary

So, a lot of excellent stuff going on this week, particularly with the audio stuff (even though I don’t fully grok it), and the game code live update. It may have been a bit longer than a week, but real life called a couple times, so I was a little bit slower, but I’m still here.