#weaving factory

We all know the feeling of pure and utter stress when, ten minutes before the deadline, you finally click the Build And Export in your game engine and then it fails to build for some reason.

To prevent issues like this, people invented automated builds, also sometimes referred to as CI/CD.

This is a mechanism that tests the project every single time a change is made (a git commit is pushed, or a Perforce changelist is submitted). Because of this, we very quickly know whenever a change broke the build, which means we can fix it immediately, instead of having to fix it at the end.

It is also useful whenever a regression happens that doesn't break the build. We can go back to a previous build, and see if the issue is there or not. By checking a few builds by way of binary search, we can very quickly pinpoint the exact change that caused the regression. We have used this a few times, in fact. Once, the weaving machines stopped showing up in the build, and with this method, we were able to pinpoint the exact change that caused it, and then we submitted a fix!

It's very useful to have an archive of builds for every change we made to the project.

There are multiple different softwares that do this kind of thing, but Jenkins is by and far the most used. Both by indies, but also large AAA studios! So knowing how it works is very useful, so that's why I picked Jenkins for the job. Again, like with Perforce, it was pretty easy to install!

Here is a list of every build run that Jenkins did, including a neat little graph :)

Configuring it was quite tricky, though. I had to create a console terminal CLI command that makes Unreal Engine build project. Resources used: (one) (two)

It took many days of constant iteration, but in the end, I got it to work very well!

I also wrote some explanations of what Jenkins is and how to use it on the Jenkins pages themselves, for my teammates to read:

Dashboard (The Home Page):

Project Page:

Now, it is of course very useful to build the project and catch errors when they happen, but if no-one looks on the Jenkins page, then no-one will see the status! Even if the website is accessible to everyone, people won't really look there very often. Which kind of makes the whole thing useless… So to solve that issue, I implemented Discord pings! There is a Jenkins plugin that automatically sends messages to a specific Discord channel once the build is done. This lets everyone know when the build succeeded or failed.

We of course already had a discord server that we used to discuss everything in, and to hold out online meetings with. So this #build-status channel fit in perfectly, and it was super helpful in catching and solving issues.

Whenever a build failed, people could click on the link, and see the console output that Unreal Engine gave during the build, so we could instantly see where the issue was coming from. And because it rebuilds for every change that is made, we know for certain that the issue can only have come from the change that was just made! This meant that keeping every change small, made it easier to find and fix problems!

Whenever a build succeeds, it gets stored on the server, in the build archive.

But storing it on the server alone is nice and all, but people can't really do anything with them there. I could personally access them by remotely connecting to the server, but I cannot make my teammates go through all that. So I needed to make them more accessible for the whole team, to download whenever they please.

For this, I wanted to create a small website, from which they can download them. Just by going to the link, they could scroll through every build and download it immediately.

I already knew of multiple ways of easily doing this, so I tried out a few.

The Python programming language actually ships with a built-in webserver module that can be used very easily with a single command. But Python wasn't installed, and installing Python on Windows is kind of annoying, so I wanted something else. Something simpler.

I often use the program "darkhttpd" whenever I want a simple webserver, so I tried to download that, but I couldn't get it to work on Windows. Seems like it only really supports Linux…

So I went looking for other, single-executable, webserver programs that do support Windows.

And so I stumbled on "caddy". I'd actually heard of it before, but never used it, as I never had a need for it before then. For actual full websites, I've always used nginx. After some time of looking at the official documentation of caddy's various configuration file formats and command-line arguments, and tweaking things, I had it working like I wanted! It now even automatically starts when the computer boots up, which is something that Perforce and Jenkins set up automatically. Resources used: (one) (two) (three) (four)

And I think that's it! Unfortunately, none of this will roll over to the next team that has to work on this project, because none of this is code that is inside our project folder.

Here’s a summary of the setup, drawn as a network map:

Future

For my study, I am currently working with a team of other students on a VR experience for a local museum.

The museum plans to have a VR headset somewhere, which visitors can put on to experience how life was like in the past.

There are lots of things to show, so the user needs some way to move from one place to another.

Sometimes, this is done with a Controller, where the user can move the joystick, which moves the camera around the virtual world.

This has a big problem, however: such smooth movement in VR is incredibly nauseating for people who are not accustomed to it, which is definitely most museum visitors.

As such, game designers in the past invented the now common mechanic of teleportation. You point the controller somewhere, click a button, and you get teleported to where you were pointing. The range is usually not very large, to keep it balanced. I have also seen a slightly different mechanic in some VR games, where there isn't a button to teleport, but it is the joystick. You point the joystick somewhere, and you can choose which direction you want to end up in after the teleport.

The concern is that this can be a bit tricky to learn how to do.

Especially if we decide to not use any controllers. To keep things simpler, we would like to not need controllers. Putting on the headset is already complicated enough, and we need to keep this experience as accessible as possible. Controllers can also get stolen or damaged.

Luckily, there are ways to do this teleportation with hand gestures only!

However, these gestures are even more complicated than the controller buttons for it.

So we racked our brains over a really nice way to handle moving around the environment.

Finally, I had an idea!

We can designate specific Points Of Interest in the environment, which will be the only places the user can be at. Then to move from one POI to the other, the user can just look over to such a POI, and they will teleport to it.

Of course, that teleportation should not be instant, otherwise that will still be confusing and disorienting.

So if the user looks at a POI, a small animation will play in front of them to show that something will happen shortly. (If they look away again, the countdown and whole teleportation process will be cancelled, naturally. They really need to stare at the POI for a solid few seconds, to make sure they really want to do it.)

After that animation finishes playing, the screen will fade to black and back. During the darkness, the user will have been teleported. The teleportation is not instant, again, to prevent disorienting the user.

I mocked up a small visual example of how this system could look like in Blender, and showed it to my team. They unanimously agreed with the system! 🥳

Here is that animation:

You will see that the Points Of Interest are marked with blue holograms, which become taller if the user looks near them.

Working together on a game is quite difficult. How do you make sure everyone is on the same page?

You could send zips with assets and code between people, but that is slow, cumbersome, and very error-prone.

This is why Source Control software was invented. With Source Control, you can store the project files on a server, and everyone who wants to work on the project files, can download the project from the server. Once they're done with their changes, they send them back to the server. Then the other people can get those changes from the server again.

There are many different Source Control softwares. Probably the best-known one is git. (Source Control, Version Control, Revision Control all mean essentially the same thing.) Git is very good for working on projects that are all just plain-text code. However, it kind of falls flat when it needs to deal with binary assets, like textures, models, and sounds. Unreal Engine Scenes and Blueprints are also binary files. So if you don't use C++, but only Blueprints, basically your entire project is binary files.

This is why I went looking for a different Version Control Software that can handle this better. At my internship, I used Subversion, which is said to work better with binary assets, but in my experience then, it still wasn't exactly great. Though that might also have been due to the fact that they barely used any of its fancier features there.

Unreal Engine itself recommends using Perforce Helix Core. (Perforce is the company, Helix Core is the VCS, but the term "Perforce" is usually used to refer to the VCS itself, due to historical reasons. Source) So I looked into it, and indeed, it seemed very suitable!

So we requested a computer from the XR Lab to use as server for this project, and I installed the Helix Core Server on it.

It was surprisingly easy to install!

I then downloaded the Helix Core Client application on my laptop, and connected to the server. The setup there took a while, because there was a lot to learn. But in the end, I did it!

Sadly, Perforce is not free, so we are forced to use the free version, which is limited to a maximum of five users. We are with ten people, so we had to choose a few "representatives" who would actually put the things everyone made into the project. We requested an educational licence from Saxion, but got told to wait. Now that the project is over, we still don't have it. We also requested an educational licence from Perforce itself, directly, but we have still not got a response. But we made do with the limitations we got.

I wrote a guide for my teammates on how to set up a workspace for the project with Perforce, and improved it multiple times based on user testing and feedback. (I sat next to my teammates while they were following the guide and tried my best to now say anything; to let the guide speak for itself. I would then take note of what went wrong and improved that section of the guide for the next user test.)

These are a few excerpts from a recording of the first time I managed to get the experience running on the standalone VR headset.