#software architecture

There's been a lot of fascinating drama around Stop Killing Games. Go read the initiative here:

It is a good initiative, and anyone who is a consumer that can, should absolutely go support it.

Jason "Thor" Hall, CEO of Pirate Software, recently had a few, let's say, "takes" on the matter (I'm trying and failing to remain neutral), which began on a stream. The stream's VOD has since deleted on his YouTube channel.

Louis Rossmann, who you might know as the largest Right to Repair activist in the US, made a response to a section of the releevant stream here:

Thor, CEO of Pirate Software, made two videos to clarify his points:

There is an argument in the video at the 2:08 mark that I will reference later.

(I recommend watching all these videos on 2x speed. You will get the same info out of them all, because especially video 2 is a lot of repetition)

Now, as mentioned above, there is one particular technical argument that bugs me about what Thor, CEO of Pirate Software, is making. Here is the full quote:

How would you keep League of Legends in a functional, playable state? You'd have to rearchitect the entire game. The game is what is called "client-server". So, in client-server models, there's a server, there's a client, and all of the math, all of the game, everything happens on the server. The client just displays it. And the reason we want to do it that way is so that you can't teleport around and do a billion damage. You don't trust the client. You trust the server. The client just displays what it's told. Right? So, if we wanted to rearchitect this, we would have to take all of that server logic, push it back out into the client, and somehow make that playable in a multiplayer-only video game. That doesn't make sense to me. So this doesn't work for all games. Why is [the initiative] calling out all games?

So, first off, yes, most games do client-server architecture for multiplayer logic, because you do trust the servers. It is an important step to curbing an entire class of cheats. It doesn't necessarily mean the client isn't malicious (for example, there are cheats for League of Legends that show a growing circle when an enemy leaves the fog of war in the minimap). However, it does mean the client doesn't know 100% of the game at any time when information is selectively fed to each client based on something like the fog of war. That's awesome.

Some games, like PlanetSide (rest in peace) and Overwatch (2) use what's called client-side hit detection. Some games, like Halo 1, employ more selective hit detection models, where only certain weapons use client-side hit detection (see https://c20.reclaimers.net/h1/engine/netcode/). Client-side versus server-side hit detection can change the overall feel of a game, and it's one of the things game developers decide on in multiplayer-only games that require it. In the case of an massively multiplayer online first person shooter (MMOFPS) like PlanetSide (2), the server simply can't calculate thousands of people's math in a reasonable amount of time, because otherwise the hit detection would otherwise feel very crappy to play, and so the math is offloaded to the client and the client says "hit" when they hit.

However, there are a few counterexamples to the specific technical argument that keeping the game playable after end-of-lifing it requires rearchitecting:

Games with dedicated servers exist - Command & Conquer: Renegade, Starsiege: TRIBES

Games where one client also hosts the multiplayer server exist - Half Life 2, Warhammer 40k: Space Marine

Private server hosting exists - World of Warcraft

Some of these games, particularly the examples with dedicated servers that can be run on user hardware, can also run as the second example.

To say keeping a multiplayer-only online game requires rearchitecting a game like League of Legends means a lack of imagination. More relevantly, it means a lack of systems thinking.

To me, it is very strange for someone such as Thor, CEO of Pirate Software, who is self-described as being a 20 year veteran of the games industry to say. I won't say skill issue, because I think there is an ulterior motive at play.

Just to hammer the point home, I drew up some crappy diagrams in Inkscape because this extremely wrong technical argument bugged me so, so much.

Here is what a client-server model looks like:

Here, you have 10 clients, each being a player of the game. Then, you have the server, run by Riot, the developer and maintainer of League of Legends.

Here is the imagination of Thor, CEO of Pirate Software, had to say on the matter on the required way rearchitect it:

Those who know their network models would understand this looks very much like a mesh network, or a peer-to-peer model. And, to be fair, some games might attempt it.

However, this isn't *usually* how games described using a peer-to-peer (P2P) model work. Most peer-to-peer models, like the architecture used in Space Marines, are often used for matchmaking. Once you are in a game, one of the clients also serves as the host (selecting by some algorithm, like randomly or whoever has the best hardware).

P2P is nice, because the company doesn't have to run servers for matchmaking at all during their lifespan (and sometimes a matchmaking server might be spun up to serve as a relay to help with network issues or help other clients find clients quickly). As we'll get into later, a client machine will also serve as the host machine. It is a perfectly fair and valid, although it comes with it's frustrations (mainly in the realm of network address translation (NAT) traversal, because your computer behind a router is not usually exposed to the wider Internet, though sometimes routers have universal plug-and-play (UPnP) set up, which makes NAT traversal much easier here).

If you've ever seen a message in the game "migrating host" because the host left, they likely use P2P matchmaking, but still use a client-server model. They can just migrate the game data to a new host using the data on the other clients as a seed for the data.

This is likely their setup for actual gameplay:

One of the clients now has a server on the same machine. Sometimes, this could be the game itself that would serve in singleplayer. However, most often, this is just a server that's lightweight enough for the client to connect to and they play that way (it's also really nice to develop and QA this way, because many server bugs will also be seen by the client).

Now, one of the disadvantages here is: Can all remote clients connect to the host that the server (and one of the clients) is running on? Again, NAT traversal issues usually play a role here. In the first few days of any game that uses this, and only this, there will likely be a lot of issues with connectivity.

Another disadvantage: The host won't have latency issues. This is why in the case of, for example, Among Us, the client host can see certain things happening (like someone is dead the moment they hit a button or reported a body), but remote player hosts might not.

Okay, so, maybe it's not possible to rearchitect something like League of Legends like this. It could reasonably be a lot of work. Here is another solution:

Looks very similar to the first architecture, doesn't it? It is! The difference is that the text "Riot" was changed to "not Riot".

This is how World of Warcraft and Pokemon Go private servers work.

-me

Never mind that "clean" code can be slow.

Off the top of my head, I could give you several examples of software projects that were deliberately designed to be didactic examples for beginners, but are unreasonably hard to read and difficult to understand, especially for beginners.

Some projects are like that because they are the equivalent of GNU Hello World: They are using all the bells and whistles and and best practices and design patterns and architecture and software development ceremony to demonstrate how to software engineering is supposed to work in the big leagues. There is a lot of validity to that idea. Not every project needs microservices, load balancing, RDBMS and a worker queue, but a project that does need all those things might not be a good "hello, world" example. Not every project needs continuous integration, acceptance testing, unit tests, integration tests, code reviews, an official branching and merging procedure document, and test coverage metrics. Some projects can just be two people who collaborate via git and push to master, with one shell script to run the tests and one shell script to build or deploy the application.

So what about those other projects that aren't like GNU Hello World?

There are projects out there that go out of their way to make the code simple and well-factored to be easier for beginners to grasp, and they fail spectacularly. Instead of having a main() that reads input, does things, and prints the result, these projects define an object-oriented framework. The main file loads the framework, the framework calls the CLI argument parser, which then calls the interactive input reader, which then calls the business logic. All this complexity happens in the name of writing short, easy to understand functions and classes.

None of those things - the parser, the interactive part, the calculation - are in the same file, module, or even directory. They are all strewn about in a large directory hierarchy, and if you don't have an IDE configured to go to the definition of a class with a shortcut, you'll have trouble figuring out what is happening, how, and where.

The smaller you make your functions, the less they do individually. They can still do the same amount of work, but in more places. The smaller you make your classes, the more is-a and as-a relationships you have between classes and objects. The result is not Spaghetti Code, but Ravioli Code: Little enclosed bits floating in sauce, with no obvious connections.

Ravioli Code makes it hard to see what the code actually does, how it does it, and where is does stuff. This is a general problem with code documentation: Do you just document what a function does, do you document how it works, does the documentation include what it should and shouldn't be used for and how to use it? The "how it works" part should be easy to figure out by reading the code, but the more you split up things that don't need splitting up - sometimes over multiple files - the harder you make it to understand what the code actually does just by looking at it.

To put it succinctly: Information hiding and encapsulation can obscure control flow and make it harder to find out how things work.

This is not just a problem for beginner programmers. It's an invisible problem for existing developers and a barrier to entry for new developers, because the existing developers wrote the code and know where everything is. The existing developers also have knowledge about what kinds of types, subclasses, or just special cases exist, might be added in the future, or are out of scope. If there is a limited and known number of cases for a code base to handle, and no plan for downstream users to extend the functionality, then the downside to a "switch" statement is limited, and the upside is the ability to make changes that affect all special cases without the risk of missing a subclass that is hiding somewhere in the code base.

Up until now, I have focused on OOP foundations like polymorphism/encapsulation/inheritance and principles like the single responsibility principle and separation of concerns, mainly because that video by Casey Muratori on the performance cost of "Clean Code" and OOP focused on those. I think these problems can occur in the large just as they do in the small, in distributed software architectures, overly abstract types in functional programming, dependency injection, inversion of control, the model/view/controller pattern, client/server architectures, and similar abstractions.

a rant that ended up flowing into "structured privileges"

Thinking about the execve(2) interface of UNIX, particularly with elevated privileges - especially in combination with the UNIX philosophy of little tools that do small jobs well.

This generalizes, actually - the same ideas of privilege segregation could apply within a process if languages and runtimes took advantage of modern kernels' support for it - at least in Linux, where processes and threads are basically the same kernel primitive, with just a few configuration differences.

Anyway, it bothers me that the default behavior is that if I start a root program, everything it executes is also root. In some sense, the ideal default for a program with a privileged effective UID and an unprivileged real ID is that the execve system call drops privileges - and if you want to keep privileges, that should be an explicit flag.

Even more ideally, this wouldn't be a flag that you, as the program with privileged effective UID or effective capability set or whatever, get to just set however you want - this should be some sort of object, like a file descriptor or a 128-bit random key or a pointer into read-only mapped memory, which is given to you by the caller who gave you elevated privileges. So your options are either "drop privileges" or "keep any privileges that your caller allowed you to keep" - notably absent from your options is an unconditional "keep privileges".

Of course, one problem here is that at some levels of abstraction, forking another process should be an implementation detail - when I call a program as root, I shouldn't have to know "this program wants to fork a child", let alone details like which child programs, how many, how deep the process tree goes, and so on.

So execve isn't exactly the right boundary here, at least not always. But here are a couple examples where the execve boundary is user-facing and explicit, and is the right boundary:

Suppose I want to run a privileged `find` program - because it needs to look inside directories that only root can read, and then I want to run some un-privileged command per file path - the most secure thing, especially if I don't have time to audit the command's source and build and binary for problems, is to prefix my exec arguments for find with something like sudo -iu "$USER" so that the child process of find drops privileges back down to my user before running the command.

For another example, suppose I want to run something like fzf on a list of paths, and then I want fzf to use `bat` to preview those files. bat could need read-only root privileges if some of those files are only root-readable, but there's no great way to give those privileges to just bat instead of fzf - at best, I run sudo once before running fzf, sudo caches my credentials, and then I set the preview argument of fzf to include a sudo call. But this isn't friendly to building larger programs: if the fzf call is inside some larger script, that script now has to either unconditionally include that sudo call, or include it or not based on either a flag that is passed in or if it already has special privileges.

Going back to how this generalizes, this same problem happens with libraries - instead of calling into another program, you call into some library, but either way, you probably don't have time to audit the whole thing, and it often doesn't need all the elevated privileges that you have - if I've got code to parse YAML inside my process, those instructions don't need any privileges besides access to the input bytes and enough memory to return the parsed result, and yet I have to go out of my way to isolate them and drop privileges if I want to be safe from some exploitable parser bug.

Ideally, code should have to go out of its way to explicitly take privileges that it either needs or can optionally use, as a kind of parameter. The whole system, from the ground up, would be built with APIs which have a bias towards dropping privileges and which require every privilege to be passed in from higher up in the call tree. Then the path of least resistance aligns with security: if you're writing code which doesn't need privileges, you don't go out of your way to take them from the caller - if you need privileges to do something, you have no choice but to reveal it in the API.

The only wrinkle here is that people should be able to write middleware which doesn't know or care about privileges, but is transparent to them. The simplest solution is to do what Go did with its context type - force middleware to take that parameter, and pass it through just in case the thing it calls needs it. This is... better than privilege use being invisible, but it's still pretty bad, especially when we generalize it to a granular "permissions" object, since middleware would need an arbitrarily permissive type: instead of knowing if we're calling something that requires privileges, we are back to calling something that might use any of the privileges we fed into it, and we can't know without inspecting the middleware.

In languages with templates, the permissions could be expressed as a template parameter, and then it's at least clear that the middleware takes whatever permissions its callee requires - and someone higher up in the call stack is defining the callee, so they have the responsibility and knowledge of those permissions. But this still isn't really enough, that's just a nice way to make the type system transparently propagate information about elevated privileges to the places where a developer is most served by knowing it.

But at runtime, we need a distinction between a usable privilege, and an unusable "sealed" privilege which is just being passed through code from a higher caller to a transitive callee which has what they need to unseal it.

This has been on my mind for a couple years now, at least. The overarching theme here is that ideally software has a reified representation of all interesting privileges, which ideally is both visible in the type system and actually enforced at runtime, and that code should be forced to be explicit whenever it uses or passes privileges to do anything.

How to write clean code by Uncle Bob (Robert Cecil Martin)

This is a short preview of the article: This is a collection of videos by Uncle Bob (Robert Cecil Martin) on how to write clean code. He is an American software engineer, instructor, and best-selling author. He developed several software design principles and co-founded the Agile Manifesto. There is more to object oriented

If you like it consider checking out the full version of the post at: How to write clean code by Uncle Bob (Robert Cecil Martin)

If you are looking for ideas for tweet or re-blog this post you may want to consider the following hashtags:

Hashtags: #Agile, #CodeBestPractices, #CommentsInCode, #ProjectManagement, #RobertMartin, #SoftwareArchitecture, #UncleBob

The Hashtags of the Categories are: #Programming, #SoftwareEngineering, #Tutorial

How to write clean code by Uncle Bob (Robert Cecil Martin) is available at the following link: https://francescolelli.info/tutorial/how-to-write-clean-code-by-uncle-bob-robert-cecil-martin/ You will find more information, stories, examples, data, opinions and scientific papers as part of a collection of articles about Information Management, Computer Science, Economics, Finance and More.

The title of the full article is: How to write clean code by Uncle Bob (Robert Cecil Martin)

It belong to the following categories: Programming, Software Engineering, Tutorial

The most relevant keywords are: agile, code best practices, comments in code, project management, Robert Martin, software architecture, Uncle Bob

It has been published by Francesco Lelli at Francesco Lelli a blog about Information Management, Computer Science, Finance, Economics and nearby ideas and opinions

This is a collection of videos by Uncle Bob (Robert Cecil Martin) on how to write clean code. He is an American software engineer, instructor, and best-selling author. He developed several software design principles and co-founded the Agile Manifesto. There is more to object oriented

Hope you will find it interesting and that it will help you in your journey

Hi! I have two main hypotheses about this.

First hypothesis: Themes.

It is entirely possible that some themes just hide the notes that a particular post has. So, if you’re seeing a post page, you might be looking at the blog theme, and the theme might just not be showing the posts. This does not apply if you’re looking at the Tumblr Dashboard or the Tumblr mobile version of a post -- those don’t have custom themes.

Sometimes the theme design just didn’t consider notes, or sometimes they only considered a certain amount of them. Or it might just be coded weirdly. Furthermore, the Tumblr theme coding allows you to show any arbitrary subset of notes. I am not exactly sure why you would choose to only show some notes, but that’s something that you can absolutely do.

Here are Tumblr Docs on the code that themes use. Right now the notes are put on the page all or none (or loaded by AJAX, meaning that they will be loaded after the page has loaded). Still, custom JavaScript (code) or CSS (styles) on the page is something that you control and that gives you, as a coder, power to hide any of them.

Second hypothesis: Eventual consistency.

I wrote a bit about it in an old answer about the no-notes post, but here goes another try.

Tumblr is a really big platform. As such, it is impossible for Tumblr to keep everything running from a single server. This means that they need to provide the content from different servers at the same time. This also means that some of that information (such as the notes for a post) must be present in multiple places at the same time… and that means, in turn, that when a new note comes in, it will be sent to one of those first. If you look at the post, and the second server is the one showing you the info, then you won’t see the notes.

(Disclaimer 1: It is a lot more complicated than this, I’m simplifying how it works.)

(Disclaimer 2: I’m making several assumptions -- only Tumblr employees know exactly what’s going on.)

Similar to Youtube and the views not being updated in real-time, the servers, later on, will catch up and all of them will show the right notes. That’s why the mechanism is called eventual consistency: it might be consistent at times, but it eventually settles down the differences.

The strategy, put simply, is a trade-off to be able to handle really big systems with a lot of data.

– Alpha

When you hear stories about the most gigantic projects having a microservice architecture, you are tempted to introduce dozens of tiny applications that would work for you, like house elves, invisible and undemanding. However, system architectures lie on a spectrum.

What we imagine is the extreme end of that spectrum: tiny applications exchanging many messages. At the other end of the spectrum you imagine a giant monolith that stands alone to do too many things. In reality, there are many service-oriented architectures lying somewhere between those two extremes.

In a nutshell, a microservice architecture means that each application, or microservice’s code and resources are its very own and will not be shared with any other app. When two applications need to communicate, they use an application programming interface (API) — a controlled set of rules that both programs can handle. Developers can make many changes to each application as long as it plays well with the API.

This idea comes in many flavors, with different shares of the monolith architecture. In this post, we are going to discuss one of such variations of microservice architecture, known as Hexagonal Architecture.

The first key concept of this architecture is to keep all the business models and logic in a single place, and the second concept — each hexagon should be independent.

What is Hexagonal Architecture?

Invented by Alistair Cockburn in 2005, Hexagonal Architecture, or to call it properly, Ports and Adapters, is driven by the idea that the application is central to your system. All inputs and outputs reach or leave the core of the application through a port that isolates the application from external technologies, tools and delivery mechanics.

Allow an application to equally be driven by users, programs, automated test or batch scripts, and to be developed and tested in isolation from its eventual run-time devices and databases. Alistair Cockburn

Hexagonal Architecture draws a thick line between the software’s inside and outside parts, decoupling the business logic from the persistence and the service layer. The inside part makes up the use cases and the domain model it’s built upon. The outside part includes UI, database, etc. The connection between them is realized via ports and their implementation counterparts are called adapters. In this way, Hexagonal Architecture ensures encapsulation of logic in different layers, which ensures higher testability and control over the code.

Architecture components

Each side of the hexagon represents an input — port that uses an adapter for the specific type. Hence, ports and adapters form two major components of Hexagon Architecture:

Ports

A port is a gateway, provided by the core logic. It allows the entry or exiting of data to and from the application. The simplest implementation of a Port is an API layer. Ports exist in 2 types: inbound and outbound.

An inbound port is the only part of the core exposed to the world that defines how the Core Business Logic can be used.

An outbound port is an interface the core needs to communicate with the outside world

Adapters

An adapter transforms one interface into another, creating a bridge between the application and the service that it needs. In hexagonal architecture all communication between the primary (which use system to achieve a particular goal) and secondary actors (which system uses to achieve primary actor’s goals) and application ports is done with the help of adapters. Therefore, adapters can also be of two types:

Primary Adapters

The primary or Driving Adapters represents the UI. It is a piece of code between the user and the core logic. They are called driving adapters because they drive the application, and start actions in the core application. Examples of a primary adapters are API controllers, Web controllers or views.

Secondary Adapters

The secondary or Driven Adapters represent the connection to back-end databases, external libraries, mail API’s, etc. It is an implementation of the secondary port, which is an interface. These adapters react to actions initiated by the primary adapters.

Domain Model

The third component of the architecture is the domain model, a conceptual model that represents meaningful concepts to the domain that need to be modelled in software. The concepts include the data involved in the business and rules the business uses in relation to that data.

Benefits of Hexagonal Architecture

High Maintainability, since changes in one area of an application doesn’t affect others.

Ports and Adapters are replaceable with different implementations that conform to the same interface.

The application is agnostic to the outside world, so it can be driven by any number of different controls.

The application is independent from external services, so you can develop the inner core before building external services, such as databases.

Easier to test in isolation, since the code is decoupled from the implementation details of the outside world.

Our implementation of Hexagonal Architecture

One of Sciforce’s projects required to separate often changing external elements from internal ones that might lessen the impact of change and simplify the testing process.

The standard architectural template is based on the Spring set of frameworks:

The Core contains objects that represent the business logic of the service. Typically for Hexagonal Architecture, the core knows nothing about the outside world, including the network and the file system. All communication with the outside world is handled by Inbound and Outbound gateway layers.

In our application, a Port is a Groovy interface used to access either a Core or an Outbound object and an Adapter is an implementation of a Port interface that understands how to transform to and from external representations into the Core’s internal data model.

You can swap out an Adapter in gateway layer: for example, you can enable accepting messages from RabbitMQ instead of HTTP clients with no impact on the Core.We can also substitute Service Stubs for outbound gateways increasing the speed and reliability of integration tests.

Example: Simple application

In the example, we show a scheme of a simple application that uses the Spring Framework to accept a REST request with some text, converts the text to lowercase and saves it to MongoDB.

At the first step, when the client sends an HTTP request, the RestController is responsible for handling it. In terms of Hexagonal Architecture, this controller serves as an Adapter that communicates the request from the HTTP protocol to the internal domain model (a simple string in this case). It is also responsible for calling into the Core via the Port and sending the results back over the client.

The ConversionService is the object that the inbound Adapter (RestController) invokes via the ConversionPort interface. The service itself doesn’t access anything outside of the process: all it needs to do its job is to access the in-memory objects. After performing all the necessary processing (converting the text to lowercase), it delegates the task of storing the results to the outbound PersistencePort.

The MongoDBGateway is the implementation of the PersistencePort. It knows how to adapt the Core’s internal model, which is plain text, into a form that MongoDB can handle. Though the example is basic, in more sophisticated systems, it might implement exception, logging and retry logic in the code.

This example shows only 3 objects but in an actual application, you would have multiple objects in play. For example, a single REST controller would respond to different URLs by calling different services which then call different outbound gateways.

Conclusion

To sum things up, the main idea of Hexagonal Architecture is decoupling the application logic from the inputs and outputs. It helps to free your most important code of unnecessary technical details and to achieve flexibility, testability and other important advantages that make your working process more efficient.

However, like other architectures, hexagonal architecture has its limitations and downsides. For instance, it will effectively duplicate the number of classes on your boundary.