#network engine

I’ve recently given tours of nengi to 3 developers. The feedback has been very helpful.

The top requested features/changes so far are:

demo (and api) for multi-instance game worlds

demo of sending tilemaps via nengi

webpack support in demos, build/run scripts, es6 module support

division of demos into client, server, and shared code

code sample for authenticating players

code sample for saving/persisting a game world

more support for 3 dimensional space (z-axis, different ways of calculating entities within view of a player)

a more complete game example that goes beyond nengi features for newer devs to disect (player, enemies, camera, medium sized world, map data, animations)

Also on the TODO list for the current game that I'm working on are improvements to clientside prediction, serverside lag compensated collisions, and polishing+modularizing the current entity interpolator. These probably won't be seen in the published version of nengi until after I've tested them.

BTW if you're using nengi currently and trying to make use of multiple instances as one game world, a player can travel between instances the same way the player gets into the instance to begin with: via connecting. So it is possible to have a player step into a cave entrance (for example) and then have the server detect this and tell the client to connect to another instance. This is a manual process at the moment. The feature that I may add is a streamlined version of the same idea, perhaps with an api like instance.transferClient(whoToTransfer, instanceToTransferTo).

If you make a multi-instance game in nengi I recommend being very tidy and modular with the game logic, such that it can be used across all relevant instances. Like in a classic mmorpg.. much of the game's core logic applies in all areas of the game world, with only the terrain, creatures, and players varying. Perhaps a few instances are slight exceptions... maybe the big city of the game is optimized for higher populations by having a lower tickrate and disabling the CombatCollisionSystem. Maybe an arena instance is optimized for responsiveness via a higher tickrate, and also carries slightly different rules.

I'm also debating a feature where a client can connect to multiple instances at the same time as a means of hiding the seam between the areas of the world controlled by one server and another.

ETAs are largely unknown. v1 prediction/compensation is going to be released with my new game. I hope to add multi-instance support and do a big code reorg around the same time (6-8 weeks)

I've begun separating out the network magics from the BadStar game from the game itself. I'm calling this thing nengi (net engine) and I'm leaning towards releasing it as open source. The core of it is pretty narrow in focus; it takes a javascript object on the server and sends any changes that occur to the client's browser. The technologies used are node.js and binary websockets. It is a huge optimization over json.

There's also this whole enginesque bit about deciding which objects are near the player, exposing a component-esque pattern to work with game objects, and all the fancy lag compensation stuff. I've made this stuff already, but I haven't figured out how I'd offer these things without interfering with people's game designs. 

Anyways, back to the better understood part of nengi. So far the developer writes a list of properties that should be networked, as well as their types, and the rest is somewhat automagic.

As a simple example, there might be a gameObject of type Troll. Perhaps its data looks like this:

Troll: { x: 25 y: -229 velocity.x: 2 velocity.y: 20 ai: new DefaultEnemyComponent() ai.target.id: somePlayer.id ai.state: EntityState.Enraged hp: 24 maxHp: 100 }

For nengi to network this object, it needs a netSchema added. Here's an example:

Troll.netSchema = { 'id': Binary.UInt16 'type': Binary.UInt8 'x': Binary.Int16 'y': Binary.Int16 'ai.state': Binary.UInt8 'hp': Binary.UInt8 'maxHp': Binary.UInt8 }

This schema lists which properties of troll should be networked, and what type of number each of these properties is. The id and type are mandatory, everything else can be whatever you want it to be. It is also worth noting that Troll has numerous properties that aren't in the netSchema -- its entirely up to you which parts of the object matter to the client, and which should just stay on the server.

By specifying the exact type (Int32, etc), the developer can get game objects down to a very small number of bytes. It's also up to the developer to enforce these decisions, or pick different types. For example, if the EntityState enum happened to have more than 256 possible ai states, then choosing to represent it with one byte (UInt8) would be insufficient, and would cause an error.

Using the troll and netSchema above, nengi will keep all players up to date should the troll experience any changes in its x,y (movement) its state (aggro, peaceful, anything), and its hitpoints. This includes some potentially neat optimizations, where if only one thing about the troll changes, nengi will send a very minimal amount of data. For example if the troll took 5 damage, but did not move nor experience a change in its 'ai.state', nengi would simply send the following data:

1, 4, 19 which has the meaning: { gameObjectId: 1, propertyChanged:'hp', newValue:19 }

This would be 4 bytes, instead of the full object (which itself is optimized to 10 bytes in this scenario). This type of optimization becomes more essential for larger objects that have many properties.

This automagic object syncing is the core feature of nengi. However nengi has a bit of engine to it, and its changing by the day so I'll just quickly list the current featureset:

[done] classic game loop with adjustable rates (20 fps, 60 fps, etc)

[done] game object + component system, where each component can run logic via an update function called on each cycle of the game loop

[done] quadtree, used to decide which game objects exist within view of the player (it is from this that the network code decides which objects need sync'd). Also usable for collisions. 

[partial] utf8 text as a binary data type

[partial] instancing support, a game runs as an instance, and multiple instances can run in the same server process. The opposite, sharding, is awesome but I have no intention to make this anytime soon -- it would be pretty specific to a game's implementation.

[partial] rudimentary collisions, with lag compensation via a "historian" (a player with 500 ms of latency can be checked for collisions against older positions of game objects from half a second in the past, thus more closely resembling what they see on their own screen). This totally works, but I worry that the implementation isn't generalizeable enough to be a neutral enginesque feature.

[prototyped separately] gameObject.IsPredicted = true, a flag that lets the client run the exact same logic as the server for a gameObject, rather than wait for updates from the server. It is easy to do this in a naive way, but it may be too specific to certain types of games, and not ever really work quite right. Might not be included.

[prototyped separately] entity interpolation. Easy, technically. Not entirely sure how I'm going to expose this feature while keeping it decoupled from game-specific logic. Might not be included.

It may be the case that I need to release 3 separate things: 

1) a nengi 'object syncing library,' with the core feature, but devs would need to manually call all of the appropriate functions to wire it into a game, nothing automagic

2) a nengi 'component based engine,' which wraps #1 and exposes a barebones game engine w/ automagic networking -- just follow a component based pattern, write the netschemas, and everything works. Just a note, I'm not making a full on game engine. It's just a game loop, two base classes called 'GameObject' and 'GameComponent' alongside the network code. Its just a template, and I'm not sure how applicable it would be for other devs.

3) a nengi demo, which includes #2 and also shows an example client (prediction, interpolation, etc) with a pixi.js frontend. Maybe I'll just take a small piece of BadStar and change the artwork.

So yeah, a lot of unknowns! Also a lot of things done and functional to some degree. I'm glad that I now have a small lib that can do the network stuff for both BadStar and the zombie game from way back when.