This project implements the simulation and networking strategies described in Glenn Fiedler's article series Networked Physics. It's written in C++ using OGRE 3D with ODE for physics simulation and a client-server architecture as opposed to the producer-consumer one used in the articles.
Multiplayer.GitHub.mp4
Get the latest release for your platform of choice, as long as that choice is x64 or ARM64 on Windows. Run this to start a local server:
Server.exe --numPlayers <N>
Run this once per client on the server machine:
Client.exe
To have server and clients in different machines, pass the server address to both server and client. Run the executables with --help to see all available options.
Once running, you can change the networking strategy via the Debug panel in the server. The same panel, in both client and server, contains options to simulate different network conditions and tweak an unreasonable amount of parameters.
I'm using Glenn Fiedler's own yojimbo library for establishing connections and sending reliable and unreliable messages over UDP. The reliable channel is used only for control messages (e.g. start simulation, pause/resume). Messages for networking the simulation itself are sent unreliably with a custom resend/ack system when required.
Messages are bitpacked using serialize.
There are three strategies for networking the simulation, explained in detail in the original articles: deterministic lockstep, snapshot interpolation and state synchronization.
Clients send inputs, server steps simulation and sends the inputs used back to clients, clients step when they receive step message from server. Uses very little bandwidth but requires determinism across clients and server. Implemented as in the reference but with a different approach to the playout delay buffer.
Conceptually, this is the step message sent to clients:
Step Message
{
PlayerInput inputs[AllPlayers]
}
Clients send inputs, server steps simulation and sends a snapshot of the state back to clients, clients update local state with snapshot. Doesn't require determinism, as clients don't step the simulation, and facilitates late joining but consumes more bandwidth than lockstep. Implemented using linear interpolation between states and delta compression for snapshots without the incremental improvements from the reference.
Step Message
{
Position positions[AllObjects]
Orientation orientations[AllObjects]
}
Clients send inputs, server steps simulation and sends the inputs used and the state of a few objects, clients step the simulation with inputs from server and correct local state of received objects. It doesn't require determinism, because divergent states are eventually corrected, and allows for fine grained bandwidth control, as you can choose how many objects to sync per step. Implemented with controllable bandwidth and object state quantization on both sides but no visual smoothing or delta compression.
Step Message
{
PlayerInput inputs[AllPlayers]
Position positions[SomeObjects]
Orientation orientations[SomeObjects]
Velocity linearVelocities[SomeObjects]
Velocity angularVelocities[SomeObjects]
}
I use Ogre with a few, very simple materials and shadow mapping. No frills, just like the reference (but worse). Pre-integrated Dear ImGui is used for UI plus ImPlot for fancier plotting.
I started using Ogre's Bullet integration but found that I couldn't get as close as I wanted to the reference simulation. Specifically, I couldn't get the friction behavior required for the tumbling movement of the player cube. I knew the reference used ODE so decided to give that one a go.
ODE integration was straightforward but required lots of parameter tweaking to get a reasonably close behavior. I'm stepping the simulation single threadedly and passing a known seed to the random number generator to achieve determinism across x64 and ARM64 builds.
I knew debugging distributed issues would be challenging and good instrumentation critical. I looked into different options (Tracy, Perfetto, ETW) and ended up going for Palanteer as it was straightforward to integrate, gave me the control I needed and supported merging traces from different processes. This, combined with running server and clients in the same machine, allowed me to sequence and visualize events across processes easily.
I ended up contributing back to a bunch of my dependencies. A lot of it was about getting them to run on ARM64 on Windows.
Yojimbo and related libraries:
- Detect endianness on ARM64 on Windows
- Unified memory layout of BitReader across debug and release builds
- Generate build targets for all Windows architectures
- Fix sodium build error in x64
Ogre:
- Upgraded SDL2 to latest version to fix ARM64 support on Windows
- Build Bullet without "_Debug" postfix so find_package finds it
Ode:
Palanteer:
Many thanks to all the open source contributors that have made this project possible and specially to Glenn Fiedler for putting together the very useful Networking Physics article series and yojimbo library.
This project uses the following open-source libraries:
- Ogre: Object-Oriented Graphics Rendering Engine, a modular C++ renderer for custom engine development.
- yojimbo: a network library for client/server games written in C++.
- ODE: Open Dynamics Engine, a library for simulating articulated rigid body dynamics.
- Palanteer: a set of tools to improve the quality of software, for C++ and Python programs.
- Dear ImGui: a graphical user interface library for C++.
- ImPlot: an immediate mode plotting library for Dear ImGui.
- inifile-cpp: a single header-only ini file en- and decoder for C++.
- argparse: a command line argument parser.
Seven clients and a server running in the same machine using snapshot interpolation:
I'm a video, click me!