CosimGym is a powerful orchestration framework that seamlessly bridges the gap between complex co-simulation environments and Reinforcement Learning (RL).
The Challenge: Modern engineering systems—such as power grids, building energy systems, and robotic swarms—often rely on multiple interacting subsystems that are best modeled with specialized tools. Co-simulation (powered by the HELICS middleware) enables these heterogeneous models to run simultaneously and exchange data at every time step, providing a unified system-level perspective.
The Solution: By natively integrating Gymnasium, CosimGym translates complex publish/subscribe data exchanges into the standard reset() and step() paradigm. This allows RL agents to directly interact with, learn from, and control realistic, physics-based simulations without the heavy burden of networking and synchronization boilerplate.
The full CosimGym Documentation is available at energycenterlab.github.io/CosimGym.
With this framework you can setup co-simulation scenario by plugging your own models or reuse the community shared ones. The main orchestration mechanism relies on HELICS middleware and by wrapping those low level functionalities enables a standard and reproducible approach to plug and co-simulate models. In addition this framework is intended to be a Reinforcement Learning gym for training and testing your agents on composable realistic phisical environments.
- 📜 Declarative YAML Scenarios: Define complete simulation architectures, timing windows, multi-federation setups, and RL hyperparameters purely through YAML configuration files—no hardcoded orchestration required.
- ⚙️ Automated Orchestration: The
ScenarioManagerhandles the heavy lifting: starting HELICS brokers, spawning individual federate processes, distributing configurations via Redis, managing time synchronization, and ensuring graceful shutdowns. - 🧩 Plug-and-Play Models: Rely on a standardized
BaseModelinterface and a centralized Model Catalog to easily drop in custom physics models, data readers, FMUs, or arbitrary RL algorithms. - 🤖 Gymnasium Compatibility: The built-in
HelicsGymEnvwrapper automatically binds HELICS pub/sub variables to standard Observation and Action spaces, making it out-of-the-box compatible with popular RL libraries (e.g. stablebaseline3, RLlib). - 🔄 Flexible Workflows: Run standard physics-only co-simulations, conduct live online RL training (with built-in agents like DQN or SAC), or evaluate pre-trained policies within a single unified framework.
- 📊 Built-in Dashboard: Monitor simulation metrics and analyze RL performance through an interactive, ready-to-use Streamlit dashboard, including a live view for data as it streams during a run.
- 🔌 Digital-Twin Interfaces: Mirror any federate's data to MQTT for live observability, or swap a simulated federate for an
interfacefederate bridging to real sensors/actuators — a one-line, config-only sim-to-real change (see Digital-Twin Interfaces & Live Streaming).
In order to start using this repository there are different options listed and explained here Installation
Simple example of runnable test cases (see full list for scenario names and how to run them):
- Simple spring mass damper system (single & multi-federation) -->
simple_test(Case 0) - Building + Heatpump + Weather + PID controller -->
bui_hp_test_base(Case 1) - Building + Heatpump + Weather + RL agent (DQN/SAC) -->
bui_hp_DQN/bui_hp_SAC(Cases 2 & 3) - PV + Battery + Load + Weather + RB controller -->
pv_batt_test_base(Case 4) - PV + Battery + Load + Weather + RL controller (DQN/SAC)-->
pv_batt_DQN/pv_batt_SAC(Cases 5 & 6)
Published Applications: Incoming
This repository is in an early prototype stage. Testing is still ongoing, and the codebase is under active development. Expect changes, refactoring, and new features in the near future!
Contributions, issues, and feature requests are welcome. Feel free to open an issue or submit a pull request. A discussion session will be open so in this first stage start brainstorming there. This repo is meant to be a collective place for multidisciplinary exchanges.
incoming
This project has been supported by the following institutions and organizations:
 Politecnico di Torino |
 Energy Center Lab EC-lab |