Online observability for reinforcement learning training. RL-Insight connects training-side metrics, RL state traces, and service dashboards across distributed rollout and optimization workloads.
RL-Insight focuses on the online observability path that RL training needs most:
- One-command server startup: install dependencies and start the RL-Insight server with Prometheus, Tempo, and Grafana with
rl-insight server installandrl-insight server start. - Trainer and rollout metric aggregation: collect key actor, rollout, and transfer queue metrics in one monitoring view while keeping training-side instrumentation lightweight.
- Grafana dashboards for RL workloads: provide ready-to-use dashboard structure for training metrics, rollout behavior, engine metrics, and RL state timelines.
RL-Insight has two metric sources. Training code reports framework-internal signals through the Python API, and RL subsystems such as rollout engines and transfer queues register their own /metrics endpoints through the metric aggregation interface. The RL-Insight server coordinates the training side and manages Prometheus, Tempo, and Grafana so metrics, traces, and subsystem signals converge into unified RL dashboards.
rl-insight.demo.mp4
- [2026/06/16] RL-Insight officially supports Online Monitor, including one-command server startup, trainer and rollout metric aggregation, and Grafana dashboards for RL workloads.
Start with the guide that matches your current setup:
| Document | What it covers | When to use it |
|---|---|---|
| Server Installation | Prometheus, Tempo, and Grafana service setup, including supported Linux platforms, direct installation, offline installation, and existing service binaries. | Use this first if the monitor services are not installed or you need to verify the server environment. |
| Quick Start | A full smoke-test flow: install the Python package, start the monitor stack, emit sample metric/trace data, and open Grafana. | Use this after the services are ready, or when you want to validate the monitor path end to end. |
Recommended order:
- Prepare the server services with Server Installation.
- Run the end-to-end flow with Quick Start.
RL-Insight manages three open-source services locally on Linux:
| Service | Purpose | Default port | Required version | Installer version |
|---|---|---|---|---|
| Prometheus | Metric storage and queries | 9090 |
>= 2.30.0 |
2.54.1 |
| Tempo | Trace storage and query API | 3200 |
>= 2.0.0 |
2.6.1 |
| Grafana | Dashboards and trace exploration | 3000 |
>= 13.0.0 |
13.0.0 |
rl-insight server install downloads supported Linux binaries into ~/.rl-insight/services. rl-insight server start runs the RL-Insight server with Prometheus, Tempo, and Grafana with data persisted under ~/.rl-insight/data by default.
rl_insight/ exports the online monitor public API, so training code can import one module:
| API | Use |
|---|---|
init(project=None, experiment_name=None, config=None) |
Enable monitoring once per process and attach global labels. |
metric_count(name, amount=1.0, documentation="", **labels) |
Increment a Prometheus counter. |
metric_gauge(name, value, documentation="", **labels) |
Record the latest value for a gauge. |
metric_histogram(name, value, documentation="", **labels) |
Add one sample to a histogram. |
trace_state(state_name, state_lane_id=None, **labels) |
Record a named RL state interval. |
trace_op(name=None, extra_labels=None, **static_labels) |
Decorate a synchronous function and emit one duration span per call. |
finish() |
Reset in-process monitor state. |
Configuration can be passed to insight.init(config=...) or through environment variables:
insight.init(
project="verl",
experiment_name="ppo-smoke-test",
config={
"server": {
"namespace": "rl_insight_monitor",
"backend": "ray",
"url": "http://<server-ip>:18080",
},
"prometheus": {
"metrics_report_port": 9092,
},
},
)Useful environment variables:
| Variable | Purpose |
|---|---|
RL_INSIGHT_SERVER_URL |
RL-Insight server URL, for example http://<server-ip>:18080. |
Offline timeline, heatmap, and parser utilities are kept under recipe/; see Recipe README for that workflow.
- Quick Start: install RL-Insight, start the services, instrument code, and open Grafana.
- Server Installation: Linux service requirements, supported OS/CPU combinations, and version policy.
- Default server config: bundled ports, retention settings, and service config paths.
- Recipe README: offline timeline, heatmap, and parser utilities.
See CONTRIBUTING.md.