Feature/update cloud example#345
Conversation
There was a problem hiding this comment.
Pull request overview
Updates the “Cloud Environment with Re-imaging” example notebook to reflect the updated observation/action space and demonstrates new greedy heuristics plus a custom token/attention-based actor-critic architecture for RLlib PPO. Also documents the update in the v1.3.0 release notes.
Changes:
- Refresh the environment configuration (observations/actions/lookahead) and reward logic in
cloud_environment_with_reimaging.ipynb. - Add greedy heuristic baselines (re-imaging and single-picture) and a custom RLlib
RLModulewith attention-based actor/critic definitions. - Add a release note entry indicating the example has been updated.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| examples/cloud_environment_with_reimaging.ipynb | Updates the cloud re-imaging tutorial with new observation/action definitions, heuristics, and custom actor-critic/RLModule code. |
| docs/source/release_notes.rst | Adds a release note entry referencing the updated example notebook. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
LorenzzoQM
force-pushed
the
feature/update_cloud_example
branch
from
552e14c to
ddd4323
Compare
LorenzzoQM
force-pushed
the
feature/update_cloud_example
branch
from
ddd4323 to
0fcd701
Compare
LorenzzoQM
force-pushed
the
feature/update_cloud_example
branch
from
0fcd701 to
a41a458
Compare
| " dict(prop=\"v_BN_P\", norm=7616.5),\n", | ||
| " dict(prop=\"battery_charge_fraction\"),\n", | ||
| " dict(prop=\"solar_angle_norm\"),\n", | ||
| " dict(prop=\"omega_BN_B\", norm=0.03), # 0,1,2 - CHECKED\n", |
There was a problem hiding this comment.
Those "CHECKED" comments should be removed.
| " log_level=\"INFO\",\n", | ||
| " log_level=\"WARNING\",\n", | ||
| " failure_penalty=0.0,\n", | ||
| " # disable_env_checker=True, # For debugging\n", |
There was a problem hiding this comment.
I think the "disable_env_checker" doesn't exist anymore and we need to remove this.
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Training and custom actor-critic modules\n", |
There was a problem hiding this comment.
I feel like this code is too much for one example script. Have you thought about splitting the code into two? For cloud with re-imaging example. And training with new architecture script?
Description
Closes #342
Updates the cloud with reimaging example script to incorporate recent changes and new actor-critic network architecture.
Type of change
How should this pull request be reviewed?
Future Work
Update with the related publication information once available.
Checklist