Play the policy's trajectory in the harness, one command per channel per round - #591
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b0fa7eefdd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 279b4fda30
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 442f35e626
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dddf23b43f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 61b19835b9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 381978aef8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
381978a to
8c1a074
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 757c3898d6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ebc7bafd2f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c1e822dad0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0595b577f8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
0595b57 to
18161be
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 18161bea13
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 66c5d394c1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 49829b8763
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d1c7c7bc44
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
d1c7c7b to
f12d4bb
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4b3cee928d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8cfe38c4c6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
8cfe38c to
b31eef8
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b31eef8d67
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 201f759faa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d6a4699d0c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 330b312f49
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a7061c3cce
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4f49190e5e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
`seal` closes through `end` instead of repeating it, `_take` asks `_effect_time` for the instant it was already recomputing, and the cached `_descriptor` gives way to the embodiment that owns it.
`TrajectoryPlayer` wrapped one deque behind three methods; the harness now keeps `dict[str, deque]` and plays it in `_play`, where the collapse-to-last rule is stated.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d5b4401668
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The declared-vs-wall latency mode was re-tested in four places; it is now the choice of one `_Charge` at episode start. `_InferenceWorker` takes the session, the executor and the future, so `_take` drops from 22 lines to 9 and stops carrying a future it always had.
`_trial_terminal` narrows `_deadline` itself instead of trusting the caller's guard, so the task-less case is stated where it applies. The test stubs get the annotations their `None`-initialised fields always needed.
The body named what the name did: two already-named calls.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3f592edb3b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… checkers `earn-its-place` now covers functions and states the single-user test across the shapes it applies to; `stranded-definition` cites it instead of carrying it, leaving existence to one rule and placement to the other. `check-rules` prompts are four fixed parts. Anything an author adds is drawn from knowing the change, which is the knowledge that hid the violation from them.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c43832d8d9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
`inference_latency` is a flag: a sim either charges each call the wall time it took or holds the world for it; a real rig always pays wall time. With the constant charge gone the release condition is "the call returned" in both modes, so `_Answer`, the `_Charge` classes and `_take` go, and the mode is one bool on `_InferenceWorker` — `throttle`'s timeout and `effect_time`'s wall term. `_step` is the algorithm in one place: throttle and reschedule on the call in flight; with none in flight, submit on a fresh observation and give it the rest of the round. Rollout-start bookkeeping moves to the first observation, the deadline drop into `_reschedule`, `_running` into `_worker`. The golden keeps its fixed 50 ms latency through a test-local `_SimulatedLatency` policy that holds each chunk on the world clock; the golden file is unchanged.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7f620e2c80
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 588a459933
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2036bc0efc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…its user `_InferenceWorker` takes the clock and stamps `_t0_ns`/`_wall_t0` before `new_session`, so the `now` it hands the session reads a trial instant from the moment the session exists rather than raising on an unset field. `submit` and `throttle` read the held clock instead of taking one per call. `_owned` and `_report_abandoned` become static methods above their sole callers; `_append` inlines into `_record`. `keys.DESCRIPTOR` replaces the `'descriptor'` literal at the harness and its readers.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7620f1a5b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
`test_harness.py`, `test_golden_pipeline.py` and `test_ds_writer_agent.py` each carried a `roboarm.State` fake over the same four properties. They now share `FakeRobotState` in `drivers/roboarm/tests/fakes.py`, beside the interface it fakes, with `make_robot_state` for the stationary identity-rotation case the two policy tests want. The merged fake copies on read, which the golden producer relied on and the other two are indifferent to. The golden file is unchanged.
`main` moved trajectory playing out of the adapter and into the harness (#591), so `EnvAdapter.action` takes the command messages alone and holds the last one per channel: the proxy drops the clock it used to pass, reads the receivers without a default and skips the empty ones, and `WireCommandAdapter` reads grip straight off `_held` with no sticky `_grip` to freeze across a cancelled trajectory. This branch's `protocol.ACTION_*` constants stand over the wire literals on main's side of every one of those hunks, and `_step_env` keeps its assertion that a step follows a connected reset. `main` also names the encoded observation codec — `ObservationCodec.WIRE_NAME` — which `to_spec` now emits in place of the literal, alongside this branch's `lowercase_task` arg. `close` takes main's `_CLOSE_ACK_TIMEOUT` and the test that pins it, both of which the env-server tests import next to this branch's `serve_subprocess` bind test. The baseline keeps this branch's shrink: the `client.py`, `proxy.py` and `server.py` entries main still carries are fixed here, and basedpyright reports no error without them.
1 participant
Summary
The wire between harness and driver carries the single command due now, not a future trajectory. The plan stops at the harness:
TrajectoryPlayermoves above the wire, every driver executes the latest updated message and holds otherwise, and emission time is execution time.reduce/_combineand the player'sreducehook go; last-wins is the only collapse, and the[]-means-cancel protocol becomes the harness's own. Every driver-side player is deleted — franka, yam, kinova, so101, dh, robotiq, the MuJoCo sim and the env-server adapter all execute onupdated.TrajectoryPlayerper command channel, one emit per channel per round, paced to the next waypoint (capped at the poll period). Command timing granularity becomes the round — ~10 ms real, one control period in sim — against 20–50 ms waypoint spacing.inference_latency; what the policy stack gets isnow, a clock reading the instant the in-flight call's output takes effect. A scheduling wrapper stamps its chunk atnow()and never learns the mode._bump_schedule_end(and its TODO),_inference_delayand the post-inference shift are deleted rather than ported.t0 + charge, with the schedule playing on the way there. A wall charge (inference_latency=True, and every real rig) can hold nothing still, so the world runs no further ahead of the call than wall time has. A real rig ignores the key entirely.keys.ROBOT_FAULT(and without the arm's own entries, which a faulted arm cannot give). The newStopOnFaultwrapper — outermost, ahead of the scheduler, in every built-in stack — answers the empty trajectory and resets the sessions below, so recovery plans afresh instead of resuming a chunk stamped before the fault.TrajectoryOverrideSerializeris deleted, and with it the whole self-timestamped-stream mechanism it was the only user of (Timestamped,StatefulSerializer.flush, the STOP-time flush loop, the writer's expand-one-message-into-many branch). Command signals register as plain per-sample. This closes Eval recordings truncate the command signals before the episode ends, so a recorded episode cannot be replayed to its own outcome #580 by construction: nothing infers which prefix of a buffered chunk actually ran.RTC and temporal ensembling are the point of the substrate but are not in this PR: they are wrappers that hand back a new trajectory before the old one is exhausted, and
test_harness_keeps_playing_while_a_call_is_in_flightpins that the harness keeps emitting through such a call.Not included: replay reproducing
eval.success(the other half of #580).replay_record.mainseeds the sim fromepisode.static['mjSTATE_INTEGRATION'], but evals record sim state as a signal, and there is no scorer over a replayed episode — both are separate changes.Test plan
uv run --locked pytest --no-cov— 1092 passed, 8 skipped.uv run --locked ruff check .,env UV_PROJECT_ENVIRONMENT=.venv-typecheck uv run --locked --exact basedpyright(0 errors; the baseline only shrinks), andutilities/check_basedpyright_baseline_ratchet.py— all clean.ROBOT_FAULTset and the arm's entries absent;StopOnFaultanswers[], does not call the model, and lets the next sound observation re-plan instead of waiting out the pre-fault chunk.golden_pipeline.json.gzregenerated: every driver executes onupdated, and the fake arm's injected fault now stops the plan instead of being played through. Review the diff for shape, not values.An end-to-end pass with a real policy, which no unit test substitutes for.
pi05_liberoserved from a stockpositro/openpi:main— the CI image built from this branch's own base commit, declaringchunked_schedule | restrict_image_sizeand knowing nothing ofstop_on_fault, so both arms met an identical server. Only theharness, drivers and env server ran from this branch.
main,libero_object, seeds 0–2 over all 10 tasks (30 trials each): 30/30 both arms. Pairedtrial by trial, verdicts agree 30/30 — neither arm wins a trial the other loses. Medians, branch vs
main:path 1132.4 vs 1139.6 mm, span 6.50 vs 6.90 s, max step 12.62 vs 12.75 mm, speed p95 0.238 vs 0.241 m/s;
median per-trial |diff| in path length 27 mm. No recorded frame on either arm has the arm sitting still
between rounds (stall fraction 0.000 on all 60), which is the failure the per-tick wire could have
introduced. Motion is compared through
robot_state.ee_pose, recorded identically on both sides;robot_commandis not comparable across arms by construction, sincemainrecords the planned trajectorythrough
TrajectoryOverrideSerializerand this branch records what each round emitted.--inference_latency=0.3: command timestamps are reproduced exactly across two runswhose real inference latency differed — the schedule is a function of the charge alone. The action values
diverge, and that is the accelerator, not the harness: two freshly started servers, same
jax.random.key(0), same observation, answer differently at 6e-4 (openpi splits its sampler key per call,so within one process the RNG state also carries across sessions). A 6e-4 perturbation compounds through a
closed loop, so sample-for-sample equality of a whole rollout is not achievable against any GPU-served
policy, by any harness.
test_a_constant_charge_keeps_the_trace_off_the_machine_clockpins the harness half.--inference_latency=Truewithpositronic eval timing-report, 10 trials: real-time factor 99.4%(sim-s per wall-s) — the world runs no further ahead of the call than wall time has — over 263 inference
calls at p50/p95 367.3/369.0 ms,
policy_wait56.2% of pass wall.Not run, for want of the hardware: by hand on the rig, FINISH while a call is in flight and an induced arm
fault — the two paths the decisions above changed. Both are covered by tests, neither by a physical rig.
That pass also surfaced a hang the harness's own error path had been masking, fixed here: a policy error
propagates correctly, but the world then tore down through
EnvConnection.close, whose unboundedrecvwaitedon a LIBERO process already wedged in its own destructor — for as long as the run had. The acknowledgement is
now bounded, an unanswered goodbye ends the run, and a peer that took the request and never replied is logged
rather than swallowed.
test_close_gives_up_on_a_peer_that_never_answersblocks on the exact frame theproduction thread dump named.
Filed, not fixed here:
eval timing-report --run_dirdoes not expand~(#615).