positronic eval timing-report --run_dir=~/evals/timing fails as if the run carried no telemetry:
ValueError: no telemetry under ~/evals/timing/telemetry (recorded without --timing?)
The run was recorded with --timing and the sidecars are there. ~ simply reaches Path unexpanded, so
root / TELEMETRY_SUBDIR names a directory that does not exist. Passing the absolute path works:
positronic eval timing-report --run_dir=/home/vertix/evals/timing
The error message names the one cause that is not the actual one, so the reader goes looking at how the run was
recorded rather than at the path they typed.
timing_report in positronic/cli/eval/timing_report.py builds the local path with
Path(run_dir), with no expanduser. The s3:// branch beside it goes through pos3.download, and
eval run --output_dir goes through pos3.sync, which is why the same ~ works there — a
~-relative path is the documented form in docs/evaluation.md, where the quickstart writes to
~/evals/libero and would be read back by this command.
Worth checking whether any other CLI entry point takes a filesystem path straight to Path without the same
treatment, rather than fixing this one call site alone.
Found while running the real-policy eval pass for #591; unrelated to that change.
positronic eval timing-report --run_dir=~/evals/timingfails as if the run carried no telemetry:The run was recorded with
--timingand the sidecars are there.~simply reachesPathunexpanded, soroot / TELEMETRY_SUBDIRnames a directory that does not exist. Passing the absolute path works:positronic eval timing-report --run_dir=/home/vertix/evals/timingThe error message names the one cause that is not the actual one, so the reader goes looking at how the run was
recorded rather than at the path they typed.
timing_reportinpositronic/cli/eval/timing_report.pybuilds the local path withPath(run_dir), with noexpanduser. Thes3://branch beside it goes throughpos3.download, andeval run --output_dirgoes throughpos3.sync, which is why the same~works there — a~-relative path is the documented form indocs/evaluation.md, where the quickstart writes to~/evals/liberoand would be read back by this command.Worth checking whether any other CLI entry point takes a filesystem path straight to
Pathwithout the sametreatment, rather than fixing this one call site alone.
Found while running the real-policy eval pass for #591; unrelated to that change.