Fail fast when overlap wait times out

[TianyeGGBond]

Context

_wait_for_overlap_engines_offloaded is the safety gate before training resumes on GPUs that may have just been used by inference engines. If the engine state probe fails or the wait times out, continuing can surface later as a less-informative train wake-up OOM.

Change

• raise when engine state polling fails
• raise when engines do not reach a safe state before the timeout
• raise when the free-memory gate times out

Validation

• python -m py_compile rlix/pipeline/miles_pipeline.py

[JunzheJoe]

All three conversions (get_engine_states failure, state-poll timeout, free-mem timeout) are warn-and-proceed → raise, which matches the fail-fast-only operational policy and closes the F2 review finding about the 60s overlap wait being warn-only — a timeout here previously let _before_training proceed into a likely wake_up OOM with only a log line as evidence.

Two notes:

• Scope check (good): the nvidia-smi-unavailable branch (min_free_gb is None counting) is untouched — that fail-open is a separate item pending Tao's per-item verdict, so no overlap.
• Sequencing: this collides textually with #19 (same warn blocks, #19 keeps them as warns while refactoring). Suggested order: #19 first, this rebases onto it — the raises land unchanged in the refactored function. LGTM.

[JunzheJoe]

Follow-up after learning (via #21's closure) that _wait_for_overlap_engines_offloaded was rewritten on the base branch (howard989, 4af2fc4). Reconciled against current HEAD — the three hunks now split:

• Hunks 1–2 still apply and are still wanted: on the rewritten HEAD, Phase 1 is still warn-only on both paths (get_engine_states failure → warn + return; state timeout → warn and fall through to Phase 2), so the warn→raise conversion remains the right fail-fast change.
• Hunk 3 is superseded: the free-mem wait loop it converts no longer exists — Phase 2 is now the whole-GPU residual used gate, which already raises when the threshold is exceeded. Nothing left to convert there.
• The one remaining fail-open in the new Phase 2 (nvidia-smi unavailable → warn + skip the gate) is deliberately out of this PR's scope — it's on the pending fail-open list awaiting per-item sign-off.

So: rebase, keep hunks 1–2, drop hunk 3. Approval stands for that reduced scope.

[Kynnet]

change seems good, but needs to be implemented for updated miles_pipeline, this version is behind