Add: Worker-level chip bootstrap orchestration for distributed L3

[ChaoWao]

Summary

Wires ChipWorker.bootstrap_context into the Worker factory so an L3 Worker(level>=3, chip_bootstrap_configs=[...]) brings up every chip child's communicator during init() and surfaces a ChipContext list to orch code before the first run().

Note on terminology: the L3 in the title refers to runtime hierarchy Level 3 (chip-level Worker) — see docs/hierarchical_level_runtime.md. Earlier split commits (#608, #610) used (L2)/(L5) tags as split-step labels, which collides with the Level-0..Level-6 hierarchy; this PR drops those labels from new and touched code.

• ChipContext dataclass in task_interface — device_id / rank / nranks / device_ctx / local_window_base / actual_window_size / buffer_ptrs: dict[str, int]. The per-buffer dict is built by zipping cfg.buffers with the result's buffer_ptrs, so orch code addresses a named window slice without tracking list indices. A length check before the zip raises RuntimeError on a parent/child buffer-count mismatch instead of silently truncating.
• Parent: per-chip ChipBootstrapChannel mailbox (4096 B shared-memory, zero-filled so state starts IDLE) allocated pre-fork. Parent polls each channel with time.sleep(0.001) + 120 s soft timeout; on the first ERROR raises RuntimeError(f"chip {idx} bootstrap failed: {channel.error_message}") and best-effort SIGKILLs every forked child + unlinks every shm so init() raises cleanly without leaking state. chip_contexts is a property that raises before init().
• Child: new _chip_process_loop_with_bootstrap runs bootstrap_context first (channel publishes SUCCESS/ERROR), then enters the same task/control poll loop as _chip_process_loop. try/finally runs shutdown_bootstrap then finalize on SHUTDOWN. Bootstrap failure returns via os._exit(0) so the parent's waitpid isn't confused by a non-zero exit code layered on top of the channel's error.
• Teardown ordering: _worker.close() → SHUTDOWN → waitpid → unlink sub/chip/next-level mailboxes → bootstrap mailboxes unlinked last, because chip children touch their ChipBootstrapChannel inside shutdown_bootstrap() + finalize().
• The original _chip_process_loop and _Worker scheduler wiring are untouched; the bootstrap path is gated on a non-None chip_bootstrap_configs argument and runs eagerly at init() time instead of the usual lazy _start_hierarchical() on first run().

Does not extend to level-4+ recursive Worker children — the _next_level_workers fork path is unchanged; adding distributed bring-up for nested Workers is a follow-up.

Testing

• tests/ut/py/test_worker/test_worker_distributed_sim.py — happy path + error path (bogus placement triggers RuntimeError) + chip_contexts-before-init guard + __init__ validation (level<3 reject, length-mismatch reject).
• tests/ut/py/test_worker/test_worker_distributed_hw.py — 2-card hardware smoke, drives Worker(level=3, chip_bootstrap_configs=[...]) end-to-end, asserts each rank's device_ctx != 0, local_window_base != 0, actual_window_size >= requested, and buffer_ptrs == {"x": local_window_base}. No comm_barrier — HCCL 507018 stays off the critical path. Lives under tests/ut so the ut-a2a3 job picks it up without xdist's per-worker device-slicing (which would break a 2-device request under tests/st).
• pytest tests/ut/py/test_worker with chip_bootstrap_configs=None paths — 59 green, no regression.

Ref: #571 (split), builds on #608, #610.

[gemini-code-assist]

Code Review

This pull request introduces worker-level chip bootstrap orchestration (L6). It adds the ChipContext dataclass and updates the Worker class to support asynchronous bootstrap of chip children via shared-memory mailboxes. Key changes include a new child process loop that executes bootstrap_context, a timeout-based polling mechanism in the parent to collect results, and enhanced cleanup logic to prevent shared-memory leaks on failure. New hardware and simulation tests are also provided. Feedback is provided regarding a potential silent truncation issue when zipping buffer pointers, suggesting an explicit length check to ensure data integrity.