Skip to content

feat(live): add hands-free Codex voice calls for OMP panes - #164

Closed
Ad3Digital wants to merge 2 commits into
AltanS:mainfrom
Ad3Digital:feat/omp-live
Closed

feat(live): add hands-free Codex voice calls for OMP panes#164
Ad3Digital wants to merge 2 commits into
AltanS:mainfrom
Ad3Digital:feat/omp-live

Conversation

@Ad3Digital

Copy link
Copy Markdown

Summary

Add opt-in, hands-free Codex Live calls to local Oh My Pi (OMP) panes in Herdr. The phone carries the microphone and speaker while requests execute in the existing OMP session, so the operator can keep working away from the desk. This is bidirectional realtime audio, not speech-to-text pasted into the composer.

  • Mobile call sheet with start, mute/unmute, end, live transcripts, audio-playback recovery, and a visible-page wake lock; all six UI dictionaries are updated.
  • OMP companion extension owns Codex OAuth, signaling, and the realtime sideband. Delegated requests enter the current OMP conversation; progress and results return to the voice session.
  • Collie proxies bounded control requests to a token-protected, loopback-only companion. Discovery credentials and Codex OAuth are not returned to the browser. Existing identity, device, same-origin, and read-only gates remain in force.
  • Session pinning, a 45-second lease, and cleanup on cancellation, disconnect, session change, and shutdown prevent abandoned or cross-session calls.
  • Community Windows installer preserves the existing configuration/front door and supports Install, Status, and Restore. The Windows readiness probe now supplies the configured trusted identity instead of misreporting an authenticated bridge as unavailable.

Trying it

Requires a local Herdr pane, OMP with a working Codex OAuth account and Live access, and a secure browser context. Tested with OMP 18.1.10 and Bun 1.4.0.

  1. Build/start this Collie checkout and load the companion in a Herdr-hosted OMP process, for example:
    omp --extension /absolute/path/to/collie/omp/extension.ts
  2. Open the pane in Collie, choose Live call → Start call, and allow the microphone. /live-mobile in OMP reports companion readiness.
  3. If using global OMP extension configuration, restart existing OMP processes once; reloading an already-running process does not pick up the newly configured global path.

For an existing community Windows installation, contrib/windows/install-omp-live.ps1 performs the reversible cutover; -Action Restore restores the original service. No private configuration, account tokens, machine URLs, recordings, or generated build artifacts are included in this PR.

Verification

On the published branch in an isolated native Linux filesystem under WSL, using Bun 1.4.0:

  • Version consistency, root/web typechecks, and full-tree oxlint: pass.
  • Bun suites for bridge, CLI, scripts, and OMP (separate invocations, matching the current test script): pass.
  • Full web Vitest suite: 5,035 passed, 30 existing todo cases; production Vite/PWA build: pass.
  • After adding the shared Live discovery fixture, the affected chat/API/ack suites were rerun: 141 passed, with typecheck and lint passing again.
  • Shell suites collie-ctl.test.sh and check-tag.test.sh: pass.

Known upstream test failure: scripts/collie-cli.test.sh fails its empty external-call-log assertion with two tailscale status --json calls. The identical failure was reproduced independently on unmodified upstream ba39c05c6350a52bcb0a88f118cd0680ff85a1c5 (1.5.1), using the same environment and Bun installation. This unrelated suite was not changed, so the combined root test command is not claimed green.

Deployment smoke test on Windows 10 before rebasing onto 1.5.1: real Portuguese test audio over the installed HTTPS front door reached Codex Live, delegated a file write to OMP, and received spoken confirmation. WebRTC audio traffic, mute/unmute, track disposal on end, cancellation during pending microphone acquisition, stale-session/auth/origin refusals, and restore/reinstall were exercised. The operator subsequently confirmed the feature works in their own setup. The rebase preserved the Live implementation and resolved only the upstream test-command split.

Scope and caveats

  • Local OMP/Herdr panes only; no claim of pack-peer or other-harness support.
  • The browser should remain open and the screen awake. Background/locked-screen continuity depends on browser/OS policy and is not promised.
  • This uses the Codex Live backend/protocol used by OMP, not the general public OpenAI Realtime API. Account availability and upstream protocol changes can affect it.
  • Protocol code adapted from OMP retains its MIT attribution. Implementation and automated verification were AI-assisted; real usage was confirmed by the operator.

Base branch

  • Opened against main.

Checks

  • All combined checks pass — the independently reproduced upstream CLI shell failure above remains.
  • Fork PR: no release/version bump or CHANGELOG entry; versioning is left to the maintainer.

Suggested changelog: Add optional Codex Live voice calls for local OMP panes, with mobile audio controls and a reversible Windows installer.

@github-actions github-actions Bot added area: bridge Bun/TS bridge — Herdr socket, state engine, polling, REST area: web Phone PWA — UI, mirror, composer, service worker platform: windows Windows host — named-pipe bridge, contrib/windows area: harness Per-agent adapters and send guards (claude/codex/pi/omp) area: security Auth gates, pairing, CSP, exposure surface enhancement New feature or request labels Sep 4, 2026
@AltanS

AltanS commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Sorry this doesnt fit what i want to focus on right now. Feel free to chime in on the discussion around STT to explain exactly how you're using this feature and why it should be included.

@AltanS AltanS closed this Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: bridge Bun/TS bridge — Herdr socket, state engine, polling, REST area: harness Per-agent adapters and send guards (claude/codex/pi/omp) area: security Auth gates, pairing, CSP, exposure surface area: web Phone PWA — UI, mirror, composer, service worker enhancement New feature or request platform: windows Windows host — named-pipe bridge, contrib/windows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants