Skip to content

Add a view-mode selector to the Roku Device View (WebRTC playback)#833

Draft
chrisdp wants to merge 3 commits into
masterfrom
feature/device-view-modes
Draft

Add a view-mode selector to the Roku Device View (WebRTC playback)#833
chrisdp wants to merge 3 commits into
masterfrom
feature/device-view-modes

Conversation

@chrisdp

@chrisdp chrisdp commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

Adds a source-mode selector to the Roku Device View. A dropdown switches the panel between Screenshot (default, unchanged) and live stream/capture modes; more can be added later. Everything runs in the webview, with no extension host changes. The selected mode and a per-mode last-used URL persist in webview storage.

Modes:

  • Screenshot: the existing ECP screenshot view.
  • WebRTC (WHEP): POSTs a recvonly SDP offer, applies the answer, renders the track in a <video>, and DELETEs the session on Stop.
  • Video URL: native <video src> for progressive files.
  • MJPEG: native <img> multipart/x-mixed-replace.

Capability detection

Webview media support varies by platform and Electron build, so on load the panel probes what actually works and only offers supported modes (WebRTC gated on RTCPeerConnection; Video URL kept only if a native container plays). Native <video> support is measured with a real load-probe (load a tiny embedded clip, watch for loadeddata vs error), because canPlayType reports the codec allowlist without knowing whether the container demuxer is present. The Video URL mode shows the detected support inline.

VS Code webview media support (what we measured)

  • VS Code's bundled ffmpeg (Chromium 148 / Electron 42) has the MP4 demuxer but no Matroska/WebM demuxer, identically on macOS, Linux, and Windows. Native WebM file playback fails everywhere, regardless of codec.
  • H.264/AAC mp4 plays (AAC via the OS platform decoder).
  • WebRTC plays both H.264 and VP8/VP9, because it uses RTP plus its own bundled decoders and never the container demuxer. This is the portable path for a live device view.
  • Reference: Webms with supported format not previewable in VSCode microsoft/vscode#195758 (closed as upstream/electron).

Follow-ups

  • Cursor-location node inspection overlay (hover highlight + node info from Screenshot mode) available in all modes
  • Additional source modes as needed
  • Configurable ICE servers (custom STUN/TURN) for WebRTC streams that can't connect over the default public STUN alone
  • Auth support for WHEP endpoints that require it (e.g. Authorization: Bearer <streamKey>)
  • Auto-reconnect on dropped WebRTC connections
  • Audio starts muted for autoplay reliability; add an unmute affordance and remember the preference
  • Recent-URL history instead of a single remembered URL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant