Skip to content

fix(mediaplayer): target-exact seeks, clean backward seeks and true VOD end#979

Open
towneh wants to merge 6 commits into
BasisVR:developerfrom
towneh:fix/mediaplayer-seek-fixes
Open

fix(mediaplayer): target-exact seeks, clean backward seeks and true VOD end#979
towneh wants to merge 6 commits into
BasisVR:developerfrom
towneh:fix/mediaplayer-seek-fixes

Conversation

@towneh

@towneh towneh commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

A batch of VOD seek and end-of-stream fixes in the native plugin. It started from the 2026-07-20 test night reports; verifying those on Windows and Quest surfaced the rest. All the symptoms below are present in the current release binaries, and the fixes are C and C++ only.

  • Seeks played the keyframe run-up instead of landing at the target. The pace clock re-anchored to the first delivered sample after a reposition, so the run-up from the preceding keyframe played out in real time. On sparse-keyframe files a mid-GOP seek could replay from as far back as the file start. The pace clock now re-anchors at the seek target, the run-up floods at decode speed, and both decoders decode it for references while dropping it from presentation. HLS-TS seeks become target-exact rather than segment-aligned as a side effect.
  • Backward seeks lost the first second or so of audio. Until the first post-seek frame presents, the audio serve clock still describes the pre-seek timeline, and on a backward seek that stale (higher) clock trimmed freshly banked post-target audio as long-stale. The clock is invalidated at seek and re-derives from the first post-seek present.
  • Paced sources ended a couple of seconds early. ENDED fired when delivery finished rather than when presentation did, and the video decoder's reorder tail (seconds of content at low frame rates) was never flushed out. The demux loop now drains presentation before raising ENDED and notifies the decoder end-of-stream (MFT drain on Windows, an EOS input plus bounded output pump on Android).
  • An HLS VOD stopped seeking after playing to the end. The fetcher thread exited at the endlist, so a later seek had nothing left to serve it. It now parks at VOD end and a seek revives it; end-of-stream is raised by the reader instead, so a parked fetcher can't be mistaken for a finished stream. An HLS VOD that cannot seek (fMP4 variant) still exits rather than parking forever.
  • Backward seeks flashed pre-seek content and bounced the seek bar. Three routes in, each closed: pre-seek tail AUs delivered across the seek (dropped by an engine-side gate mirroring the audio one, plus an await-keyframe gate in the decoders); frames still in flight through the AImageReader listener when the flush clears the ring (dropped by a seek-generation tag carried in the sub-microsecond digits of the surface timestamp); and the await-keyframe gate itself opening at the keyframe's submission, while a slipped tail AU's post-flush garbage frame was still inside the codec with a pre-seek PTS past the target. The gate now stays closed until the output carrying the keyframe's own PTS emerges, with a bounded drain backstop so a dropped or re-stamped output degrades to one stale frame rather than wedging video.
  • A seek landing exactly on a banked frame's PTS could end playback without showing it. The presentation-pending probe compared ring PTS against the position value, which a seek snaps to the target before anything presents, so an exact-target frame read as already shown. It now compares against the last genuinely presented PTS.

TESTING.md's seek rows now spell out target-exact landings, the sparse-keyframe adversarial case, and seeking from the tail of an HLS VOD.

Required checks

All boxes below must be ticked before this PR can merge. If a check is genuinely N/A, tick it anyway and explain under Notes.

  • Tested — I built and ran this locally. The change works in the editor and (where relevant) in a built player.
  • Transform access is combined and limited — In hot paths, transform reads/writes go through TransformAccessArray or are otherwise batched. I have not added per-frame transform.position / transform.rotation / transform.localPosition calls inside loops. Whenever I need both position and rotation, I use the combined APIs — SetPositionAndRotation / SetLocalPositionAndRotation for writes, GetPositionAndRotation / GetLocalPositionAndRotation for reads — instead of two separate property accesses; the combined call does one local-to-world matrix traversal instead of two.
  • Addressables used for asset/memory loading — Any new asset loads go through Addressables. No new Resources.Load, no direct asset references that pull large content into memory on scene load.
  • No new GetComponent / AddComponent where avoidable — Where unavoidable, the result is cached on a field, and any GetComponent<T> is replaced with TryGetComponent<T>(out var x) — bare GetComponent will be denied. TryGetComponent is the modern API (Unity 2019.2+) and skips the Editor-only GC allocation GetComponent causes when a component is missing: Unity wraps the null return in a managed "fake null" object so its overloaded == operator can still detect destroyed C++ objects, and constructing that wrapper allocates; TryGetComponent returns a bool plus out parameter and never builds the wrapper. None of these calls run inside Update, LateUpdate, FixedUpdate, jobs, or other per-frame code paths.
  • Per-frame work is scheduled through BasisEventDriver — Any new per-frame work hooks into BasisEventDriver rather than adding standalone Update / LateUpdate / FixedUpdate callbacks on a MonoBehaviour.
  • Anything added to BasisEventDriver is bulletproof, or guarded by try/catchBasisEventDriver runs the single per-frame tick that drives the whole framework (network apply, local player sim, blendshapes, JigglePhysics, nameplates, and more) as one sequential chain. An unhandled exception anywhere in that chain aborts the rest of the tick, so every step after the throwing one is silently skipped for that frame. New work added to the driver must either be guaranteed not to throw, or be wrapped in a try/catch that contains the failure and surfaces it through BasisDebug — logged once / rate-limited, never every frame (see the existing HVRBasisBuiltInAddresses.Simulate() guard for the pattern). Expect this to be scrutinized closely in review.
  • Considered jobification — I asked whether this work can be moved to a Unity Job (Burst-compiled where possible). If it can, it is. If it cannot, the reason is in Notes.
  • No needless { get; set; } properties or access lockdowns — Public fields are fine; Basis is meant to be read and modified freely, so don't wall things off private/internal without a real reason. Don't wrap a field in { get; set; } when the accessors do nothing — property accessors have a real performance cost vs direct field access, and the lead maintainer prefers plain fields (or a method / setter-only property when only the setter needs logic) over a noop-getter pair. For .Instance singletons, callers reassigning Type.Instance is allowed; if that would break your code, log a warning or throw — don't block the assignment. Locking down access is not your call.
  • Camera access goes through BasisLocalCameraDriver — Code that needs the local camera (transform, projection, rig data, etc.) pulls it from BasisLocalCameraDriver rather than looking one up itself. Don't roll a separate camera discovery path.
  • Logging uses BasisDebug — All new logging calls go through BasisDebug.Log / BasisDebug.LogWarning / BasisDebug.LogError (with an appropriate LogTag) instead of UnityEngine.Debug.Log / Debug.LogWarning / Debug.LogError. BasisDebug routes through Basis's tagged, color-coded logger and respects the project-wide LoggingDisabled toggle so logging can be killed at runtime; bare Debug.Log calls bypass that and will be denied.
  • No scene-wide discovery for dependencies — New code is architected so it does not need FindObjectOfType / FindObjectsOfType / GameObject.Find / FindGameObjectsWithTag to locate what it depends on. References are wired in — registered through an existing manager/driver, injected at init, or passed in by the caller — rather than discovered by scanning the scene at runtime. If a scene scan is genuinely unavoidable, justify it under Notes.
  • No allocations in hot paths — Per-frame code (Update / LateUpdate / FixedUpdate, simulation loops, jobs, anything called once per frame or more) does not allocate. No new on reference types, no LINQ, no string concatenation/interpolation, no boxing, no foreach over interface-typed collections. Allocate once at init and reuse the buffer.
  • No debugging in hot paths — No log calls of any kind on per-frame paths, including BasisDebug. Hot-path logging floods the console and incurs cost on every frame regardless of whether the message is filtered out downstream. If a hot-path log is needed while iterating, gate it behind #if UNITY_EDITOR and remove (or leave gated) before merge.
  • Hot-path collection access is optimized — Cache .Count (lists) / .Length (arrays) into a local int before the loop instead of re-reading the property each iteration. Prefer T[] (with a separate length int when the array is over-sized) over List<T> where the data is hot — Unity's mono BCL doesn't expose CollectionsMarshal.AsSpan(List<T>), so a list can't be fed into Span<T> / unsafe paths cleanly. Where the perf justifies it, drop into Span<T> / ref locals / Unsafe.As / unsafe pointer code to skip bounds checks and copies, and call out the invariants you're relying on under Notes so reviewers can sanity-check them.

Testing details

Tick the platforms you actually tested on. Leave the rest unticked — these are informational and do not block merge.

  • Windows
  • Linux
  • Android
  • iOS
  • macOS

Input / control mode coverage:

  • Tested in VR (note headset under Notes)
  • Tested in desktop / non-VR mode
  • Tested with phone controls (mobile touch input)
  • N/A — change does not touch player/XR/input code

Where applicable, confirm these flows still work after your changes:

  • Hot-switching (desktop ↔ VR mode swap at runtime)
  • Avatar swapping
  • Server swapping (joining / leaving / changing servers)
  • N/A — change does not touch any of the above

Notes

  • The change is entirely native (C and C++ in Native~) plus TESTING.md; no C# is touched, so the Unity-specific required checks are ticked as N/A on that basis.
  • Verified per TESTING.md's seek and end-of-stream rows on both halves. Editor (Windows): forward and backward seeks on a normal-GOP MP4, a sparse-keyframe MP4 (keyframes only at 0s and 31s, the adversarial case) and an HLS-TS VOD, each played to its true duration. Quest Pro build: the same three lanes with backward seeks mid-file and from the tail, landings and ends confirmed against the diagnostics CSV (single clean position jump per seek, ends at true duration).
  • Known remaining issue, pre-existing and tracked separately: on audio-only MP3, the reported position stalls after a forward seek and can overrun the duration after a backward one. The audio itself seeks and plays correctly; it's position bookkeeping in the MP3 demuxer, and it predates this branch.
  • Both shipped binaries (Windows x64, Android arm64-v8a) are rebuilt from this branch's source.

towneh added 6 commits July 22, 2026 21:10
…eyframe run-up

A container seek repositions the byte source to the sync sample at or
before the target, and the pace clock then re-anchored on the first
delivered sample — so the whole run-up played out at 1x. On a
sparse-keyframe file the gap can be tens of seconds: a mid-GOP seek
visibly restarted at the previous keyframe (or, with the present clock
pinned, sat silent at the target) for as long as the gap was wide. A
1.1MB 46s capture with keyframes only at 0s and 31.25s reproduced it on
every seek inside the first 31 seconds.

Re-anchor the pace clock at the seek target when a demux leg takes the
seek, so the run-up reads as late and flows at decode speed while
everything from the target onwards paces at 1x as before. Both decoders
then drop decoded frames short of the target instead of banking them -
they exist only as references - so the run-up is never shown and the
present clock releases on the first frame at or past the target. Audio
needs nothing: the demuxer's audio cursor already lands at the target,
and the PCM serve's clock-gated trim eats anything earlier.

HLS-TS seeks inherit the same mechanism through the TS demuxer's
take_seek, so they now land target-exact rather than at the start of
the containing segment.

Verified with a standalone decode harness against the sparse-keyframe
repro (forward into the gap, backward, past the far keyframe), a
normal-GOP progressive MP4, and a TS-HLS VOD: the run-up floods through
in about a second, post-target frames bank for presentation, and audio
resumes immediately. Presentation needs an in-Editor pass (the harness
cannot drive Unity's render tick).
…n backward seeks

The audio serve clock re-derives from video presents, and between a seek
and the first post-seek present it still describes the pre-seek
timeline. On a backward seek that clock is ahead of the target, so audio
banked during the settle reads as long-stale and the clock-gated trim
discards it - audibly, video resumed about a second before audio did,
with the trim counter climbing through every settle. Forward seeks were
immune only because their stale clock sits behind the target, which
makes the serve hold rather than trim.

Invalidate the offset at the seek notify on both platforms so the serve
holds in either direction: post-seek audio banks through the settle and
releases in sync with the first presented frame. Audio-only sources are
unaffected - their offset never leaves the hold state to begin with.
ENDED fired when delivery finished rather than when presentation did, and the
video decoder's reorder tail (seconds of content at low frame rates) was never
flushed, so paced sources ended early. The demux loop now notifies the decoder
end-of-stream (MFT drain on Windows, an EOS input plus a bounded output pump
on Android) and drains presentation before raising ENDED: done once the
decoder holds nothing more to show or serve and the reported position has
settled, with an absolute cap as the escape hatch for a consumer that never
presents.

The presentation-pending probe compares against the last genuinely presented
PTS rather than the reported position, which a seek snaps to the target
before anything presents: a banked frame whose PTS lands exactly on the
target would otherwise read as already shown, and ENDED could fire without
showing it.
The fetcher thread exited at the endlist, so a seek after a VOD played to its
end had nothing left to serve it. The fetcher now parks at VOD end and a seek
revives it; end-of-stream is raised by the reader once the generations have
settled, so a parked fetcher no longer reads as a finished stream, and
producer_done again means the thread actually exited. An HLS VOD that cannot
seek (the fMP4 variant) still exits rather than parking forever.
Backward seeks could flash pre-seek content and bounce the reported position:
any pre-seek frame that slips through carries a PTS past a backward target, so
it not only presents but can end the preroll cut and let the keyframe run-up
present at decode speed. Three routes in, each closed:

- Pre-seek tail AUs delivered between the seek request and the demuxer taking
  it are dropped at the engine sink by the seek_taken gate (mirroring the
  audio gate), and an await-keyframe gate in the decoders covers the HLS path
  the engine gate cannot see.
- Frames still in flight through the AImageReader listener when the seek
  flush clears the ring are dropped by a seek-generation tag carried in the
  sub-microsecond digits of the surface timestamp. The tag is the feeder
  thread's generation, which only advances at the flush, so a pre-seek frame
  drained after the seek posts still carries the old tag.
- The await-keyframe gate opened at the keyframe's submission, while a
  slipped tail AU's post-flush garbage frame could still be inside the codec.
  It now holds until the output carrying the keyframe's latched PTS emerges,
  with a bounded drain backstop so a dropped or re-stamped keyframe output
  degrades to one stale frame instead of wedging video. This applies to both
  decoders: Adreno emits post-flush mid-GOP garbage directly, and the Windows
  MFT reorder pipeline has the same post-submission window.
Built from this branch's source; verified on Quest Pro (backward-seek matrix
across progressive MP4, sparse-keyframe MP4 and HLS-TS VOD) and in the
Editor (same three lanes, forward + backward seeks, true-duration ends).
@towneh towneh added the bug Something isn't working label Jul 22, 2026
@towneh
towneh requested a review from dooly123 July 22, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant