gfx: caching + offscreen infrastructure (pre-scene, from #2109) by jcelerier · Pull Request #2119 · ossia/score

jcelerier · 2026-07-14T05:43:00Z

Extracts the self-contained gfx infrastructure from #2109 that compiles against master independently of the coupled "scene concept" rework. Landing these first shrinks the eventual scene-rework PR and makes it reviewable.

These are additive: new files, wired into CMake, not yet referenced by any consumer — the scene rework (a later PR) wires them up. They are included here so that PR is smaller and each piece can be reviewed on its own.

Stacked on #2117 (test infrastructure); will retarget to master once #2117 merges.

Commits

gfx: add AssetTable + TextureLoader — a shared cache for decoded image/texture assets (Gfx/AssetTable.*, Gfx/Graph/TextureLoader.*). Depends only on QImage/QRhi/ossia::hash_map.
gfx: make staged-but-never-acquired AssetTable entries evictable (follow-up to the above).
gfx: add OffscreenDevice + RhiPreviewWidget + a Metal buffer-copy backend — offscreen QRhi rendering + a preview widget, using the existing Graph/GfxContext/BackgroundNode APIs.

Deferred to the scene-rework PR (Plan D)

The GPU memory arena (GpuResourceRegistry + OffsetAllocator submodule) — hard-depends on SceneGPUState.hpp, which the scene concept introduces.
The pipeline-cache — only a minimal QRhi load/store is separable; its async-flush hook is entangled with the RenderList rebuild, so the whole piece lands with the scene rework.

Validation

Builds cleanly against master (all plugins).
ctest: 17/17 (no regression; the additions are not yet on any exercised path).

🤖 Generated with Claude Code

test_unit_gfx_assettable: AssetTable CPU logic + TextureLoader upload paths on QRhi's Null backend (no GPU needed). tests/hardware/: capability-probe framework (AJA/DeckLink/Magewell/PipeWire/ display) + ScoreHardwareTests.cmake — gated, run only on capture rigs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

FIX-ISF (libisf/src/isf.cpp — applies on top of this branch's ISF/CSF feature rework, hence #2120 not #2119): - write_isf: seekp(0,end)-before-tellp corrupted the emitted descriptor - use-after-move on m_sourceVertex -> guard with .empty() - parse_shadertoy_json: GLSL45 preludes + iMouse vec4(0,0,0,0) - glsl_sandbox: duplicate-TIME uniform guard - replace_identifier: word-boundary match Tests: test_unit_isf_importers (all 5 importer branches); golden-image render regression (16 pinned JS-corpus cases, llvmpipe, SSIM/PSNR); timeline-driven scenario ramp; gfx resource/leak soak (DISABLED — guards a still-unfixed render-clock teardown UAF); live-edit churn corpus. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

FIX-ENC: - NV12.hpp: UV plane encodes into an R8 target (16-byte-tight rows), was mis-sized producing garbage chroma on readback - YUVPlanar.hpp: p420_10 -> RGBA8-packed 16-bit LE on the Qt<6.10 GL path Tests: test_unit_video_pixel_format (pixfmt plane math + avutil mapping); test_integration_encoder_matrix (every encoder on a real QRhi backend, byte-checked); test_regression_offscreen_teardown (full-app offscreen render + /stop + /exit exits 0 — guards 1228382 #2119 + 32ad555 #2121 teardown UAFs); VideoDecoderTester + video-decoder-sweep.sh; EncoderTester ctest. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

An unconfigured Text process rendered nothing: the UBO default position was {0.5,0.5} (TextNode.hpp), a clip-space offset added to the text quad in the vertex shader, which shoved the default text ~half a screen up+right — off the top edge — until Position was manually set. Default to {0,0} (screen centre). Found + verified by the P3-text render harness (untouched Text now renders 1508 visible px; the harness's default-visibility check is a hard assertion). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

FIX-ISF (libisf/src/isf.cpp — applies on top of this branch's ISF/CSF feature rework, hence #2120 not #2119): - write_isf: seekp(0,end)-before-tellp corrupted the emitted descriptor - use-after-move on m_sourceVertex -> guard with .empty() - parse_shadertoy_json: GLSL45 preludes + iMouse vec4(0,0,0,0) - glsl_sandbox: duplicate-TIME uniform guard - replace_identifier: word-boundary match Tests: test_unit_isf_importers (all 5 importer branches); golden-image render regression (16 pinned JS-corpus cases, llvmpipe, SSIM/PSNR); timeline-driven scenario ramp; gfx resource/leak soak (DISABLED — guards a still-unfixed render-clock teardown UAF); live-edit churn corpus. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

Headless llvmpipe golden+value harness for the Gfx::Text process: grabs the untouched-defaults frame (hard off-screen-default visibility assertion — guards the #2119 default-position fix), then live-edits text/font/size/position/scale/ color over OSC /script and asserts pixel VALUES per case (coverage, bbox ordering across sizes, centroid movement, channel dominance, blank empty string, unicode/CJK/tofu/2000-char) + golden refs (compare.py strict). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

FIX-ENC: - NV12.hpp: UV plane encodes into an R8 target (16-byte-tight rows), was mis-sized producing garbage chroma on readback - YUVPlanar.hpp: p420_10 -> RGBA8-packed 16-bit LE on the Qt<6.10 GL path Tests: test_unit_video_pixel_format (pixfmt plane math + avutil mapping); test_integration_encoder_matrix (every encoder on a real QRhi backend, byte-checked); test_regression_offscreen_teardown (full-app offscreen render + /stop + /exit exits 0 — guards 1228382 #2119 + 32ad555 #2121 teardown UAFs); VideoDecoderTester + video-decoder-sweep.sh; EncoderTester ctest. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

(cherry picked from commit 1944985)

stage() inserted a DecodedAsset with refcount==0 and added its bytes to m_total_bytes but left in_lru=false, so it never entered the cold LRU. Since trim()/maybeAutoTrim() evict only from the cold pool, an asset that was staged (e.g. a decoded glTF baseColor.jpg) but whose material was removed before any acquire() stayed in the table forever -> unbounded growth over a session. Seed freshly-staged entries into the cold LRU (in_lru=true, m_cold_bytes += byte_size) so an unconsumed stage is trimmable. acquire() already splices a cold entry back out of the pool and decrements m_cold_bytes on first use, keeping the byte accounting consistent. (cherry picked from commit a7464ff) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT

(cherry picked from commit 4745b5f)

An offscreen BackgroundNode output is owned by the Window device, so on shutdown WindowDevice::disconnect() destroys it (and its QRhi) before Graph::~Graph runs — the async node-removal queue is never drained during teardown. This left the graph with a dangling m_outputs pointer plus a RenderList over an already-freed QRhi (deleteLater on a null QRhi; dynamic_cast over a freed node in ~Graph), and double-freed the node, which is owned by BOTH the device (m_node) and the GfxContext `nodes` map (register_node). Add GfxContext::destroyOutput(), called synchronously from ~offscreen_device while the node and its QRhi are still alive: it releases the output's RenderList via Graph::destroyOutputRenderList (which drops it from m_outputs), removes it from m_nodes, and .release()s the double-owned nodes-map entry so only the device frees the node. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

…ails (cherry picked from commit ee1a189)

(cherry picked from commit 42e44a5)

test_unit_gfx_assettable: AssetTable CPU logic + TextureLoader upload paths on QRhi's Null backend (no GPU needed). tests/hardware/: capability-probe framework (AJA/DeckLink/Magewell/PipeWire/ display) + ScoreHardwareTests.cmake — gated, run only on capture rigs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

An unconfigured Text process rendered nothing: the UBO default position was {0.5,0.5} (TextNode.hpp), a clip-space offset added to the text quad in the vertex shader, which shoved the default text ~half a screen up+right — off the top edge — until Position was manually set. Default to {0,0} (screen centre). Found + verified by the text-render harness (untouched Text now renders 1508 visible px; the harness's default-visibility check is a hard assertion). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

Qt's tga handler only accepts TGA 2.0 files ending in the "TRUEVISION-XFILE." footer; plain original-spec TGAs — still produced by common tools — were silently rejected even though the image loaders advertise *.tga. When a decode fails and the header plausibly describes a TGA, retry with the footer appended. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

GfxContext::destroyOutput() released the output's RenderList and dropped the node, but never removed it from m_renderClocks. A TimerClock holds raw OutputNode* and its tick iterates them calling canRender()/render(), so the freed node stayed reachable: tearing an output down pumps the event loop (closing an inspector preview, removing a device, shutting down), the next queued tick fired, and it dereferenced freed memory. The asynchronous REMOVE_NODE path already unregisters from the clocks and drops emptied ones; do the same here, before the node goes away. Found by the gfx resource/lifetime soak, which is disabled precisely because it reproduced this use-after-free. With the fix it runs its 60 create/destroy cycles ASAN-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

Two more lifetime bugs on the output-teardown path, both reachable by removing a device or closing an inspector preview. Graph::removeNode is, in its own words, "a pure pointer erase": it leaves m_edges holding Edges that point at the removed output's Ports. Ports are owned by their Node, so once the device freed the output, ~Graph -> clearEdges() deleted those Edges and ~Edge unlinked them from freed Ports -- a heap-buffer-overflow read in Edge::~Edge, reproducible by adding and removing a device a dozen times. destroyOutput now calls removeNodeAndEdges() first, exactly as the asynchronous REMOVE_NODE path does. RhiPreviewWidget held a bare GfxContext*. An inspector widget can be destroyed after the context it points to: InspectorPanel de-selects widgets with deleteLater(), forceCloseDocument's processEvents() does not flush that delete (it is posted from a nested scope, so Qt reposts it), and the document -- hence ~DocumentPlugin, hence ~GfxContext -- is deleted before the main loop finally delivers it. detach() then called unregister_node() on freed memory. The sibling texture-preview path (TextureOutletBackgroundRenderer) already guards its plugin with a QPointer; do the same here so the existing null check holds. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

In VSync mode the on-screen render loop re-drives itself solely through QWindow::requestUpdate(). On xcb that is an unconditional timer, so the loop self-heals. On Wayland requestUpdate() is frame-callback gated: if the compositor withholds a frame callback past QT_WAYLAND_FRAME_CALLBACK_TIMEOUT (default 100ms) -- occluded window, blanked/DPMS output, another surface on top, or heavy per-frame work -- Qt marks the window inexposed and the loop, having no other driver, produces no frames. The render itself is fine: an offscreen readback of the same frame returns correct pixels, so the symptom is a black on-screen window despite correct rendering. Confirmed by flipping only the VSync setting on a KDE-Wayland session, and by a minimal QRhi repro whose requestUpdate loop dies under a forced 1ms frame-callback timeout while a timer-driven loop is unaffected. Gate the vsync loop off when the Qt platform is Wayland so it takes the timer path. The compositor supplies frame pacing there, so no tearing results from dropping swap-chain vsync. xcb, eglfs and the embedded backends are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

Replace the timer->set<OutputNode*> map with RenderClock: one TimerClock per distinct manualRenderingRate owning the coalesced set of outputs it drives, and a DisplayVSyncClock wrapping the swap-chain vsync callback. Behaviour is unchanged; the clocks are torn down before the timer pool is rebuilt so their dtors release the shared timers back to a still-live pool. The teardown paths added earlier in this branch already unregister outputs from the render clocks, but the abstraction they use landed further up the series; this brings it down to where it is first needed. Dropping an output's edges on teardown needs Graph::removeNodeAndEdges, which is part of the incremental render-list rework, so that call moves to the commit that introduces it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

coveralls · 2026-07-25T20:28:46Z

coverage: 15.406% (+0.1%) from 15.311% — planc/gfx-infra into master

jcelerier mentioned this pull request Jul 14, 2026

gfx: scene-concept rework (the coupled core of #2109) #2120

Open

jcelerier force-pushed the planc/gfx-infra branch from 1228382 to 1b6651e Compare July 17, 2026 11:58

jcelerier force-pushed the planc/gfx-infra branch from 1b6651e to 08516ad Compare July 17, 2026 14:10

jcelerier changed the base branch from tests/infrastructure to master July 18, 2026 13:07

jcelerier force-pushed the planc/gfx-infra branch from 6470247 to 2676d67 Compare July 18, 2026 13:37

jcelerier changed the base branch from master to planb/app-robustness July 18, 2026 13:37

jcelerier force-pushed the planc/gfx-infra branch 2 times, most recently from 387b649 to 65ff611 Compare July 19, 2026 03:14

jcelerier changed the base branch from planb/app-robustness to master July 19, 2026 21:59

jcelerier force-pushed the planc/gfx-infra branch 5 times, most recently from 5411148 to 2b108de Compare July 24, 2026 23:07

jcelerier and others added 9 commits July 25, 2026 07:51

gfx: add AssetTable + TextureLoader for shared decoded-asset cache

2f9a57f

(cherry picked from commit 1944985)

gfx: add OffscreenDevice + RhiPreviewWidget + Metal buffer-copy backend

3136b3e

(cherry picked from commit 4745b5f)

gfx: don't return a half-initialized render state when QRhi::create f…

9b90c44

…ails (cherry picked from commit ee1a189)

gfx: clear image inputs with no incoming edge each frame

75951c6

(cherry picked from commit 42e44a5)

jcelerier and others added 4 commits July 25, 2026 07:51

jcelerier force-pushed the planc/gfx-infra branch from 2b108de to 51e0e1b Compare July 25, 2026 11:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

gfx: caching + offscreen infrastructure (pre-scene, from #2109)#2119

gfx: caching + offscreen infrastructure (pre-scene, from #2109)#2119
jcelerier wants to merge 13 commits into
masterfrom
planc/gfx-infra

jcelerier commented Jul 14, 2026

Uh oh!

coveralls commented Jul 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

jcelerier commented Jul 14, 2026

Commits

Deferred to the scene-rework PR (Plan D)

Validation

Uh oh!

coveralls commented Jul 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants