layer-surface: clear any stale buffer when creating the layer surface

[perlowja]

The bug

A wl_surface can outlive the layer surface built on it. GTK keeps one wl_surface across hide/show and tears down only the role object, so layer_surface_create_surface_object() frequently runs on a surface that was previously mapped. wl_surface state is persistent, so a buffer that was current when the previous layer surface was destroyed is still current at that point.

The protocol requires the first commit after get_layer_surface to carry no buffer. If anything attached one in between, that initial commit carries it and the compositor kills the client:

zwlr_layer_surface_v1 error 2: "layer_surface has never been configured"
Gdk-Message: Error 71 (Protocol error) dispatching to Wayland display.

From WAYLAND_DEBUG=1, a working cycle vs a crashing one in the same session:

working  (#77 -> #64)          crashing (#64 -> #82)
destroy()                      destroy()
attach(nil); commit()          attach(nil); commit()
<nothing>                      attach(wl_buffer#79); commit()   <-- stray
get_layer_surface(#64)         get_layer_surface(#82)
commit() -> configure OK       commit() -> ERROR 2

The only difference is a frame queued by a closing animation landing ~0.7 ms after the unmap.

This isn't GTK failing to clean up

I checked before assuming. gdk_wayland_surface_hide_surface() in GTK 4.22 already calls gdk_wayland_surface_clear_frame_callback() and then wl_surface_attach(NULL) + wl_surface_commit() — that's the attach(nil); commit() visible in both traces. The stray attach arrives after all of it. GTK destroys the wl_surface only on destroy, not on hide, so surface reuse across hide/show is deliberate GTK behaviour rather than an oversight.

That leaves the gap on the library side: a new role object gets created over a surface whose buffer state nobody re-cleared.

The change

wl_surface_attach(wl_surface, NULL, 0, 0) + wl_surface_commit() immediately before get_layer_surface.

Placement matters, and the protocol is explicit about it:

Creating a layer surface from a wl_surface which has a buffer attached or committed is a client error, and any attempts by a client to attach or manipulate a buffer prior to the first layer_surface.configure call must also be treated as errors.

So clearing after get_layer_surface would be too late — the error is already committed by then — and the attach itself would be the second half of that sentence. The commit is required too: attach alone only sets pending state and would not clear the current buffer.

The surface is roleless at that point (the previous role object was destroyed at unmap), so attach+commit is legal, and it is the same sequence GTK performs in gdk_wayland_surface_hide_surface(). On a genuinely fresh surface it is a no-op.

Relationship to #94 and #119

#94 looks like this exact failure — same error, same "random after N opens", reproduces only on the gl/vulkan renderers — and was closed because Hyprland changed rather than because a client-side cause was found. #119 addressed teardown ordering and was declined as a smithay bug. This is a different mechanism: not ordering, and not compositor-side, but persistent wl_surface buffer state surviving into a new role object.

Validation

Hardware: CIX Sky1 (Radxa Orion O6N), labwc/wlroots, GTK4 4.22.4, gtk4-layer-shell 1.3.0, GLES on Mali (libmali).

Tested on metal by isolating the library as the only variable. The client is a shell build that does not contain any client-side workaround, so the library is doing all the work:

boot	shell binary	libgtk4-layer-shell	result
17:32	e23ebc08 (no client-side fix)	stock 55919c08	crashed immediately on repeated launcher opens
17:35	e23ebc08 (identical binary)	patched 0f43ad6d	repeated launcher opens + workspace pager, no crash

Same binary, same kernel, same session type, minutes apart. On the patched run the shell process is one second younger than the compositor it was launched by and stayed that way throughout testing, i.e. it never restarted.

For completeness on how the mechanism was originally established: a client-side workaround (dropping the GdkSurface on close, so each open gets a fresh wl_surface) also eliminates it, and a Wayland trace analyzer that flags a layer surface re-created over a surface still holding a buffer went from 1 dangerous re-creation to 0 across 36 layer surfaces. That workaround is proposed separately downstream, but it has now twice missed close paths that a client has to remember to cover — which is the argument for fixing it here instead, once, for every client.

Why this may not reproduce on your hardware

A hypothesis, not a claim. The bug needs the animation frame to land inside the window between unmap and the next open, bounded by when wl_buffer.release returns. On unified-memory Mali the buffer is the same physical page the compositor samples, dmabuf-imported with no copy, so release can come back almost immediately — hence the ~0.7 ms figure. On a discrete GPU the buffer sits behind an explicit VRAM transfer and that window may never open wide enough to hit.

That would also explain the gl/vulkan-only character in #94 — those are the renderers that still have a live GPU buffer attached at close time — and the "random after N opens" behaviour, since the race window is set by memory topology rather than by anything in the client.

We should be able to test that directly before long: we're bringing this shell up on an 8 GB Dragon Q8B with a GPU, and I'll report back either way. Happy to hold the PR until then if you'd rather see the cross-hardware result first.

---

Review notes

An earlier revision of this PR placed the clear after get_layer_surface. That was wrong for the reason quoted above, and an adversarial review pass on our side caught it before it reached you. Corrected in 4920e1b.

Two other candidate objections from that review, checked and dismissed with evidence:

• Could the session-lock wl_surface.attach hook swallow this null attach? No. surface_data entries are only added inside if (self) in create_xdg_surface_hook, where self = callback(wl_surface) returns a lock_surface_t*. A layer surface has none, so it never enters that table.
• Are 0, 0 offsets correct for wl_surface v5+? Yes — non-zero offsets raise invalid_offset; zero is required, and is valid on older versions too.

[wmww]

I'm sorry you're having problems with the library. My Claude was unable to reproduce this issue or find a likely cause. Please provide:

• compositor + version (labwc 0.20.1?)
• minimal reproducer code
• steps to reproduce
• full WAYLAND_DEBUG=1 log

WAYLAND_DEBUG=1 log is especially important if you're unable to make a minimal reproducer for some reason, or in case I can't reproduce due to hardware differences or something. I'm hesitant to merge the attach null buffer fix as I think it might only partially mitigate the issue, and it could exacerbate Smithay/smithay#1979.

[perlowja]

Reproducer below — 90 lines, no shell involved, fails 5/5 on stock 1.3.0 within four layer surfaces.

Two ingredients, both easy to miss

I could not reproduce this at first, and the reason is worth stating because it probably explains why you couldn't either:

1. Two or more layer windows must be cycling. With one window it never fails — 720 surfaces clean across three runs. With two it fails every run.
2. The hide must be dispatched after a paint has been queued, not before. Hiding from inside the tick callback is clean; hiding from a G_PRIORITY_HIGH idle scheduled in that tick fails. The idle stands in for a real close animation, which is what queues the late frame in production.

Miss either and the surface is always clean at role creation.

Results

Same binary, same compositor, one variable — which .so is loaded via LD_LIBRARY_PATH:

library	runs	layer surfaces	dangerous re-creations	protocol errors
stock 1.3.0-1+b1	5	4 then dies	1 per run	1 per run
this PR	5	240 (completed)	0	0

"Dangerous re-creation" = get_layer_surface() on a wl_surface whose current buffer is non-nil. I count that rather than only counting crashes, because whether the compositor actually errors is a further race on wl_buffer.release — the dangerous state occurs more often than the crash does.

The failing sequence

-> zwlr_layer_surface_v1#57.destroy()
-> wl_surface#53.attach(nil, 0, 0)                    <- GTK's clean unmap
-> wl_surface#53.commit()
-> wl_surface#53.attach(wl_buffer#64, 0, 0)           <- stray: the queued paint lands
-> wl_surface#53.commit()
-> zwlr_layer_shell_v1#46.get_layer_surface(new id zwlr_layer_surface_v1#57, wl_surface#53, nil, 3, "lsrepro")
-> wl_surface#53.commit()
wl_display#1.error(zwlr_layer_surface_v1#57, 2, "layer_surface has never been configured")

wl_surface state is persistent, so the buffer attached after the unmap is still current when the next role is created, and its first commit carries a buffer before any configure.

Environment

• labwc 0.9.5, wlroots 0.20.2
• GTK 4.22.4+ds-1, gtk4-layer-shell 1.3.0-1+b1 (verified stock with dpkg -V before each run)
• wayland-client 1.26.0, GSK_RENDERER=ngl (also reproduces with gl)
• aarch64, CIX Sky1, Mali G720, GLES

Production case that led here: two layer windows — an application chooser and a virtual-desktop chooser — opened and closed at 1080p. The reproducer uses 320x200 windows, so surface size is not a factor.

Steps

gcc -O1 -o lsrepro lsrepro.c $(pkg-config --cflags --libs gtk4 gtk4-layer-shell-0)
WAYLAND_DEBUG=1 ./lsrepro 2> wayland.log
# stock:   dies within ~4 layer surfaces with the error above
# patched: prints "completed 120 cycles per window with no protocol error"

On whether this is the right fix

I'm not arguing the layer. The measurement above says this patch removes the dangerous re-creation on the machine that reproduces it; it doesn't tell you whether clearing the buffer in the library is preferable to requiring callers to hand over a clean surface, and it doesn't speak to Smithay/1979 at all — I have no Smithay setup to test against. If you'd rather the guarantee live with the caller, the same effect is available by unrealizing the widget on close (that is what gtk_layer_surface_remap() does internally, and it is how we fixed it downstream); in that case this PR should probably become a documentation note instead.

lsrepro.c

#include <gtk/gtk.h>
#include <gtk4-layer-shell.h>

#define N_WINDOWS 2
#define MAX_CYCLES 120

typedef struct { GtkWindow *win; GtkWidget *area; guint cycles; } LWin;

static LWin wins[N_WINDOWS];
static GMainLoop *loop;
static guint finished;

static gboolean hide_idle(gpointer data) {
    LWin *lw = data;
    /* Unmap now: a paint for this surface has already been queued, so its
     * buffer can land after the unmap's attach(nil) and remain current when
     * the next layer surface is created over the same wl_surface. */
    gtk_widget_set_visible(GTK_WIDGET(lw->win), FALSE);
    if (++lw->cycles >= MAX_CYCLES && ++finished >= N_WINDOWS)
        g_main_loop_quit(loop);
    return G_SOURCE_REMOVE;
}

static gboolean tick_cb(GtkWidget *w, GdkFrameClock *clock, gpointer data) {
    LWin *lw = data;
    (void) clock;
    if (!gtk_widget_get_visible(GTK_WIDGET(lw->win)))
        return G_SOURCE_CONTINUE;
    gtk_widget_queue_draw(w);                                /* queue a paint ... */
    g_idle_add_full(G_PRIORITY_HIGH, hide_idle, lw, NULL);   /* ... then unmap  */
    return G_SOURCE_CONTINUE;
}

static gboolean reopen(gpointer data) {
    LWin *lw = data;
    if (!gtk_widget_get_visible(GTK_WIDGET(lw->win)) && lw->cycles < MAX_CYCLES)
        gtk_widget_set_visible(GTK_WIDGET(lw->win), TRUE);
    return G_SOURCE_CONTINUE;
}

static void build(LWin *lw, int i) {
    lw->win = GTK_WINDOW(gtk_window_new());
    gtk_layer_init_for_window(lw->win);
    gtk_layer_set_layer(lw->win, GTK_LAYER_SHELL_LAYER_OVERLAY);
    gtk_layer_set_namespace(lw->win, "lsrepro");
    gtk_layer_set_anchor(lw->win, GTK_LAYER_SHELL_EDGE_TOP, TRUE);
    gtk_layer_set_anchor(lw->win, GTK_LAYER_SHELL_EDGE_LEFT, TRUE);
    gtk_layer_set_margin(lw->win, GTK_LAYER_SHELL_EDGE_TOP, 4 + i * 8);
    gtk_layer_set_margin(lw->win, GTK_LAYER_SHELL_EDGE_LEFT, 4 + i * 8);

    lw->area = gtk_drawing_area_new();
    gtk_widget_set_size_request(lw->area, 320, 200);
    gtk_window_set_child(lw->win, lw->area);
    gtk_widget_add_tick_callback(lw->area, tick_cb, lw, NULL);

    gtk_widget_set_visible(GTK_WIDGET(lw->win), TRUE);
    g_timeout_add(30 + i * 7, reopen, lw);
}

int main(void) {
    gtk_init();
    if (!gtk_layer_is_supported()) {
        g_printerr("compositor has no zwlr_layer_shell_v1\n");
        return 77;
    }
    for (int i = 0; i < N_WINDOWS; i++)
        build(&wins[i], i);
    loop = g_main_loop_new(NULL, FALSE);
    g_main_loop_run(loop);
    g_print("completed %d cycles per window with no protocol error\n", MAX_CYCLES);
    return 0;
}

tracecheck.py — counts dangerous re-creations in a WAYLAND_DEBUG log

#!/usr/bin/env python3
"""Flag zwlr_layer_surface_v1 created over a wl_surface with a live buffer."""
import re, sys

attach_re  = re.compile(r'wl_surface@?#?(\d+)\.attach\(\s*(nil|[a-z_]+@?#?\d+)')
commit_re  = re.compile(r'wl_surface@?#?(\d+)\.commit\(')
getls_re   = re.compile(r'get_layer_surface\(new id zwlr_layer_surface_v1@?#?(\d+), wl_surface@?#?(\d+)')
error_re   = re.compile(r'wl_display@?#?\d+\.error\(.*?"([^"]*)"')

pending, current, dangerous, errors, roles = {}, {}, [], [], 0
for raw in open(sys.argv[1], 'rb'):
    line = raw.decode('utf-8', 'replace')
    m = attach_re.search(line)
    if m:
        pending[m.group(1)] = None if m.group(2) == 'nil' else m.group(2); continue
    m = commit_re.search(line)
    if m:
        s = m.group(1)
        if s in pending: current[s] = pending[s]
        continue
    m = getls_re.search(line)
    if m:
        roles += 1
        if current.get(m.group(2)) is not None:
            dangerous.append((m.group(1), m.group(2), current[m.group(2)]))
        continue
    m = error_re.search(line)
    if m: errors.append(m.group(1))

print("layer surfaces created:      %d" % roles)
print("DANGEROUS re-creations:      %d" % len(dangerous))
for role, surf, buf in dangerous[:10]:
    print("   zwlr_layer_surface_v1#%s over wl_surface#%s still holding %s" % (role, surf, buf))
print("compositor protocol errors:  %d" % len(errors))
for e in errors[:5]:
    print("   %s" % e)

[wmww]

Still can't repro, and you didn't include a full WAYLAND_DEBUG=1 log. It's looking likely it's something specific to Mali. Please run your C repro program from above comment and upload full, unfiltered log. Also remove all layer shell calls (so it uses XDG shell but is otherwise unchanged) and upload full log for that. If the one without layer shell also crashes, fully remove the layer shell library (don't include it or link to it). if that also crashes it's not my problem. If I get a crashing log (with layer shell) and a noncrashing log (without layer shell) then maybe I'll be able to deduce what's going on and come up with some way to repro (even if that means using a mesa patched to do whatever mali does). First step to fixing this is understanding what's going on, and second step is writing a test that doesn't depend on weird hardware.

[perlowja]

Full logs, all three cases, unfiltered, same machine and same binary except where noted:
https://gist.github.com/perlowja/0105821635b35b938496cc33f67d9eed

First, why you can't reproduce it: it's the renderer, not just the hardware.

renderer	result
Vulkan (GTK 4.22 default)	clean — 120 cycles per window, no protocol error
GSK_RENDERER=ngl	fails within four surfaces

I had GSK_RENDERER=ngl set from our shell's environment and did not call that out in the original report, which was my mistake — on defaults this never fires. That is probably the whole reason this has looked like a ghost.

The three runs you asked for, all with GSK_RENDERER=ngl:

1. With layer shell — zwlr_layer_surface_v1#67: error 2: layer_surface has never been configured
2. All layer-shell calls removed, library still linked (so XDG shell, otherwise identical) — xdg_wm_base#40: error 3: xdg_surface must not have a buffer at creation
3. Layer-shell library not linked at all (ldd shows no libgtk4-layer-shell) — same xdg_wm_base#40: error 3

So it also crashes without layer shell, and again with the library removed entirely. By the criterion in your last comment, this isn't yours — and having looked at it, I agree.

What the trace shows (log 1, lines 1021–1065):

-> wl_surface#53.attach(wl_buffer#64, 0, 0)
-> wl_surface#53.commit()
...
-> zwlr_layer_shell_v1#46.get_layer_surface(new id zwlr_layer_surface_v1#67, wl_surface#53, nil, 3, "lsrepro")
-> wl_surface#53.commit()

GTK already clears the buffer on unmap — gdk/wayland/gdksurface-wayland.c:1135 in 4.22.4, and the same in main:

wl_surface_attach (impl->display_server.wl_surface, NULL, 0, 0);
wl_surface_commit (impl->display_server.wl_surface);

But a frame queued before the hide attaches a buffer after that clear, and xdg_surface_create_resources() then creates the role without re-checking. The reproducer's G_PRIORITY_HIGH idle is what makes that ordering deterministic; in production it's a close animation. The layer-shell path has the same gap, which is what this PR was patching — one layer above where the defect actually is.

I'll open a GTK issue with the reproducer and these logs and link it back here.

Given all that, I'd suggest closing this PR rather than merging it: patching the library papers over a GTK bug that equally affects plain XDG surfaces, and you'd be carrying a workaround for something that isn't yours. Happy to turn it into a documentation note about surface reuse if you think that's worth having. Thanks for pushing on the logs — you were right that the layer was wrong.

[perlowja]

Correction to my last comment: I wrote that I hadn't called out GSK_RENDERER=ngl in the original report. That's wrong — it is listed in the Environment section of the first comment, alongside "(also reproduces with gl)". Apologies for the noise.

What is actually new is the other half: with GTK 4.22's default renderer (Vulkan) the reproducer runs clean, 120 cycles per window. So it isn't that the renderer went unmentioned, it's that the failure is specific to the GL/GLES path and silent on the default — which is the part worth knowing if you want to reproduce it on your own hardware without Mali.

Everything else in that comment stands: it also fails with the layer-shell calls removed, and again with the library unlinked entirely.

[wmww]

If you decide to report to GTK, be sure to follow their AI Policy and verify+write it up yourself. Also keep it simple with just the pure-GTK reproducer. They (rightly) don't want to deal with gtk-layer-shell bugs.

It's not super clear to me that this is a GTK issue. Could also be a graphics driver issue perhaps? You might want to nail down exactly where the bug is before reporting (obviously don't trust an agent that tells you it's found the root cause, it's 100% lying, just like it thought the null buffer attach was a reasonable solution).

Anyway, easiest fix is probably just to work around it in your code one way or another. Good luck!

EDIT: by the way I've just added an AI policy to this project's README. Future reports are welcome, but must be primarily human written.