From: Landry Breuil <landry@openbsd.org>
Subject: fail early in mesa/eglInitialize when on wayland
To: tech@openbsd.org
Cc: jsg@openbsd.org, matthieu@openbsd.org
Date: Sat, 1 Nov 2025 18:33:19 +0100

hi,

currently mesa isnt built with wayland support (which is a topic in itself).
When running on wayland, starting any gtk+4 app will crash early when
initializing, because we end up taking a libxcb codepath, eg run gtk+4-demo:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000316d2ae48aa in XGetXCBConnection () from /usr/X11R6/lib/libX11-xcb.so.2.0
[Current thread is 1 (process 381892)]
(gdb) bt
#0  0x00000316d2ae48aa in XGetXCBConnection () from /usr/X11R6/lib/libX11-xcb.so.2.0
#1  0x00000316849f7ff4 in dri2_get_xcb_connection (disp=0x31632dbc5a0, dri2_dpy=0x3161541bcc0) at /usr/xenocara/lib/mesa/mk/libEGL/../../src/egl/drivers/dri2/platform_x11.c:1720
#2  0x00000316849f7ad8 in dri2_initialize_x11_dri3 (disp=0x31632dbc5a0) at /usr/xenocara/lib/mesa/mk/libEGL/../../src/egl/drivers/dri2/platform_x11.c:1926
#3  dri2_initialize_x11 (disp=0x31632dbc5a0) at /usr/xenocara/lib/mesa/mk/libEGL/../../src/egl/drivers/dri2/platform_x11.c:2095
#4  0x00000316849e3ee5 in dri2_initialize (disp=0x31632dbc5a0) at /usr/xenocara/lib/mesa/mk/libEGL/../../src/egl/drivers/dri2/egl_dri2.c:891
#5  0x00000316849e79b1 in eglInitialize (dpy=<optimized out>, major=0x7eac247c3468, minor=0x7eac247c346c) at /usr/xenocara/lib/mesa/mk/libEGL/../../src/egl/main/eglapi.c:705
#6  0x000003163bf7675e in gdk_display_init_egl () from /usr/local/lib/libgtk-4.so.5.5
#7  0x000003163bf0c073 in gdk_wayland_display_init_gl () from /usr/local/lib/libgtk-4.so.5.5
#8  0x000003163bf76399 in gdk_display_prepare_gl () from /usr/local/lib/libgtk-4.so.5.5
#9  0x000003163bfe1217 in get_renderer_for_gl () from /usr/local/lib/libgtk-4.so.5.5
#10 0x000003163bfe0880 in gsk_renderer_new_for_surface_full () from /usr/local/lib/libgtk-4.so.5.5
#11 0x000003163bd2b543 in gtk_window_realize () from /usr/local/lib/libgtk-4.so.5.5

it can also be seen with /usr/local/lib/firefox/glxtest -w, which also calls
eglIinitialize:
https://searchfox.org/firefox-main/source/toolkit/xre/glxtest/glxtest.cpp#612

looking at the code, display->Platform is X11, since wayland support isnt
builtin:

#4  0x00000316849e3ee5 in dri2_initialize (disp=0x31632dbc5a0) at /usr/xenocara/lib/mesa/mk/libEGL/../../src/egl/drivers/dri2/egl_dri2.c:891
(gdb) p disp->Platform
$3 = _EGL_PLATFORM_X11

so, i think we should fail early if we know we're on wayland. maybe the test
should be !!getenv(DISPLAY) to negate the condition, but i've stolen it from
https://github.com/openbsd/xenocara/blob/master/lib/mesa/src/vulkan/device-select-layer/device_select_layer.c#L149
and in my limited testing, now i can run gtk4-demo or d-spy from within wayland,
and glxtest doesnt crash when firefox starts:

$/usr/local/lib/firefox/glxtest
WARNING
libEGL initialize failed
VENDOR
AMD
RENDERER
AMD Radeon Vega 10 Graphics (radeonsi, raven, ACO, DRM 3.61, 7.8)
VERSION
4.6 (Compatibility Profile) Mesa 25.0.7
TFP
TRUE
MESA_VENDOR_ID
0x1002
MESA_DEVICE_ID
0x15d8
MESA_ACCELERATED
TRUE
MESA_VRAM
2048MB
DRI_DRIVER
radeonsi
TEST_TYPE
GLX

$/usr/local/lib/firefox/glxtest -w
WARNING
libEGL initialize failed
ERROR
EGL test failed
TEST_TYPE
EGL

so, how about the following diff, for now ?

Landry

diff --git a/lib/mesa/src/egl/drivers/dri2/egl_dri2.c b/lib/mesa/src/egl/drivers/dri2/egl_dri2.c
index a9273f6d0..d7b0bd775 100644
--- a/lib/mesa/src/egl/drivers/dri2/egl_dri2.c
+++ b/lib/mesa/src/egl/drivers/dri2/egl_dri2.c
@@ -888,6 +888,8 @@ dri2_initialize(_EGLDisplay *disp)
       break;
    case _EGL_PLATFORM_X11:
    case _EGL_PLATFORM_XCB:
+      if (getenv("WAYLAND_DISPLAY") || getenv("WAYLAND_SOCKET"))
+          return EGL_FALSE;
       ret = dri2_initialize_x11(disp);
       break;
    case _EGL_PLATFORM_DRM: