A compositor-agnostic input-automation and accessibility layer for Wayland, as the X11 sunset strands hands-free voice-control users like Talon
desktop app real project •• multiple requests
Disabled Linux users who drive their computers hands-free (voice control, eye tracking, switch input) depend on X11's global APIs for simulating input, querying the focused window, and reading screen state. Distros are dropping X11 for Wayland, which deliberately scattered those primitives across libei, desktop portals, and per-compositor extensions, and the leading tool (Talon) is removing Linux support from its free tier, with the paid tier retaining X11 but not committing to Wayland, leaving many hands-free Linux users without a supported path forward. The opportunity is a single libei-plus-portals abstraction that gives assistive tech and automation tools a compositor-agnostic API again, so users do not get locked out as their distro flips to Wayland-only.
builder note Do not frame this as 'Wayland has no APIs' (KDE devs will correctly push back); the real gap is fragmentation and consent-gating across GNOME/KDE/wlroots. The win is one abstraction over libei plus portals that ships as a near-drop-in for the X11 automation AT tools assume, and you must design around absolute-pointer-position and window-PID being deliberately unavailable.
landscape (6 existing solutions)
X11 exposed global input, window, and focus APIs that assistive tech and automation tools relied on for decades; Wayland's security-first redesign split those primitives across libei, desktop portals, and compositor-specific DBus extensions, none of which is a complete, compositor-agnostic replacement. KDE developers argue the needed pieces already exist while tool authors like Talon call them inadequate and are leaving Linux, so disabled users are caught in the middle as distros phase out X11.
Talon Voice The dominant hands-free voice-control tool is removing Linux support from its free tier; the paid tier retains X11 support but Talon has not committed to a Wayland path, so users on distros dropping X11 face an uncertain future and free-tier users are losing Linux access entirely. libei (emulated input) Now adopted by GNOME and KDE for keyboard/mouse input emulation, but it does not expose absolute pointer position, the focused window's PID, or window/title querying that voice-control grammars and macros depend on. XDG desktop portals (GlobalShortcuts / RemoteDesktop) The GlobalShortcuts portal requires a user-consent dialog on each session and only permits binding registration once per session (the BindShortcuts method accepts an array, but the session can only attempt binding once), which is inadequate for the hundreds of dynamic, context-dependent commands a voice grammar registers. Behavior also varies across compositors. xdotool (Wayland port, in progress) The xdotool maintainer has an active Wayland port with virtual input working across KDE, GNOME, and wlroots, and a Talon-on-Wayland prototype has been demonstrated. It is not yet released as a stable tool and does not yet expose the focused window's PID, absolute pointer position, or window-title querying that voice-control grammars need for context-aware commands — the same primitives the Wayland security model deliberately withholds. ydotool Injects synthetic input through the kernel uinput device, so it has no awareness of windows, focus, or screen content and needs elevated uinput access; it can type and click but cannot drive window-aware automation. Orca / AT-SPI (screen reader stack) The blind-user side of the same problem: Orca runs on Wayland, but AT-SPI capabilities are inconsistent across compositors and the stack is chronically underfunded, so screen-reader users hit the same missing-primitives wall. accessibilitywaylandlinuxvoice-controlassistive-tech