https://thelastguardian.me/tags/gnome/Recent content in Gnome on thelastguardian.meHugo -- gohugo.ioen-usSun, 26 Apr 2026 00:00:00 +0000https://thelastguardian.me/posts/2026-04-26-screen-control-on-wayland/Sun, 26 Apr 2026 00:00:00 +0000https://thelastguardian.me/posts/2026-04-26-screen-control-on-wayland/<p>Second half of giving the assistant hands: driving the actual screen.</p> <p>In the <a href="https://thelastguardian.me/posts/2026-04-19-teaching-an-agent-to-use-a-browser/">last post</a> I got an agent driving a browser - Selenium against a copy of my Firefox profile for anything on my own accounts, Playwright for the clean-room jobs. That didn&rsquo;t cover native desktop apps with no DOM to drive, or the occasional site where only my <em>actual</em> running browser, with my actual session, would do.</p> <p>Instead of controlling a browser, control the screen: move the real mouse, press real keys, against whatever window is in front of me. If the agent can operate the machine the way I do when I&rsquo;m sitting at it, the browser and the native app stop being two different problems.</p> <p>On GNOME on Wayland, that took a lot longer than I expected.</p>