Ai on thelastguardian.me

I Got an AMD R9700 (32GB) for Local Inference

Sat, 16 May 2026 00:00:00 +0000

If a local model was going to be part of the assistant, I wanted to understand the machine under it. So I bought a AI-oriented GPU.

I wanted to learn the internals of LLMs and inference, and I can’t afford the multi-million-dollar racks of datacenter GPUs the hyperscalers run. A local model is a temporary stand-in for that learning: not smart enough to replace a frontier model, not some AGI/ASI waking up in my utility room, but decent enough to test chatting flows and scripts and watch how serving behaves.

Teaching an Agent to Click on Wayland

Sun, 26 Apr 2026 00:00:00 +0000

Second half of giving the assistant hands: driving the actual screen.

In the last post I got an agent driving a browser - Selenium against a copy of my Firefox profile for anything on my own accounts, Playwright for the clean-room jobs. That didn’t cover native desktop apps with no DOM to drive, or the occasional site where only my actual running browser, with my actual session, would do.

Instead of controlling a browser, control the screen: move the real mouse, press real keys, against whatever window is in front of me. If the agent can operate the machine the way I do when I’m sitting at it, the browser and the native app stop being two different problems.

On GNOME on Wayland, that took a lot longer than I expected.

Teaching an Agent to Use a Browser

Sun, 19 Apr 2026 00:00:00 +0000

For the assistant to do real chores, it needed hands. This is the first half of giving it some.

I have a pile of small browser chores that only I can do because they live behind my own logins: register a warranty on an appliance, check a dashboard that has no API, fill in some form that wants my account. None of it is hard, all of it is tedious. Could I hand that work to one of the coding agents I already run, and have it do the same clicking and typing I’d do, on my own accounts, while I was off doing something else?

Note: most of this was before Claude Cowork was generally available, and anyway I wanted to build my own generic screen control for experimentation and things that there wouldn’t be Cowork connectors for, atleast initially.

The catch: an agent that can read and write files and run shell commands still can’t see a web page the way I can, and it can’t bring my logged-in session. So: how do you let an agent drive a browser at all?

Memory That Outlives the Context Window

Sun, 12 Apr 2026 00:00:00 +0000

An assistant that forgets everything between chats can’t be trusted with anything ongoing.

I run a fleet of AI coding agents in my homelab, and for a while they all had the same flaw: every session, they forgot everything. You’d tell an agent “it’s haven, not the NUC, the NUC was retired” and it would nod along, fix it, and then three days later a fresh session would confidently SSH into a machine that no longer existed. Context windows are big now, but they’re not forever, and the moment a long session compacts or a new one starts, all that context is gone.

A Discord Control Plane for Autonomous Agents

Sun, 29 Mar 2026 00:00:00 +0000

Another piece of the assistant: being able to answer it when I’m not at the desk.

The agent stack in my homelab runs AI coding sessions unattended. A scheduler pulls recurring tasks from Todoist, launches a session to handle each one, and a control-plane timer health-checks them every couple of minutes. Most of the time I never see any of it. But unattended work still needs a human sometimes: a permission prompt, an ambiguous call, a finding worth a second opinion.

I’m not sitting at the terminal when an agent hits one of those moments. So I built a session bridge: it relays an agent’s session to a Discord thread and relays my replies back. The agent asks, my phone buzzes, I answer from the bus, the agent carries on. One thread per session, so each conversation keeps its context.

The relay itself was easy. The hard part was the two failure modes I hit making it trustworthy.

Automating Permissions Safely - Before Claude Auto Mode Was Around

Wed, 18 Mar 2026 00:00:00 +0000

One piece of the Jarvis I’m after is letting it act while I’m not watching, which means trusting it with a shell.

I run a handful of AI coding agents in my homelab without sitting over them. A scheduler hands them recurring tasks, they work, and most of the time I find out what happened by reading it after the fact. The catch is that a coding agent is only useful if it can actually run things - git, kubectl, the occasional shell one-liner - and “can run things, unsupervised” is a phrase that should make anyone a little nervous. So the real question was never whether to let the agent act. It was when it should just go ahead, and when it should stop and ask me first. I use agents less for self-contained software coding and more for operating my homelab and agentic workflows on internal and external services, so “just run it in a sandbox and allow everything” wouldn’t work for me.

When I started doing this, Claude Code didn’t have a built-in answer. Surely someone had made something like this without the often recommended “just use –dangerously-skip-permissions” mode? No? Let’s use hooks to automate permission decisions with my mental context noted down.

HomeLab 2026 - The Two Datacenters

Sun, 25 Jan 2026 00:00:00 +0000

Six years since the last post. The lab grew up - two sites on two continents, two Kubernetes clusters, GitOps everywhere, NixOS everywhere, and a GPU that talks back. A tour of what runs today and why.