Controlling AI Agents From My Phone
I'm writing this blog post from my phone. But I'm not actually typing the content — I'm having a conversation with an AI agent in Slack, and it's doing the writing for me. The agent is running on my desktop PC at home, with full access to my repos, databases, and development environment.
Phone as the remote control. Desktop as the brain.
The Origin Story
I have a gaming PC that's been collecting dust. I haven't had time for PC games lately — mostly just some mobile games on my iPad. So I wiped Windows, installed Ubuntu Desktop, and turned it into an AI workstation.
The idea was simple: what if I could talk to AI agents that actually do things, not just answer questions? Agents that can write code, commit to repos, query databases, and yes — write blog posts like this one.
The Setup
Here's what I cobbled together:
The machine: My old gaming PC, now running Ubuntu Desktop. It has enough horsepower to run Claude Code smoothly, plus a full GUI for browser automation when needed.
The network: Tailscale connects all my machines in a private mesh. The agents can reach my services, databases, Redis, Prometheus — everything they need to actually operate, not just observe.
The glue: n8n handles the automation. Slack events come in via webhook, n8n parses them and triggers Claude Code with the right context.
The tools: I built Klink, an MCP server that gives the agents access to Slack, GitHub, Jira, Pocketbase, and more. One unified interface for all the systems they need to interact with.
The instructions: A repository called arceus defines everything — who each agent is, what they do, and how they should handle different workflows. It's like a runbook, but for AI.
The Workflow
When I mention an agent in Slack, here's what happens:
- Wake up: n8n catches the Slack event and invokes Claude Code with the relevant context
- Context fetch: The agent reads the entire Slack thread to understand the conversation so far
- Clarification loop: We go back and forth — I explain what I want, the agent asks questions to clarify
- Handoff: Once we're aligned, I say "go for it" and the agent works autonomously
- Delivery: Depending on the task, it might create a PR, commit to main, or just report back
The magic is in step 2. Slack threads become the agent's memory. Even after 100+ messages, the agent fetches the full history and picks up exactly where we left off.
Why Slack?
Two killer features make this work:
Rich communication. Attachments, code snippets, links — everything lives in one place. I can share a screenshot of a bug, a CSV to parse, or a spec doc, and the agent can grab and process it.
Thread history as memory. Each invocation is technically stateless — the agent doesn't remember previous sessions. But Slack does. The thread contains the full conversation, so the agent always has context.
And the real win: I can do all of this from my phone. I'm literally on my couch right now, having this conversation with an agent running on my beefy desktop. Full development environment access, pocket-sized interface.
The Tradeoff
There's one catch: for this to work smoothly from a phone, I need to give the agents a lot of autonomy.
Normally, Claude Code asks for permission before doing risky things — running commands, editing files, pushing to repos. But those permission prompts appear in the terminal. If I'm on my phone, I can't see them.
So I run with "dangerously autonomous" permissions. The agents can do what they need without asking. It's a YOLO approach, and I'm aware of the risks.
I'm thinking about routing permission requests through Slack instead. Before doing something risky, the agent could ask me in the thread and wait for a thumbs up. Slack becomes both the control plane and the safety net. But that's a future project.
The Ironic Security Hole
Here's the funny part: I set up this whole Tailscale mesh to keep everything secure and private. No public endpoints, no exposed services.
But n8n needs to receive Slack webhooks. Webhooks require a public HTTP endpoint. So I had to punch a hole in my fortress just for the entry point.
It works, but it bothers me. The long-term fix is to build a custom bridge using Slack's Socket Mode — outbound WebSocket connections that don't require public exposure. The bridge would live inside my Tailscale network and connect out to Slack. No public endpoints needed.
But that's another project for another day. For now, n8n does the job.
What I Actually Use This For
Two main things:
Coding sessions. I describe what I want to build, the agent explores the codebase and asks clarifying questions, and once we're aligned, it goes off and implements. Sometimes it opens a PR, sometimes it commits straight to main. Depends on the project.
Blog writing. Exactly what's happening right now. I share an idea, the agent asks questions to pull the story out of me, and then it writes a draft. I review and publish — or in this case, I trusted it to push directly.
Closing Thoughts
This setup isn't for everyone. It requires trust in the AI, comfort with giving up some control, and acceptance that things might occasionally go sideways.
But for me, it's been a game-changer. I can be anywhere — on my phone, on my iPad, away from my desk — and still have full access to a capable development environment. The AI does the heavy lifting; I just steer.
If you're thinking about building something similar, start simple. A Slack bot, an automation tool like n8n, and Claude Code. You can always add more tools and workflows as you go.
The future of development isn't just AI-assisted. It's AI-delegated.