I built an on-device AI assistant in mid-2026 as a plugin on top of OpenClaw, a real open-source personal-assistant gateway. It was supposed to give me nudges — lunch, hydration, movement, bedtime, meeting warnings — hold light conversation, and read my notes vault for context. The architecture passed every security gate I threw at it. The code was clean. Seven vault escape probes passed. And the product still failed, because it was a resource hog that couldn't maintain continuity and needed babysitting to provide any value.
That failure taught me three rules I still use. This piece walks through what broke, what the mechanism was, and how another builder can apply the lessons.
The architecture directive
The project ran as a gated, multi-phase workflow: Architecture Review → Security Review → Implementation Plan → MVP build and verification, each phase its own document with a "Status: Complete" header. Two directives stood out. The first: "Read Everything. Write Almost Nothing." The assistant could read my notes vault for context, but it had almost no permission to write anywhere.
The second directive was the one that matters for this piece: "The LLM should not determine event detection." The rhythm-tracking engine that decided when to nudge me was deterministic code with no model call in it at all. The model only supplied the wording. That separation — decision logic in code, phrasing delegated to the model — was an explicit design choice, and it was the right one.
What passed
A dedicated verification pass ran at install time. The plugin registered correctly with the gateway. The local model answered a probe turn. Shell access was denied at the tool layer. A vault escape-probe suite passed 7 of 7 checks: a protected key file refused, a path-escape attempt rejected, an absolute out-of-vault path rejected, root listing hid infrastructure, and search never surfaced key material.
Security wasn't the problem.
What broke and how I fixed it
Three platform bugs, all found by testing live behavior rather than reading documentation.
The silent no-op. An SDK-imported function was a no-op end to end because the plugin loader compiled it into a separate module instance from the one the running gateway actually read. I described it in the code comment as "a parallel universe the real gateway never reads." The function looked fine, called fine, and did nothing.
The invisible tool strip. A global configuration setting silently stripped 13 of the assistant's tools before its own per-agent allow-list was even consulted. I only found it by reading a gateway log line.
The stuck login screen. The desktop app's login screen appeared permanently stuck after two rounds of fixes aimed at window focus and cached JavaScript. The real cause: a CSS rule gave the token overlay display: flex with no override to hide it, so it was painted permanently from the initial HTML parse in every build. The app was actually connecting successfully behind an overlay that only looked broken. Every earlier check had inspected a DOM property that read as fine instead of what pixels were actually on screen. The fix was one CSS rule.
The lesson I took from that last one: UI state claims require pixel evidence — a screenshot — not DOM-property evidence.
The model reliability problem
The local model's core-loop reliability was never resolved. In ordinary use it would execute a tool call and then fail to produce reply text — a known failure mode for local models handling tool calls. The project documentation lists several configuration levers to try, ending with falling back to a hosted model for critical turns. I never got it stable enough to trust.
Why it was abandoned
It saw four active build days across an 11-day span, then 55 days of no further use. In its active life it produced one journal entry, two wishlist items, and four voice-cue requests. Every security and correctness check passed, but the product was a resource hog that typically failed to provide value unless I was babysitting it, which defeated the purpose. I parked it. Revival condition: a strong, capable model that could run on my laptop gets released.
The code itself was 5,798 lines with zero automated tests — not because I skipped them, but because the thing I needed to verify (does it nudge me at the right times, does it feel helpful, can it hold context across a day) wasn't something a test suite could measure.
The three rules that came out of it
One: Put the decision logic in code; the model only chooses the words. Rhythm tracking, event detection, whether to speak at all — that's deterministic code with no model call. The model gets handed a decision that's already been made and asked to phrase it. This keeps the system predictable, testable with normal code tools, and cheap. It also means I can change the logic without retraining or reprompting anything.
Two: Keep the agent's memory out of your own notes. The assistant had read access to my notes vault for context. That design survived review, passed the escape probes, and was still a mistake. When an agent's memory and my own working notes occupy the same namespace, I have to think about information boundaries every time I write a note. The cognitive load isn't worth it. The right architecture: the agent's memory lives in its own separate store, and if it needs context from my notes, I hand it in explicitly.
Three: Watch for an SDK talking to a copy of itself. The silent no-op happened because the plugin loader created its own instance of the SDK, separate from the one the running gateway was reading. Both instances were real, both compiled fine, and only one of them mattered. This is a class of bug I'd never seen before: not a logic error, not a missing dependency, but two parallel module graphs that look identical and don't communicate. The symptom: a function that returns success but whose side effects never appear. The fix: force the plugin to import from the gateway's already-loaded SDK instance instead of bundling its own.
What I use now
I don't run a personal assistant anymore. The lessons, though, I use constantly. Deterministic decision logic with model-only phrasing is how I build every agent workflow in LEGION. The agent's working memory lives in its own tables, never mixed with the knowledge base it reads. And when an SDK function succeeds but nothing happens, I check whether I'm talking to a ghost.
Receipts
- 7 of 7 vault security escape probes passed.
- 4 active build days across an 11-day span; 55 days of silence at the time of review.
- 5,798 lines of source code; zero automated tests.
- Status: abandoned.
You can see more numbers and verification evidence on the receipts page.