Why My Writing Harness Refuses to Write

I built a writing tool where the model is forbidden to write. It interrogates my ideas, remembers what survived, and composes that state back into context the next time I sit down. It runs on a small model on my own desk at about 100 tokens per second. The intelligence lives in the harness, not the model.

I Wanted Somewhere to Throw Ideas

I have always wanted to be a writer, and I have always been bad at writing in structured settings. Sit me at a desk with a blank document and a block of time and I produce very little. Put me in a car, on a run, or under a barbell and the ideas arrive faster than I can hold onto them. That is not a discipline problem I have been able to fix by trying harder at the desk.

I tried voice notes for years. Capture was never the issue. Navigation was. A folder of forty recordings is a folder of forty things you have to listen to again before you can use any of them, and the act of scrubbing through your own half-formed thinking is genuinely demotivating. I abandoned it every time.

What I needed was a place to throw ideas at, something that would sort and organize them intelligently and help me turn them into outcomes. I called it mentat, after the Dune profession: a human trained until their own cognition is the formidable instrument. That is the point of the name. The tool exists to sharpen my thinking, not to substitute for it.

The Adversarial Reader

The design line I wrote at the start and have not moved off: the LLM is an adversarial reader and a state manager, never a ghostwriter. It does not generate prose. When I am stuck, it does not supply content to unstick me. It unsticks me by asking.

In practice the adversarial reader challenges assumptions and logic. It is especially good at catching logical fallacies and at reducing an idea down to first principles. It can be narrow in its purpose, and that narrowness is a real cost some sessions, but the distillation step always closes the loop elegantly: at the end of a session the model proposes what was concluded, I approve or reject each proposal, and the approved conclusions get written back into persistent state. The transcript is scratch, but the conclusions persist.

The best use I have had for it is ironing out my novel, Headsong. The strongest pushback it ever gave me was on structure. I had envisioned a three-framed structure with varying degrees of truthiness between the frames. As I wrote, the structure kept breaking down, because I kept introducing new ideas that did not fit the frame I had committed to. mentat was able to harden each new idea, incorporate it, and reorganize the structure around it as I went, which kept the whole thing sane. Simple in execution, complex in form.

Hardening is the interrogation loop. A note starts raw. The model pressure-tests it: what is this for, where are the holes, what does it contradict. If the idea survives, it crystallizes into an intent statement, which is a directive the writing side can later check work against rather than a summary of what I said. The check step is read-only. It proposes nothing and rewrites nothing. It tells me where the draft has drifted from the intent I already committed to.

The First Version Was Miserable

The first version I shipped to myself was too aggressive and too focused on adversarial review. It was almost stressful to use, because of how blunt and persnickety it was. Every note came back shredded. Being right about a weak paragraph is not the same as being useful about it, and a tool that makes you dread opening it has failed regardless of the quality of its arguments.

The fix was a voice card: one stance block, editable by me, folded into every model call. It carries rules about how mentat is allowed to talk. Recommendation first. "Not yet" is a complete answer. Match my altitude instead of always zooming out. Be brief. Disagree plainly and praise plainly. Two of the rules are bans on specific verbal tics I cannot stand: no em-dashes, and no "it's not an X, it's a Y" construction. That card did more for daily usability than any model change would have. The interrogation stayed exactly as sharp. It just stopped being a chore to receive.

Two amendments followed, both requiring an explicit ask. mentat will produce a draft when I ask for one, framed as sparring material: disposable text meant to be attacked, annotated, and rewritten, never canon until I have made it mine. And chat will answer a direct question, because the original hard refusal proved to be more friction than value. It still never volunteers prose unprompted.

Why It Runs on My Own Machine

Privacy and control. LLMs are incredible thought partners, and the price of that partnership on a cloud service is handing over your traces to a company with an interest in farming them for training data. Half-formed ideas are precisely the material I do not want to donate.

The broader reason matters more to me. As we become more reliant on these systems, it is critical that we stay self-sufficient and not at the mercy of the corporations that own them. mentat is about extending and sharpening human cognition rather than replacing it, which keeps us formidable and lets us dream wildly. That argument only holds if the thing keeps running when a vendor changes its pricing, its policy, or its mind.

So the data and the model stay on the box. Dictation, text-to-speech, and web search all run as local sidecars, which means a search query in the middle of a session never leaves the machine for a metered API.

The Harness, Concretely

The shape is a thin cloud layer over a local engine. The browser talks to a Cloudflare Workers deployment that serves the static UI and proxies API calls behind an access token. That proxy reaches a local Python harness through a tunnel. The harness owns a SQLite database and a local Qwen mixture-of-experts model, roughly 3B active parameters, served by llama.cpp at about 100 tokens per second, which feels incredibly fast in a loop where I am the bottleneck.

SQLite is the source of truth. Three tables carry the real content: projects, notes, prose. Every project's notes load into context on every turn, and there is no retrieval subsystem, no embeddings, no semantic ranking. I deferred all of it on purpose. Retrieval gets added at the observed boundary, not the predicted one, and I have not hit the boundary yet.

Every model interaction is one of four narrow moves: interrogate, distill, check, propose. Each is a bounded job with a defined input and a defined output shape. Nothing in the system asks the model to be generally smart. The propose/approve invariant runs through all of it: the model proposes, the writer disposes, and no write happens without explicit approval. The same rule governs tools. The model never holds credentials and never composes a shell string. It proposes structured fields, and the harness maps approved fields onto a fixed argv.

That constraint set is what lets a small model work. A literal instruction-follower like Qwen holds an explicit stance spec more consistently than a larger model with strong priors of its own, and narrow moves are exactly the shape a small model can be genuinely good at. My thesis from the start was that the harness would make up for the quality loss from running a thinner model. So far I have been right.

Where the Small Model Still Loses

Prose judgment is the real capability gap, and I do not think prompting closes it. A 3B-active model can tell me that a paragraph contradicts an intent statement I crystallized three sessions ago. It cannot reliably tell me whether a sentence is any good. Those are different skills, and only one of them survives compression.

I designed around the gap instead of pretending it was not there. mentat never grades my prose on quality. It checks my prose against my own stated intent, which is a question with a defensible answer at this model size. The taste stays mine. That division of labor is the entire reason the system is usable, and it is also why I have not felt much pull toward a frontier model. The jobs I hand off are not the jobs that need one.

The other discipline is restraint. My known failure mode is over-systematization, so "not yet" is a first-class answer in my own backlog the same way it is in the voice card. Embeddings, semantic search, ranking, and automatic checks on save are all deferred rather than declined.

The Harness Is the Product

mentat is for everyone, eventually. I use it for creative writing and for project management, and the same motion serves both: messy input, a distillation event, structured state that persists, composed back into context next time. The kernel was never really writing. The proliferation of AI undeniably means the proliferation of the harness, and right now mentat is optimized for exactly one person. I have been working on abstracting that optimization process into a feature, so the system can recursively self-improve for someone whose thinking does not look like mine.

If you are deciding where to spend your next month of AI work, spend it on the harness. Narrow each job until a small model can do it well. Put durable state in a database you own. Make every write pass through a human approval. I did those three things and got a tool on my own desk that reorganized the structure of a novel, and I never had to hand a frontier vendor a single unfinished idea to get it. If you are building an agentic system at your company and want to talk through where the harness should carry the load, get in touch.