A Brainyacts Companion · The Forward-Deployed Legal Engineer

How I
Built It.

The build behind the article, told in plain words with the technical terms running alongside. Read the left column and you will understand it. Read the sidebars and you will be able to talk about it.

Josh Kubicki · Brainyacts · July 2026

The Whole Thing In One Idea

I wrote down what I know, and taught a machine to use it.

For twenty-five years I have carried a way of reading people and their practices around in my head. It worked, but only when I was in the room. AI is the first tool that can take judgment written out in plain English and apply it consistently, without me in the room. So the build was mostly writing, not coding: getting what was in my head onto the page precisely enough for a machine to follow, then testing it until I trusted it.

The Thing Itself

What I was building, in one breath

Before the how, the what. The system has two parts, and they feed each other.

The front door · a diagnostic interview

A structured interview a client completes in under an hour. It decodes how their practice actually wins: where the work really comes from, what the market actually knows them for, and the bottleneck quietly limiting all of it. The output is a profile of their own operating pattern, the kind of read most people have never seen laid out about themselves.

The engine · a weekly intelligence desk

A one-page brief that watches that client's specific market and turns what is moving into a handful of dated, sourced signals, each already converted into a specific move ending in a single decision: do it, queue it, or skip it. The interview teaches the system who the client is. The desk uses that to say what matters this week.

Personal enough that a generic tool cannot fake it, repeatable enough that it does not need me in the room every time. Holding both of those at once is what the rest of this page is about.

Start Here

Four facts about AI that explain every decision below

Not a metaphor, just how the tool actually behaves. Each fact created one part of the build.

It's brilliant, but generic

It has read nearly everything public. It knows nothing about you, your clients, or your method. Untrained, it produces the polished average.

It has no intuition

It cannot "just tell" when something is off, and it sounds equally confident when it is wrong. "You'll know it when you see it" means nothing to it.

It has memory, but you have to drive it

Modern tools do carry memory and context. Left unmanaged, that memory is partial and uneven: it forgets, it blends, it drifts. What it holds about you has to be deliberately built, constrained, and tested.

It's fast and tireless

Minutes instead of weeks. Which makes the first three facts dangerous: it can be wrong at scale, confidently.

The question was never "how do I learn to code." It was: given those four facts, how do you make this thing trustworthy with real clients? The seven steps below are the answer.

What I Did, In Order

Seven steps, plain on the left, technical on the right

Step One

I wrote down what was in my head

Everything I do with a client by feel, I forced into writing: the questions I ask and why, the patterns I watch for, what it means when answers to two different questions quietly contradict each other, what I would say next and what I would never say. I was decoding my own heuristics, turning "I just know" into instructions a machine can follow.

Why it's necessary: everything the AI read was public, and your edge was never public. A human hire absorbs your judgment by shadowing you. This one cannot shadow anyone; it knows only what you write down.
In technical terms
Knowledge captureDomain expertise

Turning tacit expertise into explicit written instructions. In AI work this writing becomes the raw material for the system's instructions.

In this build: a set of plain-text documents covering a six-dimension diagnostic method and a practitioner pattern library.
Step Two

I defined "good" in writing, before any real work

I can read a page of analysis and know in ten seconds whether it is good. That is intuition. The machine has none, and it writes beautifully whether it is right or wrong. So I invented three imaginary clients, each a type I have seen a hundred times. For each, I wrote their complete interview answers, plus the specific findings a good analysis of those answers must catch. Then a scorecard with a passing grade, decided in advance.

Why it's necessary: if you cannot say on paper what separates good from fluent-but-hollow, you can never tell whether the system works, and you stay the bottleneck forever.
In technical terms
EvalGolden personaRubric

An eval is a repeatable quality test: known inputs, required findings, a scoring rubric, a pass bar. The imaginary clients are golden personas.

In this build: 3 personas, 12 scoring criteria, pass bar of 8.0 out of 10 with no criterion below 6.
Step Three

I told it how to think, and in what order

Left alone, the machine starts writing polished sentences before it has thought anything through. My instructions force a sequence: first work through all the answers and diagnose what is actually going on, including where the person's story contradicts itself. Only when that analysis is complete may it write a word anyone will read. Think first, write last.

Why it's necessary: beautiful writing hides empty thinking. Forcing the analysis to happen first, in a form I can inspect, keeps the work honest.
In technical terms
System promptTwo-phase pipelineStructured output

The written method becomes the system prompt: standing instructions the model reads before every task. The forced sequence is a two-phase pipeline: a hidden, structured diagnosis (data, not prose) first, then the narrative built from it.

In this build: a canonical "protocol" document handed to the model verbatim; phase one produces scores, evidence, and tensions before any narrative exists.
Step Four

I decided what it knows. I never leave that to chance

The tools have memory and context, but unmanaged memory is exactly the kind you should not trust with client work: it holds some things, loses others, and blends the rest. So I do not rely on what it happens to remember. I keep a standing briefing document, the way you would brief a consultant walking in cold: what the business is, who we serve, what is in motion, and the rules learned the hard way ("every date gets checked against the original source," "nothing longer than one page goes to a client"). It reads the briefing at the start of every session, and what it carries forward is deliberately built, constrained, refined, and tested, not accumulated by accident.

Why it's necessary: without it, you are trusting the machine's patchy recollection of your world, and it will quietly reinvent or contradict prior decisions. Most people using ChatGPT are unknowingly running on accidental memory, re-explaining what it lost and inheriting whatever it mislearned.
In technical terms
Context fileContext engineering

Deciding what the model reads before it works: briefings, rules, file maps, style constraints. Conventionally a file named CONTEXT.md or CLAUDE.md at the root of a project folder.

In this build: one briefing file plus standing "master prompt" instruction files dropped into each client's working folder.
Step Five

I made it practice until it passed, every time

I ran the three imaginary clients through the whole system and graded the results against the scorecard. Where it missed a required finding, I sharpened the instructions and ran it again. The rule: it touches no real person until it passes all three cases, every time. Eventually I handed the loop itself to the machine, with a standing order to keep iterating and never lower the bar.

Why it's necessary: a prompt that worked once is luck. Practice cases with known right answers turn luck into evidence. This is the step most people skip, and it is the difference between a demo and a system.
In technical terms
Eval harnessLLM-as-judge

A harness runs the tests automatically end to end. LLM-as-judge means a model scores another model's output against your rubric, so review scales past your own reading.

In this build: one command runs all three personas and reports pass or fail, with the standing order: "do not stop until all three pass, do not lower the threshold."
Step Six

I gave it limits, and kept my hand on every send

Trust came with a leash. Research runs on a hard budget: a fixed number of searches per person, allocated across named sources, no exceptions. Any fact that cannot be verified against an original source gets flagged, never sent. And the biggest rule: nothing reaches a real person until I have read it and hit send myself. The machine drafts; I decide.

Why it still pays: reviewing a strong draft takes minutes; producing it used to take my week. The machine does the diligence and drafting. I keep the judgment, which is what clients were paying for anyway.
In technical terms
AgentSkillHuman in the loop

An agent is an AI that takes actions (searching, fetching, filing), not just answering. A skill is the reusable instruction file defining its job, sources, budget, and required output format. Human in the loop means a person approves output before it acts in the world.

In this build: a research agent capped at 20 searches per person, required to return a fixed-format dossier where "null" means searched and found nothing.
Step Seven

Once live, I started keeping score

Every real interaction gets logged: what each client acted on, ignored, rewrote in their own words, or flatly rejected (the most useful signal of all). The record makes next month's work sharper than this month's, and it slowly becomes the one asset nobody can copy. Anyone can buy the same AI. Nobody can buy the record of how a hundred real clients actually responded.

In technical terms
LedgerGolden dataset

Every human decision logged becomes a labeled example. Over time the ledger is the dataset future automation is trained against and measured against.

In this build: a per-client ledger file logging every pick, kill, edit, and silence, with a one-line note on what it teaches.
The Part Everyone Assumes Was Hard

The website took evenings. The writing took weeks.

There is a real application behind this: the interview a client clicks through, the database holding their answers, the page where results appear, the login. I built it without writing or reading a line of code, by describing what I wanted in plain English to an AI coding tool that writes, runs, and fixes the code itself. Describe, look, redirect, repeat. The industry calls this vibe coding, and the tool is a coding agent (I used Claude Code). The stack it produced is deliberately ordinary: a standard web framework, a hosted database, one-click deployment.

57 questions

One instrument

A structured interview across six dimensions. No branching; the intelligence is in the analysis, not the question tree.

3 + 12

Practice cases + criteria

Three golden personas scored on 12 criteria. Pass bar 8.0, nothing below 6, never lowered.

~$1

Per client analysis

In computing costs. The whole build launched for a few hundred dollars on free hosting tiers.

0

Engineers hired

No developers, no data team, no IT project. The scarce inputs were written precision and patience.

One more choice worth naming: which AI to use was itself a test, not a brand decision. Early on the same work ran across open-source and frontier commercial models, graded against the same scorecard. The build settled on the Anthropic Claude family for the analysis, the research, and the coding. Because "good" was already defined, comparing models was a measurement.

If You Remember Nothing Else

Three sentences to keep

1

The machine has no intuition, so you have to write yours down. The version of your expertise that lives in your gut is invisible to AI. The version you can put on a page becomes a system you own.

2

It does not know what good is until you define good and test for it. Fluent is not the same as right. Practice cases with answers you decide in advance are what turn a demo into something you would put your name on.

3

It forgets and it drifts, so you brief it, bound it, and keep a person on every send. The goal was never to remove the judgment. It was to make the judgment go further than one room at a time.

The code was never the hard part.
Saying what you know, precisely, was.

A companion to the Brainyacts edition on the forward-deployed legal engineer · thebrainyacts.beehiiv.com

· The End ·