Replacing the SME
Interview Loop with an Agent
From SME training
to agent-verified extraction.
Complex enterprise software builds routinely depend on a subject matter expert's domain knowledge, and the standard way to get it into a build has been the SME training me directly, then a slow write-and-check loop to validate what I'd built against their expertise. That cycle was slow, and every round trip was a chance for something to get lost in interpretation.
I designed an AI-assisted alternative: a generated prompt the SME runs on their own machine that bootstraps a persistent, agent-driven workspace. That workspace conversationally extracts the domain knowledge needed, verifies it, and delivers it in the exact structure the software requires.
Knowledge transfer that depended on two people staying in sync.
The previous process put the SME's expertise through a human relay: they trained me, I built against what I understood, and then we checked the result against what they actually meant. Every step in that relay was a place for the original knowledge to drift before it reached the software.
The loop was also slow by construction. Write, check, correct, re-check: each round trip required both people's calendars to line up again, and the SME's time is the scarcest resource in the whole process.
"The SME shouldn't have to teach me their domain twice: once out loud, and once again every time I got something wrong."
SME-Trains-Human Loop
- SME trains the builder directly, synchronously
- Builder interprets and constructs from memory/notes
- Write-and-check loop to validate against SME's actual expertise
- Every round trip risks misinterpretation
- Slow: bounded by both people's calendars
Agent-Driven Workspace
- Generated prompt bootstraps a persistent workspace on the SME's own machine
- Workspace conversationally extracts domain knowledge directly from the SME
- Extraction and verification happen in the same environment
- Delivers knowledge in the exact structure the software requires
- SME's time spent talking to the workspace, not being interviewed twice
Designing for where
the interpretation gap actually lived.
The failure mode wasn't the SME's knowledge, it was the handoff. Every fix I considered still routed the SME's expertise through a second person before it reached the software, which meant the interpretation gap was structural, not incidental.
So I designed around removing the second person from the extraction step entirely: the SME talks directly to a workspace built to ask the right questions, verify the answers, and hold them in the software's own required structure, rather than talking to me and hoping I hold it correctly.
Persistence turns extraction and verification into one step
Because the workspace's filesystem holds rules, skills, and agent definitions across the whole session, it can extract knowledge and then immediately verify it against the software's own commands and test harnesses, instead of extraction and verification being two separate handoffs.
Three decisions
that made it work.
The choices behind a workflow that had to be trustworthy enough for a domain expert to rely on unsupervised.
A generated prompt instead of a shared tool install
Bootstrapping from a prompt the SME runs on their own machine meant no separate software rollout or environment setup was a prerequisite, lowering the barrier to actually using the workflow.
Persistent, not stateless
Populating the workspace's filesystem with rules, skills, and agent definitions meant the same environment could run software commands, manage version control, and execute test harnesses, keeping extraction and verification in one place instead of bouncing between systems.
Deliver in the software's exact required structure, not a generic format
Extraction that produces the precise structure the build needs removes a translation step that used to be part of my own work, and another place interpretation could have crept back in.
One workspace,
doing what two people used to.
The workspace isn't a single script, it's a persistent environment carrying rules, skills, and agent definitions that stay resident across the whole extraction session, so it can act on what it learns instead of just recording it.
That's what lets it close the loop itself: extract from the SME, verify against the software's own test harnesses, and hand off a result already in the structure engineering needs.
This one is genuinely metrics-pending. The qualitative outcome (collapsed cycle, removed interpretation gap) is real and already observed, but cycle-time and error-rate numbers aren't instrumented yet. Update this section once they are.
What's next
for this one.
The workflow is built and working, but I haven't yet put numbers on what it replaced: how much faster the loop actually is, how many round-trips it eliminates, how often the old process produced a correction. Those are the metrics worth instrumenting next, because the qualitative story is strong but a portfolio case study is stronger with evidence behind it.
I'd also want to test how the workspace holds up with a second SME on a different domain, to see how much of this generalizes versus how much was shaped by the first one.
"I designed this to remove myself as the bottleneck between an expert's knowledge and the software that needs it. The next test is whether it holds up without me shaping every workspace by hand."
Michaela Hoffman