Don't Let AI Write Your Docs - Let It Interview Your Team

We built a clean core product - a pipeline that ingested bank statements, ran analytics, and handed off structured output. Good documentation, clear boundaries, sensible defaults.

Then we built around it. Custom output formats for specific clients. Modified indicator sets. PDF parsers. Each one reasonable at the time. After a while, data compliance, orchestration, and monitoring had grown into something none of us could hold in our heads at once.

When it came time to clean it up, I tried the obvious thing first: let the AI read the codebase and generate context. Fed it the repo. Asked it to summarise the architecture. It produced something coherent-looking and almost entirely wrong - not factually, but structurally. It described what the code did. It had no idea why it was shaped that way.

I scrapped the generated docs and did something slower: interviewed myself. Went section by section through the architecture, wrote down decisions and their reasons, surfaced the constraints that weren’t visible in the code. Then used that as context.

The difference was not subtle.

The problem with AI-generated context

A 2025 study of 401 open-source repositories found AI-generated context files showed 1–5% improvement over baseline, sometimes negative, while increasing reasoning token use by 20%. Generic, comprehensive-seeming, and missing the one thing that matters: the decisions your team made and why.

LLMs already struggle with unstructured data - too much irrelevant information causes weak reasoning and hallucination. AI-written context adds volume without signal. It reflects the system’s surface, not its history.

What actually helps

The same study found that human-curated project information consistently outperforms AI-generated context. Not because humans write better prose, but because they know which constraints were load-bearing, which decisions were reversible, and which shortcuts are now permanent.

An IBM study of 669 developers found up to 47% improvement in response accuracy with well-structured documentation. Context engineering research adds +13% task accuracy and +19% accuracy with 12% less latency versus unstructured input.

AI tools need constraints, decisions, and boundaries - not descriptions of what the code does. The code already does that. What’s missing is the reasoning that preceded it. When that’s absent, AI proposes changes that are technically coherent and architecturally backwards.

arc42 maps to what AI needs

arc42 is a documentation template with 12 sections covering context, constraints, solution strategy, architecture decisions, cross-cutting concerns, and risks. It was designed for human architects. It turns out to be almost exactly what AI tools need too.

Most documentation answers “what does this do.” arc42 forces the harder questions: What are the boundaries of this system? What constraints are non-negotiable? What decisions were made, and what were they trading off against? That last one, Architecture Decision Records, is where most free-form wikis fall apart. ADRs are the difference between an AI that understands why a compliance boundary exists and one that helpfully refactors it away.

There’s a structural reason it works better for retrieval too. arc42 documents in named, scoped sections. Retrieval-augmented systems chunk and query against structure. A wiki page titled “System Overview - updated March, April, June, also see related pages” does not give them much to work with.

Free-form wikis also degrade. Teams add to them, contradict them, abandon them. arc42 makes gaps visible - if a section is blank, something is missing. If a decision changed, there’s a place to record it. The documentation stays queryable because it stays honest about what it doesn’t know.

Let the LLM interview your team

The role changes: from writer to interviewer.

Instead of asking an AI to summarise your codebase, ask it to interrogate your team against the arc42 sections it can’t infer from code alone. Humans write the output, humans verify it. The AI draws out what would otherwise stay in someone’s head.

A few prompts that work in practice:

“Walk me through every external system this touches. For each one, tell me what format data arrives in and what guarantees you need to make about it before passing it downstream.”

“Name a decision made in the first year that you’d make differently now - and tell me why you didn’t change it.”

“Where in this system could a new engineer introduce a compliance violation without realising it?”

“Name the three scenarios that would cause an on-call incident at 2am.”

These aren’t arc42 form-filling. They’re the questions that surface tacit knowledge - the constraints, the history, the implicit rules that live in engineers’ heads and nowhere else. arc42 gives that knowledge a place to land.

This is the difference between the 47% and the -5%.

What arc42 still misses

A March 2026 paper - Describing Agentic AI Systems with C4 - identifies a specific gap: arc42 and C4 handle traditional software well, but they have no vocabulary for agents. No way to express agent roles and boundaries, memory structures, tool access, interaction protocols, or quality gates. At the container level, artifact exchanges between agents become invisible.

The paper proposes seven new constructs layered on top of C4’s component level: <<agent>> stereotypes, <<task>> regions, shared <<datastore>> memory, decision nodes for quality gates, explicit orchestration vs. choreography markers, and structured tables for prompt specifications, separating static role descriptions from task-specific instructions.

The community is extending arc42, not replacing it. That matters. The foundation holds; it just needs new vocabulary for a new kind of system.

Where this leaves you

Arc42 won’t make your AI 47% smarter. But it’s the closest thing to documentation that AI tools can actually use - because it separates what you built from why you built it, and the second part is what AI consistently lacks.

Use arc42 as the structure, the LLM as the interviewer, write the answers yourself. Your docs become something an AI can reason from, not just retrieve.

At some point an AI agent will be making decisions about your system at 2am without you in the room. The question is whether your documentation was written for that moment, or written for the sprint review where nobody read it either.