AI LLM Vibe Coding Technical Debt

Update 2025-11-20: Improved article structure.

All Your Coworkers Are Probabilistic Too

There is a conversation I've had more than once in my career. Usually after a feature demo.

PM: "This isn't quite what I wanted."

Dev: "Well, it is what I thought we agreed on."

We wrote a ticket, discussed it in a meeting, and thought we understood each other. A week later, somebody had built something related to what was said, but it wasn't what the other person meant. We hadn't communicated enough of the intent and context to notice that we were imagining different things.

When people complain about large language models, I often think of those conversations.

I've worked with enough people to know that expressing what you want is difficult. Sometimes you don't really know yet. You have a rough idea, and seeing somebody try to implement it helps you figure out the rest. Given how many software problems require exploration, that might be all you can reasonably start with.

So we clarify the user story, leave comments on the pull request, explain something again, and try another version. Sometimes several. For over twenty years, I've seen work critiqued and revised every day. But when an LLM needs correction, some people take that as evidence that working with it is pointless.

Of course, how often it gets things wrong matters, and so does how much work it takes to fix them. But having to explain yourself more than once is hardly a new problem in software development.

I think part of the frustration comes from what we expect of a machine. A compiler doesn't improvise or claim to have fixed something it hasn't fixed. Now we have tools that need context, make assumptions, and sometimes confidently produce nonsense. I'm familiar with that behavior. I just didn't expect it from my editor.

On a Hacker News thread about specification-driven development, somebody described an LLM as an "unreliable compiler". I like that comparison. Give two engineers the same specification and you'll get two different designs. Give the same engineer the same task three months apart and they probably won't write the same code either.

Different doesn't necessarily mean wrong. There may be several reasonable ways to fill in what I haven't specified. Ignoring an explicit requirement or claiming to have checked something without checking it is another matter. I wouldn't excuse that in a colleague by calling it creativity either.

Working Through It

I wrote about building this site with LLM assistance in I Let AI Build My Website. Visual changes were particularly frustrating with the older GPT model I was using in VS Code. It would change the CSS and report success, but the result still wasn't what I wanted. With more recent models, I've found this less of a problem.

Things improved when I asked it to write a plan first and commented on that before it changed anything. I also told it to run the standard build and check tasks and inspect the output. It still needed guidance, but we were getting somewhere.

That shouldn't have been much of a revelation. With colleagues, we discuss constraints, tradeoffs, examples, and what a useful result would look like. We also talk about what doesn't matter. Giving a model the same information helps too. People used to call that "prompt engineering." I still think of it as writing a better spec.

A lot of people tell me they tried an LLM for coding and gave up because it kept making mistakes. When I ask how they used it, the answer is often that they wrote a long prompt, waited for a large amount of code, and were disappointed by the result.

I've had better results with smaller steps and more feedback. I don't want a colleague to disappear for two weeks and come back with a surprise either. Something can look reasonable on its own and still fit badly with the rest of the system. I'd rather find that out before there's too much of it.

Some people call discussing specifications before changing code a return to waterfall. I don't see it that way. Writing something down wasn't the problem. Treating it as fixed and getting feedback far too late was. The agile projects I've seen work had plans too. We changed them as we learned more.

Letting Go of the Details

I've been managing or supervising people for over ten years now. At some point, to be any good at it, you have to let go of the idea that everything will be done the way you would have done it yourself. That's hard when you're used to being in control of every little detail.

It's quite easy to spend time correcting something just because it isn't how I would have written it. But does the change actually improve anything? Or am I making somebody redo acceptable work because I have a preference?

I can tell people what matters to me and explain why. If I have a good argument and manage to communicate it, they can use that reasoning when making their own decisions. I don't need to prescribe every detail. The same approach helps with models, and it leaves me more time to think about the things that do need my attention.

Models don't challenge me very often, though. They can be far too enthusiastic about whatever I've proposed and quite happy to make something silly work. So I have to be careful about what I insist on. An unnecessary constraint can create a lot of unnecessary work, especially when nobody questions it.

One of my standard instructions is to criticize my design and technology decisions, explain alternatives, and suggest better approaches. Some models do this better than others. Asking doesn't guarantee that they'll notice when I'm talking nonsense, but it helps.

And I have to be willing to listen. If I ask for criticism and dismiss it as soon as it conflicts with what I wanted, what was the point? I need to allow myself to be challenged, including on things I feel quite sure about. That matters just as much with people. Being the manager doesn't make my argument better.

Context Has to Live Somewhere

Early on, I assumed that working inside my editor meant the agent would find everything relevant. It didn't. It would apply a pattern in one part of the codebase and ignore three other places that used the same pattern.

Agents can search for information using tools, so I don't have to put the whole repository into a prompt. But being able to look through the files doesn't mean they'll find the right ones. The model can only work with a limited amount of context at once, and what ends up in that context matters.

I started writing project rules in documents such as docs/system-design.md and docs/coding-style.md. Through .github/copilot-instructions.md, and later AGENTS.md, I told the agent to read them for each task. I kept the overviews short and pointed it at relevant files instead of hoping it would work out what mattered.

Eventually I made README.md the starting point for all of it. Everything needed to work within the project should be discoverable by following links from there. Now it's almost enough to tell the agent to read the README and follow the instructions.

I've spent years in companies with large, aging, poorly maintained codebases where onboarding was predictably miserable. Every new engineer had to reconstruct the system from scattered comments, half-remembered conversations, and whoever was still around to answer questions. Much of what they needed had never been written down.

We needed coherent documentation, a clear place to start, tests that helped us understand what was safe to change, and some explanation of how we worked. Those are the same things I'm now providing for agents. That doesn't solve every onboarding problem, but if an agent can find what it needs without me explaining everything again, I've probably made the repository easier for the next person too.

People shouldn't have to rely on tribal knowledge to work around technical debt. We tolerate it for a long time anyway. LLMs make the consequences harder to ignore.

What I Still Have to Take Care Of

When I explain an architectural decision to a colleague, I expect them to remember the important parts. If we have the same conversation three times and they keep making the same mistake, we'll have a different conversation.

With a model, I can't assume a fresh session has that shared history. The tools around it can store information and bring it back into context, but it has to be stored somewhere. My preference for composition over inheritance in this project, or my objection to adding another YAML parser, needs to be written down if I want it to carry over reliably.

Over time, I stopped thinking of this as teaching the model. I was putting instructions and checks into the project so I wouldn't have to keep explaining myself. That's useful work even when I'm the only person working on it. I forget why I made decisions too.

Responsibility doesn't transfer as easily as instructions. A model doesn't take it personally when I discard its work, and it doesn't take responsibility when I accept it. "The AI did it" doesn't explain away a bad deployment. Somebody allowed that code into production. Delegating the implementation doesn't remove my responsibility to judge whether the result is good enough.

The amount of work I can delegate is changing too. Hiring more people takes time and money. With agents, I can run several attempts at a feature in parallel and compare the results. Simon Willison has described working with multiple coding agents this way. When I first wrote this article, I hadn't seriously tried it. But generating more code also gives me more to evaluate, so the way I check the work matters.

Tests, linters, CI, code review rules, and branch protections already help us catch mistakes made by people. They help with generated code too. Willison makes this point in his article on "vibe engineering": good engineering practices give agents a much better environment to work in. Without them, they can make an existing mess worse very quickly.

People aren't particularly good at staying vigilant for long periods. Machines can run the same check a thousand times without getting bored. I'd rather automate what I can check automatically and spend my attention on whether we're solving the right problem.

That's also what I want to spend my time on when working with people. Explain what matters, give them what they need, look at the result, and be prepared to find out that my own idea was wrong. I don't need everything to look as if I wrote it. I need to understand whether it works and whether we can keep working with it.

Most of what has helped me work with LLMs would have helped the people in those old codebases too. We already had good reasons to make their work easier. Maybe doing it for the machines will finally get us to do more of it for each other.

Comments

Comment on this article
Names are not protected without a secret.
Identifies your name and lets you delete comments.
or drop anywhere in this form JPEG, PNG or static WebP, up to 1 MB.
Loading security question...
No comments yet. Be the first to comment!