AI LLM Software Engineering Productivity

The Cost of Working Without AI

Recently, a discussion at work about reducing AI costs and limiting each employee to a single tool made me think about what access to these tools is actually worth to me.

My answer was thousands per month. Possibly as much as my salary, considering what they allow me to get done. That probably sounds excessive. But looking at how I work now, I have a hard time arriving at a much smaller number.

I was an early adopter of LLMs. Working with them feels so natural now that it's easy to forget how laborious and slow much of my work was just half a decade ago.

I've always considered myself a productive engineer and manager. Now I do considerably more, and in several areas I do it better. Taking AI away would mean giving up a substantial part of my output.

I mostly use Claude and Codex, but the particular tool matters less here than what I use it for. The discussion about AI's value often seems to stop at writing code.

Product Ownership and Jira

A substantial part of my responsibility is being the technical product owner for Exasol Personal. I author most of our Jira epics: taking feedback from users and requests from product management, refining and discussing them, working through designs and prototypes, and reflecting the resulting work in Jira.

In a typical month, that means roughly a dozen epics and 50 user stories and tasks, with proper descriptions, acceptance criteria, half a dozen fields assigned correctly, and links to related work.

I've worked with many product owners over the years. At the level of detail and quality I maintain now, this would traditionally have been a full-time job. With AI, I get that done better than anybody I've ever worked with, and in about 30% of my time.

I still make the decisions, but I can delegate much of the work involved in developing and documenting those ideas. Without that help, I'd either do less of my other work or prepare the team's work less thoroughly. Probably both.

Finding Information

I also use LLMs to find information, of course. For public information, I find that much faster than searching Google and working through the results. Internally, I use them to find Jira issues related to a particular bug, search codebases, or work through large log files. Finding the information I need used to take a lot of my time too. Being able to delegate those searches is another huge time-saver.

Writing and Presenting

While the biggest chunk of writing I produce happens in Jira, there is more like architecture documentation, development guidelines and release announcements. The information already existed, scattered across various places. But I still had to collect it, decide what I needed, and make sure I hadn't forgotten anything.

I use LLMs for this kind of writing now. I appreciate efficient communication and have spent quite a bit of time tuning their output to retain my tone and style. I regularly tell them to be more concise, leave out irrelevant tangents, or stay at a higher level.

There is something sad about the prospect of everybody's writing starting to sound alike. But let's be honest: for quite a few people, using an LLM to help with communication is a substantial improvement, much like using a spell checker. I'd just prefer that we don't all acquire the same enthusiastic vocabulary in the process.

I've also been using AI to prepare presentations at the end of our work cycles for quite a while, and increasingly for other presentations and diagrams.

I always enjoyed working out architecture diagrams. Moving boxes around, fiddling with arrows and colors, aligning everything, and updating the same information in several places later could take forever, though.

Now I describe the facts and relationships in text and have an LLM produce the presentable version. I still review and refine it, but getting to something worth showing takes much less time.

Writing and Reviewing Code

I almost never write code by hand anymore. After spending so much of my life doing exactly that, it is still a slightly strange thing to say.

I'm still producing features and fixing issues, by proxy, faster than ever. For some of that work, my estimate is literally ten times faster. I haven't run a controlled study of myself.

In our platform team, I've created reusable instructions and workflows, usually called skills, for feature delivery and code review. The whole team uses them now, guided by global and repository-specific contribution guidelines we've developed.

The review skill finds more than I would find on my own, from little details to some higher-level issues. I still look at pull requests, but concentrate on specifications, architecture, API design, and conceptual problems. I spend much less time checking individual conditionals or looking for typos.

The skills and guidelines take work to develop and refine, but that effort benefits the whole team without me personally repeating every instruction or performing every review.

Using the Product as a User Would

We also routinely run automated product evaluations to find friction in new features. Our CPO, Alexander Stigens, gave me the idea, and I've since refined and expanded it into an LLM skill.

The agent creates personas with goals the software is intended to help people achieve, then tries to achieve them. It works from publicly available information, without access to our source code or internal documentation.

That restriction matters. If the product only makes sense after reading an internal design document, a user has a problem. I want the evaluation to encounter that problem too.

The agent reports what it finds without fixing the product. We review the findings and feed the useful ones back into implementation.

We can target the whole product or particular parts, including performance, software updates, and error recovery. We also ask it to do some of the silly things people can easily do: delete files, press buttons in the wrong order, or use Task Manager to kill an application.

The agent misses problems, particularly bad UX, so I still manually test our products often. Complex products may need several evaluations. But it finds a lot, and getting those findings while we're still working on a feature saves time and helps us make the experience smoother.

Keeping Things Moving While I Do Something Else

Infrastructure and CI work often involves long-running processes that need occasional intervention: analyzing failures, making corrections, restarting, or interpreting results. I can't reliably predict when they'll need my attention. I used to either interrupt my other work or leave things waiting until I was available.

Now I give an agent a clear goal and let it handle much of that process asynchronously. I check in and intervene when needed, but the work keeps progressing while I concentrate on something else. That's worth quite a lot on its own.

I Was Already Used to Delegating

I think this transition felt relatively natural because I'd already spent years managing people: explaining goals, discussing constraints, reviewing results, and asking for changes. I was also used to somebody implementing things differently from how I would have done it. You have to get comfortable with that to delegate anything substantial.

I've always felt limited by how fast my hands, eyes, and mouth could keep up with what I wanted to get done. Before LLMs, so much of my day went into carrying out things I'd already thought through. Typing them, assembling them, checking them, moving them from one place to another.

I still take pride in creating the guidelines and frameworks within which the agents work, and review the results much as I would an employee's work. When something goes wrong, I ask what I need to improve in the instructions, tools, infrastructure, or feedback I provided.

The agents help me improve those things too. But I can't accept a result and then disclaim responsibility because a model produced it.

In return, I get more time for thinking, deciding, and judging, and more of that work actually making it through to completion.

What Would I Lose Without It?

If I had to stop using AI tomorrow, I honestly think I'd lose about 70% of my current output. That's my estimate across my work as a whole.

The remaining work would suffer too: less complete Jira issues, less attention to presentations, fewer product evaluations, and slower feature delivery. There aren't enough hours in the day to maintain all of this manually at the same volume and quality.

Everybody's work is different. I don't expect every employee to get the same benefit. But this is what these tools have become worth to me, and I don't see much reason to be coy about it.

To my employer, that access could be worth as much as my salary, perhaps more. Losing a substantial part of the work I've described would cost considerably more than a subscription.

Other people in my industry have access to these tools too. That changes how much work we can reasonably expect to get done.

For me, cutting access to useful AI tools to save money would be like turning off build automation to save on compute, then paying engineers to do that work manually. You can make that bill smaller and still end up paying far more to get the work done.

Comments

Comment on this article
Names are not protected without a secret.
Identifies your name and lets you delete comments.
or drop anywhere in this form JPEG, PNG or static WebP, up to 1 MB.
Loading security question...
No comments yet. Be the first to comment!