Most conversations about AI productivity start with the same question: does AI help developers finish work faster? I'm not sure this is a useful question. It's too generic.
It mixes up two completely different things. There's using AI for work I already know how to do. And then there's using AI for work I would have no idea how to do on my own. Those aren't the same kind of productivity gain at all but we treat them the same. Coding is where this debate is loudest right now, but the same thing applies to any AI tool. The question I care about is much simpler: what would I have done without AI at all?
Faster work vs. new work
Say I ask an agent to write something in a programming language I've never used. Without AI, that task starts with me learning the language. I need to understand the conventions, figure out the tools, and get comfortable enough to start. Then I still have to do the actual work.
API integrations are another good example. I remember when I had to block out a whole morning just to read the API documentation before I could even start planning the integration. Now an AI agent can read the entire documentation in less than two minutes and start building right away. I don't even have to learn how that API works.
AI gives me the missing knowledge and the execution at the same time. So yes, the gain can be insane. A task that started with me staring at my computer not knowing where to start ends with a working result. Now take a task I know well. I could already do it without AI, so the most the tool can save me is the time I would have spent doing it myself. There's no learning curve to remove. The ceiling is just much lower.
A lot of the analysis I see seems to miss this. Ask developers whether AI makes them feel faster and they'll usually think about familiar work, because that's the work they can compare to how they worked before. We end up measuring how much quicker AI makes known work while missing the work it makes possible in the first place.
Who gets to judge?
There's another thing going on here, which is trust. When I ask an agent to do something I don't know how to do, I need it to bring knowledge along with the execution. I can check the obvious result, but I can't confidently review every decision because I don't know enough to have a strong opinion about every decision.
At some point I'm basically saying, "I'm going to trust you on this one." That's probably not always wise, but honestly, it's what I do. We all do it, some of the time. The AI does the work and, to some extent, judges its own work. Then it ships.
When I know the task well, I become the annoying client. No, I wanted it this way. Why did you use that pattern? Read this part again. Give me another pass. Redo that. The agent may have produced the first version quickly, but now I'm steering, checking, rewriting, and pulling my hair out while it tries again. So yes, it generated the code fast, but I still spent a bunch of time reviewing it.
I call this the review paradox. I often move faster with AI on work I don't know, because I don't know enough to second-guess it. On work I know well, every opinion becomes another review loop. In a weird way, knowing too much can become a productivity curse.
Maybe part of the review paradox is just our own hubris. A few years ago, it was easy to assume I knew better than the model. At the speed these things are improving, I'm much less sure. More and more often, I may be the dumb one in the review loop.
The more I know, the more I review.
Why I fight AI over databases
I notice this most when I'm building software. I've designed data models before and I've lived through database migrations, so I have opinions about how data should be stored. Maybe too many opinions. If an agent proposes a data model, I'm unlikely to accept it without a fight. I might insist on Postgres instead of NoSQL. I might want one entity in its own table, a relationship shaped a certain way, or the data saved according to a pattern I already trust.
There are probably several perfectly valid ways to build the same model. Would the agent's first version really be that bad? Maybe not. It might work just fine. But I know the domain, so I bring my taste and my scars into every decision. Each opinion creates another loop. Someone who has never designed a data model might accept the first working version and move on. On paper, they look far more productive with AI than I do.
Now take a very senior engineer, someone with far more experience than me. They've seen more failures, they know more edge cases, and they have stronger views about what good code looks like. Of course they're more likely to review, rewrite, and reject the agent's work. A survey can easily record that as a smaller productivity gain, even if all that judgment leads to better code. So what is the survey really measuring? The tool's ability, or how willing the person is to accept its first answer?
The part we don't measure
For work I already know, AI writes the code faster and I still want to read the code. The gain is real, but review eats into it. For work I don't know, AI removes part of the learning cost and supplies knowledge I didn't have. It can turn something too slow, too expensive, or simply beyond me into something I can actually finish.
If we want a useful measure, we should ask what people can now do that they couldn't do before. We should include the learning they didn't need to complete first and the tasks that became cheap enough to try. Then, for familiar work, we should count all the time spent steering and reviewing the agent. I don't have a neat formula for that. But minutes saved on familiar tasks only capture a small part of what AI changes.
For me, most of the value has come from being able to try things I wouldn't have tried before. That's the bit I want us to measure.
Written with ❤️ by a human (still)