AI helps those who help themselves

I’m going to be grumpy today and complain about something trivial: I’m frustrated by how much AI is touted as the ultimate multiplier to any business without explaining what it actually takes to get good results. It takes hard work, work that has been established as good practice and followed by the top software shops for decades. And yet from what I gather, this is exactly the kind of work most eager AI buyers have been avoiding since well before generative AI was a thing.

Consider this: each new session with AI starts from scratch. “Hi, I’m Claude, how can I help?” This is your new hire. They’re sharp, they’re willing, and they will vanish into the void the moment the session ends. This is a temp worker, not an engineer that will hopefully stick with you soaking up organizational knowledge. Even if they are trained to be experts in your domain, they will still need introduction into your specific problems, constraints, requirements, and legacy baggage—all the institutional knowledge that is in your head. You can summon an infinite number of them, limited only by your wallet, but their usefulness is capped hard by how good your onboarding is. And this is a pre-generative-AI problem. Anyone who has onboarded an engineer knows it can take months before they become effective.¹ The human at least has incentives to learn. Each new AI session will test the efficiency of your onboarding process again.²

Consider this: the magic of large language models is to a large part compression of human knowledge. It is truly astonishing how LLMs can be partners in a conversation about almost anything. Yet there’s a limit to any compression scheme, and there are natural biases in LLM training. Write your system in Clojure and the model will quietly try to drag you to bash.³ Maintain a legacy Java 8 codebase and it will keep reaching for Java 17 features.⁴ Run on a closed platform and the performance of your LLM will be seriously degraded.⁵ Old problem again — hiring humans for legacy or niche stacks was always painful.⁶

Consider this: LLMs are not omniscient and there’s always a non-zero chance they’re wrong. Some mistakes are funny,⁷ some cost you business.⁸ So you need ways to keep the errors manageable, and… well, you are probably starting to notice the pattern: this is also a problem we have been solving since long before generative AI. We do it with code reviews, security boundaries, unit tests, manual QA, change control. But when humans are both authors and reviewers, the scale of necessary quality assurance processes is easy to match: since the same people perform both activities, they naturally control the rate of code to the rate of necessary reviews. When AI produces code at volumes escaping human review capacity, strong quality control becomes much more important.⁶

Consider this: while almost everyone agrees generative AI is transformative for the economy, there is no agreement how to do it well. It takes just a quick peek at social media to see dozens of strategies, many of them getting the moment a new Claude Code feature drops in. And again, we have seen this in the past. The 1980’s were the original transformational decade for business computing. Businesses everywhere wanted in, and a great many of them failed at it. Then there was enterprise IT in 1990’s, Agile in 2000’s, DevOps and Cloud in 2010’s. These four decades brought us valuable practical knowledge: how to structure the IT stack (reusable platforms,⁹ buy vs. build¹⁰), how to organize people (Agile,¹¹ Team Topologies¹²), how to let them communicate (version control, ticketing, knowledge bases, postmortems), or how to measure delivery (DORA¹³). Yet there were also a great number of failed directions (Lisp Machine,¹⁴ CASE tools,¹⁵ CORBA,¹⁶ IBM Watson¹⁷, microservices-by-default¹⁸, to name just a few). Any business that wants to follow the frontier needs to know how to learn from own failures, because there will be many of them.

No institutional knowledge, or no onboarding into it. Complex legacy systems. Lackluster QA. Failing retrospectives. My today’s thesis is: many organizations willing to go all in on AI are exactly organizations with existing unrecognized fundamental gaps in their operations. These organizations may be thinking that this move will help them reduce competitive pressure they are likely feeling, and yet also they are the ones where benefits from adopting AI will be the most underwhelming.¹⁹

AI helps those that help themselves.