I also haven't quite figured out how to build a billion dollar business with autonomous agent swarms that work while I sleep. I have, however, seen how strengthening S/PDLC fundamentals throughout an organisation, rethinking workflows from the ground up, and figuring out how to test and measure progress lead to positive AI leverage.
At its most basic, the software / product-development lifecycle (S/PDLC) is a hypothesis-testing system. We rarely know for sure if a change/feature will add value to a user (product), or to the team (productivity) and we don't want to waste money if we're wrong. So first prize is to always build and ship the smallest (lowest dev cost, lowest risk, minimal loss if we're wrong), highest value (often qualitative guesswork, it's an art) slice of work. And then we need to do this over, and over, day after day.
It turns out all of this is bloody hard.
The point is your best chance is small tests, frequent shipping, rapid iteration. That is the heart of Agile and continuous discovery.1 Small, small, small. Cut scope. Reduce complexity. It's much harder to remove than to add even as humans, and AI loves doing more, so if you don't already have this nailed down, adding AI into the mix will absolutely bring you a lot of pain extremely quickly.
More software is not the same as more value.
Below is one way to think about the human/AI collaboration at various stages of the PDLC. Form a hypothesis, ship the smallest slice that can test it, and learn.2
Sort and summarise the research you give it, surface themes, draft interview guides and generate options worth testing.
Stay close to customers. Write a hypothesis small enough to test, define what better means and decide whether the evidence is good enough to invest.
A testable hypothesis, observed customer or engineering behaviour, and a clear intended outcome.
What happens in production becomes evidence for the next hypothesis. Then the cycle starts again.
Like I said on the home page, you can bolt agents onto a system designed around humans, and what you'll get is an amplified version of the same organisation.
“AI-first” means redesigning work around AI's strengths, i.e. what you should stop doing. It requires a mindset shift not dissimilar to what I've seen when individual contributors move into their first leadership roles. There are things you need to learn how to let go of, so that you are not the bottleneck.
You need to learn a new way of working, with an old technique.3
Select any part of the loop below:
“If you don't know where you're going, any road will get you there”.
Token use (don't do this), training attendance, lines of code produced (definitely don't do this), time saved. Those can all go up while revenue, quality and delivery stay put.
Value evidence is a baseline and a way to see whether your experiments produce the results you wanted. Without that, you cannot tell a useful change (outcome) from a busy one (output), which makes it really difficult to decide whether to keep, revise or reverse what you just did.
Good luck measuring knowledge worker productivity directly. No single number can tell you whether what you're doing is working. The best we have, outside of increased profitability, are proxies. But at least we have proxies. And nothing stops us from coming up with or refining specific metrics you care about.4