The age of Agile must end
30 years ago the technology industry attempted to import Lean practices — it failed. Instead of “continuous improvement,” progress halted. Agile is incompatible with UX research, design, and scalable development. It always will be. It’s time to create a new operational standard.
As startups refocus on “operational efficiency,” let’s emphasize that efficiency is a way of working, not your headcount.
Agile development has been the #1 operational principle of tech for over 20 years, unchecked, unquestioned. Yet, it has fundamental flaws that have been glaring at us all along.
Flaw #1: Humans are not machinery.
Flaw #2: Design is not inventory.
Flaw #3: Product can’t be defined by what can be accomplished by an arbitrary number of people of arbitrary skill and experience in a two-week sprint.
Let’s take a look backwards at how we timeboxed ourselves in:
- 2001 — Agile
- 1995 — Scrum
- 1988 — Lean
- 1960’s — Toyota Production System (TPS)
From the Beginning
The TPS evolved for decades, from the ’40s through the ’70s, allowing Toyota to achieve and maintain its dominant position among world carmakers. These brilliant beginnings focused on eliminating muda (waste), mura (inconsistency), and muri (overburden).
The elimination of waste was the most thoroughly documented. Among the eight types of waste, the most detrimental were excess inventory — both finished product and raw materials, and idle machinery. In other words — stuff you’ve spent money on that isn’t making you money. Toyota’s solutions were logistical in nature and came to be known as “just-in-time manufacturing.”
“Receive materials just in time for the assembly line to process them just in time for the finished product to meet customer demand.”
A partner in excellence to TPS was “The Toyota Way.” This philosophy called for continuous improvement, including overcoming challenges towards a long-term vision, kaizen (innovating on operations), and my personal fave, genchi genbutsu. Meaning “real place, real thing,” this principle meant, “go and see for yourself,” make decisions accordingly. It also encouraged a bottom-up approach to improvement, gathering insight from every employee, no matter their level.
America worked its magic and turned The Toyota Way into the equivalent of Cup Noodles. Ok, maybe I’m being unfair. Just-in-time manufacturing was well-applied stateside to… manufacturing. It was rebranded as Lean in a 1988 article, “Triumph of the Lean Production System” — the lineage was clear.
Lean and its forebears are perfect for the manufacture of physical goods, particularly when there is an established market and a mature product. It relies on predictability of both demand and supply chain. When it comes to creating a new product, manufacturers relied on different processes in their R&D departments. Whether it was Genentech developing synthetic insulin or Proctor & Gamble developing the Swiffer, the process was long and fairly linear. These involved deep research and market analysis ahead of product development.
The Waterfall Scapegoat
The burgeoning software industry largely followed these processes; applied to digital products, they became known as Waterfall methodology. The term Waterfall has become a catch-all to describe a linear development process that culminates in a fully realized product release. The legit criticism is that the big expensive product release can still be a dud. As such, Waterfall is used as a foil to argue that iteration, testing, and reaction to the market are necessary on a shorter timeline.
In the 1990s, practitioners in the technology industry explored ways to apply Lean to digital products. Unfortunately, from the get-go, they misdiagnosed the problem. Time to market is important but I’d argue more so from a cash flow perspective, particularly for startups. Lean proponents threw out the baby with the bath water. Note how Scrum and Agile literature makes little to no mention of research or strategy. Design got a bad rap as the lengthy time-wasting phase of Waterfall. Lean frameworks are all about build and release.
Around this time, there was a Kaiju vs. Mecha style fight between proponents of Lean and those who had been practicing Waterfall development. Both the monster and the robot failed to consider users’ needs and instead stomped on them and destroyed the world.
What’s the difference? Just the diagram — Agile representing a piece, Waterfall representing a whole.
The practice of iterating on a living product in the market is a luxury of software — updates can be pushed at any time; assembly lines and tooling don’t need to be revamped. Every technology product should be built in stages but it also must be built towards an end goal and a fully realized product. Success requires strategy, research, design, and genchi genbutsu, just as The Toyota Way intended. The Agile vs. Waterfall argument is a false dichotomy: a company can have a long-term strategy and carry it out while still releasing the product incrementally, iteratively.
The 30-Year-Old Error
As product owners and engineers attempted to apply lean manufacturing principles to software development, they found ways to mimic an assembly line. Requirements would be determined just in time and then a cross-functional team would sprint to release working software. The term Scrum was an apt choice — it’s from rugby, the formation of a row of dudes rushing across the field tossing the ball back and forth. The approach to software development has the same level of finesse and strategy.
You’ve all had to deal with Scrum, here are the reasons why it is the way it is:
- It has to be timeboxed into a Sprint to make sure software can be released into the market in an “iterative” cycle.
- Sprint Planning happens right before the sprint because that’s when you’ll have the latest insights on what to focus on, just in time.
- There’s a Daily Standup to have on-the-fly “operational improvements.” (Only developers are allowed to speak and discussion is not allowed because that leaves the machinery idle.)
- At the end of the sprint there’s a Review to view working software and to determine what was not completed or what needs improvement.
- There’s also supposed to be a Retrospective to discuss what went well and what didn’t go well, in the interest of increasing the quantity of output.
- The Backlog isn’t core to the Scrum practice, it’s an artifact of the collateral damage and detritus of everything jettisoned during the sprint.
You can see how Scrum is fairly illogical already, but then it gets so much worse.
A few of the creators of Scrum got together with more dudes who had put together other Lean development practices. With their forces combined, they created the Manifesto for Agile Software Development. The Marxist undertones are likely intentional: they were stating that the workers would own the means of production.
The [Fr]Agile Manifesto
As far as manifestos go, this is one of the more pathetic documents ever created. It lists four “Values” and twelve “Principles.” They may have been well-intentioned at their inception — that intention was to treat developers as humans, not cogs in a machine — but now we’ve gone full circle. These principles have devolved into something caustic for organizations. Viewed now, they’re variations on a theme of how a scrum pod can absolve themselves of any accountability.
Highlights (and my translations):
- Individuals and interactions over processes and tools. (Yeah, we’re gonna do things however we want).
- Welcome changing requirements, even in late development. (We can also change our minds whenever).
- Projects are built around motivated individuals, who should be trusted. (Leave us alone, take what you get).
- Working software is the primary measure of progress. (The fact that we produced something, anything, is all that matters).
- Simplicity — the art of maximizing the amount of work not done — is essential. (We are going to do as little as possible).
- Best architectures, requirements, and designs emerge from self-organizing teams. (Don’t tell us what to do).
Now, this is not an attack on our engineer friends. Most of the developers I work with are passionate about doing things well and feel constrained to produce subpar work based on sprint timelines and promised deliverables. Still, some technical leaders who have been indoctrinated into the Agile cult feel quite at liberty to follow the above principles. Requirements are more suggestion than rule, designs are up for interpretation, anything is fair game to be cut if the end result still “works.”
The combination of Agile principles and Scrum practices is disastrous for startups. These are operational directives from management; designers, PM’s, and engineers are not self-organizing and choosing to work this way. It’s all in the name of an “MVP” and time to market; this is what happens. Every. Time.
Research & Design create solutions to customer problems but what gets built always pales in comparison.
Product Managers are rushed to create the next shiny object on the roadmap… just as little of it as possible.
Engineers are treated like machinery on an assembly line, always expected to be producing and delivering, incentivized to cut corners in order to get it “done.”
Leadership is left scratching their heads why the product is so wonky and ineffective.
Far from eliminating muda, Agile and Scrum create only waste. It’s a muddy slop of burnout, tech debt, design debt, a ballooning backlog, hard-coded front-end logic, and an ever-present threat of a complete refactoring.
How can we get out of this cycle?
Just stop.
You can’t shoehorn a customer-centered solution into a sprint.
- Determine what needs to be built.
- Determine how best to build it for scalability and future proofing.
- Then build it, however long it takes. (It’s not going to take two years but neither is it going to take two weeks. Releases can be sliced and diced, so long as the product gets built to spec.)
- Incentivize excellence, not corner cutting.
- Invest in foundational efforts like design systems and a clean tech stack so future efforts can be more efficient.
- Invest in UX and depth over breadth of feature sets.
- Invest in operational efficiency the right way, rather than chopping headcount and expecting the same quantity of output.
Otherwise, only one thing can give — quality.
