When Agile isn’t Agile
Signs your organization has missed the point of Agile and it’s become another buzz-word!
Treating Agile as just a method not part of a mindset, or culture, is a common contradiction in orgs trying to become Agile organizations. This approach is like following a recipe for a dish and believing that it makes you a chef. A chef knows the Why of cooking — which ingredients and methods do what, and why. Not grasping the Why of Agile is behind many failures to transform into an Agile organization.
Agile in Name Only
There was a cook who, whenever roasting lamb, cut the leg tip after having watched his mother do it, thinking it made a difference. Decades later, he learned that her reason for trimming was that she only possessed a small roasting pan. As with all things in life, if you don’t understand the Why of something, then you’ll likely miss the point. The same is true of Agile.
Many organizations claiming to be on the path to Agile transformation are still replete with steering committees, waterfall plans by proxy (Powerpoint “Gantts”) and similar Agile anti-patterns. Such things might have their place, but are often overused whenever’s there’s a failure in grasping the Why of Agile.
The opposite to Agile is Waterfall, or “Big Bang” planning. As Sims & Johnson point out in their book The Elements of Scrum, Winston Royce described Waterfall as early as 1970, noting that it was an example of how not to do things. He went on to describe an iterative process, although he didn’t call it Agile. I suspect he called it common sense because he understood the Why.
The Why that Royce understood came from an observation. He recognized the folly of planning too far ahead when faced with the specter of uncertainty. He understood that we cannot eliminate it. Instead, we must embrace uncertainty. This is the core of an “Agile mindset” — the Why of Agile. If you take away nothing else from this article, then note this: if you think of Agile as just another method that project folks do, you’re missing a trick. If you think of it as a tool for embracing uncertainty, then you’re on the right track.
Uncertainty Is The Norm
Everybody knows from experience that projects never go to plan. Things going adrift is the norm and yet we tend to act as if the reverse is true. Consider how often the prevailing management culture is to apply undue pressure and emphasis upon deadlines as if they are the sole objective of work. Or, consider the undue demand for details that simply aren’t knowable, at least with any usable accuracy. I think part of this comes from clinging to the old maxim that the ability to hit a deadline is only as good as the detail of the plan. But the result is often the overproduction of documents and meetings that border on “planning the plan”, not to move the needle forward, but to assuage higher-ups of their doubts. This is the wrong way to cope with uncertainty.
The natural state of affairs in any complex organization is that we should, most of the time, be uncertain about future events. Moreover, we ought to expect uncertainty to be increasing due to the hyper growth of digital assets. It has become so trivially easy to create assets via digital tools, that the rate of creation far exceeds any reasonable rate of maintenance and management. It is estimated that up to 80% of work is just doing “meta work” — i.e. joining the dots between assets, tasks and what’s in people’s heads (“Tribal Knowledge”). In case it isn’t obvious, this isn’t productive work.
Agile as Heuristic
As world-renowned heuristics expert Gerd Gigerenzer points out, nearly every decision we make in life is under the condition of uncertainty. Typically, we have incomplete information. That’s why humans rely upon heuristics, or rules of thumb. At its core, Agile is rooted in a heuristic which says that in the face of uncertainty, it’s better to take smaller steps and get feedback often whilst course corrections are affordable. This is the “common sense” that Royce had noticed. And note that this applies to any project, software or not. It is the uncertainty in the face of complexity that matters.
Uncertainty and risk are not the same thing. This is a common confusion. Risk is about weighing up the probabilities of various outcomes when they are all known. Uncertainty is about not knowing what some outcomes, or associated probabilities, might be. Managers who are “risk averse” typically overreact to uncertainty, which can undermine the Agile process. Their reaction is often to insist upon details in order to “de-risk”. However, there are no such details to be had if, indeed, uncertainty prevails.
The only real solution here is to trust in the process and trust in the people running it. But this is made all the more complicated by a subtle side-effect of Agile, also related to heuristics, which I call the “intuition gap”.
Agile and The Intuition Gap
Iteration via smaller steps allows workers to exercise better intuitive judgment about their work. Working closer to the problem space (i.e. hands-on) they are able to work with what I call the “intuitive flow” — their intuitions about solutions are fairly accurate. This flows from being in touch with the problem at “intuition-scale” — i.e. where various parameters are connectable via gut feel.
A major problem occurs when workers with “intuitive flow” make proposals or decisions about their work based upon intuition. Higher-ups, in trying to assess such proposals, are faced with what I call the “intuition gap” — they cannot intuit what the worker intuits.
The results are often calamitous because the higher-ups will insist upon detailed justifications for proposals as a means to bridge this gap. Obviously, intuition is lost in translation — workers cannot articulate their decision-making process because it involved gut-feel. This makes matters worse when higher-ups begin to doubt the efficacy of workers who cannot articulate their reasoning clearly. Poor articulation gets misinterpreted as incompetence.
This is the exact opposite of what ought to happen. If a worker is using Agile or some other hands-on method to navigate at intuition scale, then the worker ought to be more trusted, not less. Lack of details, per Gigerenzer, is actually a good thing because that’s how intuitions work best. Attempting to back-fill with details can throw the intuition out of whack.
In fact, heuristics work not just by knowing which dots to join, but by knowing which dots to ignore. As Gigerenzer says: “In contrast to the widely held view that less processing [“detailed planning”] reduces accuracy, the study of heuristics shows that less information, computation, and time can in fact improve accuracy.” He cites plenty of examples in his literature.
To bring this to life, let me use a sailing analogy.
Imagine that the worker is a skilled sailor navigating the Atlantic towards Dublin with her hand on the yacht’s tiller whilst tacking and jibing via iterations, sensing the wind, rudder play and other factors in order to intuitively course correct. A manager’s insistence upon a detailed justification is like asking the sailor to explain a set of differential equations used to calculate the wind shear. Bonkers!
Of course, no such calculations were ever made by the sailor. But the sailor is acting as if they were, via intuition. Asking the sailor to convert their intuitions into equations is the equivalent of insisting upon detailed justifications.
But the omission of information is equally important. Say the yacht is nearing a sand bank that could possibly ground the keel. The sailor in “intuitive flow” is ignoring this information because it’s irrelevant. She already knows that her course has no danger of grounding. However, the higher-up, not knowing what the sailor knows, sees the sand bank and raises the alarm.
Unwritten is Not Undone
The double-whammy is that the intuitive process, because it is unwritten and unrecorded, causes higher-ups to assume that either nothing is happening, or it’s not happening fast enough because they cannot see detailed justifications. This, again, is the exact opposite to what ought to happen. Remember what Gigerenzer said: “less information!”
The failure of higher-ups to appreciate the intuitive advantages of following Agile, in addition to its concrete aspects (e.g. stories, velocity etc) can lead to a lack of trust and an opposing cultural approach that serves ultimately to unhinge the potential gains. Higher-ups will intervene with various corrective measures that are based upon their view, absent of the hands-on intuitions available to the team. Returning to our sailor, perhaps she suddenly appears to be headed for Iceland instead of Dublin. The higher-ups immediately intervene without knowing that the sailor was forced to alter course temporarily in order to avoid a storm.
You might think that merely providing information about the storm will assuage the higher-up’s concern. But this isn’t always so. The Dunning-Kruger effect kicks in and suddenly the higher-up thinks they know better what action the sailor ought to have taken. In other words, we are back to asking the sailor to explain and justify every small decision as if they were all calculated like a set of differential equations, absent of intuitions. [Note that this is what happened to the pilot of Flight 1549 after he landed on the Hudson River, as told in the movie Sully.]
Moreover, these types of intervention are susceptible to cognitive biases that tend to take hold in such situations where the higher-up is unable to cross the intuition gap. For example, if the worker is junior, a higher-up might interpret the lack of detailed justifications as lack of experience or insufficient technical knowledge.
Anyone reading this article who has ever worked in a hands-on team close to the action will know what I am talking about. Your team is happily getting on with the work and yet higher-ups are making all kinds of odd interventions based upon their biases. How many times have you said, or heard your co-workers say, in relation to higher-ups: “They don’t get it!”
This expression — “get it” — couldn’t be more apt. Lacking the same intuitions as the workers, higher-ups literally don’t get it. And now you can see my particular choice of words — higher-ups — makes sense. Managers are usually higher up in terms of floating above the scale of intuition as it applies to the project’s details.
Agile Mindset
So what, then, is an Agile mindset and how does it solve the maladies laid out above? It mostly comes down to trust, in the process and the people, plus an acceptance of uncertainty.
Uncertainty is the norm, yet the Agile process, if set up correctly, smooths out uncertainty over time. It provides concrete mechanisms for taking action and it provides localized data that lessen uncertainty. The process goes hand in hand with working at the “scale of intuition”. In other words, you should expect the hands-on workers to be in touch with certain intuitive decision-making processes that lead to actions without the need for detailed justifications.
Insisting upon details in lieu of uncertainty is the wrong move and the wrong mindset. Intervening with suggestions and course corrections that are based upon information that you think the team is ignoring, like that approaching sand bank, is usually a bad move. The irony is that such well-intended interventions can end up causing the sailor to hit the sandbank by reverting to some mechanistic process that interferes with intuitive flow. This exacerbates the tendency towards judging the worker as incompetent.
Conclusion
Complexity is on the rise. The days of wanting detailed justifications and accounts of work are over. This approach isn’t scalable in today’s modern workplace. The scalable approach is to decentralize work and put the decision making into the hands of the workers, empowering them via trust and knowledge of how processes like Agile actually work. Not only do they provide concrete means to get things done, by handling uncertainty, they also allow workers to operate with “intuitive flow” in ways that may not translate into detailed justifications.
