The Enduring Agile Enterprise

Engineers within large agile organizations are frustrated. Making decisions at a strategic, or even tactical, level is hampered by poor situational awareness. Individuals and teams find it difficult to identify the most important issues in their area of concern, let alone the solution in general. Comprehending the interactions between issues is even harder, and making projections about future states based on their best current assessment is a black art which only the most experienced, and long-serving, employees can manage effectively.

When an organization has more than a few self-organizing agile teams collaborating on a single offering, it becomes difficult to produce a coherent release. Trade shows, earnings calls, and competitors offerings demand a coordinated response, but, with each team focused on different objectives and continuously “distracted” by urgent patches, and emergency releases, predictable release scheduling becomes a near impossible task.

Onboarding new employees takes months when internal systems are an obscure alphabet soup of acronyms with no high-level map of the overall solution. When vital operational information is mixed with outdated and misleading content in unstructured wikis, new employees struggle to get traction. Favoring working code over documentation is excellent for a single, fast-moving, team working on a new solution, but when the overall system is spread across many poorly documented code repo’s that have evolved over many years, finding the information you need becomes an archaeological expedition.

I am frequently asked “Is everyone this messed up?” to which I answer “No, only the most successful companies are this messed up! It takes time and effort to get like this!” Few organizations have a solution to this problem. Indeed, what many perceive as a problem may be the unavoidable price of enterprise agility, steady growth, and staff turnover. Rapid adaptation to emerging needs arises from a steady churn. Evolution proceeds through a series of wasteful experiments. However, there may be a way to reduce the pain. Many of these inefficiencies have the same root cause.

The Agile Manifesto famously stated:

With the experience of enterprise-scale agile, we can see that the Agile Manifesto is a small-scale solution that does not solve long-term problems. Specifically, the Agile Manifest does not solve the problem of staff turnover. Software Engineers typically stay in their jobs for 2 to 3 years before moving on. Most software organizations experience something like a +15% turnover per year. Agile projects lasting 6 to 9 months can often avoid turnover issues. Engineers consumed by a project tend not to look around for new opportunities. Together, these factors mean the agile manifesto works well for single teams on single projects, but in an organization with many teams and projects, the manifesto is itself the cause of long-term problems.

Valuing “Individuals and interactions over processes and tools” is fine if the individuals stick around. When individuals move on it is often only the processes and tools that remain.

Valuing “Working software over comprehensive documentation” is great if detailed knowledge of the software is already in the heads of the developers. However, when developers move on, the knowledge goes with them, all that remains is the code. Software inevitably breaks, leaving broken code with no documentation, which is the worst possible combination. Reverse engineering a complicated piece of broken code to fix a defect, with little understanding of what the code was originally intended to do, is fraught with peril.

Valuing “Customer collaboration over contract negotiation” works as long as the customers and the software developers are around to keep each other honest. When either party moves on it becomes hard to honor half-remembered agreements that were never written down. While negotiating contracts is seldom fruitful, some effort to document what we were trying to produce can avoid many subsequent misinterpretations.

“Responding to change over following a plan” is an act of faith that turns to hubris as the challenge scales. For a single skilled team, it can be justified. However, if ten-plus agile teams are trying to coordinate a coherent product release in time for next year’s trade show, trusting to responsiveness will inevitably lead to teams thrashing as they all try to adapt to each other’s changes. To be successful a project with so many moving parts requires a strategy for resolving conflicting demands. Agreed goals and interim objectives, clear ownership of responsibilities, internal commitments to deliverables and dates, and a robust escalation procedure to resolve the inevitable problems is one way to manage such conflict. Internal marketplaces where resources can be traded is another.

All this is not to say the Agile Manifesto is wrong. However, it is insufficient for large organizations with many teams. It is missing mechanisms to harvest, improve and transfer knowledge between employees as they join and ultimately leave the organization. Institutional memory is crucial to the success of enterprises that aim to survive for the medium to long-term.

Starting from 100 employees and growing at a modest 5% per quarter the company will reach about 325 employees in 6 years, by that time:

Starting from 100 employees and growing aggressively at a rate of 10% per quarter the company will reach about 1000 employees in 6 years, by that time:

The more aggressively you grow the worse the knowledge management problem becomes. Without an effective plan to harvest and redistribute knowledge, aggressive growth companies can easily forget how they became successful in the first place.

Preserving institutional knowledge is essential if organizations want to retain agility and velocity as they scale. Knowledge lives in the heads of people; it is generated from information that is itself created by processing discrete, unorganized objective facts or observations.

In Enterprises, knowledge becomes embedded in static representations like documents, and wikis. Knowledge also becomes embedded in dynamic systems, processes, and working practices. Managing the transfer of knowledge from peoples heads to these static and dynamic representations and back again is essential for knowledge to persist as people move through the organization and the organization grows.

Institutional knowledge management requires an explicit strategy that identifies a few key areas of focus. Organizations should not attempt to retain or control all knowledge; institutional learning is as much about retiring outdated information, as it is about capturing and retaining new information. Knowledge turnover is essential for organizational learning. By identifying a few key focus areas, organizations can ensure they retain knowledge in vital areas while allowing new knowledge to replace outdated thinking.

Enforcing knowledge management top down has a poor track record. Instead, it must be encouraged to emerge bottom up, as the people who directly benefit either share or consume valuable information. Organizations must incentivize these individual transactions if they wish to ensure the emergent benefits.

To retain knowledge, information must be acquired, revised, classified, utilized, and repeatedly evaluated by the people who use it. Providing recognition and rewards to staff who undertake these activities is essential. If knowledge management is not recognized as valuable, it will not happen. The problem with the Agile Manifesto is that it’s only solution for knowledge transfer is either word of mouth or code. In reality, there are other, often more effective mechanisms for capturing and transferring knowledge. Large enterprises must use all these methods if they are to learn and endure.

Related posts:

5 Popular Relationship Goals That Are Surprisingly Toxic| Unhealthy couple goals that subtly destroy relationships. Find out if you have these goals.

Last digit of GST Number, i.e. checksum digit, is still a mystery to many people and GST experts. This article is for those enthusiast who want to know the mathematics behind that 15th digit. The…

You just closed your round of financing, money is in the bank, it’s time to celebrate and party! 🔥🎉 It’s also time to dump your entire financial documentation in the trash where it belongs. Who…