We assume that large and complex projects will be risky. Complexity, scale and risk are bound together in the way we think, feeding off one another. It may be, though, that complex projects are risky not just because of complexity but also because of the way we choose to approach them. The interaction between complexity and project management raises many interesting issues. This is just one of them that has wide ranging implications.
When faced with a challenging project, most of us default to detailed analysis and forecasting. The more challenging the project appears the more analysis we will do – breaking down work into smaller parts, building up bigger schedules and estimating costs in more detail. Then we set out to enforce that plan from the start to the end.
If a project fails, we tell ourselves that there was insufficient effort devoted to analysis and control. There have been many articles and papers over the past few years using words to the effect of 'If the scope had been clearly agreed and change had been controlled, we could have avoided this disaster', but perhaps it isn’t always that straightforward.
Complexity and complication
Close control is feasible, in principle, if we can understand the cause effect relationships at work in a project. If we understand these relationships then, whenever we deviate from our original plan, we will understand what caused the deviation and be able to take steps to bring the work back on track.
However, if a project is inherently volatile and exists in a state of real complexity, this might not be possible. The important distinction between a complicated project, where cause effect relationships can be found, and a complex project, where it may be useful to consider an alternative approach from the outset, is outlined in this short video.
Where there are multiple independent stakeholders whose behaviours affect one another and affect the way the project develops, while at the same time the stakeholders’ intentions are affected by what happens on the project, it might not be possible to understand once and for all how the various forces at work interact, let alone exercise tight control. This is complexity that goes beyond being complicated. Each day might see the course of the work being driven by different issues as the work itself and the people who influence it change and interact with one another.
When our orderly plans are overtaken by creeping changes or disruptive events, we console ourselves by blaming it on complexity. While complexity may actually be the root of many of our problems, most of us are not mentally prepared to work with it and, for some, it can just be a convenient excuse. It is quite rare for anyone to suggest that detailed analysis and tight control might not have been the best way to tackle a challenging project, but this is worth contemplating.
Approaching complex projects
By approaching complex tasks as if they were orderly, we set an expectation that the work can be closely controlled. We invest effort in developing detailed plans that will be overtaken by events. Despite the number of papers and speeches we see devoted to empowerment, responsiveness and agility, some organisations’ cultures still invest so much in control and precision that they leave themselves poorly prepared to recognise and respond to changes that fall outside the framework on which those plans were based. It is understandable that we complain when key stakeholders change their priorities or circumstances that we really could not have anticipated crop up, but we still have a job to do. Dealing with these changes and surprises is part of that job.
Throwing ever more detailed and intense analysis at a project in which stakeholders (owners, contractors, investors, regulators, neighbouring communities) will shift ground and modify the way the work is carried out, might not only be wasteful but also builds a false sense of security. This is not to say that we should give up on plans, forecasts, monitoring and control. However, there is a point beyond which not only do we see a diminishing return on our efforts but we might actually be limiting our capacity to sense and respond to the forces at work in our project. So much effort can go into looking for what we expect to happen that we don’t see something important, that we could not have anticipated, until it is too late.
This effect has been described nicely in an INSEAD paper. The same phenomena can be seen in the introduction of new information systems and other changes into an organisation. Any large undertaking that relies on knowledge and information artefacts as much as or more than on physical deliverables can face these challenges. The physical deliverables of a project may be very tangible, such as civil infrastructure, but the design exists as knowledge and data.
Applying a strongly ordered approach to a complex project can actually generate risk rather than quell it. We focus on a predefined set of performance indicators so we miss subtle shifts we are not measuring. Laying down a detailed plan from here to the project’s end, we constrain the creativity required to react swiftly when circumstances change in ways that the reporting systems do not pick up quickly.
For example, we might find staff churn resulting in a gradual but steady reduction in skill levels, leaving the team without the intellectual capacity to maintain progress. All the easy installation activities might be completed first so the rate of progress will decline as we near the end of the job and have to take on the tougher tasks. Rising levels of conflict that are not seen as particularly important to the technical content of the work might break out in staff losses and a breakdown in essential working relationships. Each of these specific examples can be monitored, once they have been mentioned, and could easily find their way into a risk register and be managed through conventional means. The problem is that there may be such a large number of subtle and potentially significant factors of this kind that we cannot hope to anticipate them all.
If we take a moment to ask whether our project is basically ordered and controllable, even if it is complicated, or is actually complex and fluid, we can adopt an appropriate approach. It is likely that some parts of a large project will lend themselves to tight planning and control while other parts are better suited to something more flexible.
The Agile movement in software, systems and related areas has recognised this challenge but it has yet to take hold in other areas. Wherever knowledge and information are at the core of our work, the effect of complexity is worth considering.
Disrupting default behaviour
When we face complexity, we may need to disrupt our reflex tendency towards exhaustive analysis and rigid control. This has to be a conscious effort, especially for those of us who have worked for decades in environments where detail and a systematic approach to complicated systems was seen as the answer to any challenge. We can start to position ourselves to deal with complex tasks by:
- Avoiding premature convergence on fixed ideas, allowing uncertainty and ambiguity to persist in our own minds and in the shared understanding of the project team
- Reminding ourselves that some systems, or parts of systems, can be influenced but not controlled
- Watching for and investigating weak signals of emerging deviations rather than relying solely on rigid lagging indicators, filtered through static reporting schemas and layers of management with vested interests in reporting on-target performance
- Locking down any parts of the work that we can control, piece by piece, as we find them, while tolerating and remaining responsive to those that remain complex.
The Cynefin framework provides a means to make sense of the difference between order and complexity. Ideas flowing from this framework provide guidance on how to approach each of type of work: obvious, complicated, complex or chaotic.
If we are lucky enough to face a project in which all the moving parts are predictable, we can plan it in detail, exercise tight control and expect to achieve a successful outcome. If we strike a project, or significant part of a project, in which intentions and relationships will shift as we carry out the work, and the way we carry out the work will stimulate stakeholders to change their behaviour, an incremental and responsive approach is likely to be more fruitful.
Hitting a fluid complex project with detailed plans and tight controls will not lock down the behaviour of the project’s major stakeholders but it could lock the project management team into an inflexible mindset and daily pattern of work that cannot keep up with unpredictable changes in the project. Rather than reduce risk, this will amplify it and reduce our capacity to achieve our goals. The effort to control risk might leave the team trying to tame a tiger while stuck in a straitjacket.