A government transport agency undertook a lessons learned process to ‘look back’ at the tendering process for a recent highway project to be procured as a public private partnership (PPP). The agency’s business standards require that, ‘after any significant event or change, a suitable root cause analysis will be conducted to learn lessons from both successes and failures’.
The objectives were:
- To identify lessons learned and define the actions that should be taken to ensure that successes could be repeated and failures could be prevented
- To provide input to an agency-wide procurement strategy for major projects.
This second point was important in shaping the way the process was implemented. At the time this work was undertaken, the agency was not familiar with PPP procurement processes, but it recognised that PPPs would be increasingly important for major projects in the future. It was anticipated that this exercise would identify many specific and detailed matters, but it was important that the primary focus be on more general lessons. This would enable the agency to take a strategic, long-term view as it developed its PPP approach and processes.
With this in mind, two workshops were conducted, one to learn lessons and a second to focus on how risks should be managed in PPPs in the future, consistent with the agency’s established risk management framework.
The case demonstrates how root cause analysis can support new or innovative initiatives.
The PPP project involved upgrading, operating and maintaining several hundred kilometres of a major highway for a 25-year period. The capital costs were anticipated to exceed USD 1 billion.
Following an analysis of options and preparation of a value-for-money analysis and business case, and with the approval of the government, the agency decided to implement the project as a PPP with private financing. This particular highway upgrade was suitable for PPP procurement because:
- It is an important strategic route
- It was at a suitable stage in project preparation
- A feasibility study had been carried out, preliminary designs were available and most key approvals had been secured
- It had suffered many years of delay under conventional procurement processes
- The existing road was in a poor condition and had a terrible safety record.
The company selected to manage the PPP concession would receive specified annual payments, subject to deductions for failing to meet strict performance targets relating to availability, quality and safety.
The expected PPP project and contract structure is shown in Figure 1. Figure 2 shows the government organisation structure for the project.
A key objective for the PPP procurement was to maximise value-for-money by:
- Incorporating effective quality- and cost-based competition in the bidding process
- Attracting consortia of world-leading highway designers, developers, operators and financiers
- Encouraging innovation in design, construction, financing, operations and maintenance
- Enforcing high standards of quality, safety and operation through the payment mechanism
- Providing incentives to the concession holder to complete construction as soon as possible and in the sequence of highway segments preferred by the agency
- Providing incentives to the concession holder to optimise design, construction, operations and maintenance over the life of the highway asset
- Avoiding excessive pricing for risk caused by an inappropriate allocation of project risks over the concession period.
Some of the main stakeholders and their objectives and concerns were identified and discussed before the first workshop. The main stakeholders were:
- The agency
- Institutional equity investors
- Private equity investors
- Design and construct prime contractor
- Operations and maintenance contractor
- Utilities and support services
- Local communities.
External and internal factors
Some of the factors that might influence the success of the PPP are noted in Table 1 and Table 2. (These tables only contain selected items from a more detailed analysis.)
The global financial crisis intervened
Credit availability was reduced
Finance costs increased
Construction costs fell, linked to a contraction of the market
Lenders’ appetite for risk reduced: lenders became more conservative, and some risks that were normally taken were now pushed away
Some banking arrangements were exclusive
Some banks were committed exclusively to one consortium
The agency insisted that these arrangements be dissolved after a preferred bidder was selected
A PPP is more expensive initially than a conventional project
A PPP has a front-loaded cash flow profile, different from a conventional project
Initial procurement costs were expected to be outweighed by more efficient project delivery and cheaper operations and maintenance
The payment schedule was designed to allow bidders flexibility in how they might deliver the highway, but with clear incentives to align with the agency’s preferred sequence
The agency was unfamiliar with a PPP approach
Guidelines for pre-qualification and bid evaluation were developed as the project progressed
There was no model within the agency for risk analysis and risk allocation
There was no established PPP unit, procurement process or probity rules
Communication between functions was difficult initially
The procurement function’s lack of experience with the PPP approach made progress difficult
The role of procurement was not specified clearly
There was initial conflict between the Project Committee and the procurement, finance and legal functions
The tendering process was generally well managed
The bidders had a favourable view of the process
Documents were to a high standard, well structured and easy to follow; they will form a useful model for future PPPs
The data room and document management was not so good –the procurement function had no experience in managing such information
Five world-class consortia were shortlisted for bidding
Root cause analysis workshop
The lessons learned workshop examined the successes and failures of the procurement activity for the PPP project, across the three phases of work that had been completed at the time:
- The initial preparatory work, before the issue of the Request for Quotation (RFQ)
- Issue of the RFQ
- Evaluation of responses to the RFQ.
Participants in the workshop were members of the agency project team, its specialist external advisers, and members from other key functions in the agency.
The participants identified nine successes to be analysed, and a further seven matters where the outcomes were below what was desired. Root cause analyses, using the fishbone (Ishekawa) method, were undertaken for the matters considered the highest priority in terms of potential improvements for the agency.
Figure 3 shows an extract from the fishbone for one of the items examined in detail, described as: We did not review draft documents internally at an appropriate pace for the project. Three major lessons were derived from the discussion, relating to the project management process in the agency as a whole and the composition of the PPP project team. A set of seven actions was developed to embed improvements associated with the lessons.
Three general lessons for the agency were derived from this specific root cause analysis.
- All major procurement exercises should start with a risk assessment that leads, in part, to the development of a communications plan for all relevant internal and external stakeholders. It is particularly important to communicate with key decision makers so that they clearly understand the context and strategy behind a particular procurement before they are asked to decide on the award of contract.
- Each major procurement exercise should be run as a project, with a project plan, timelines and resource requirements.
- The resource impact on the agency should be assessed as part of project and procurement planning to ensure that suitable and sufficient resources (including office space) are available to support, manage and derive maximum return from the project.
Risk management workshop
The objectives of the second workshop were to:
- Understand the general risk management needs and requirements for PPP projects
- Review and agree the updated risk register and risk treatment plan for this specific PPP
- Develop a generic risk management plan for future PPP projects.
Participants were representatives from the main delivery and support functions in the agency: highways, transport planning, procurement, finance, legal, public and community relations and the Program Management Office.
The workshop followed the process in ISO 31000 Risk management – Principles and guidance. The briefing note for this workshop included the context material for the first workshop and its outcomes.
The workshop identified and documented 110 risks. Discussion and subsequent analysis generated a register containing, for each risk:
- An identifier for the risk
- A description of the risk
- Ratings for the consequences, likelihood and level of risk
- Comments on how the risk might be treated, and whether by the agency or the concession holder
- The proposed allocation of risk between the agency and the concession holder
- A cross-reference to relevant clauses in the concession agreement.
The allocations of risks to the agency or the concession holder were clear in most cases. However, there were some risks for which sharing was agreed to be a better option:
- Traffic volumes above or below the forecast band
- Movements in debt margins between the time of bidding and award of the concession
- Movements in inflation and exchange rates
- Refinancing gains on completion of construction and the substitution of project funding by long-term debt.
A revenue model based on availability of the road rather than tolls avoided transferring the risk associated with traffic volumes to the concession holder, a risk the market would not otherwise have accepted in this case.
Procurement using a PPP was new for the transport agency. It was familiar with traditional procurement processes but it had not employed more innovative contract structures. Few permanent staff had direct experience of PPPs; this experience was confined primarily to new members of the team created for this specific project.
Learning lessons was particularly important under these circumstances. The outcomes from the first workshop influenced the entire procurement strategy for the agency, as well as providing more detailed tactical lessons and improvements.
The risk workshop facilitated detailed discussion about the allocation of risk. Like many government entities, the agency had previously used procurement approaches that allocated most if not all risk to its contractors and suppliers. By considering risk explicitly, participants started to understand the implications of their risk allocation decisions, and particularly the implications for the whole-of-life cost of the highway asset they wanted to create.
The workshops served to raise awareness about non-traditional contracts, in an organisation that previously had been highly risk-averse. The participants were able to relate the experiences so far in this PPP to the more conventional approach they had been following previously but which had not generated the outcomes they desired.
Raising awareness was enhanced by the selection of workshop participants. As noted above, they were drawn from all of the main functions in the agency who were concerned in some way with procuring highway assets. This assisted in spreading the message about the potential benefits of non-conventional procurement processes like PPPs, and what the agency would need to do to realise those benefits.
For this specific project, involvement in the workshop process allowed all the participants to agree and gain consensus about the lessons learned and the best way forward, promoting support for the implementation of recommended actions. This is a common feature of workshops that are focused on a well-defined outcome and are facilitated well.
Establishing the context
Establishing the context is an important step in many management processes, as it sets the scene for subsequent analysis and the development of options for decisions. It serves this function for root cause analysis, just as it does for risk management.
The international standard IEC 62740-2015 Root cause analysis (RCA) specifies two steps that are equivalent to establishing the context in ISO 31000 Risk management – Principles and guidelines:
- Initiation: based on the knowledge available on the focus events, the successes or failures of interest, determine the need to carry out RCA and define the purpose and scope
- Establishing facts: Collect data and establish the facts of what happened, where, when and by whom.
The analysis of stakeholders and the external and internal context described here satisfied the needs of both the root cause analysis workshop and the risk assessment workshop, despite their different objectives.
Making the most of innovation
As noted above, this case study examined an approach to procurement that was innovative for the agency, and the analysis had a powerful effect on its future strategic direction.
Root cause analysis provides an important way of learning lessons that may have significant implications when an organisation has done something different or tried a new initiative. The significance lies in the ability to review outcomes, identify potential improvements and recognise what might be done to enhance strategic advantage. In this sense, learning lessons through a formal root cause analysis process is an integral part of the Plan-Do-Check-Adjust management cycle.
On the other hand, root cause analysis can be used to promote innovative approaches to day-to-day operations. The actions taken in response to an identified cause may take the form of doing something completely different, either to enhance a cause with potentially beneficial consequences or to eliminate a cause that may lead to undesired outcomes. However, generating actions that are innovative does not depend on root cause analysis alone – it requires sound facilitation that is planned and structured carefully to foster divergent thinking and the dispassionate evaluation of new ideas.