Summary

This case study concerns a sewerage upgrade project in its pre-feasibility phase. It outlines a simple qualitative risk assessment for the project, the development of treatment actions, and an initial evaluation of commercial and pricing options. The case draws lessons about the value of risk management for procurement, and some of the pitfalls of residual risk calculations.

Introduction

Project background

The wastewater system for a municipality was approaching capacity. The catchment covered a wide area and served a large and growing population. It included suburbs with a range of residential, commercial and industrial land uses and a Central Business District (CBD).

The main spine sewer through the area was built nearly a century ago. It was heavily overloaded, major overflows were common in wet weather and the frequency and extent of overflows were projected to increase even in dry weather.

The preferred option for augmenting the system was the construction of a new gravity relief sewer tunnel. The tunnel would be 4 km long, with a possible extension of 0.5 km. It would have a finished diameter of just under 2.5 metres. The preferred alignment passed through mixed ground conditions, including a stretch of variable and soft ground.

The project was in its late pre-feasibility phase. Final technical options had not been agreed, although the tunnel option was clearly preferred. Commercial and procurement arrangements had not been finalised and no contracts had been drafted.

Broadleaf worked with three stakeholder groups:

• The asset owner: the wastewater system belonged to the municipality, which provided capital for the upgrade
• The operator: the system was operated by a separate entity, notionally owned by the municipality but in practice independent in technical and most commercial matters
• The project team: the specialist team, largely from the operator, tasked with ensuring the upgrade was implemented successfully.

The objectives of the risk assessment were to identify and set priorities for the risks associated with the project and to develop action plans for treating the most important ones.

A separate assessment of commercial options was required, to contribute to the operator’s strategic planning for its commercial future. The outcomes required from the assessment were:

• A list of the main commercial options for delivering the project
• A preliminary analysis of the benefits and costs of each option.

Approach

The approach to risk management was aligned with ISO 31000 Risk management – Guidelines (Figure 1). We worked with the project team to develop scope and context material. We then facilitated two structured workshops: the first addressed risk assessment, the identification, analysis and evaluation steps; the second, held several weeks later, addressed risk treatment. Participants in the workshops were members of the project team and specialist advisers from the asset owner, the operator and external organisations.

Scope, context and criteria

Purpose

The first step in the risk management process had several purposes:

• To establish the scope of the risk assessment and the strategic environment in which the risk assessment was taking place
• To identify a set of criteria, related to the objectives of the project, the project’s success criteria, for measuring the consequences of identified risks
• To define a set of key elements for structuring the risk assessment workshop.

Scope

The scope of the risk assessment covered the remaining phases of the project:

• Technical feasibility, when the detail of the preferred option would be finalised and agreed
• Procurement, when the preferred commercial approach to delivering the project would be selected and implemented
• Delivery, when the project would be implemented
• Handover of the completed asset to the operations team.

Criteria

The objectives of the asset owner and the operator were reviewed and condensed to a set of success criteria for the project (Table 1). These were required for the analysis of the consequences of risks later in the process.

Key elements

We prepared a set of key elements, linked to the main activities of the project (Table 2) to provide a structure for the risk identification task in the main workshop. These key elements helped the team to focus their attention and ensure that all important aspects of the project were considered.

Commercial options

The commercial options for the project formed a core part of the context for the risk assessment. They were analysed in parallel with the other risk management activities. An initial structure was developed as part of the first (assessment) workshop. Details were discussed and agreed at the second (treatment) workshop.

The structure for the commercial evaluation was based on a set of five delivery options (Table 3), combined with three pricing options for the commercial relationship between the asset owner and the operator (Table 4), and a further three similar pricing options for the commercial relationship between the operator and a supplier of tunnelling and other construction services. This resulted in a total of 45 scenarios, not all of which were anticipated to be feasible.

The advantages and disadvantages of each option for the asset owner and the operator were considered, with the discussion recorded in templates like Table 5. The advantages and disadvantages were allocated an indicative rating on a 5-point scale, according to an estimate of the financial benefits or costs that might be realised. These were combined to form an overall attractiveness rating for each option, to generate a triage of Good, Marginal and Poor options.

This form of options analysis using qualitative measures provides a framework within which the participants can share and clarify their thoughts quickly. Where detailed quantitative information is required, this can be developed later to refine the initial assessment.

There was a clear correlation between the perceived benefits and costs for the scenarios. Those scenarios offering the highest rewards generally also had the highest potential costs and risks.

The initial structure of 45 scenarios was reduced after considering individual delivery and pricing options and their combinations. Some scenarios were eliminated because they were commercially infeasible, and some were combined because the distinction between them had no meaning in an Alliance delivery structure. Table 6 shows the 24 feasible combinations that remained, with the outcomes of the evaluation and triage signified using the Good, Marginal and Poor designations with pricing designated as Fixed, Risk sharing, and Schedule of rates.

The analysis and evaluation had several limitations:

• The analysis was squeezed into the time allocated for an assessment of the project risks. In consequence, the time available was restricted, limiting the depth of coverage to a preliminary assessment only.
• The analysis was conducted in a forum consisting of people whose skills lay primarily with the project and technical tunnelling matters. Although many had good commercial knowledge, that was not the main reason for their participation.
• The analysis was recognised as a preliminary assessment, a prioritising and culling process to identify viable commercial arrangements to be further examined by the operator in seeking to achieve its goals.

The asset owner and the operator were not considering a commercial arrangement without further in-depth analysis of the commercial imperatives and the associated risks. The 14 scenarios initially evaluated as Poor would probably not receive any further effort, but each of the remaining ten scenarios with initial ratings of Good or Marginal were considered worth examining in more detail.

Risk assessment

Workshop process

For each element in Table 2, risks were identified in a structured brainstorming workshop. Risks were defined as events or circumstances that could arise and affect the success of the project.

For each risk:

• Associated controls were noted, conditions or measures that would affect the consequence of the risk and the likelihood of that being experienced by the project
• A rating was developed for control effectiveness, considering the design and implementation of the controls compared to the best that could be expected for wastewater organisations like this, not compared to perfection as that is unrealistic
• The consequences of the risk were rated in terms of the potential effects on the project criteria in Table 1, taking into account the existing controls and their effectiveness
• The likelihood of those consequences arising was rated, again taking the controls into account
• An assessment was made of the potential exposure, the largest consequences that might arise were all the controls to fail in a credible manner
• An initial level of risk was derived from the consequence and likelihood ratings, initially as a triage of Major, Medium and Minor risk.

All the risks, controls and ratings were reviewed and confirmed in a final workshop session, and individual risk owners were allocated for all the Major and Medium risks.

Many risks were allocated initially to the Project Manager, often because the direct responsibility was unclear or potentially shared. This was a matter of convenience in the workshop. In practice, many of these risks were delegated subsequently to other members of the project team.

Outcomes

146 risks were identified during the workshop and the subsequent review: 16 Major, 55 Medium and 75 Minor (Figure 2). The relatively large number of Major and Medium risks reflected the early stage of the project’s definition, the early stage of development of project controls and hence the high degree of uncertainty.

Most of the Major risks, those with both high consequences and high likelihood, related to approvals processes. Most of the Medium risks were technical, particularly associated with tunnelling tasks and equipment, or were related to commercial procurement matters.

There were no risks identified for several of the elements, those that were regarded as business-as-usual for the operator.

Risk treatment

Process

After the assessment workshop, designated risk owners for the 16 Major risks were tasked with identifying and analysing options for treating them, and developing draft recommendations and action plans to address the recommended options, using templates like Table 7. The draft templates were discussed in detail during the second workshop and updated to include the main matters raised. Participants considered the plans in detail, recommended additional options where appropriate, and assessed the residual risk to the project if the new controls were implemented fully.

Responsibilities for actions were not all allocated within the project team. Some of the commercial procurement risks with potentially organisation-wide implications, above the level of the project, were later allocated more properly to executives in the asset owner and the wastewater company.

Outcomes

Many of the options and actions that were recommended were routine matters that would be undertaken in any project like the one being discussed, due to the operator’s application of regulation, policy and good practice. They had not been included in the risk assessment as existing controls due to the early stage of the project at which the assessment was undertaken. After reviewing these ‘normal’ project controls, and additional controls that would arise as a result of implementing other recommended actions, the risk analysis was revisited. This generated a new level of risk, sometimes called the residual risk.

A set of risks related to construction processes and acceptance was identified. They were grouped for attention because it was likely they would need to be addressed in the contract documents. They might be disaggregated later as they involved different kinds of activities, but it was more useful to consider them as a group at this early stage of the project.

After considering planned treatments, there were only four risks with residual ratings of Major, three of which were directly concerned with delays in internal and external approval decisions. Approval delays could all have large negative consequences for the project and all were rated as likely to arise; clearly significant additional effort would be required to address approval matters. Of the other risks rated as Major in the first workshop, six were reduced to Medium and six to Minor.

Lessons

Procurement support

The risk assessment conducted here, during the pre-feasibility phase of the project, preceded any significant procurement activities. Risk management at this early stage provides structured input to several procurement processes, including gathering market information, preparing Request for Tender (RFT) documents, evaluating tender responses and preparing draft contracts.

Risk management helps to identify matters that should be included in RFT documents, and particularly in specifying the contractor experience, capabilities and processes that are necessary for dealing with the risks of the kind anticipated in the project. As well as requiring prospective contractors to set out their risk management capabilities, RFT documents usually ask for a list of the major risks that are anticipated and how they would be addressed; the internal risk assessment helps to evaluate the thoroughness of tenderers’ responses and, indirectly, their appreciation of the challenges the project presents and their risk assessment processes.

The draft contract should contain the preferred allocation and pricing for risks, and particularly any that are significantly different from those anticipated in ‘normal’ construction activities. For example, in this project there were risks associated with two technical areas that might be incorporated in specific contract terms and conditions:

• Tunnelling itself requires particular expertise, but there are additional requirements and controls when tunnelling in variable ground conditions and below residential and CBD areas
• Interconnections and break-ins to the existing wastewater system require care, particularly in the case of old, legacy assets whose physical condition may not be known precisely when the contract is signed.

Problems with residual risk

We calculated residual risk, the level of risk anticipated if the treatments were implemented fully, in this case because the client insisted on it – it was part of the organisation’s risk management system recommended by a large, well-known consulting firm. We prefer not to use residual risk, for some of the reasons set out in Table 8.

General lessons

This was a relatively straightforward risk assessment. The project had a few special features but, with good project management, there were no obvious fatal flaws.

Like many projects that are concerned with the delivery or upgrade of physical assets, success would be determined not just by the outcomes of the delivery phase but by the benefits achieved by the asset itself after it was handed over to the operators. The criteria for success in Table 1, which underpinned the measures of consequences used in the risk analysis, reflected what would be considered ‘good outcomes’ for the operating sewer as well as for project delivery.

For any risk assessment, the development of a comprehensive set of key elements is fundamental to the efficiency and effectiveness of the risk identification activity. With too few elements, risks tend to be stated in quite general terms and it is often difficult to be precise enough to develop treatment plans that are specific and targeted. With too many elements, discussion is usually very detailed too, and not always related to important matters, so time is not used as fruitfully as it might be. In this case the elements in Table 2 were sufficiently detailed to provide a good structure for brainstorming and effective use of time in a one-day workshop.

The value of the risk assessment in setting priorities for risk treatment was clear to the project team. They were able to focus on the important matters first and develop plans that were expected to achieve appropriate benefits quickly. After the first treatment workshop they conducted further work to develop plans for treating the risks classified as Medium, as well as for the four Major risks that still warranted attention.

It is pleasing to note that the new sewer tunnel is now in operation and achieving the outcomes that were desired by the stakeholders.