Skip to main content.

Qualitative risk assessment for an urban toll road

Summary

This case study describes the conduct and outcome of a risk assessment for an urban toll road, to be delivered as a public private partnership (PPP). Only a high-level view was required, so detailed risk identification was not needed, but understanding was critical.

The case illustrates a simple risk assessment process, with an extended risk analysis stage that supported more detailed control assurance. It did not identify many risks, but it generated increased confidence and understanding among the key stakeholders in the project and the work that had been undertaken so far.

Background and scope

Project background

The project involved the construction of a new urban toll road linking two existing highway corridors. The new link would be a mix of new underground roads in bored tunnels, existing upgraded surface roads, and new surface roads that would require property to be purchased, some by compulsory acquisition.

The project was a key infrastructure priority for the government. Government road and delivery agencies had worked to relatively tight approval deadlines, and further tight deadlines were expected for construction.

The project would be delivered as a public private partnership (PPP), with a mix of public and private sector funding. Toll revenues and their timing would be important for the success of the project.

Scope and objectives of the assessment

As part of its due diligence activities, one of the funding agencies sought to understand the main risks associated with delivery of the project. Only a high-level view was required, so detailed risk identification would not be needed, but understanding was critical, so discussion was to be encouraged.

This case study describes the conduct and outcome of the risk assessment, based on a workshop with project representatives and stakeholders. Broadleaf prepared for and facilitated the workshop then analysed the results and reviewed them with the agency’s team.

Process outline

The process followed in this case was based on the international standard ISO 31000 Risk management – Principles and guidance, outlined in Figure 1.

Figure 1: Risk management process

The risk management process

Establishing the context

Discussions were held with senior personnel to establish the context and set the scene for the risk assessment. The outcomes from this stage of the process were summarised in a briefing note for participants in the workshop that followed. This covered four main areas:

  • Stakeholders
  • Factors affecting risk
  • Rating scales
  • Key elements or topics for risk identification and analysis.

Stakeholders

Direct stakeholders and other interest groups affected by the project may seek to influence the design or the approach to implementation. They determine how the outcomes are perceived and whether it will be regarded as a success. Some of the major stakeholders and their concerns are noted in Table 1. The list is not intended to be exhaustive.

Table 1: Stakeholders

Stakeholder

Interests and concerns

Government

Project seen as success by the community at large and all financial obligations are met in a timely manner; project seen as meeting government infrastructure objectives

Transport and road users

Limits on the disruption created by the construction work; overall improvement in traffic flows; tolls they can live with; freedom from unintended consequences when traffic flows adjust to the new capacity

Neighbouring communities and businesses

Minimal disruption during construction; improved travel times for them and their customers when the road is complete

Neighbouring property owners

Certainty about what will happen; fair compensation for compulsory acquisition

Construction industry

Opportunity for short and long term employment; business opportunities; injection of funds into the industry sector

Factors affecting risk

Some of the most significant internal and external factors that give rise to and influence the risks associated with the project are listed in Table 2 and Table 3.

Table 2: Internal factors

Factor

Notes

Decision making and approvals

There are several government agencies involved in the project. As tenders are received and evaluated, private sector entities will be involved as well. Each entity has its own practices for decision making and granting approval to proceed within the areas of the project for which they are responsible. Some of these decision-making and approval processes interact with one another.

Integrity of project plans and project governance

The business case may not have been as thoroughly tested as desired. The governance arrangements for the project have not been finalised yet.

Design maturity

The design is not complete and work on detailed construction plans may result in changes.

Tunnelling and general construction

Tunnel boring is a well-established construction method but some uncertainty always remains about ground conditions and machine performance. Each tunnel’s progress rests on the performance of one complex machine and the team that operates and supports it.

Associated construction works are expected to be straightforward, although interactions with neighbouring landholders and other stakeholders may present some challenges.

Scale

This is a large project that will involve many people during construction, both directly and in supporting roles.

Table 3: External factors

Factor

Notes

Road usage forecasts

The performance of the road depends strongly on economic and demographic developments in the regions that it serves, as well as patterns of public transport usage.

Public expectations

Changes to traffic patterns lead to emotive community responses. The project will have temporary and long-term effects on neighbouring communities, businesses and roads.

Toll levels must be seen as fair and providing value for money.

Stakeholder influence

There are several major stakeholders directly involved in the project, and there are many interest groups that will be affected by it. They may seek to influence the design or the approach to implementation.

Environmental and other approvals

Some approvals have not yet been finalised and more will be required as construction starts.

Economic conditions

Local economic conditions, as well as in adjacent regions, may affect the availability and price of labour, plant and other resources required for construction.

Rating scales

Rating scales are developed in the context stage of the risk management process to support risk analysis. They provide a way to describe or rate the effectiveness of existing controls, the consequences of a risk, the likelihood of those consequences affecting the project, the overall risk rating associated with a particular consequence and likelihood rating and the potential exposure of the project if those controls do not function as expected.

Consequence and likelihood ratings were made in the context of existing controls. The effectiveness of controls, shown in Table 4, was a measure of how well the risk is controlled by the set of existing controls, as a whole, relative to what is practicable for the agency (rather than an unrealistic concept of perfect control).

Consequence scales in Table 5 were used to describe the magnitude of the impact of a risk if it should affect the project. The five levels in the scales run from almost negligible to so significant that anything greater is difficult to contemplate for this project. (Note that Table 5 shows only a selection of the consequence criteria.)

Most risks could affect more than one criterion. The rating assigned by the workshop participants was based on the criterion against which the risk had the greatest impact.

A simple 5-point likelihood scale was used to describe the likelihood of the consequence being felt by the project. For any risk, the risk rating was expressed as the combined effect of the consequence and the likelihood that this consequence would be felt, using Table 6.

The consequence scales in Table 5 were also used to describe the potential exposure of a risk. This rating was used for setting control assurance priorities.

Table 4: Control effectiveness rating

Rating

Description

1

Controls are as good as realistically possible, well-conceived and implemented as well as they can be

2

Controls are generally well designed and well implemented but some improvement is possible in their design or in how effectively they are implemented

3

Controls are well conceived but some are not implemented effectively

OR

While the implementation is diligent, it is clear that better controls could be devised

4

There are significant gaps in the design or in the effective implementation of controls – much more could be done

5

Virtually no credible control relative to what could be done

Table 5: Selected consequence criteria

Rating

Cost

Schedule

Functional performance

Financial performance

1

Government steps in or imposes a new management entity with additional public sector funding

Completion slips by more than two years

The problems the road has generated are seen to far outweigh its benefits

The project would be sold at a loss for the private sector and public sector funding would be written off

2

Additional public sector funding required to complete the project

Completion slips by one to two years

Serious permanent traffic problems can be blamed on the project

Investors’ money and public sector funding would be written off

3

Scope deferred or cut to allow the core project to be completed

Completion slips by a year

Persistent non-critical traffic problems can be blamed on the project

The return to investors is very poor

Loan repayments are deferred for several years

4

Contingency funds severely depleted

Externally visible intermediate milestones slip without affecting the completion date

Isolated disturbances to traffic cause persistent but localised concerns

The return to investors is disappointing, loan repayments are delayed

5

Minor draw down of contingency funds required

Minor delays, not visible outside the project, would be absorbed by replanning

Isolated disturbances to traffic cause temporary concern

Returns to investors and loan repayments are both delayed a little

Table 6: Risk rating

Key elements

The topics that formed the agenda for the workshop are listed in Table 7.

Table 7: Key elements

Key element

Notes

Program management

Program governance and assurance; program and project complexity; management of multiple stakeholders

Procurement

Interlocking decisions and associated approvals; engagement of the prime contractor; decisions within the contractor’s organisation

Community and public

Community perception and acceptability; activities of public stakeholders and focused interest groups

Design

Design completion and necessary changes

Environment

Pervasive or particularly high profile environmental factors that might affect the design and construction of the project (while avoiding detail appropriate for an environmental risk assessment)

Construction

Human, plant and financial resources, spatial concentration

Operation and maintenance

Aspects of operations that might affect traffic flows, toll revenues, operating costs, financial viability or loan repayments

Assessment workshop

Overview

Establishing the context, the first step in the risk management process in Figure 1, was completed prior to the workshop in draft and reviewed at the start of the workshop. The workshop focused on Steps 2, 3 and 4: risk identification, analysis and evaluation.

In the workshop, the following steps were followed:

  • Risks and associated controls were identified and documented
  • The effectiveness of the controls was examined and rated
  • The consequences of the risk were rated, and the likelihood of that level of consequence arising, given the controls and their effectiveness
  • The potential exposure to the risk, the greatest consequences were all the controls to fail, was examined and rated.

Focus on risks

When people take part in risk identification, the opportunity to engage in open thinking and make fresh connections between ideas can lead to many matters that are not risks coming to mind. This was important as it served to enhance understanding and communication within a core group of project stakeholders, one of the objectives of the process for this project. To make best use of the time in the workshop, we distinguished between facts, decisions and uncertainties. We also deliberately left risk treatment for a later stage. This is illustrated in Figure 2.

Figure 2: Facts, decisions and uncertainties

Risk assessment outputs

The workshop identified and assessed 14 risks, many of which encompassed several related areas of concern. Six risks were assessed as Medium and eight were assessed as Low. Importantly, the risk assessment identified no High or Extreme risks, once existing controls were taken into consideration. This finding was confirmed in a review at the end of the workshop.

Table 8 illustrates the result of the risk assessment, by consequence and likelihood ratings.

Table 8: Risks by consequence and likelihood

Participants expressed the view that the relatively conventional nature of the project and the effectiveness of existing controls was the reason none of these risks were greater than Medium priority. This might have been asserted prior to the workshop but the analysis supported that opinion with a process that others could understand. If anyone sought to challenge the position, they could do so within a structured framework rather than simply arguing about personal impressions.

Of the 14 risks assessed, all were found to have largely or partially effective controls (Table 9). This reflects the earlier effort devoted to creating a set of controls that were well suited to the risks of the project. To the extent that treatment actions were to be developed, attention should be given to the risks nearest the bottom right hand corner of the risks in Table 9, those with the highest ratings and the least effective controls. In this case there were no Extreme or High risks, so control improvement efforts were to be directed to the Medium risks, as resources permitted.

Table 9: Control improvement priorities

A guide to control assurance priorities, making sure controls are maintained and will operate as expected, is provided by the relationship between risk rating and potential exposure shown in Table 10. Risks towards the upper left corner of this matrix are currently seen as low priority, but if controls were to fail the consequences, represented by the potential exposure, would affect the project. These are priority risks for control assurance to ensure that existing controls remain in place and effective.

Table 10: Control assurance priorities

Causes of risks, controls and impacts

Several more detailed analyses were also undertaken. The diagram in Figure 3 illustrates some of the relationships between the main causes of risk and their impacts described in the risk register, and a table like Table 11 shows the relationship between individual controls and individual risks. (Remember, the ratings for control effectiveness, consequences and likelihoods relate to the whole set of controls that are associated with a particular risk, but assurance activities usually focus on individual controls. There is not usually a one-to-one relationship between controls and risks.)

Analyses like these were used to assist in understanding the implications of the risks and where to focus effort to support existing controls. The root causes of the impacts of risks lie to the left and bottom of Figure 3 while the other factors are to some extent driven by these primary factors. Table 11 allowed those controls that affect several risks to be identified.

Figure 3: Causes of risk and links to impacts

Table 11: Individual controls and risks

Lessons

Consequence scales

The consequence scales in Table 5 did not focus entirely on direct impacts for the project. They also included descriptions based on broader thinking about what might happen if an event arose, including government responses. The consequences were still linked directly to the key objectives for the project, just expressed in a different way that seemed more suitable for this project and the workshop participants. As in all risk management activities, tailoring the process to suit the context and purpose is recommended.

Extended risk analysis

The risk analysis described here did not focus solely on consequences and likelihoods and the associated levels of risk. It used an extended process that included analysis of control effectiveness and potential exposure, additional ratings that can be derived in a workshop with only marginal extra effort but which generate significant additional insights.

The analysis also developed the picture of links between risks in Figure 3 and between risks and individual controls in Table 11. These expanded on the risk register, which was merely a list of risks, their causes and consequences. Figure 3 helped to develop a deeper understanding about how a small number of high-level risks were related to one another, risks that were each quite broad in terms of the concepts they encompassed and with a range of possible initiating causes and potential outcomes. Table 11 provided a further guide to the more important controls and assurance priorities.

Risk registers are often criticised because they are too simplistic. While simplicity is often valuable, particularly when a detailed analysis of closely defined risks is required, a risk register alone cannot paint a sufficiently comprehensive picture needed for a high-level view of a large project. The participants reported that Figure 3 and Table 11 were valuable in augmenting their understanding of the risks and the project, as well as providing specific guidance.

Workshop outcomes

The workshop only identified 14 risks, and no high or extreme ones, and no urgent actions. Nevertheless it contributed significantly to the project. It provided key stakeholders with far greater confidence in the project and the work that had been undertaken to that time, as well as giving them greater insights and understanding.

The focus following a risk assessment is usually on treatments required to improve existing controls. Due to the nature of this project and the fact that, according to the assessments provided in the workshop, risks were fairly well controlled at present, this assessment drew more attention to the need for control assurance rather than treatment planning.

There are risks that, if existing controls were to fail, could have significant undesirable consequences for the project and its stakeholders. It was recommended that the controls upon which the current risk ratings relied be reviewed to ensure they were as strong as was assumed in the assessment. Further recommendations were that mechanisms be put in place to review the controls regularly (and in some cases frequently) to ensure that they remained effective, especially where they relied on the behaviour of independent stakeholders outside the agency’s direct control.

Other recommendations related to periodic reviews by the agency, together with more formal reviews at project milestones, to consider:

  • For each risk, the progress and effectiveness of control assurance activities
  • Obtaining and documenting assurances that the existing controls remained adequate and effective, including audit and other assurance processes
  • Understanding the lessons learned from events and occurrences, whether successes or failures, that might indicate the adequacy and effectiveness of existing controls
  • Any changes to the context such as stakeholders’ objectives and external or internal factors
  • Any new or emerging risks that required further discussion, analysis and treatment.