This case describes the estimation of a risk-based liability cap for a high-value, complex procurement contract. An estimate of potential liabilities, based on scenarios that might arise and their implications, provided a sound basis for negotiating liability terms in the contract.
Although this case concerns a Government agency buying equipment from the private sector, we have also undertaken similar analyses with private-sector owners about to embark on significant procurements.
We conducted a detailed quantitative analysis of liability scenarios, their likelihoods of occurrence and their cost implications for Government, to develop a quantitative justification for specific contractor liability limits. Both Government and the contractor agreed the calculations were a good basis for negotiations. Both parties accepted the outcomes.
Risk-based calculations allowed the parties to the contract to focus on what might happen in some detail, thus leading to better informed and hence more acceptable contract negotiation outcomes.
Many contract templates and general contract formats for large and complex procurement activities, particularly in the public sector, require contractors to assume liability for loss or damage to material and facilities due to poor design, poor manufacture or negligent behaviour. Contractors take particular notice of the liability clauses of contracts as they specify the limit of liability that a contractor may encounter in the conduct of the work. Contractors endeavour to keep the limit of liability to a minimum for obvious commercial reasons. This reduces their costs and exposure, and makes the contract more acceptable from a corporate perspective.
Some small and medium suppliers might be unable to bid for certain jobs simply because they would lay themselves open to liabilities that they could not bear if they were to materialise or because they simply cannot obtain insurance coverage for large values. This can limit competition unnecessarily if the liability limit stipulated in the proposed contract exceeds the amount really needed.
This case examines the liability requirements for a contract for the procurement of high-value, high-technology equipment for a Government agency. The draft contract included provisions that outlined the contractor liability requirements and set limits of that contractor liability in broad financial terms. Precise values of the limits were not included in the draft contract, as they were to be determined and agreed between the parties before contract signature. This aspect of the contract was intentionally left open for negotiations between the Government and the contractor.
The analysis described in this case study was undertaken for the Government during the industry solicitation process. It allowed the Government to arrive at a risk-based estimate of the liability limit and provided a justified basis from which to conduct negotiations with the preferred tenderer.
Cost and reasonableness of uncapped liability
There are many examples where Government seeks unlimited liability in contracts. Indeed unlimited liability has been a default position for many projects entering into negotiations. However, unlimited liability causes contractors considerable difficulty in gaining corporate approval to enter into an agreement under such terms, and they may result in large increases in the costs to fund the required financial instruments providing that coverage, costs that are passed on to Government in the form of higher bids.
In some cases, contract negotiations have been frustrated due to an inability of the parties to agree to liability limits, or to move away from the Government’s initial requirements for unlimited liability.
Parties to negotiations can move away from an opening position of unlimited liability through careful consideration and assessment of the likely events and consequences that may lead to damages where the contractor may be held liable. Such consideration must usually be based on a detailed knowledge of the contract scope of work and the nature of any services and deliverables.
The approach taken for this activity involved brainstorming with project staff and specialists, to develop a range of possible scenarios that might lead to damages for which the contractor would be liable. The scenarios were estimated in terms of the best, worst and most likely outcomes (three-point estimates). The probabilities of the scenarios occurring were estimated, based on the project team’s knowledge of occurrence rates, and an overall contractor liability limit was calculated. Figure 1 illustrates the approach.
The first stage of the task was to develop a range of scenarios that might result in the contractor being liable for damages. A group of experienced project staff were brought together to brainstorm the scenarios. They were senior members of the bid preparation team, which included members of the user organisation.
To assist the process, the scenarios were split into three groups:
- Events that might occur whilst the equipment and systems were under the responsibility of the contractor
- Those that could arise during initial test and trials
- Those that might arise when the equipment and systems were in operation under Government control.
No limitation on timeframes was imposed and participants were asked to consider events that might occur during the full life of the equipment and systems being procured.
A full list of the scenarios was documented in some detail, including those scenarios identified but not quantified due to their low relevance in the circumstances surrounding this project and contract.
Scenarios that were considered to have been the fault of, or caused by, the Government were set aside along with those that would not result in damage to the Government.
Scenarios that resulted in injury or death were specifically excluded from this activity as they were covered under other provisions within the contract, and the law prevents any limitation on damages in these cases.
Once a range of scenarios had been identified, participants were asked to estimate the probability of each event occurring. Estimates of probability were generated from past knowledge of similar systems and operations, design parameters and objectives, known failure rates and, in the absence of other data, best guesses.
Probabilities were estimated as single values.
Best, worst and most likely costs
In each scenario, a range of cost outcomes (damage suffered by Government) was estimated. Participants considered:
- The best case, B, where only minimal damage occurred
- The worst case, W, the maximum plausible outcome in terms of damage
- The most likely case, L, a reasonable and expected scale of damage given due consideration to controls within the contractor's and the Government’s organisations.
Estimates of cost were based on engineering and contractor rates, major sub-systems costs and all-up equipment replacement costs in the case of total loss.
The three-point distributions were interpreted in two ways (Figure 2) each evaluated separately.
- A triangular distribution was assumed, to provide a conservative view of the outcomes. This distribution is conservative because it has 'fat' tails, with a reasonably high probability of an outcome greater than the most likely outcome L in the common case where the distribution is positively skewed.
- A Pert distribution was assumed, to provide a less conservative view, in which the distribution is concentrated around the most likely estimate L. This gives less weight to the extremes, and in particular it gives less weight to the potentially large losses that might arise near the worst case W.
The purpose of using two distributions was to provide a degree of sensitivity testing for the model assumptions.
Combining scenario information
The probabilities and damage estimates for the scenarios can be interpreted as probability trees. Figure 3 illustrates the principles with a simple case involving two scenarios (or risks), with probabilities P1 and P2 of arising and associated costs C1 and C2. Simple probability trees, shown at the top of the diagram, can represent the individual scenarios. The probability tree in the lower part of the diagram represents their combination. This allows probabilities and costs to be calculated for the four possible outcomes with these two scenarios.
A spreadsheet model was built in which the costs C1 and C2 were distributions, and two sets of calculations were performed in parallel, one using the triangular and the other the Pert distribution assumptions. The model extended the calculation in Figure 3 across the full set of scenarios developed in the workshop. The model used the Excel spreadsheet add-in @Risk, a simulation package that combines the probability and distribution information and generates results in tabular and graphical forms based on a Monte Carlo simulation.
The arrangement in Figure 3 is appropriate for scenarios that are independent. Different structures were used to represent scenarios that were mutually exclusive or that demonstrated other more complex relationships.
The scenarios were developed in an initial workshop, then reviewed and refined. The liability simulation model was refined and run a number of times as the scenarios evolved. Because the extreme, high-loss end of the output distribution was important for setting the liability limit, the model was run over a large number of iterations to establish consistent results in the tails of the output distribution; the results in this case are all from simulations of 50,000 iterations.
Figure 4 shows the simulation outputs for the combined effect of all scenarios, for the Pert and triangular distribution assumptions. The graph shows the probability that damages will be less than any specified total cost. It illustrates how, at the higher percentiles, the liability level increases dramatically while the marginal probability of such a liability exposure reduces. Figure 5 shows the same information in inverse form as the probability of damages greater than any specified total cost arising. As expected, the triangular distribution is far more conservative than the Pert assumption. For this contract, the figures also showed that, for a reasonably high level of confidence, a realistic and hopefully affordable level of liability limit might be established.
A more conservative option
While the estimates of probability in the scenarios reflected the best knowledge of project staff, it was clear that the estimates could be inaccurate by an order of magnitude. It was agreed with the Project Manager that the model should also be run for a more conservative position, where the probability of occurrence of any scenario was ten times more likely, resulting in a higher limit of liability.
Figure 6 shows the results of this sensitivity analysis over the same number of iterations.
Figure 4 and Figure 5 indicate that a reasonable limit of contractor liability for this contract, given the scenarios developed and probabilities estimated, would be somewhere in the vicinity of $18 million, for a confidence level of 99.99%.
Figure 6 indicates that, should the estimates of probability be generally increased ten times, a liability limit of approximately $20 million would be more appropriate for the same level of confidence.
A liability limit of $18 million was defensible and justifiable, based on the scenarios and the model used, the project team’s experience, and the probabilities and costs estimated. A limit of $20 million was defensible and justifiable, should project estimates of the likelihood of occurrence prove to be too optimistic.
As the actual liability limit was yet to be negotiated and agreed, it was recommended that the project use the more conservative limit of $20 million as an opening position for negotiations. Should there be pressure during negotiations to reduce the limit, the figure of $18 million could be used as a survival position, not to be reduced further.
To download a copy of this case study, please click here.