Skip to main content.

Control effectiveness

Introduction

Control effectiveness (CE) represents the total effectiveness of all the controls that act upon a particular risk. This includes those controls that affect the likelihood of the risk (sometimes called ‘preventive controls’ when the controls act to reduce the likelihood of negative consequences) and those that affect the consequences (sometimes called ‘mitigating controls’ when the controls act to reduce the negative consequences). It is also sometimes (but more rarely) applied to separate controls within a suite of controls.

CE reflects not just the ability of controls to theoretically treat a risk, but also their actual effectiveness in terms of consistent, complete, reliable and timely operation. In this way the measure represents the fit of the actual controls to the ‘design intent’ for risk treatment, particularly in terms of changing the causal aspect of the risk (this is called ‘adequacy’) and the actual effectiveness in practice (called ‘effectiveness’).

CE can be expressed in quantitative or qualitative terms and can be either an absolute or a relative measure. It is normally reported together with a measure of residual risk.

Benefits and costs

Cost-benefit analysis is generally used to determine the need for further risk treatment. The general test is that the net benefit obtained by changing the magnitude of the risk from one level to another should be greater than the net cost of bringing about the change by the imposition of further controls or improvements to existing controls. This is also the general principle behind the ALARP (as low as is reasonably practicable) criterion used in workplace health and safety legislation.

Generally the ‘breakeven’ level of risk where the costs and benefits are equal is described as that which is (just) ‘tolerable’. Under an ALARP approach, a further weighting is applied to the benefits such that further risk treatment is not required only if the costs greatly outweigh the benefits. The UK case of Edwards vs. National Coal Board gives the dictum:

'Reasonably practicable' is a narrower term than ‘physically possible’ and implies that a computation must be made in which the quantum of risk is placed in one scale and the sacrifice involved in the measures necessary for averting the risk (whether in time, trouble or money) is placed in the other and that, if it be shown that there is a great disproportion between them – the risk being insignificant in relation to the sacrifice – the person upon whom the obligation is imposed discharges the onus which is upon him.

With quantitative risk analysis, the change in risk can be represented by a change in the absolute value of the CE.

Qualitative risk assessment

In a qualitative risk assessment the level of risk is often described by a label or a description. The analysis normally involves the consideration of factors that represent a range of consequences and their corresponding likelihoods. A consequence rating is normally chosen from a table of ranges of values for different types of consequences, then a corresponding rating for likelihood is chosen from another table.

Sometimes the likelihood rating is obtained by combining a rating that reflects the base frequency of an initiating event together with a rating that represents the conditional probability that the selected consequences will arise if the event occurs. More usually, however, frequency and probability are combined within the look-up table and a single factor is chosen.

In qualitative assessment, the risk rating normally represents the residual level of risk (taking into account the existing controls and their level of effectiveness) and is a relative not an absolute measure. The risk rating is used for ranking a set of risks for a particular business, activity, project or operation to determine the priority with which they are considered for treatment.

Under qualitative risk assessment, CE can relate to the effectiveness of individual controls, but ordinarily it is a measure of the effectiveness of all the controls in place that act on and influence a particular risk.

As qualitative risk assessment involves relative and not absolute measures of risk, CE in qualitative risk analysis is always a relative measure of the control actually being achieved compared with that which is achievable by the organisation. In this way, qualitative CE is, in effect, an expression of cost-benefit analysis: it compares the current level of control for a risk against that which could be achieved, at the point where the costs of further control just outweigh the benefits of further control. It therefore follows that if the current level of control is almost as good as what can be cost-beneficially achieved, then CE is high and the risk is almost at a level that can be tolerated. If much more control is possible compared with what is reasonably practicable or achievable, then CE is low and the risk cannot be tolerated.

In this way CE and a qualitative residual risk rating are linked. A risk may have a high residual risk rating, but that may be tolerable if the CE is high. If the CE is low, a high residual risk rating cannot be tolerated and should be reduced. Once the risk has been treated and the level of control improved the final risk rating will represent that which is tolerable.

A recent publication from the Institute of Internal Auditors and Standards Australia defined CE as:

a relative assessment of actual level of control that is currently present and effective, compared with that which is reasonably achievable for a particular risk.

CE in qualitative risk assessment can be expressed as a label or a numerical factor that is chosen from a table giving ranges of levels of effectiveness. Sometimes the factor used is expressed as a percentage. Examples are shown in Table 1 and Table 2.

Table 1: Example of a guide to CE as a percentage of that reasonably achievable

Description

CE

‘Just getting started’, ’A lot of work still to be done’

20 – 30%

‘About half way there’

50 – 60%

‘Most things in place and working, but some more still to be done’

75 – 80%

‘Nothing more to be done except review and monitor the existing controls’

Over 90%

Table 2: Example of a qualitative CE scale

Control effectiveness

Guide

Fully effective

Nothing more to be done except review and monitor the existing controls. Controls are well designed for the risk, and address the root causes. Management believes they are effective and reliable at all times.

Substantially effective

Most controls are designed correctly and are in place and effective. Some more work to be done to improve operating effectiveness or management has doubts about operational effectiveness and reliability.

Partially effective

While the design of controls may be largely correct in that they treat most of the root causes of the risk, they are not currently very effective.

or

Some of the controls do not seem correctly designed in that they do not treat root causes. Those that are correctly designed are operating effectively.

Largely ineffective

Significant control gaps. Either controls do not treat root causes or they do not operate at all effectively.

None or totally ineffective

Virtually no credible control. Management has no confidence that any degree of control is being achieved due to poor control design or very limited operational effectiveness.

Assessment of CE

The IIA Professional Practices Framework says that:

Among the responsibilities of the organisation’s managers is the assessment of the control processes in their respective areas. Internal and external auditors provide varying degrees of assurance about the state of effectiveness of the risk management and control processes in select activities and functions of the organisation.

As auditors have to verify management’s assessment of risk and CE, they should report their assessment using similar scales and measures.

Under the Turnbull Rules of the London Stock Exchange, Clause 24 states that:

Reviewing the effectiveness of internal control is an essential part of the board's responsibilities. The board will need to form its own view on effectiveness based on the information and assurances provided to it, exercising the standard of care generally applicable to directors in the exercise of their duties. Management is accountable to the board for monitoring the system of internal control and for providing assurance to the board that it has done so.

Clause 28 also states:

The reports from management to the board should, in relation to the areas covered by them, provide a balanced assessment of the significant risks and the effectiveness of the system of internal control in managing those risks. Any significant control failings or weaknesses identified should be discussed in the reports, including the impact that they have had, or may have, on the company and the actions being taken to rectify them. It is essential that there be openness of communication by management with the board on matters relating to risk and control.

Similarly the Australian Securities Exchange Guidelines on Corporate Governance, states at Principle 7:

A listed entity should disclose ... the processes it employs for evaluating and continually improving the effectiveness of its risk management and internal control processes.

It is clear that CE represents an important measure that can be used to represent management’s assessment of the current level of control compared with that which is reasonably achievable. If management assess the CE is as low, it is incumbent upon them to take steps to further treat the risk by either improving the effectiveness and adequacy of existing controls or by providing further controls. If management report a low CE as part of a governance report, then they should also explain what further risk treatment they plan so that the level of risk will become tolerable.

References and notes

  1. ‘Adequacy of risk management, control, and governance processes is present if management has planned and designed them in a manner that provides reasonable assurance that the organisation’s objectives and goals will be achieved efficiently and economically.’ IIA Professional Practices Framework, Glossary.

  2. ‘Effectiveness of risk management, control, and governance processes is present if processes are operating in a manner that provides reasonable assurance that the organisation’s objectives and goals will be achieved.’ IIA Professional Practices Framework, Glossary.

  3. Edwards v National Coal Board [1949] All ER 743 (CA)

  4. HB 158: 2010, Delivering Assurance based on ISO 31000:2009, Risk Management, Standards Australia.

  5. The Financial Reporting Council, Internal Control, Revised Guidance For Directors On The Combined Code, October 2005, London, ISBN 1-84140-724-0.

  6. Corporate Governance Principles and Recommendations, 3rd Edition, ASX Corporate Governance Council, March 2004.