Skip to main content.

Risk-based monitoring of human services delivery

Background

In one of the states of Australia, the Government agency with overall responsibility for the oversight of child care services wished to develop a risk management approach to monitoring the compliance of child care service providers with the relevant regulations. This had to be done in the context of two competing sets of expectations: there are judicial and community demands for additional regulatory measures to protect children in child care, and there is a clear expectation in the industry and in the community that Governments should do nothing that might increase the costs of care in such a way that the supply of child care services become limited or unaffordable.

This case describes an application of risk management in the development of a protocol and procedures for monitoring the providers of child care services. As a by-product, it also developed a self-assessment tool for use by the management committees of child care centres to improve their performance.

This case concerns child care services, but a similar approach could be applied to regulating the delivery of any community service: aged care, health care, disability support and so on.

Introduction

Child care

Child care plays a central role in most developed economies. Child care services include long day care, pre-school care, occasional care, home-based care and mobile child care, and are provided by a large number of organisations, including commercial operators. Together, these services look after large numbers of children under the age of 6, while parents (over 70% of them in some areas) go to work. The child care industry employs significant numbers of people, and it receives significant funding each year from a wide range of sources. Like all industries, child care is affected by the social, economic and political environment within which it operates, and it is subject to regulation, competition and the operation of market forces.

Agency roles

The agency that undertook the work described in this case is responsible for regulating the provision of child care services in its state. Regulators all use similar tools to monitor regulatory compliance: regular inspections, random audits, complaints investigations and unannounced visits. These are all used in monitoring regulatory compliance in child care. However, much of the agency’s monitoring work has, historically, relied on a close association between service providers and agency staff. The view was that if staff got to know service providers really well, they would be in a position to notice and deal with problems early and be able to advise providers on how best to meet regulatory standards.

The risks in this approach are that agency staff form too close an association with the providers and fail to administer the regulations, or that the public or Government Ministers develop this perception. Clearly, if your primary task is to see that the law is complied with, having a close relationship with the people required to comply could compromise your ability to take an objective view. But if staff don’t have intimate knowledge of how services are operating, a number of questions arise:

  • How does the agency know which services need to be inspected and visited?
  • What indicators provide a warning to an agency manager that a service needs attention?
  • Are there indicators that can trigger specific agency responses that are most likely to reduce the risks for the agency?

What regulations can and can’t do

Regulations deal largely with inputs: the height of the fences, the size of the playground, the number of staff on duty and their qualifications. Regulations do not, by themselves, capture all the risks present in an industry, nor do they provide a way of measuring whether or not they are being managed. Regulations cannot, for instance, set standards around the extent to which service providers contribute to the well-being of children or children’s development, although parents expect that child care providers will contribute to both of these aspects of their children’s lives, and it is incumbent on the agency to at least see that the services that are licensed do no active harm to children, either physically or psychologically.

Not only are regulations not suited to establishing standards and rules around these aspects, they can sometimes obscure risks. For example, the main reason for requiring fences and self-locking gates in child care centres is to reduce the risk of a child walking out of a centre without anyone noticing. Recently a three-year-old boy did just that – he simply left the centre and walked home. The child care staff didn’t notice that he had been gone for three hours. They had simply not considered such a possibility; after all, their fences and gates complied with the regulation. They had not instituted any specific processes for managing the risk of a child going missing, because they had never focused on the practical, operational risks that are inherent in a situation where 30 or more children under 6 are gathered together.

From the agency’s point of view, this centre complied with all the regulations: they had adequate fences and gates, they maintained the attendance records and signing out procedures required by the regulations and they had the right number and kind of staff on duty. What the service had failed to do was to properly supervise the children.

Competing requirements

The business of the regulator is to safeguard the interests of consumers, without regard to the profitability of the industry. It is subject to several competing pressures.

  • As the regulator, it is subject to increasing scrutiny and required to justify its decisions and its existence, in an environment of regulatory reform
  • As a Government agency, it is subject to increasing pressures to maintain or even reduce its budgets, despite increasing numbers of service providers and growing demands on its regulatory activities and its personnel
  • There are judicial and community demands for additional regulatory measures to protect children in child care, but there is a clear expectation in the industry and in the community that Governments should do nothing that might increase costs such that the supply of child care becomes limited or unaffordable, particularly for families with lower incomes
  • Early childhood professionals cite research findings that suggest very high standards should be prescribed for the quantity and mix of equipment used in child care centres and for the qualifications of staff, but service providers, quite reasonably, object that no recognised national or international standards exist in either of these areas
  • Service providers have a legitimate expectation that the regulator will treat them even-handedly, but this can only be done by making regulatory standards concrete and measurable and so limit the discretion that can be exercised by regulatory staff
  • However, making the regulations tighter, so that they can be measured objectively, often means that they unduly restrict creativity and flexibility, both of which are being demanded by families, and additional restrictions can increase delivery costs.

Objectives

The agency with overall responsibility for the oversight of child care services wished to develop a risk-based approach to monitoring the compliance of child care service providers with the relevant regulations. They planned a set of risk assessment workshops to identify relevant risks.

One of the aims was to use the risks identified in the workshop process, with a focus on the high-priority risks, as indicators in the monitoring of compliance, and to further use them as the basis for establishing a comprehensive risk assessment system. The objective of the risk assessment described here was to identify and examine the potential risks in the provision of child care services, and to develop an agreed priority listing of them, as a fundamental input to the process, prior to a full-scale implementation.

This assessment would necessarily take account of perceived risks to the main stakeholders, including the various providers of child care services, the children and their parents, the providers of funding and the regulators.

Approach

Overview

The general approach to the risk management exercise was based on the initial steps in the process described in the international standard ISO 31000 Risk management – Principles and guidance.

Two workshops were held. The purpose of the first workshop was to brief the participants and to undertake the first stage in the risk management process, establishing the context. The second workshop addressed risk identification, analysis and assessment. Twenty participants attended from the main stakeholder groups, including the agency, service providers and parents.

Establishing the context

The purpose of the initial context establishment process was to develop a detailed structure and set of rating scales for the risk identification, analysis and assessment tasks to follow.

A list of stakeholders was identified. Initially, they were grouped in four categories:

  • Service providers
  • Children and parents
  • Funding providers
  • Regulators.

From the list of stakeholders, a large set of areas of interest and concern was derived. These were then consolidated into a shorter list of criteria for analysing the impacts of risks:

  • Physical and psychological safety of children
  • Child development
  • Provision of quality services
  • Compliance with regulations and legislation
  • Funding and associated impacts
  • Impacts on the staff and management of service providers.

Key elements

Fifteen key elements were identified to structure the risk assessment task. They were based on four main ‘issue areas’: safety of children, health, management of the service and child development. Within each area there was a group of topics referred to as contributors that were used as the key elements (Table 1).

Table 1: Key elements

Issue area

Contributor (key element)

Safety

Physical safety, including emergency procedures

Physical environment

Authorised access to children

Abuse of children

Health

Health, including emergency treatment and procedures

Hygiene

Nutrition

Management

Numbers, including staff/child ratios and group size

Records and confidentiality

Staff skills and conditions

Service policies

Child development

Development programme

Staff/child interaction

Equipment

Special needs, including disability, cultural, religious, language and dietary needs

Risk identification, analysis and evaluation

The purpose of the risk identification, analysis and evaluation process was to identify and set priorities for the major risks arising in the provision of child care services. The desired outcome was a list of key risks, with agreed priorities. This would allow the agency and others concerned with child care to identify the practical actions required to minimise the identified risks and maximise the achievement of the critical success measures.

The identification workshop involved a facilitated brainstorming exercise with a group of stakeholders. It was addressed in two passes.

The first pass focussed on risk identification and analysis. In the morning and early afternoon, the participants addressed each key element:

  • A brief description of the element was provided, to ensure everyone was using a common set of definitions and assumptions
  • They identified the relevant risks for each key element
  • For each risk, an assessment of its potential impacts was made and the likelihood of those impacts arising was assessed
  • From the likelihood and impact assessments, initial risk priorities were derived.

In a second pass, later in the afternoon, the participants reviewed each identified risk and its priority then agreed its final priority. Impacts were rated twice (Table 2). The first impact rating was based on the physical and psychological safety of children. The second impact rating was linked to the other critical success measures.

Table 2: Impact ratings

Rating

Child safety

Other impacts

A

Potential fatality, major long-term trauma

Very high, most criteria may not be achieved

B

Major injury, ambulance attendance, or major long-term psychological harm

High, most criteria threatened

C

Moderate injury, doctor required, or moderate long-term psychological harm

Moderate, some criteria affected

D

Minor injury, first aid sufficient, or minor psychological harm

Minor, easily remedied

E

Negligible

Negligible

Risks leading to psychological harm or to children not achieving their human potential posed particular difficulties. This impact area was included during the workshop in the ‘safety’ impact as ‘psychological harm’, but this was not wholly satisfactory and led to some initial confusion and inconsistency. As a result, the scale used for rating safety impacts was extended during the workshop to make more explicit the importance of potential psychological harm to children. This was a minor weakness in the process – in future workshops of this kind this area should be dealt with more explicitly.

The likelihood and impact ratings were used to generate initial priorities. Because there were two distinct impact ratings for each risk, there were two priority ratings: one for safety and one for the other criteria.

During the final part of the workshop, the participants reviewed the initial priority ratings to generate a single agreed priority rating for each risk. The need for action by service providers or intervention by the agency determined the agreed risk priority (Table 3). The priorities are indicators to assist in making decisions about what action might be warranted if it is determined that service providers are not managing risks adequately.

Table 3: Priority rating

Rating

Priority for action by the provider or the agency

Extreme

Extreme risks that are likely to arise and have potentially serious consequences requiring attention as a matter of urgency, or require an investigation with the intent of taking legal action

High

High risks that are likely to arise and have potentially serious consequences requiring urgent attention, or investigation to decide whether to remedy or prosecute

Medium

Medium risks that are likely to arise or have serious consequences requiring attention, or the agency would tell the provider to fix the risk in a stated timeframe

Low

Minor risks and low consequence that may be managed by routine procedures, or the provider would negotiate an agreed timeframe to fix the risk

Workshop outcomes

Over 300 risks were identified, and initial priority ratings were determined in relation to safety and other impacts. The participants reviewed each risk, and an agreed priority was allocated. The spread of risks and agreed priorities across the key elements is shown in Figure 1.

Figure 1: Risks by element and agreed priority

The varying nature of the stakeholders represented in the workshop and the differing importance of particular issues made some aspects of the priority assessment difficult. Some issues were central to some stakeholders, but irrelevant to others. Participants generally contributed their views strongly in relation to areas of concern to them, and did not for issues that were less relevant to them. The implication of this is that individual risks have agreed priorities that reflect the view of the participants most concerned about that subject, i.e. the most conservative or ‘risky’ agreed rating among the participants. This is reflected in the high percentage of Extreme and High ratings for risks in each element.

A further implication is that not all risks have the same priority for all kinds of service providers. For example, risks associated with night-time care and pets were important for family day care and home-based care providers, whilst commercial providers were concerned about having skilled staff who could plan and implement appropriate programmes.

Across all risks, 54% were rated as Extreme or High, indicating that the participants regarded the majority of the risks identified as very important and worthy of serious management attention.

It must be noted that the large proportion of high risks does not imply that risks are being ignored, over-looked or not treated seriously now. Rather it reflects their inherent importance in the provision of child care services, irrespective of current policies, practices and controls. The challenge for both the agency and the service providers is to ensure that current policies and practices do in fact address all the high-risk areas.

Overall, risks associated with the key elements physical safety, abuse and numbers (staff to child ratios and group sizes) all received a high proportion of Extreme or High ratings, reflecting the importance placed on these areas by the participants. Key elements related to access, nutrition, staff and equipment for development each contained some Extreme or High risks, although the risks associated with these elements received low ratings overall.

Initial and agreed priorities

There was an apparent disparity between the initial risk ratings and the agreed priorities, in the sense that the agreed priorities seemed far higher than would be warranted by the initial ratings.

The initial assessments took account of the likelihoods and impacts of risks, and the outcomes that were observed in practice. In a sense, these assessments reflect actual risks as they arise in child care services, given the current policies, practices and controls in place now. However, the agreed priorities reflected a different view, taking more account of the potential exposure in the absence of controls, often with a greater importance placed on the potential impacts rather than balancing between likelihood and impact: risks with high potential impacts were rated highly, even if the likelihoods of those impacts arising were low.

The differences between the two sets of ratings may be related to the tables used to set the initial priorities. For example, if the priority ratings were modified to increase the weighting placed on high consequences the apparent discrepancies would be largely removed, at least overall, although many individual differences in ratings would remain.

A better solution would be to use an extended analysis structure that rated the maximum potential impact for each risk, if all the controls were to fail. A formulation like this may have avoided any confusion the participants felt about the analysis process and its outcomes.

Sources of risk

After the workshop, Extreme and High risks were grouped in loose categories relating to their initiating mechanisms (Table 4). We usually find that grouping by source of risk is far preferable to grouping by impact, as it leads directly to treatment actions that address underlying causes and help prevent risks.

This was not intended to be a definitive set of categories, nor a reflection of agency policy, but rather an indication of risk sources that might be linked to potential classes of risk treatment actions.

Table 4: Categories of risk sources

Category

Classes of treatment actions

Risks arising in the physical environment

Regulation, inspection, capital and maintenance funding, procedures

Risks arising specifically on excursions

Procedures, guidance, staff training, adult supervisor training

Risks relating to the suitability and selection of staff

Staff selection procedures, counselling

Risks arising because staff do not or cannot perform as required, for many reasons: insufficient numbers, insufficient training, inadequate knowledge or skills, or inadequate implementation of procedures

Provision of additional resources, staff and management training, procedures, supervision, financial measures, regulation, inspection

Breach of licence conditions

Regulation, inspection, staff training

Risks arising through inadequate management, policies and procedures

Regulation, guidance, inspection, procedures, financial measures, resources, management training, staff training

Risks arising from people normally outside the service environment

Physical security, procedures, education of the community, supervision

Risks relating to the accuracy and maintenance of records

Regulation, inspection, procedures, staff training

Risks arising from inadequate development programmes

Regulation, inspection, guidance, staff training, resources

Health and hygiene risks not already covered

Regulation, inspection, guidance, staff training, resources, education of families

Risks associated with children and families having special needs

Regulation, inspection, guidance, staff training, resources, education of families

Subsequent steps

Subsequent steps for the agency

As an important part of its role as a regulator and provider of funding, the agency compared the identified risks with existing policies, practices and controls, focussing on the Extreme and High risks, to identify the key gaps. The risks that are considered Extreme or High vary according to the kind of service provider – these distinctions were not made in the workshop nor this case study, but they are important for the agency.

Initially the focus was on gaps between the identified high-priority risks and the policies and regulations, i.e. on what should be in place. Subsequently the emphasis moved to field examination of the implementation of those policies and regulations in relation to high-priority risks, i.e. on what is actually in place and working in practice.

The target of each of these comparisons, between identified high-priority risks and existing policies and controls, was the development and implementation of action plans to address the identified gaps where high risks were not being treated adequately.

One set of action plans involved the development of indicators for the monitoring of compliance, based on the high-priority risks identified in the workshop. These were used as the basis for establishing a comprehensive risk assessment system.

Subsequent steps for service providers

Although the risk management study was intended primarily for the agency, it also provided the basis for a management tool for service providers.

Service providers should be taking immediate action to minimise risks to the children in their care, as part of their general duty of care. Like the agency, they should compare the identified risks in their service areas with their existing policies, practices and controls, again focussing on the Extreme and High risks, to identify the key gaps.

In practice, the diversity of providers in the industry means that the development of a formal tool for providers will probably fall to the agency.

Lessons

Detailed discussion

There are several important lessons that emerge from this case study.

The priorities that are developed during risk analyses are only a guide for managers. They do not mandate anything, and decision-makers can over-ride a priority if they have a justifiable reason for doing so. The differences between initial and agreed priorities in this case reflected, in part, the analysis process that was used, but they also reflected a genuine desire by the participants to take a broader view than can be encapsulated in simple scales of impacts and likelihoods and mechanistic methods for combining them. This is the way that a process based on the standard ISO 31000 is intended to function.

Because the child care industry is so diverse and fragmented, with a mix of large and small centres spread across many urban and regional areas, most of the participants were from ‘umbrella’ organisations, such as industry groups representing different kinds of service providers and organisations representing parents. This had many benefits:

  • The industry representatives each had a broad perspective across their part of the sector. They were able to present sector-wide views rather than views rooted in the specific detail of any particular child care provider.
  • The industry representatives reported back to their members on the workshop, its intentions and its outcomes. They provided positive feedback about the process and its inclusiveness, which paved the way for the agency’s subsequent fieldwork and made that phase of the agency’s policy development far easier.
  • The agency had set out to conduct an inclusive and transparent process. The informal feedback from all participants was that this approach was very welcome, and that they were looking forward to participating in the next phases and to seeing and acting on the outcomes.
  • Agency staff felt that the workshop increased their understanding and would improve their interactions with their client service providers.

Depending on the situation, sometimes stakeholders are part of the solution and sometimes they are regarded as part of ‘the problem’. In this case, having a wide cross-section of stakeholder participants in the workshop was very valuable. Their different perspectives generated rich discussions and new insights for everyone, they adopted a very positive and cooperative approach and they were able to reach agreement on most matters.

This did not come about by chance.

  • The positive attitude had been fostered by the initial invitation to the workshop issued by the agency, and emphasised in its follow-up telephone calls and meetings. These early communications set the scene, explained the purpose of the workshop and the outcomes the agency was seeking. They generated a positive mindset before the workshop even started.
  • The workshop introduction focussed on the purpose of the meeting and the benefits for all participants in the child care sector, and the workshop process was designed to elicit constructive discussion and cooperative behaviour.
  • Of particular importance was the attitude of the senior agency personnel in charge of the regulation and inspection of child care services. They genuinely wanted to help everyone in the sector to obtain good outcomes, and this was clearly apparent to the participants.
  • The participants were treated explicitly as part of the solution, their inputs had been sought explicitly and were being listened to, and there was a clear path by which the outcomes from the workshop would lead to benefits to them. Not all steps on that path were clear, but the direction was, and so was the agency’s desire to move along it.

The outcomes from the workshop were intended to support the agency’s work, but it was recognised that they would be very valuable for the managers of child care centres. The agency undertook to develop risk-based monitoring protocols for centre managers and supervisory boards in parallel with their own protocols. The value of the outcomes for centre managers was a benefit that had not been anticipated in the initial design and planning of the process, and the agency’s undertaking reinforced the positive views of the workshop participants about the value of the exercise and the benefits of the agency’s policy development activities.

Summary of lessons

Involving stakeholders in a risk assessment can be very valuable. However, the stakeholder-engagement process must be designed and managed carefully. Generating a sense of inclusiveness and transparency about the purpose of the activity are particularly important. Careful selection of the stakeholders who are invited is also important.

Clear and well-thought-out communications, particularly about the mutual benefits that are being sought, set the scene and encourage cooperative behaviour. Workshop processes should be designed to foster collaboration and cooperation.

Sometimes workshops generate unexpected insights and benefits. It is always worthwhile keeping an open mind about workshop outcomes and how they can be used.

Risk analysis outcomes provide guidance for managers. They inform decisions, but do not mandate particular courses of action and they should not constrain decision makers.

Client:
Public sector regulator of child care services
Sector:
Public sector and government business
Not for profit
Services included:
Risk assessment and risk treatment
Stakeholders
Risk assessment