Measuring and Monitoring Impact…
There can be serious and unintended consequences of simply using numerical targets and KPI’s as measures in complex systems.
Focusing on achieving numbers can lead to ignoring other issues that need attention, or creative ‘work arounds’.
- quantitative measures on the number of phone calls answered in a call centre resulted in calls being answered but nothing being resolved
- health systems measures – how many hours before a patient is seen, numbers on surgery waiting lists – all have resulted in creative ways of recording what has happened to meet the targets that do not reflect reality (the waiting list before the waiting list for example).
Numbers without narratives – the what without the why provide no basis for action. The lack of context promotes speculation, favours biased choices of ‘answers’ and promises of solutions that have little substance.
Accountability and transparency are high priorities.
Based on our own work, and conversations with clients, the characteristics of the effective monitoring and feedback processes for complex systems that we co-create:
- address the whole system,
- are concerned with impact over time
- do not attempt to attribute cause and effect
- start with developing a shared understanding of the current state
- provide insights that give options
- are cost effective, dynamic and provide real time feedback
- the insights that emerge are the basis for action, improvement and feedback for both service delivery and policy
- develop quantitative and qualitative measures are appropriate, and meaningful to those who need to engage and report
- are scalable – and appropriate for small and large systems