Much has been said about leadership development and developing leaders within organisations has become the focus of much training and development.
Leadership development in itself is complex and programs addressing it are multi-faceted. Participants of such programs experience the learning quite differently given their individual learning styles, personalities and environment in which they are working. Leadership development is not just feeding in a person at one end of the program and turning out a leader at the other end. Many influences will impact on the individual’s experience and the shaping of their leadership capabilities will continue past the end of the program.
So the question of how do we evaluate this impact is one that needs some consideration. Kirkpatrick and Brinkerhoff offer valid models of evaluation, but for leadership development, this needs to be taken further.
After having grappled with the notion of evaluating leadership development programs over time and the need to reflect the complexity of experience in the evaluation, Complexability was contracted to evaluate a project funded by the Bill and Melinda Gates Foundation. This particular program was a leadership development program for emerging library leaders across the South Pacific. Whilst there were expected outcomes for the program, the ongoing experience of the participants was valued as part of their leadership journey.
The approach integrated appropriate evaluation theory, with an innovative approach to impact evaluation that is flexible and meets the requirements to evaluate outputs, outcomes and impacts of the project.
The use of Sensemaker™ enabled us to capture the experiences of the participants on an ongoing and real-time basis. It was also particularly useful as the participants were geographically spread throughout the South Pacific.
The method actively engaged participants, mentors and steering committee members in reflective learning through feedback loops, innovative thinking and improvement plans. It broadly involved:
- Collaborative design and testing of the narrative collection tool
- Online narrative collection using Sensemaker ™ from participants (Mentors and Steering Committee) at mutually agreed critical phases. Data collection became embedded in the learning process
- Reports highlighting insights that emerged from the narrative and necessary quantitative data. The reports provided the basis for feedback loops encouraging reflection on emergent issues and subsequent modification of the program, thereby engaging in a process of continuous improvement.
- Online discussion/workshops with participants to develop actions from the insights and make program adjustments. The impact of the adjustments were monitored through the ongoing narrative process.
This methodology not only provided value for money but its key to success lay in the process reflected the program’s commitment to innovation, collaboration, learning and continuous improvement. It was also flexible, able to adapt to the inevitable changing and emerging circumstances over the life of the project, and contributed to demonstrating success in all aspects of the Results Framework as determined by the Bill and Melinda Gates Foundation.
