How we evaluate the impact of the National Workforce Centre for Child Mental Health

Our evaluation approach

We take a participatory approach to both monitoring and evaluation. Emerging Minds’ operational teams play a central role in identifying and monitoring key indicators, and also work with the Emerging Minds Research and Evaluation team to co-design theory and logic models, and conduct evaluation studies.

Emerging Minds’ advisory forums, user advisors and subject matter experts also contribute to the design and delivery of evaluation activities.

Our research partners have contributed specific independent research and evaluation studies, and also provide advice on the overall National Workforce Centre evaluation framework.

The extent of complexity and continuous improvement within the National Workforce Centre makes it necessary to have a responsive evaluation approach that feeds findings back into program development in a timely way. Emerging Minds’ evaluation activities are therefore conducted continuously and planned as part of program development and innovation. Use of a Developmental Evaluation¹ approach enables the Research and Evaluation team to better integrate ongoing learning and reflection, and to work alongside program units within their timeframes and strategic priorities.

Embedding evaluative thinking

In the following video (5 minutes, 8 seconds) Emerging Minds’ Dr Melinda Goodyear (Chief Advisor, Policy and Impact) talks about embedding developmental evaluative thinking throughout the organisation.

Our program logic

The program logic for the National Workforce Centre has three streams of activities and outcomes which look for engagement and changes at the practitioner level, the organisational level and at the systems and policy level (Figure 2). Under this logic, the National Workforce Centre program works towards an overall service system(s) which is equipped to identify, assess and support children at risk of and experiencing emerging mental health difficulties.

In the short term, our program logic anticipated that the National Workforce Centre would register changes in professionals’ knowledge and awareness regarding prevention and early intervention to support child mental health, while gradually developing their skills, confidence and competence. As the program transitions to mid-term after five years of operation, focus also shifts towards embedding changes in practice, and systems drivers to support workforces and organisations toward this end.

National Workforce Centre program logic

See accessible data from Figure 2. National Workforce Centre program logic flowchart

Monitoring, evaluation and quality improvement

With many components of in the National Workforce Centre now well established, there has been a shift to monitoring (rather than evaluating) key performance areas including:

• program reach
• engagement of target audiences
• relevance of and satisfaction with resources and products
• post-learning impacts.

Monitoring data is used to inform continuous quality improvement of National Workforce Centre program components, contributes to data analysed for evaluation purposes, and is reported six-monthly to the Department of Health, Disability and Ageing.

Emerging Minds’ evaluation resources are targeted at assessing new and developing National Workforce Centre program components – including innovative resources, engagement and implementation support strategies – and investigating evaluation and research questions that remain of interest. Consistent with the National Workforce Centre program logic, evaluation is now increasingly focused on measuring and understanding changes in practitioner behaviour within various workforce groups, and in the mechanisms of change within organisations and service systems, while seeking to identify early- to medium-term impacts.

1. Patton, M. Q. (2011). Developmental Evaluation: Applying complexity concepts to enhance innovation and use. Guildford Press.