Canadian Research Working Group

Research at Work...

Presentations have been made at provincial, national, and international conferences, describing our evaluation framework and some of the initial field-testing that has been done. These have been received enthusiastically and it seems to us that people in the field are eager to embrace the ideas and try to implement them in their work settings.

Common Indicators

Transforming the Culture of Evaluation in Career and Employment Services

The project was undertaken by the Canadian Research Working Group on Evidence Based Practice in Career Development (CRWG) and the Life-Role Development Group (LRDG) in partnership with the Canadian Career Development Foundation (CCDF) and with the provinces of Saskatchewan, Québec and New Brunswick.  The overall objective of the research was to develop and test a performance measurement tool that captured both elements of services already being captured as well as elements of services identified by front line practitioners and research studies as having impact on employability outcomes but not usually captured and reported.

The research questions addressed in the empirical research were:

What common indicators are applicable across different client contexts, different client groups, different agencies and different interventions?  and

What statements about service effectiveness can be made by tracking common indicators of inputs, processes and outcomes?  If numbers permit it will be possible to aggregate data to begin to address the ultimate question: What kinds of interventions in what contexts produce what kinds of results?

The research in all three provinces has demonstrated the robustness and the practicality of the CRWG model in actual career and employment services settings.  The “common indicators” included in the online tool have been received by practitioners in all provinces as accurately reflecting their scope of practice and as making sense across divergent service settings.  The potential for such an online tool to contribute to quality service and to a transformation of evaluation in career and employment services is significant.

The study has demonstrated the potential of a tool to provide answers to the question: with whom, in what context, with what types of services do we achieve what results?  There are revisions, improvements, refinements and additional experimentation needed in order to be able to do the analysis necessary to respond satisfactorily to this question but this study has demonstrated significant progress.

Lead researcher for New Brunswick and Saskatchewan was initially Dr. Bryan Hiebert and subsequently Dr. Dave Redekopp, Life-Role Development Group.  Lead researcher for Québec was Dr. Guylaine Michaud.

EVIDENCE-BASED EMPLOYMENT SERVICES: COMMON INDICATORS (PHASE 2 – December 2015)

This project builds on Phase 1 (2013) to test PRIME (Performance Recording Instrument for Meaningful Evaluation) as a vehicle to strengthen data collection, service provision and – ultimately – client outcomes.  PRIME:

  • Collects and analyzes a much richer, wider range of client outcome and progress indicator data;
  • Responds to the needs of government, supervisors, frontline workers and clients by using data to inform policy and practice ;
  • Supports quality service by coaching frontline workers to conduct a consistent and high quality client needs assessment and by providing access to tailored resources and tools; and
  • Can be used collaboratively with clients (avoiding the need to enter data separate from meeting with clients).

Built on the technical platform of ARMS, PRIME serves as a model for using evaluation as an underpinning to support quality service and stronger service outcomes.

The report can be accessed by clicking here.