Program Evaluation Overview
Getting Started
Water First has embarked on a program evaluation journey to better understand our program strengths, challenges, and opportunities. The early stages of our evaluation work involved consultation with Water First staff, management, and Indigenous Advisory Council members, as we worked together to map out our plan and approach. We collectively determined that the overall goal of the evaluation would be to help us make informed, strategic decisions about program delivery, growth, management, and change, and to demonstrate program effectiveness.
Logic Models
Logic models – also known as program profiles – are critical to evaluation planning. They provide a visual roadmap to help us better understand how each Water First program area works: they articulate what each program is trying to accomplish, and what the program is doing to get there.
After developing a logic model for each of Water First’s three program areas (Drinking Water, Environmental Water, and Schools Program), we identified key program indicators to answer guiding evaluation questions over time.
Clarifying program goals:
The logic model
An important first step for evaluation planning.
Inputs
Resources needed to operate each program area at Water First.
Activities
If we have access to these resources, then we can accomplish our planned activities.
Outputs
If we accomplish our planned activities, then Water First should be able to deliver the planned amounts of service intended.
Outcomes (short- and medium-term)
If we accomplish our planned activities, then Water First should be able to deliver the planned amounts of service intended.
Outcomes (long-term)
If benefits are achieved, then certain changes can be expected to occur over time. The intended result over time ties back to program growth, program goals, and long-term outcomes.
Data Collection/Gathering
We have adapted current resources and created new ones to build a set of data collection tools that address our evaluation goals, answer the evaluation questions, and help us to track key indicators over time.
Water First surveys and community engagement tools will gather data from key stakeholders, including past and present community partners, interns, students, and staff. These data collection tools will be used at all stages of program delivery and community partnerships to continue to strengthen our relationships and deepen our understanding of community priorities, strengths and goals.
Why Evaluation?
Conducting ongoing evaluation will strategically track change while drawing on key stakeholder feedback to inform program adaptation, design, and delivery. With the information gathered through this process, we are better positioned to guide program delivery and expansion in line with community-identified objectives and priorities.
The goals and activities associated with evaluation are as follows:
- Evaluate ability to meet goal(s), aligned with Water First’s vision and values.
- Demonstrate action toward planned outcomes and outputs, and identify key successes and gaps in program design and delivery.
- Interpret data collected and apply learnings to support strategies, as well as evidence-based decision-making, program adaptation and continuous improvement, and program growth.
- Share the findings more broadly.
- Provide another measure of accountability to community partners, participants, donors, and other key stakeholders.
Water First Goals and Evaluation Indicators
Drinking Water Program
Program Primary Goal
To train Indigenous youth and young adults to become certified water treatment operators, in order to provide safe, clean drinking water for future generations.
Success will be measured through the impact on relationship-building and education and training delivery at the individual and community level.
Outcome indicators will identify:
- Do co-created programs address community priorities and support interns in meeting their individual goals.
Output and outcome indicators will identify:
- Information about partnerships, program reach, and intern engagement post graduation.
- Community feedback on if the project was implemented in respectful and collaborative partnership and how the partnership could have been strengthened.
Output and outcome indicators will identify:
- Information about interns, training received, increase in knowledge and skills, and employment after graduating.
- Community partner satisfaction and the impact on the local drinking water plant and community capacity.
Environmental Water Program
Program Primary Goal
To recruit, support, and train Indigenous youth and adults to gain employment in water resource stewardship.
Success will be measured through the impact on relationship-building and education and training delivery at the individual and community level.
Outcome indicators will identify:
- Do co-created programs address community priorities and support participants in meeting their individual goals.
Output and outcome indicators will identify:
- Information about partnerships, program delivery and reach, and participant engagement post graduation.
- Interest from communities to participate in future trainings.
- Community feedback on if the project was implemented in respectful and collaborative partnership and how the partnership could have been strengthened.
Output and outcome indicators will identify:
- Information about participants, training received, increase in knowledge and skills, and employment after graduating.
- Community partner satisfaction and the impact on local capacity and technical knowledge in water resource stewardship.
School Water Program
Program Primary Goal
To engage Indigenous students, from K-12, to showcase pathways to careers in aquatic science and drinking water sectors.
Success will be measured through the impact on relationship-building and education and training delivery at the individual and community level.
Outcome indicators will identify:
- Do co-created programs address community / staff / teacher priorities.
Output and outcome indicators will identify:
- Information about partnerships, program delivery, and reach.
- Interest from communities to participate in future trainings.
- Community feedback on if the project was implemented in respectful and collaborative partnership and how the partnership could have been strengthened.
Output and outcome indicators will identify:
- Program delivery and reach, and community partner satisfaction.
- Increase in student knowledge of water science and awareness of water science career pathways.
