Output, Outcome, Pre/Post Test, Logic Model, Metrics, Indicators, Client Satisfaction, Move the Needle, Data Driven Results, Quasi-Experimental Model, T-Test…
If any of these terms or phrases sends shivers down your spine, then you are probably involved in program evaluation.
Even for larger nonprofits, with the personnel, software and resources to undertake rigorous program evaluation, this process can be daunting. But, if you are reading this blog, more than likely you are part of an organization that lacks the necessary means to undertake long-term, systematic evaluation of your program that can isolate the effects of your activities from other factors.
And that… is just fine. You can run an amazing program that makes a huge difference in the lives of people in your community without having an intensive evaluation program in place.
But, if you do have requirements for evaluation in place and find your organization struggling, do not fret because you are not alone. A recent study indicated that many small to mid-sized nonprofits have difficulty with various aspects of program evaluation despite the ample amount of research on how to evaluate program performance. According to a survey conducted by Idealware, a Maine-based nonprofit, “while there is an abundance of theories and terminology around program evaluation in the sector, nonprofits lack the practical framework to make sense of them and to put the processes in place” (Idealware 2014).
While this article can only begin to scratch the surface on the realities of program evaluation, there are some points that organizations should keep in mind when thinking about how to grade their performance.
1.) Program evaluation is an ongoing process that informs you about what you are doing well and indicates where you need to make improvements. You should be planning your program around the evaluation process not to satisfy funders, but because it is a way to discern if you are really having the effect on the community that you intended to have.
2.) All your staff needs to be evaluation focused. While some people might be officially in charge of evaluation, everyone who works with programs should be the mindset that they need to prove how what they are doing actually makes a difference.
3.) Software and training are critical. Evaluation will often require data, and data is only as good as the people who generate it. Consistent coding of information that is uploaded into the proper software not only will save you time and energy but is often the lynchpin of effective program evaluation.
4.) Finding the right metrics for your program is critical. Too often, I see programs that have metrics for evaluation that come nowhere close to capturing the scope of services or the impact that it has on people’s lives. In some instances, this can be resolved by adding metrics and providing more data during reporting but, at other times, you may need to push for your evaluation to include information not captured by data (e.g. client testimonials or site visits that show your operations in action) so that funders and/or stakeholders get the full story.
5.) Don’t be afraid to ask for help. Discerning the short and long term impact of your program is not an easy task, but you are not on your own. Connect with other professionals at similar organizations, find blogs or online forums that have a national following and ask questions, or look for people with the skills you need who might be willing to volunteer their time. While I can’t promise you I will have any or all the answers, I am certainly willing to talk through some of your issues and point you in the direction of the right resources. Shoot me an email, firstname.lastname@example.org.
For the entire Idealware report, called The Reality of Measuring Human Service Programs: Results of a Survey, click here.