There are many types and ways of evaluation: needs assessments, monitoring activities, formative, participatory and summative evaluations — all of which are explained in the chapter on evaluation in our Executive Directors Guide.
It is important to be very clear about what you are evaluating, why you are evaluating, and for whom you are evaluating. Paying attention to these questions will help you shape your evaluation into a more useful tool.
The type of evaluation most commonly requested by foundations is called outcome evaluation. Outcome evaluations assess the effectiveness of a program in producing change. Outcome evaluations focus on difficult questions that ask what happened to program participants and how much of a difference the program made for them.
Impact or outcome evaluations are undertaken when it is important to know whether and how well the objectives of a project or program were met.
For example, outcome questions for a smoking cessation program might include:
Over-emphasizing outcome evaluation at the cost of other types, especially process evaluation, is a disservice to nonprofits and the sector. Why? Because process evaluation allows a nonprofit to look at how it develops itself, its structures, its supporting programs like communications and marketing, and even fund development to get to the outcomes everyone wants it to achieve.
In other words, process evaluations document the process of a program’s implementation. Process evaluations help stakeholders see how a program outcome or impact was achieved. The focus of a process evaluation is on the types and quantities of services delivered, the beneficiaries of those services, the resources used to deliver the services, the practical problems encountered, and the ways such problems were resolved.
Taken a step further, process evaluation can also look at the processes of program, management, and infrastructure together to judge the capacity of an organization to deliver on its promised outcomes.
For example, process evaluation questions might include:
Information from process evaluations is useful for understanding how program impact and outcome were achieved and for program replication. Looking at outcomes without analyzing how they were achieved fails to account for the human capital (over-worked staff) involved in getting to good outcomes and the true costs of the program.
Evaluating the “input” (the very first column in a logic model) is just as valid as evaluating the last columns about outcomes. It is called a “logic” model after all — and logically there is a chain of cause and effect which means if we have the right resources at the very beginning of the chain (inputs) then we assume we will be able to get to the outcomes to which we aspire.
GET THE WHOLE PICTURE
If process evaluation were used in combination with financial analysis, a much clearer picture of the cost of doing business would emerge — as would the case for more general operating and infrastructure support.
Accountability is the key reason for evaluation. Most small and mid-sized nonprofits conduct formal evaluations because it is required by their funders. Government agencies, foundations, the United Way and even individual donors are scrutinizing nonprofits more closely for accountability and effectiveness. Program participants, boards, staff, and the public are also demanding greater accountability from nonprofit organizations.
Nonprofit organizations are finding that it makes sense to place new emphasis on evaluation in order to:
Participatory evaluation enables users of services to take part in designing and implementing the evaluation of those services. The style and values of the organization and the programs being evaluated will determine who participates in the process.
A flatter, more inclusive organization is likely to include more people in the evaluation process. Different types of programs or activities will call for different types of evaluations and different levels of involvement from different people.
Generally, the board of directors is most accountable for ensuring that evaluation is taking place, while the executive director is most responsible for managing the evaluation process.
Executive directors have the role of cultivating:
These skills and competencies have to be present within the executive director in order to successfully lead an organization that continually learns, innovates and improves on behalf of its clients and mission.
Leadership must exude a spirit of belief in the importance of evaluation. If evaluation is dismissed by the ED as an annoyance forced upon the organization from the outside, staff will feel the same way. The executive director must also understand the level of evaluation that is appropriate. It does not make sense to burden program staff with evaluation plans that are so time-consuming that they don’t have time to run the program.
Be aware of how ready your organization is to implement, create and implement new systems and structures.
For updates on future reports, related webinars, and ongoing discussions about nonprofit evaluation practices, subscribe to our newsletter.
