Planning Your Evaluation

An evaluation plan is a written document that describes how you will monitor and evaluate your program, as well as how you intend to use evaluation results for program improvement and decision making. The evaluation plan clarifies how you will describe the "What," the "How," and the "Why It Matters" for your program. 

Because most people associate evaluation with measurement of program results, program planners often do not think about evaluation until after the program is up and running. Instead, evaluation should be planned from the beginning, as the program is being developed. Incorporating evaluation will sharpen everyone's thinking about the program: its mission, its goals, its objectives, and the activities designed to meet those objectives. Used in this way, evaluation planning can be a valuable management tool.

Evaluation should begin during program planning and implementation and continue through program completion.

Evaluation is the process of assessing whether a particular prevention program or policy is implemented and working as intended, and it allows program planners to revise programs and policies as needed. Systematic evaluation is the process of collecting, analyzing, and reporting this assessment information in a proactive, planned way.

Evaluation should begin during program planning and implementation and continue through program completion. The five steps of a sound evaluation plan are:

  1. Describing the intervention and creating a logic model. Logic models, an essential first step in evaluation, are diagrams that describe how and why program activities lead to program goals.
  2. Identifying process measures to document the nature, extent, and quality of program implementation.
  3. Identifying outcome measures to assess a program's success in terms of short-term, intermediate, and long-term objectives.
  4. Selecting a research design to outline when and where data will be collected to study the effect of the program.
  5. Analyzing and reporting results of the evaluation to strengthen or improve the program and disseminate findings.

There are three different types of evaluation: process, outcome, and impact.

Just as no one program or policy can solve all campus issues, evaluation is not a one-size-fits-all process. Using various types of evaluation at different times helps ensure you're measuring every facet of program planning and implementation.

  1. Process evaluation: This type of evaluation answers the question, "What are we doing?" Process evaluation assesses whether the prevention effort is being implemented as designed. This type of evaluation allows you to monitor progress and make corrections in implementation along the way. Also, if an intervention fails, process data allows evaluators to see where in implementation the failure occurred.
  2. Outcome evaluation: "Where are we going?" Outcome evaluation looks at whether each program and policy is accomplishing its short-term and intermediate objectives. The two types of outcomes to consider are changes in behavior, and changes in structure or functioning of the environment. A logic model will help you identify the outcomes you want to measure.
  3. Impact evaluation: "What effect are our efforts having?" Impact evaluation measures if your efforts are achieving their ultimate goal of reducing student AOD abuse and its consequences and violent behavior.

For examples of what to measure during each type of evaluation, please see What Should You Measure? on the Center's Data Collection and Management webpage.

Creating a logic model is an essential first step in evaluation.

A logic model (also called an evaluation map) is a diagram that shows program planners' commonsense understanding of how and why program activities lead to program goals. A logic model can show you at which points data collection is crucial, and help you determine what types of data to collect.  Logic models begin with specific strategies and activities and relate them to intermediate and long-term outcomes. These outcomes can be measured and assessed to determine if the program is meeting intended goals.

Why Use a Logic Model?

A logic model outlines the logical sequence of events that leads from program activities to intended impacts. A logic model lays out each step that is expected to occur in a program, and serves as a visual program plan. Logic models help every aspect of program planning, and assist with evaluation efforts.  Creating a logic model helps everyone on a program team get on the same page regarding activities and outcomes, allowing everyone to work together more effectively.

Logic models also serve as the basis for the evaluation plan. Evaluators can decide when and how to measure each step in the logic model. For instance, if a program's goals are not being achieved, the logic model can show which activity is not working. In addition, a logic model can show how desired outcomes result from specific activities.

When Should I Create a Logic Model?

The best time to construct your logic model is while you are considering what program activities to choose to lead you to your intended goals. Creating a logic model during program planning in this way will show if your proposed activities will lead to your ultimate goals, such as decreases in AOD use and violent incidents. Your first draft of a logic model may show flaws in the sequence of the program, and allow you to revise activities before they are implemented, saving time and money.

What Does a Logic Model Look Like?

There is no one way to draw a logic model. Many begin with program activities on the left side, progressing to long-term outcomes on the right. The middle of the model depicts the series of events connecting the activities to the outcomes.

References

 

Featured Resources

Intends to help those interested in the safety and security of a school district, college, or university with insight into the use and impact of social media. The guide provides information on social media and its uses in an educational setting, as well as the benefits and challenges; threat alert services and procedures to consider when a threat is received; prevention strategies and guidelines you can share; and developing a social media policy. Social media tips and resources you can share with your community are also included.   

The National Center for Campus Public Safety

Summarizes a meeting hosted by the U.S Department of Education with college campus leaders from across the nation to discuss the issue of racial harassment on campuses and to identify solutions for fostering supportive and inclusive educational environments. During the meeting, Nancy Cantor, Chancellor of Rutgers University-Newark, issued a statement calling for an end to racial harassment on campuses and for institutions of higher education to take the lead in making a social change on their campuses.

 

U.S. Department of Education