Evaluation is the process of assessing whether a particular prevention program or policy is implemented and working as intended, and it allows program planners to revise programs and policies as needed. Systematic evaluation is the process of collecting, analyzing, and reporting this assessment information in a proactive, planned way.
Evaluation should begin during program planning and implementation and continue through program completion. The five steps of a sound evaluation plan are:
- Describing the intervention and creating a logic model. Logic models, an essential first step in evaluation, are diagrams that describe how and why program activities lead to program goals.
- Identifying process measures to document the nature, extent, and quality of program implementation.
- Identifying outcome measures to assess a program's success in terms of short-term, intermediate, and long-term objectives.
- Selecting a research design to outline when and where data will be collected to study the effect of the program.
- Analyzing and reporting results of the evaluation to strengthen or improve the program and disseminate findings.
Just as no one program or policy can solve all campus issues, evaluation is not a one-size-fits-all process. Using various types of evaluation at different times helps ensure you're measuring every facet of program planning and implementation.
- Process evaluation: This type of evaluation answers the question, "What are we doing?" Process evaluation assesses whether the prevention effort is being implemented as designed. This type of evaluation allows you to monitor progress and make corrections in implementation along the way. Also, if an intervention fails, process data allows evaluators to see where in implementation the failure occurred.
- Outcome evaluation: "Where are we going?" Outcome evaluation looks at whether each program and policy is accomplishing its short-term and intermediate objectives. The two types of outcomes to consider are changes in behavior, and changes in structure or functioning of the environment. A logic model will help you identify the outcomes you want to measure.
- Impact evaluation: "What effect are our efforts having?" Impact evaluation measures if your efforts are achieving their ultimate goal of reducing student AOD abuse and its consequences and violent behavior.
For examples of what to measure during each type of evaluation, please see What Should You Measure? on the Center's Data Collection and Management webpage.
A logic model (also called an evaluation map) is a diagram that shows program planners' commonsense understanding of how and why program activities lead to program goals. A logic model can show you at which points data collection is crucial, and help you determine what types of data to collect. Logic models begin with specific strategies and activities and relate them to intermediate and long-term outcomes. These outcomes can be measured and assessed to determine if the program is meeting intended goals.
Why Use a Logic Model?
A logic model outlines the logical sequence of events that leads from program activities to intended impacts. A logic model lays out each step that is expected to occur in a program, and serves as a visual program plan. Logic models help every aspect of program planning, and assist with evaluation efforts. Creating a logic model helps everyone on a program team get on the same page regarding activities and outcomes, allowing everyone to work together more effectively.
Logic models also serve as the basis for the evaluation plan. Evaluators can decide when and how to measure each step in the logic model. For instance, if a program's goals are not being achieved, the logic model can show which activity is not working. In addition, a logic model can show how desired outcomes result from specific activities.
When Should I Create a Logic Model?
The best time to construct your logic model is while you are considering what program activities to choose to lead you to your intended goals. Creating a logic model during program planning in this way will show if your proposed activities will lead to your ultimate goals, such as decreases in AOD use and violent incidents. Your first draft of a logic model may show flaws in the sequence of the program, and allow you to revise activities before they are implemented, saving time and money.
What Does a Logic Model Look Like?
There is no one way to draw a logic model. Many begin with program activities on the left side, progressing to long-term outcomes on the right. The middle of the model depicts the series of events connecting the activities to the outcomes.
