Data needs to indicate what the problems are, what programming has been implemented, and what was accomplished. At each stage, your evaluator can provide important resources to help in this process.
For example, in the needs assessment stage before program implementation, evaluators may review:
- Survey data about current levels of alcohol and other drug use and violent incidents on campus and in the surrounding community in order to determine needs and issues that are unique to that environment
- Observational assessments of specific campus or community environments to determine contexts in which problems occur
- Incident reports from campus and community sources to determine magnitude of problems, as well as where and when the problems occur
These data sources can assist you in determining which specific alcohol, other drug, and violence-related outcomes should be the focus of efforts. In addition, they can help you develop a logic model to outline the processes by which problems occur and the mechanisms through which these problems can be reduced.
During implementation, evaluators collect process measures such as:
- Utilized resources (staff, volunteers, funds) to see how they compare with what was initially planned
- Activity levels and the number of people or groups served by a program to determine if it reached the intended population
After implementation, evaluators analyze outcome measures including:
- Changes in behavior, both intermediate (e.g., wait staff refusing to serve underage patrons) and long term (fewer students driving while intoxicated)
- Environmental outcomes, such as observations of rowdy behavior by students at bars, parties, or on the street
Institutions of higher education and funders may have specific requirements that prohibit data collection unless specific procedures are followed. Prior to collecting data, determine whether your institution has an institutional review board (IRB) that governs human subject protections. Nearly all universities have an IRB; state and community agencies may have their own IRB or may have an arrangement with another institution. In addition to following IRB guidelines, review general considerations for protection of human subjects and ensure that the study protocol is designed in accordance with these guidelines.
Evaluation is especially vital in an era of fiscal constraint. Long-term financial support for prevention work, whether it comes from outside funding sources or is part of a college's regular budget, will be available only if evaluation results warrant it. Evaluation is also important to help program planners revise their programs and policies so that they can focus their efforts on implementing the most effective strategies. Unfortunately, very few campus-based prevention programs have undertaken an evaluation that meets rigorous methodological standards, leaving them vulnerable to attack and fiscal cutbacks.
The difficulty is not that prevention coordinators are unaware of the need for evaluation, or that they are worried about their program failing to measure up. Rather, most coordinators do not feel equipped to conduct a rigorous evaluation.
The Center web site includes many resources that can assist coordinators in the process of designing and implementing evaluation efforts. Your campus may also have resources to assist you such as institutional research offices or faculty with expertise in evaluation issues. In some cases you may have resources to hire an evaluator. The National Center for Safe and Supportive Learning Environments provides technical assistance and training in areas including evaluation.
Methods for Assessing College Student Use of Alcohol and Other Drugs, Higher Education Center for Alcohol, Other Drug Abuse, and Violence Prevention, 2008.
