To help EDSCLS users more readily interpret and compare the results that the EDSCLS generates, the U.S. Department of Education (ED) has benchmarked scale scores, beginning with VM 3.0 (released in December 2017). As such, the EDSCLS now produces graphs showing three performance levels into which the benchmarked scale scores may fall:
The benchmarking allows users to compare performance levels no matter the topic or domain. For example, if students in a school are in performance level 1 (Least Favorable) for Engagement and performance level 2 (Favorable) for Safety, it suggests that they perceive the school is not doing as well in engaging students as it is in providing for safety.
The benchmarked scale scores were created using item parameters based on a Rasch model. The data used in the analysis were from about 100 schools across the nation that administered the EDSCLS and voluntarily shared their data with the U.S. Department of Education. The data includes completed student, instructional staff, and noninstructional staff data collections. Please note, the parent/guardian survey is not designed to produce scale scores. Due to the brevity of the survey, parent/guardian data should be examined at only the item level.
Before creating the benchmarked scale scores, individuals’ EDSCLS scale scores were centered at 300, within a range of 100 to 500, by trimming scores at the 4th or 5th standard deviation for the staff and student surveys, respectively. This psychometric benchmarking fixes the cut scores at 300 and 400 for the points separating levels 1 and 2, and levels 2 and 3, respectively. In order to maintain the full spectrum of the variation, the individual scores are not trimmed within the range of 100 to 500. However, for consistency and ease of understanding in the EDSCLS platform reporting, the aggregate scale scores are reported in the range of 100 to 500.
For more information for how the benchmarks were created, see the Benchmark Report.
The aggregate scale scores are categorized into the following three benchmarked performance levels:
- Level 1 (Least Favorable): This benchmarked performance level indicates that respondents have the least favorable perception of a school climate domain or topic area. The score is less than 300, within a possible range of 100-500, which means that the most likely answer to each positively valenced question in the scale is Disagree or Strongly Disagree. Likewise, the most likely answer to each negatively valenced question in the scale is Agree or Strongly Agree.
- Level 2 (Favorable): This benchmarked performance level indicates that respondents have a favorable perception of a school climate domain or topic area. The score is 300- 400, within a possible range of 100-500, which means that the most likely answer to each positively valenced question in the scale is Agree. Likewise, the most likely answer to each negatively valenced question in the scale is Disagree.
- Level 3 (Most Favorable): This benchmarked performance level indicates that respondents have the most favorable perception of a school climate domain or topic area. The score is more than 400, within a possible range of 100-500, which means that the most likely answer to each positively valenced question in the scale is Strongly Agree. Likewise, the most likely answer to each negatively valenced question in the scale is Strongly Disagree.
In order to improve perceptions around school climate overall, and especially for sub-groups of respondents, we encourage users to access the School Climate Improvement Data Interpretation Resources. (These resources are currently being updated to reflect the new benchmarked scale scores.)
Since the item parameters have been recalibrated, benchmark scale scores calculated using the new EDSCLS VM should not be compared with the older scale scores, known as “legacy” scales scores, which were calculated using earlier versions of the EDSCLS VM. When measuring school climate survey data trends (e.g., comparing school year 2016-17 data to what you will collect for school year 2017-18), you will need to convert your previous “legacy” scale scores to the benchmarked scale scores. In other words, the benchmarked scale scores produced by the EDSLCS platforms/VMs from now on are not comparable to the scale scores produced by previous platforms/VMs. Users have two options to convert the old “legacy” scale scores to the benchmarked scale scores:
If you have raw data from previous administration(s), you can either:
- Import raw data from previous platforms/VMs to the platform/VM 3.0 (or later version) and rerun the legacy scale scores in the VM. This will give you the benchmarked scale scores for prior administrations as well as the current one so they can be compared. (Remember, you should not compare benchmarked scores to legacy scores.) See the latest User Guide’s Section 2.2.11: Exporting and Importing Respondent-Level Results for instructions.
- Use the R code for benchmarked scale scores to convert legacy scale scores to benchmarked scale scores. The updated R codes for benchmarked scores are provided on the EDSCLS Measures page.
If you do not have raw data from previous administration(s) and need to measure data trends, contact the EDSCLS Help Desk at EDSCLS@air.org or 866-730-6735.
Changes to the Nonistructional Staff Survey
Based on the recalibration results, the following item was dropped from the environment domain and the instructional environment topic scales in the noninstructional staff survey:
NENVINS141: Staff at this school expect students to do their best all the time. The item was dropped because no respondents provided a “Strongly Disagree” response.
The survey has a similar item in terms of wording and the response distribution (NENVINS140: Staff at this school feel that it is a part of their job to prepare students to succeed in college).
Changes to the VM Platform
The underlying database of EDSCLS VM versions 2.7 and higher has a slightly different structure for the staff surveys than the EDSCLS VM 2.6 database because a new survey item was added to the staff surveys. Therefore, raw response data to staff surveys exported from EDSCLS VM 2.6 cannot be imported into EDSCLS VM 2.7 or any other subsequent versions of the VM. For more information about importing and exporting data, see User Guide Section 2.2.11 Exporting and Importing Respondent-Level Survey Results.
