The EDSCLS will not be used to establish a national data collection and reporting system. Through the EDSCLS platform, school, district, and state education agencies independently administer the EDSCLS surveys, which means that there is no one organization (e.g., NCES) overseeing the data collection and ensuring that consistent procedures are implemented across schools. This lack of centralized oversight has the potential to threaten the validity of the results and interpretation of scale scores. Therefore, the EDSCLS platform has been programmed with a focus on building in key features that should be implemented in a uniform manner across all collections.
B.1 Availability of Survey Link and Automated Dissemination of Usernames
The platform allows respondents to access their survey through a URL that can be accessed from any location with internet access and from all standard operating systems and browsers. However, the student surveys are intended to be administered in schools to help control the environment of the student respondents.
To allow potential respondents to follow a link to the survey platform, the platform offers education agencies the option of disseminating usernames by e-mail directly from the platform. The e-mails can be generated and sent directly from the platform; they contain log-in information for each e-mail address and/or a link to the log-in page of the survey (the link will be the same for every respondent). Although e- mail addresses are matched with randomly generated usernames, these e-mail addresses are deleted from the database after the data collections are closed and, therefore, will not be linked to respondents within the platform to protect respondent confidentiality.
B.2 Informed Consent
The EDSCLS is designed as a voluntary survey and incorporates informed consent/assent procedures; therefore, the need to obtain consent was considered during the development of the platform. For adult populations, obtaining consent is generally a straightforward process by which respondents receive information regarding the survey (including the purpose of the study, the extent to which confidentiality will be maintained, and other elements of informed consent) and, upon indicating their agreement to participate, begin the survey. For student populations, however, obtaining informed consent can require additional steps. Local education agencies administering the surveys should follow the appropriate procedures regarding parental consent that are used for any kind of surveys or testing in the school or school system. In the survey platform, students are presented with assent language that is appropriate for their age. This assent process is built into the surveys; similar to the adult populations, students are provided with information about the data collection and asked to participate. The prefilled consent/assent language can be edited by local education agencies based on their situations or practices.
Draft Confidentiality Language for School Climate Survey
Under the Protection of Pupil Rights Amendment, 20 U.S.C. § 1232h(c)(1)(A), any school district that will be administering the School Climate Survey must have developed and adopted policies, in consultation with parents, that allows parents to inspect, upon request, the Survey before it is administered or distributed by a school to a student and have in place procedures for granting such parental requests. The school or school district also should ensure that parents understand how the Survey responses will be used.
For privacy and data quality reasons, the U.S. Department of Education recommends that the Survey responses only be used in an aggregated, de-identified manner. However, if the school or school district administering the Survey decides to link the Survey responses with additional student information in a way that would make the Survey responses identifiable to a student, then the Survey Administrator should clearly indicate in advance that the Survey responses will be maintained in individually identifiable form. Furthermore, the Survey Administrator should clearly indicate that any individually identifiable information that is so maintained will be protected under the provisions of the Family Educational Rights and Privacy Act (FERPA), 20 U.S.C. § 1232g, and may be disclosed in identifiable form to third parties in accordance with FERPA. For example, as part of the students’ education records these linked Survey responses may be disclosed to parents upon request, to school officials with legitimate educational interests, or to other third parties for studies to improve instruction or to evaluate federally or state- supported education programs.
B.3 Spanish/English Bilingual Feature
The EDSCLS platform offers Spanish and English versions of the parent and student surveys. The EDSCLS platform allows these respondents to toggle between English and Spanish versions of questions at any point in the survey, giving them the option to answer different questions in different languages. The instructional staff and principal/noninstructional staff surveys are offered in English only.
B.4 Survey Page Design
An important design issue of web-based surveys, with implications for question-level response rates, is the number of questions displayed on each survey page. If too many questions are included on each survey page (or if the questions are poorly formatted to accommodate different screen sizes and resolutions), it is more likely that a respondent will be required to scroll down or across the page to view the question. This, in turn, can increase the likelihood that a respondent will fail to answer these questions. The EDSCLS platform contains only one question per page to allow for responding to surveys on mobile devices, such as smart phones or tablets.
B.5 Real-Time Submission Rates
The EDSCLS platform provides access to real-time submission rates at the school level. These submission rates are determined based on the number of respondents in each population who have logged in to the survey and the education agencies’ reports of the number of expected respondents in each population (i.e., the number of respondent usernames generated).
The need to maintain respondent confidentiality and data security means that the platform cannot supply education agencies access to information about which respondents have completed the survey directly from the platform. The platform does, however, allow authorized users to export a list of usernames by completion status (e.g., a list of usernames not used). This information can be used by education agencies that have linked EDSCLS usernames to respondents to determine completion status at the respondent level and target nonresponse follow up as needed.[1]
B.6 Data Export
To maximize the utility of the EDSCLS data, the platform allows authorized users to download respondent-level data for all questions and scales. The initial usernames are included only with the student data. Those education agencies that have created a crosswalk between usernames and student identifiers can link individual-level student data to extant data (such as student administrative records) for further analyses. The data download process requires that the EDSCLS administrator read and acknowledge a statement that these data contain private, identifiable information and must be handled in accordance with applicable confidentiality and privacy regulations. The data export functionality (and associated data import functionality) also allows schools to send data to districts for the purposes of creating district-level reports and allows districts to send data to states for the same purpose.
B.7 Platform Features Considered But NOT Included
Based on the recommendations and requests of key informants (and the experience of AIR’s staff in administering school climate and other school-based surveys of students, parents, and staff), several design features were considered for integration into the EDSCLS platform. These platform features were considered advantageous but could not be implemented because they conflicted with the platform goals described above or were outside the scope of EDSCLS’s objectives. Below we briefly describe these features, their potential advantages, and why they were not directly integrated into the EDSCLS platform.
B.7.1 Preloading of Respondent Data
Education agencies may desire a survey platform that allows respondent data to be preloaded into the platform rather than requiring respondents to enter the information. However, such a design feature could not be implemented for three reasons. First, risk to the confidentiality of responses would increase if EDSCLS responses were linked to identifiable records within the platform. Second, it gives the impression that the EDSCLS surveys and platform can be used to create a federal database of identifiable student information, which, if implemented, would violate federal regulations. Third, accommodating preloaded data to be used during survey administration would require significant custom programming— e.g., not all schools use the same staff role titles, so it would be difficult to ensure that all preloaded data would route staff to the appropriate role-based survey questions—that is outside the scope of EDSCLS’s objective.
B.7.2 Reporting of Data by Detailed Respondent Subgroups
Enabling education agencies to break down the survey results by respondent characteristics (such as gender, grade, race/ethnicity, gifted and talented status, special education status, and English language learner status) would allow them to assess perceptions of school climate in more detail and potentially create more targeted programs and policies to improve school climate. However, the potential for a breach in respondent confidentiality and data security outweighs the potential benefit. Additionally, statuses such as gifted and talented, special education, and English language learner may not be reliably reported by respondents. As such, the platform is only designed to allow agencies to view the student survey results by limited demographic characteristics, such as grade, gender, and race/ethnicity. In order to maintain respondent confidentiality and data security, appropriate U.S. Department of Education Disclosure Review Board standards have been applied to resultant reports generated from the EDSCLS platform. If education agencies wish to perform more detailed subgroup student reporting, respondent- level data can be exported from the platform for this purpose, leaving the education agency responsible for ensuring proper confidentiality protections.
B.7.3 Customization of Survey Content
Allowing end users to customize their survey administrations by modifying or deleting questions may be an attractive option to obtain maximum utility from the EDSCLS platform and surveys. However, modification or deletion of survey content could potentially affect the psychometric properties of the EDSCLS scales. Specifically, the questions that are offered in the EDSCLS are intended to produce psychometrically validated scales, and it is crucial for the statistical validity of those scales that all questions remain in the survey. The platform allows users to add questions, in the same multiple-choice form, to the end of the surveys. The platform will not provide any reporting on these additional questions, and it’s not possible to import data from additional questions into a data collection. However, users can export the data files and perform additional analysis and reporting outside of the system.
[1] The ability to track submission rates does not mean administrators can track individual responses. Upon the completion of a data collection, when the results can be viewed, username credentials for parents, teachers, and noninstructional staff are removed from the data and replaced with random IDs to decouple answers from personally identifiable information. Administrators do have the option of maintaining student usernames so that they can link individual student responses to other datasets, such as achievement levels.