Fueled by evidence on the strong relationships between school climate and academic achievement, teacher satisfaction, health outcomes, and socio-emotional skills, states and districts are increasingly trying to measure school climate. School climate covers both tangible and intangible attributes, including relationships among students and staff, school discipline, student engagement, and safety. It is typically measured using surveys of staff, students, and sometimes parents. Some districts use climate surveys for accountability, either as part of a formal system in which consequences are linked to school performance, or in the presentation of results for public scrutiny in school report cards.
Maryland and a few other states such as Montana, Iowa, and New Mexico have included climate as a measure of school performance in statewide accountability plans under the federal Every Student Succeeds Act (ESSA). Researchers from the Regional Educational Laboratory (REL) Mid-Atlantic and staff from the Maryland State Department of Education (MSDE) developed a school climate survey that will be administered to students and staff in all schools across the state in spring 2019. This is one of the first times that a school climate survey is being used for statewide accountability purposes. To ensure a successful rollout, REL Mid-Atlantic helped MSDE conduct a field test of the survey in fall 2018. Currently, REL Mid-Atlantic and MSDE are analyzing the data from this test to see whether the survey requires any necessary revisions before its statewide administration.
What can we learn from Maryland? Designing one of the first surveys ever used for this new purpose presents unique challenges. Our experience suggests approaches to overcome the top two:
- Challenge #1: Identify aspects of school climate appropriate for accountability and relevant to diverse stakeholders. For use in a statewide accountability system, a school climate measure must (1) reflect aspects of a school that promote student success, (2) represent a quality or characteristics that schools can feasibly influence, and (3) apply to a diverse set of schools. These criteria ensure that schools across the state can take steps to improve their scores and, in turn, student outcomes.
Approach: Examine the research and involve stakeholders. After reviewing empirical evidence and theory, REL researchers and MSDE staff identified aspects of school climate that met the first two criteria. REL Mid-Atlantic and MSDE then convened district representatives to discuss which climate aspects were most relevant to their districts. This process enabled MSDE to focus on dimensions of school climate that met all three key criteria. For example, district representatives strongly endorsed including measures of bullying, school participation/engagement, and respect for diversity, but not measures of a school’s physical environment.
- Challenge #2: Create a reliable and valid survey that isn’t burdensome for respondents. Respondent burden is a key consideration for school districts already administering statewide assessments. Moreover, many districts in Maryland are already administering their own school climate surveys and would like to limit time spent on a state-mandated school climate survey. Although these two factors point to a briefer survey, a shorter instrument might be less reliable and valid.
Approach: Draw on items from existing surveys and seek input from stakeholders. To develop a survey that captures key aspects of school climate, REL Mid-Atlantic and MSDE drew on items from existing and validated surveys, such as those gathered in this compendium of school climate surveys. For each aspect of school climate, MSDE selected the fewest number of items that would lead to a reliable measure, prioritizing those that were conceptually or empirically most important. School district representatives provided input to ensure that the items would also be useful for their districts. The final student and staff surveys take about 10 minutes to complete and pose minimal burden to respondents. By conducting a field test and analyzing the resulting data, REL Mid-Atlantic and MSDE will be able to verify that the survey is reliable and valid before using it as an accountability measure.
What’s next? Continuing our researcher–practitioner partnership, REL Mid-Atlantic and MSDE plan to analyze data from the first year of administration to further validate the survey and develop an overall index of school climate. Stay tuned for updates on this exciting work!
Cross-posted from the REL Mid-Atlantic website.
