Measures of principals’ performance based on student test scores do not accurately predict the impacts of these school leaders on student achievement, according to a new report and issue brief from Mathematica Policy Research. The study, conducted for the Institute of Education Sciences at the U.S. Department of Education, examined the accuracy of test-based measures of performance that states and districts already implement or could widely implement. The researchers found that although some measures that capture student growth were slightly related to principals’ future impacts, none of the measures could accurately identify principals who are most successful in helping students succeed academically.
The research team conducted the study using student and principal data from Pennsylvania over seven years. With these data, the researchers calculated principals’ performance ratings based on several types of measures. They subsequently identified schools at which one principal replaced another, and measured the change in student achievement at each of these schools. For each school, the researchers assessed the extent to which the principal with the higher rating actually produced higher achievement.
Findings from the study include:
- Performance measures that reflected students' end-of-year achievement, without accounting for their past achievement, provided no information for predicting principals' impacts in the following year.
- Performance measures that accounted for students' past achievement by measuring their growth provided, at most, a small amount of information for predicting principals' impacts in the following year.
- Averaging performance measures across multiple recent years did not improve their accuracy.
“Research on principal performance measures to date has been scant, so this study fills an important knowledge gap,” said Hanley Chiang, a senior researcher at Mathematica and principal investigator of the study. “Although many states are using test-based measures, we found that only some measures have any information at all for identifying effective principals—and even those measures are quite inaccurate.”
The study builds on Mathematica’s previous work suggesting that students score higher on state assessments when their schools are led by more effective principals. For nearly 50 years, Mathematica has worked with decision makers across the public and private sectors to produce evidence that helps improve societal well-being in areas from health to education to employment.
Read the report and issue brief.
