Measurement error reduces the statistical reliability of the performance measures used to identify low-performing schools. Using data provided by the Pennsylvania Department of Education, the study team used Bayesian hierarchical modeling to improve the reliability of these measures and demonstrate the...
Related Content
Related Publications for Brian Gill
-
Stabilizing Subgroup Proficiency Results to Improve the Identification of Low-Performing SchoolsFeb 28, 2023
-
The Psychological Science Accelerator’s COVID-19 Rapid-Response DatasetFeb 11, 2023
In response to the COVID-19 pandemic, the Psychological Science Accelerator coordinated three large-scale psychological studies to examine the effects of loss-gain framing, cognitive reappraisals, and autonomy framing manipulations on behavioral intentions and affective measures.
-
Interpreting and Using School Performance Data: Do's and Don'ts for 2022 and BeyondSep 30, 2022
The accountability provisions of the Every Student Succeeds Act are back in effect after 2 years of pandemic-related disruptions.
-
In COVID-19 Health Messaging, Loss Framing Increases Anxiety with Little-to-No Concomitant Benefits: Experimental Evidence from 84 CountriesSep 26, 2022
The COVID-19 pandemic (and its aftermath) highlights a critical need to communicate health information effectively to the global public.
-
What Should the Future of Educational Accountability Look Like?Sep 14, 2022
Educational accountability for schools based on their performance of their students has been a core element of federal policy since the passage of the federal No Child Left Behind Act (NCLB) two decades ago.
-
Impacts of Home Visits on Students in District of Columbia Public SchoolsNov 23, 2021
This study examined the impacts of structured relationship-building teacher home visits conducted in grades 1–5 as part of a family engagement program in the District of Columbia Public Schools.
-
Predicting Early Fall Student Enrollment in the School District of PhiladelphiaOct 12, 2021
Predicting incoming enrollment is an ongoing concern for the School District of Philadelphia (SDP) and similar districts with school choice systems, substantial student mobility, or both.
-
Using Promotion Power to Identify the Effectiveness of Public High Schools in the District of ColumbiaSep 01, 2021
This study estimated the promotion power of public high schools in the District of Columbia.
-
Exploring the Potential Role of Staff Surveys in School Leader EvaluationAug 02, 2021
The Mid-Atlantic Regional Educational Laboratory partnered with the District of Columbia Public Schools (DCPS) to explore the potential use of teacher surveys in school leader evaluation.
-
How Much Have Students Missed Academically Because of the Pandemic? A Review of the Evidence to DateJul 30, 2021
This report is part of a series that aims to provide a definitive account of the best available evidence on how the COVID-19 pandemic has affected America’s students.
-
Hybrid Instructional Approaches and COVID-19 Infection Trends in a Sample of Pennsylvania School Districts, Fall 2020Jan 08, 2021
This report describes COVID-19 infection trends in a sample of Pennsylvania school districts implementing hybrid instructional approaches in fall 2020, comparing those trends to community-level infection trends.
-
The Transition to ESSA: State and District Approaches to Implementing Title I and Title II-A in 2017–18Dec 08, 2020
This report provides a brief look at the transition toward Every Student Succeeds Act (ESSA), as the foundation for an ongoing assessment of how the law is being carried out.
-
Can Charter Schools Boost Civic Participation? The Impact of Democracy Prep Public Schools on Voting BehaviorNov 01, 2020
This study examines the impact of Democracy Prep on voter registration and participation in the 2016 election.
-
Operating Schools in a Pandemic: Predicted Effects of Opening, Quarantining, and Closing StrategiesSep 16, 2020
To help educators make more informed decisions about the best strategy for reopening schools in their community, Mathematica ran complex models of 108 different school situations, including urban, suburban, and rural settings.
-
Changing the Principal Supervisor Role to Better Support Principals: Evidence from the Principal Supervisor InitiativeJul 21, 2020
In this report, researchers from Mathematica and Vanderbilt University describe the PSI experiences of districts, principal supervisors, and principals; the PSI’s effects on teachers’ perceptions of principals’ performance; and lessons learned from the initiative.
-
Considerations for Reopening Pennsylvania SchoolsJun 29, 2020
As educational institutions plan for re-opening in Fall 2020, the Pennsylvania Department of Education (PDE) needs to provide guidance to assist schools in developing policies and procedures for mitigating the spread of COVID-19 and ensuring the safety of their students and staff.
-
The Promotion Power Impacts of Louisiana High SchoolsMar 05, 2020
This report describes the data and methods used to measure Louisiana public high schools’ promotion power, which is a school’s effect on the long-term success of its students as indicated by high school graduation, college or career readiness, college enrollment and persistence, and earnings.
-
The Promotion Power Impacts of Louisiana High Schools (Executive Summary)Mar 05, 2020
This summary describes a study that measured Louisiana public high schools’ promotion power, which is a school’s effect on the long-term success of its students as indicated by high school graduation, college or career readiness, college enrollment and persistence, and earnings.
-
How Does School Choice Affect Racial Integration?Nov 20, 2019
This brief examines how school choice options, including charter schools, vouchers, magnet schools, district-wide choice, and inter-district choice, affect the racial and economic integration of students.
-
How Does School Choice Affect Student Achievement in Traditional Public Schools?Nov 20, 2019
This brief describes how two types of school choice—charter schools and private school vouchers—affect student achievement in traditional public schools.
-
Changes in HBCU Financial Aid and Student Enrollment After the Tightening of PLUS Credit StandardsJun 19, 2019
The authors analyze changes in financial aid and student enrollment at historically Black colleges and universities (HBCUs) that occurred after the U.S. Department of Education increased the credit history requirements necessary to obtain Parent Loans for Undergraduate Students (PLUS).
-
A Life Lesson in Civics: How Democracy Prep Charter Schools Boost Student VotingApr 11, 2019
Education in the United States has a foundational public purpose: to prepare students for effective citizenship.
-
Year 2 Report of the Atlanta Public Schools Turnaround StrategyFeb 01, 2019
This is the second report on Year 2 of the Strategy, following an interim report (Hallgren et al. 2018) that presented detailed implementation findings in targeted and partnership schools after the second year of the Strategy (2017–2018).
-
The Impact of Democracy Prep Public Schools on Civic ParticipationApr 19, 2018
Using randomized admissions lotteries to conduct an experimental analysis, we find that the Democracy Prep charter-school network produces substantial positive impacts on rates of voter registration and election participation after students become old enough to vote.
-
The Atlanta Public Schools Turnaround Strategy After One Year: High Impact Tutoring and the Purpose Built Schools PartnershipOct 16, 2017
This report is the first of two interim annual reports addressing specific components of the Turnaround Strategy.
-
Changes in Financial Aid and Student Enrollment of Historically Black Colleges and Universities After the Tightening of PLUS Credit Standards: An Update for the 2013/14 School YearFeb 09, 2017
This follow-up study looks at changes in financial aid and enrollment after the summer of 2013, when the Department of Education changed the appeals process for families denied PLUS loans.
-
Implementation of Title I and Title II-A Program Initiatives: Results from 2013-14 (Executive summary)Jan 19, 2017
This report uses survey responses from states, districts, principals, and teachers to examine the implementation of Titles I and II of federal K-12 education policy across the country in spring 2014.
-
Implementation of Title I and Title II-A Program Initiatives: Results From 2013–14Jan 19, 2017
This report uses survey responses from states, districts, principals, and teachers to examine the implementation of Titles I and II of federal K-12 education policy across the country in spring 2014.
-
The Content, Predictive Power, and Potential Bias in Five Widely Used Teacher Observation InstrumentsNov 01, 2016
This study seeks to inform decisions about the selection and use of teacher observation instruments using data from the Measures of Effective Teaching project.
-
Reimagining Accountability in K-12 EducationOct 19, 2016
The authors review rule-based, market-based, and professional accountability alongside outcome-based accountability, using evidence from the laboratory and the field to describe how each can produce favorable or unfavorable effects.
-
Can Student Test Scores Provide Useful Measures of School Principals' Performance?Sep 29, 2016
This report describes findings from a study examining the accuracy of four test-based measures of principal performance for predicting principals’ contributions to student achievement in future years.
-
Performance Measures Based on Test Scores – How Well Do They Predict Principals’ Impacts? (In Focus Brief)Sep 29, 2016
This focus brief summarizes findings from a study examining the accuracy of four test-based measures of principal performance for predicting principals’ contributions to student achievement in future years.
-
Do Principals' Professional Practice Ratings Reflect Their Contributions to Student Achievement? Evidence from Pennsylvania's Framework for LeadershipJun 28, 2016
We examined Pennsylvania’s Framework for Leadership (FFL), a tool for measuring and evaluating principals’ professional practices.
-
Charter High Schools' Effects on Long-Term Attainment and Earnings (Journal Article)Jun 21, 2016
Since their inception in 1992, the number of charter schools has grown to more than 6,800 nationally, serving nearly three million students.
-
Is School Value Added Indicative of Principal Quality? (Journal Article)Jun 21, 2016
In this paper, the authors isolate principals’ effects on student achievement growth and examine the extent to which school value added captures the effects that principals persistently demonstrate.
-
Testing College Readiness: Massachusetts Compares the Validity of Two Standardized TestsJun 21, 2016
The state of Massachusetts introduced a system of standardized testing in its public schools three years before the federal No Child Left Behind Act of 2001 mandated such practices for all 50 states.
-
Value-Added Models for the Pittsburgh Public Schools, 2014-15 School YearMay 03, 2016
At the request of Pittsburgh Public Schools and the Pittsburgh Federation of Teachers, Mathematica has developed value-added models that aim to estimate the contributions of individual teachers and schools to the achievement of their students.
-
Student Selection, Attrition, and Replacement in KIPP Middle Schools (Journal Article)Mar 01, 2016
Skeptics of the KIPP (Knowledge Is Power Program) charter school network argue that these schools rely on selective admission, attrition, and replacement of students to produce positive achievement results.
-
Measuring School Leaders' Effectiveness: Findings from a Multiyear Pilot of Pennsylvania's Framework for Leadership (Stated Briefly)Jan 21, 2016
This study analyzes the score variation, internal consistency, score stability, and concurrent validity of the Framework for Leadership (FFL)—Pennsylvania’s tool for evaluating schoolleaders’ effectiveness—during its pilot implementation in 2012/13 and 2013/14.
-
Measuring School Leaders' Effectiveness: Final Report from a Multiyear Pilot of Pennsylvania's Framework for LeadershipJan 21, 2016
This series of reports examines the accuracy of performance ratings from the Framework for Leadership (FFL), Pennsylvania’s tool for evaluating the leadership practices of principals and assistant principals.
-
Replicating Experimental Impact Estimates With Nonexperimental Methods in the Context of Control-Group NoncomplianceJan 01, 2016
A growing literature on within-study comparisons (WSC) examines whether and in what context nonexperimental methods can successfully replicate the results of randomized experiments.
-
Online Charter Schools' Operational and Instructional Practices: Highlights of Findings (In Focus Brief)Oct 27, 2015
Online charter schools—also known as virtual charters or cyber charters—are publicly funded schools of choice that deliver student instruction via telecommunications.
-
Inside Online Charter SchoolsOct 27, 2015
That National Study of Online Charter Schools is organized into three separate, topical report volumes. In Volume I, Mathematica describes the universe of online charter schools, the students they serve, and their operations.
-
For Massachusetts Students, PARCC and MCAS Exams Comparable in Predicting College Outcomes (In Focus Brief)Oct 13, 2015
A new brief by Mathematica Policy Research shows that students’ scores on the existing high school assessment in Massachusetts predict college performance as well as scores on a new test that was recently developed by a consortium of states to align with Common Core standards.
-
Policy Brief: Calibrating Performance Standards Across State Lines: How PARCC Predicts College ReadinessOct 13, 2015
A new report by Mathematica Policy Research shows that students’ scores on the existing high school assessment in Massachusetts predict college performance as well as scores on a new test that was recently developed by a consortium of states to align with Common Core standards.
-
Predictive Validity of MCAS and PARCC: Comparing 10th Grade MCAS Tests to PARCC Integrated Math II, Algebra II, and 10th Grade English Language Arts TestsOct 05, 2015
A new report by Mathematica Policy Research shows that students’ scores on the existing high school assessment in Massachusetts predict college performance as well as scores on a new test that was recently developed by a consortium of states to align with Common Core standards.
-
Making Effective Use of Postsecondary Data in K-12 Education SettingsSep 30, 2015
Services such as the National Student Clearinghouse StudentTracker for High Schools provide states, districts, and schools with the capability to examine college enrollment, persistence, and completion data on students they previously served in K–12 settings.
-
Alternative Student Growth Measures for Teacher Evaluation: Implementation Experiences of Early-Adopting DistrictsJul 21, 2015
Throughout the country, school districts are scrambling to adhere to new state requirements for teacher evaluation. More than 40 states have mandated that some measure of student achievement growth be included in teacher evaluations.
-
Re-Imagining Accountability in K-12 Education: A Behavioral Science PerspectiveMay 13, 2015
The policy debate on accountability in education can be informed by the literature from behavioral science, which makes clear that accountability comes in many forms and that it can produce positive or negative effects.
-
Changes in Financial Aid and Student Enrollment at Historically Black Colleges and Universities After the Tightening of PLUS Credit StandardsApr 14, 2015
The federal Parent Loans for Undergraduate Students (PLUS) program provides loans to parents of dependent undergraduate students to help pay for education expenses.
-
Stated Briefly: Changes in Financial Aid and Student Enrollment at Historically Black Colleges and Universities after the Tightening of PLUS Credit StandardsApr 14, 2015
The year after the U.S. Department of Education tightened credit standards for Parent Loans for Undergraduate Students (PLUS), both the share of PLUS participants and the loan dollar amounts declined substantially at historically Black colleges and universities (HBCUs).
-
Sensitivity of Teacher Value-Added Estimates to Student and Peer Control Variables (Journal Article)Jan 12, 2015
Teacher value-added models (VAMs) must isolate teachers’ contributions to student achievement to be valid. Well-known VAMs use different specifications, however, leaving policymakers with little clear guidance for constructing a valid model.
-
Measuring School Leaders' Effectiveness: An Interim Report from a Multiyear Pilot of Pennsylvania's Framework for LeadershipDec 17, 2014
This study examines the accuracy of performance ratings from the Framework for Leadership (FFL), Pennsylvania’s tool for evaluating the leadership practices of principals and assistant principals.
-
Mathematica's Evaluation of The Equity Project Charter School: High Salaries for Teachers, Positive Impacts on Student Achievement (In Focus Brief)Oct 24, 2014
This In Focus brief provides a snapshot of the study and findings.
-
Does Student Attrition Explain KIPP's Success?Sep 01, 2014
The Knowledge Is Power Program (KIPP) is a network of charter schools designed to improve the educational opportunities available to low-income families. KIPP schools seek to boost their students’ academic achievement and ultimately prepare them to enroll and succeed in college.
-
A Conceptual Framework for Data-Driven Decision MakingJun 02, 2014
The Bill & Melinda Gates Foundation, which has funded four initiatives promoting strategic use of data at the state and district levels, requested that Mathematica develop a conceptual framework for data-driven decision making (DDDM) based upon knowledge generated from Mathematica’s evaluation of the...
-
Value-Added Models for the Pittsburgh Public Schools, 2012-13 School YearMay 05, 2014
At the request of Pittsburgh Public Schools and the Pittsburgh Federation of Teachers, Mathematica has developed value-added models that aim to estimate the contributions of individual teachers and schools to the achievement of their students.
-
Alternative Student Growth Measures for Teacher Evaluation: Profiles of Early-Adopting Districts (Executive Summary)Apr 30, 2014
This REL Mid-Atlantic study looks at eight districts that have been using alternative measures for teacher evaluations or performance-related compensation systems.
-
Effectiveness of Supplemental Educational ServicesApr 30, 2014
One of the modifications of the Elementary and Secondary Education Act (known as the No Child Left Behind Act) gave parents of low-income students in low-performing schools a choice of Supplemental Educational Services (SEdS).
-
Alternative Student Growth Measures for Teacher Evaluation: Profiles of Early-Adopting DistrictsApr 30, 2014
This study describes how eight early-adopting school districts are using two types of alternative measures for assessing teacher effectiveness: value-added models that use the results of end-of-course assessments or commercially available tests in statistical growth models instead of typical state assessments,...
-
Do KIPP Schools Boost Student Achievement?Jan 30, 2014
This article measures the achievement impacts of 41 Knowledge Is Power Program (KIPP) charter middle schools nationwide and found consistently positive and statistically significant test-score effects in reading, math, science, and social studies.
-
Supporting Effective Data Use (In Focus Brief)Dec 30, 2013
This brief focuses on supporting effective data use. The other two briefs focus on developing a coherent plan for effectively using data and on considerations for developing or enhancing a data system.
-
Developing a Coherent Plan for Effectively Using Data (In Focus Brief)Dec 30, 2013
This brief focuses on developing a coherent plan for effectively using data. The other two briefs focus on considerations for developing or enhancing a data system and on supporting the effective use of a data system.
-
KIPP Middle Schools: Impacts on Achievement and Other OutcomesFeb 27, 2013
This report shows that Knowledge Is Power Program (KIPP) middle schools have significant and substantial positive impacts on student achievement in four core academic subjects: reading, math, science, and social studies.
-
KIPP Middle Schools: Impacts on Achievement and Other Outcomes (Executive Summary)Feb 27, 2013
The Knowledge Is Power Program (KIPP) is a rapidly expanding network of public charter schools whose mission is to improve the education of low-income children.
-
The Impacts of Philadelphia's Accelerated Schools on Academic Progress and GraduationNov 23, 2010
This report evaluates the impacts of Philadelphia’s accelerated schools—alternative high schools serving students at high risk of dropping out—on enrollees’ graduation rates and on rates of credit accumulation for recent enrollees.
-
Teacher and Principal Value-Added: Research Findings and Implementation PracticesSep 14, 2010
This report summarizes research findings and implementation practices for teacher and principal value-added models (VAM) as a first step in the Team Pennsylvania Foundation’s pilot project to inform the development of a full, statewide model evaluation system.
-
Student Characteristics and Achievement in 22 KIPP Middle SchoolsJun 30, 2010
This report presents the first findings from a multi-year evaluation of the Knowledge Is Power Program (KIPP), a network of charter schools for disadvantaged students.
-
Student Characteristics and Outcomes in Alternative and Neighborhood High Schools in PhiladelphiaApr 14, 2010
This report presents descriptive analyses of the enrollment patterns, background characteristics, test scores, and graduation rates of students enrolled in three types of public high schools in Philadelphia: alternative disciplinary schools; alternative accelerated schools for students who struggled...
-
Charter Schools in Eight States: Effects on Achievement, Attainment, Integration, and CompetitionMar 30, 2009
Although charter schools are growing in number, debate continues about whether they provide a better education than traditional public schools.