School shopping websites can play a powerful role in shaping parents’ decisions about the school their children attend, according to a new Mathematica study. Basic design decisions, such as the order in which schools appear and whether data are presented graphically, nudge parents toward one type of school or another and can affect which schools flourish or fail in a school-choice market.
The way school directories and report cards currently provide information varies in both substance and style. For example, some provide lengthy and detailed profiles for each school, and others opt for more concise and modest profiles. Some present information primarily with numbers, and others use graphs or icons such as color-coded letter grades. Mathematica’s paper, funded by the Walton Family Foundation and based on a study for the Institute of Education Sciences, highlights the importance of making conscious choices about information design and considering the effects on choosers’ knowledge, attitude, and behavior.
Existing research has examined how parents choose schools and how information design influences behavior. It also suggests that information matters in school choice. To provide deeper insights into how best to present school choice information to parents, Mathematica, through an online randomized experiment, tested the impact of five design factors on choice of school, understanding, ease of use, and satisfaction:
- Format. School information was shown three ways: using only numbers; combining numbers with icons; or combining numbers with graphs.
- District average. A benchmark of the district’s average value for key measures of school performance was included or omitted.
- Source of information. Parents’ opinions, as an additional source of information for measures related to academic quality and school safety, were included or omitted.
- Amount of information. The total amount of information was shown three ways: low; high; or low with progressive disclosure, which means that the amount of information was low by default and users had the option to click and reveal additional information in the high-information version of the display.
- Default sort order. The default sort order for schools was by distance from home or by academic quality.
The research team asked 3,500 low-income parents of school-age children across the United States to pick from among a list of hypothetical schools using 1 of 72 different variations on a school shopping website that was created for the experiment and assigned at random to each participating parent.
The following results from the study suggest that varying how the information appears can influence the school chosen:
- Changing the default from sort by distance to sort by academics had the strongest effect, leading parents to select a school with higher academic performance even though it might be half a mile farther from home. The difference in academic performance was large—five percentile points—which is equivalent to an improvement of 0.20 standard deviations in the performance of selected schools.
- Changing the default order to sort by academics also led parents to choose schools that were more than half a mile farther from home (2.3 versus 1.7 miles, on average).
- Using icons to represent data instead of graphs or numbers led parents to choose schools with higher academic performance.
- Presenting concise summaries instead of detailed displays also led parents to choose schools with higher academic performance.
- Including parent survey results with other measures of school safety led parents to choose schools with higher ratings on safety.
Design choices influenced parents’ satisfaction and understanding of these websites, although most effects were subtle. Still, there are trade-offs to consider. For example, parents found the numbers-only display easier to understand than icons or graphs, but it also reduced their satisfaction with the website.
“Education officials and designers of school-choice websites should pay attention to how they present information,” said Study Director Steven Glazerman. “Presentation can, intentionally or unintentionally, nudge parents to choose higher or lower quality schools.”
Documents from the study include the following:
“Nudging Parents to Choose Better Schools: The Importance of School Choice Architecture,” Mathematica Working Paper 65.
“Presenting School Choice Information to Parents: An Evidence-Based Guide.”
“Presenting School Choice Information to Parents: An Evidence-Based Guide—Appendix.”
