Big Data’s impact on policy research is on my mind every day. And, of course, I’m not alone. This was apparent when I attended the recent Association for Public Policy Analysis and Management (APPAM) annual conference, where presenters discussed the role of data in research and policymaking in education, early childhood, health care, housing, employment, and so many other key areas that affect public well-being.
Big Data is not just about quantity - it inspires creativity in research design.
The conference came on the heels of the release of the fall issue of APPAM’s Journal of Policy Analysis and Management, which features a special section on Big Data and public policy that I co-edited with Julia Lane from New York University. This section includes three papers that we selected from about 60 submissions for a Big Data workshop at APPAM’s 2015 fall conference. (The workshop was a great example of the interest in Big Data—we expected 40 to 50 registrants and wound up with 178!)
Technology is transforming the research landscape by enabling us to collect new types of information from sources such as administrative records, mobile devices, and sensors—and by supporting our ability to analyze this tidal wave of data. This was clear from the trend we observed in reviewing the submissions to the journal: Big Data not only provides researchers with more information to analyze, it also profoundly influences how they conceive of and design studies that address critical policy issues. Big Data doesn’t just help researchers find new answers—it inspires us to ask entirely new questions.
Take, for example, two of the papers we selected for the journal. For one paper, “Connections Matter: How Interactive Peers Affect Students in Online College Courses,” the advent of virtual classrooms provided researchers with an original twist on the widely studied topic of peer effects in higher education.
Using data collected from an online introductory psychology course run by DeVry University, the authors tracked peer interactions through more than 400,000 online posts submitted by more than 11,000 students. Among the findings: students in virtual classrooms with more frequent peer interactions have higher academic outcomes, and students in more interactive classrooms also are more likely to enroll and take more credits in the following semester.
These results indicate that interventions to encourage student engagement and interaction may help promote the effectiveness of online learning. The paper also shows how technology can influence the design of a research study, as the access to rich data through online discussion threads—combined with new technology to analyze the discussion data—empowered the researchers to take a novel approach to investigating how students’ interactions affect their educational success.
Another paper featured in this issue, “Empowering Consumers Through Data and Smart Technology: Experimental Evidence on the Consequences of Time-of-Use Electricity Pricing Policies,” also demonstrates how to use newly available data. In this case, researchers analyzed data collected in more than 1,000 households from applications that allow customers to track and respond to fluctuations in prices. Three treatment groups of homes received access to a website that displayed information on their electricity consumption and prices in real time. One group had access to the website only, another received an in-home wireless device that displayed the same information as on the website, and a third was given a smart thermostat programmed to respond automatically to price changes.
Perhaps not surprisingly, the researchers observed that the homes with the smart thermostat recorded the greatest shifts in energy consumption based on pricing changes, even though all the homes had access to the same pricing information. This study provides evidence of the effectiveness of smart technology—and serves as another example of how researchers can use emerging data sources to devise new experiments to better understand behavioral trends and impacts.
As these papers—and many other investigations across the policy research spectrum—show, Big Data is not just about quantity. The technology that gives us an unprecedented ability to collect and analyze rich, timely information is inspiring quality and creativity in research design that can lead to important insights for decision makers charged with serving our society.
