Testing Web and Paper Modes on a National Survey of Disability Beneficiaries

In 2023, Mathematica administered Round 8 of the National Beneficiary Survey using computer-assisted telephone interviewing (CATI) technology. To test alternative survey modes, we conducted a random assignment experiment using self-administered web and paper modes with sample members who did not respond to the telephone mode. The findings suggest that the web and paper modes increased the survey’s completion rate, with a larger increase for the paper mode (by 4.6 percentage points). The largest impacts for the web mode were among people ages 18 to 29, women and those with cognitive or intellectual impairments. The largest impacts for the paper mode were among those ages 18 to 29 and those ages 50 and older. Future research might address potential mode effects; questionnaire design, including how to incorporate cognitive screeners in web and paper modes; and the optimal sequence of offering alternative modes before, alongside, or after outreach for a telephone survey. The experiment demonstrates that alternative survey modes can be effective in generating additional responses from a group of resolute nonrespondents.