Tips to Quickly Switch from Face-to-Face to Home-Based Telephone Interviewing

Editor’s note: This post has also been translated into French and Spanish.

Across the United States and around the world, in-person survey data collection has been halted to protect against COVID-19 spread; now what? Researchers are scrambling to make sure we collect data that can be used for policymaking during and after the crisis. Building on Mathematica’s recent experience carrying out telephone surveys internationally, here are some tips for converting a computer-assisted personal interview (CAPI) to a home-based computer-assisted telephone interview (CATI) in resource-poor environments. Researchers in the United States can find some useful tips here, too!

1. Safety first. Connect interviewers with supervisors by phone—no going into offices. Encourage all staff to practice the World Health Organization’s guidelines for hygiene and social distancing.
2. Transitioning to remote data collection requires different equipment and some tweaks to the process. At a minimum:
   • Interviewers and supervisors should have functioning cell phones, dedicated SIM cards, and headsets with microphones to support data collection, as well as for distance training and ongoing coaching and feedback.
   • Interviewers should have reliable internet connectivity from a 4G adapter or a hotspot data plan to transmit surveys.
   • Support interviewers in identifying a quiet, comfortable space to work in their homes, and a safe way to charge phones and tablets.
   • Create case-tracking sheets in Google Docs or develop a short, separate self-administered survey that interviewers can complete and upload daily to track their cases.
   • Create a data security plan that complies with local and international standards.
3. You can’t just use the same questionnaire for remote data collection.
   • Make a concise opening for interviewers to clearly introduce themselves, articulate the purpose of their call, and describe why the respondent should participate.
   • Create an oral consent process (including information on the study) to help respondents decide.
   • Review the questions and drop any that are not essential or cannot be collected over the phone (such as GPS location, interviewer observations of house and neighborhood, and so on).
   • Respondents will not have face-to-face visual clues, so add scripts to help with the transition between modules.
   • Keep interviews to no more than 20 minutes. Ten minutes is better.
   • As always, end the survey with a thank you, but add a statement that a supervisor might call to verify that the survey took place.
   • Create status codes for number not in service, no answer, busy, appointment, breakoffs, and so on. Use your CAPI software to capture appointments and new contact information.
   • Submit all protocol and survey changes to your ethics committee or institutional review board (IRB) and request speedy review.
4. Interviewers need to be retrained with a focus on gaining cooperation through a quick introduction, and supervisors need to prioritize timely feedback.
   • Practice the new concise opening and responses to questions from hesitant respondents. Using trained, in-person interviewers is likely easiest for converting to telephone surveys, but interviewers must master the fast, compelling introduction before contacting sample members.
   • Create FAQ scripts to use with reluctant respondents.
   • Revise training manuals, provide them electronically, and plan Zoom or Skype sessions to retrain. Use mock interviews to practice and role-play repeatedly.
   • Create scripts for use in a wide variety of scenarios, such as when leaving a voice message, when other family members answer the phone, when requested to ensure the legitimacy of the survey, and when dealing with reluctant respondents. Practice them!
   • Test all interviewers’ skills to select correct responses and to speak clearly in each language they will be using for the survey.
   • Create practice interviews, have interviewers upload responses and check for anomalies. Retrain and retest as needed.
   • Provide one-on-one feedback to interviewers as needed; daily, if possible.
   • Create supervisor-moderated groups in tools like WhatsApp to support peer learning and ongoing group training.
5. Test! Test! Test!
   • Ensure all technology works.
   • Test each part of the survey, from the introduction and consent to the questionnaire modules, closing, case status and tracking, and survey synchronization.
   • Pilot the survey. Use a convenience sample if needed to make sure interviewers and supervisors can use the technology correctly and are comfortable with it. Test the question wording, case management protocols, interviewers’ abilities to conduct the survey, and supervisors’ abilities to provide monitoring and feedback.
   • Drop or readjust questions that don’t work, adapt preambles and protocols, and retest.
6. Monitor carefully.
   • Create a back-check interview for supervisors and a method for selecting cases to call back to verify that the survey took place with the correct respondent. Conduct back-checks with at least 20 percent of cases per interviewer. Quickly compare the data between the original survey and the back-check survey and determine a protocol for how to handle discrepancies. 
   • Review key variables daily and look for expected and unexpected patterns. Check durations, which may gradually shorten, but check filter variables quickly and regularly for patterns of selecting no at filters before lengthy modules. Plan extra back-checks when suspicious patterns arise.
   • Explore options for a third-party to listen in on interviews or for recording by discussing directly with your mobile provider. Be sure that the consent script includes this possibility and IRB has approved. If you plan to record interviews, create a protocol for saving, uploading, handling, and reviewing these audio files. Audio files are large; transmitting and reviewing them takes considerable time.
7. Focus on maintaining high response rates.
   • Allow multiple sessions to complete the survey if necessary.
   • For incentives such as phone credit, create a plan for when credit is provided, such as only for reluctant respondents or particularly high-value respondents. Have supervisors or a data collection manager handle the credit process. Work with mobile providers to arrange and track credit. Keep careful records of who receives credit, how much they receive, and the results. Consider adding experiments to your incentive protocol, such as varying the amount of incentives or randomly assigning incentives to the samples.
   • Don’t be afraid to make many attempts on each valid number if necessary. You might be successful after 5 or 10 call attempts (or even more)!
   • Consider an SMS or WhatsApp message to respondents before the interview. Include a phone-in number to a supervisor to set appointments or complete the survey on the spot. Also consider experiments for any advance messaging protocols, such as randomizing which sample members receive SMS or WhatsApp reminders.
   • Depending on the size of the sample, send announcements via local radio programs or sector- or region-specific WhatsApp groups about the survey. Provide a number, Facebook page, or website for respondents to check the validity of the survey, including a link to the official permission letter.
8. To adjust for undercoverage, use statistical techniques such as multilevel regression with poststratification, which we used with Short Message Service (SMS) surveys in Africa and Asia.

With planning, researchers can keep collecting the vital data needed at this unprecedented moment, and interviewers can use phone surveys from the safety of their homes. Mathematica has carried out phone surveys in several African countries with samples of farmers; vocational education graduates; households; and small, medium and large businesses. Questionnaires ranged from 10 minutes to one hour in length. Successful examples include Namibia, where we used airtime as an incentive and achieved a 73 percent response rate among youth a year after graduating from vocational education programs. In Senegal, we used mobile headsets and tablets originally programmed for face-to-face surveys to collect data from rural farmers and completed 80 percent of the 3,500 cases by phone. In Benin, we are moving our team out of the call center and implemented implementing distributed CATI to protect our supervisors and interviewers.

Our commitment to protecting our research teams—while collecting the data necessary for evidence-based policymaking for poverty alleviation—is paramount. For additional resources, the Cross-Cultural Survey Guidelines provide checklists and references on all aspects of data collection. J-PAL and Innovations for Poverty Action have created a checklist for transitioning from in-person to telephone interviews.