Indonesia: Total Sanitation and Sanitation Marketing Study

The Bill & Melinda Gates Foundation contracted with Mathematica to conduct a retrospective implementation study of the Total Sanitation and Sanitation Marketing (TSSM) program. TSSM was designed to strengthen the supply of and demand for sanitation and improve the enabling environment for sanitation interventions.

Mathematica’s study sought to understand how the TSSM program was implemented in East Java, Indonesia; learn more about how program monitoring data were collected; and reconcile some apparent discrepancies between program monitoring data and impact evaluation data involving program progress. The study used a mixed-methods approach that drew on both qualitative and quantitative data. We relied heavily on qualitative data gathered from a large cross-section of stakeholders at the national, provincial, district, subdistrict, village, and hamlet (sub-village) levels. Mathematica met with members of the implementing team and their partners, as well as sectoral stakeholders at the national and provincial levels, to frame the context of TSSM.

To understand program implementation and outcomes in the field, Mathematica selected six districts and conducted site visits to meet with officials, health staff, and other stakeholders at the district, subdistrict, village, and hamlet levels. We also held focus groups among households from hamlets that received the program and those that did not to learn about determinants of their sanitation behavior. The team selected the communities and districts in a way that ensured they were geographically dispersed and captured a range of levels of progress toward open defecation free outcomes.

Mathematica conducted household surveys of sanitation practices in 36 hamlets, most of which had been visited as part of the qualitative data collection. Twenty randomly selected households in each hamlet were surveyed, leading to a total sample of 720 households. In addition to analyzing this data, the team also analyzed existing monitoring data provided by the Water and Sanitation Program.