Next Article in Journal
Systems Approach to Eastern Baltic Coastal Zone Management
Next Article in Special Issue
Field Investigation and Economic Benefit of a Novel Method of Silver Application to Ceramic Water Filters for Point-Of-Use Water Treatment in Low-Income Settings
Previous Article in Journal
Studies on River Training
Previous Article in Special Issue
Biosand Filter as a Point-of-Use Water Treatment Technology: Influence of Turbidity on Microorganism Removal Efficiency
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Water
Volume 12
Issue 11
10.3390/w12113095
water-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Impact Evaluation of an SMS Campaign to Promote Household Chlorination in Rural Haiti

by
Michael Ritter
1,2,3,*,
Eveline Camille
2,
Christophe Velcine
2,
Rose-Kerline Guillaume
2,
Jean Marcel Casimir
2 and
Daniele S. Lantagne
1
1
Department of Civil and Environmental Engineering, Tufts University, Medford, MA 02155, USA
2
Deep Springs International, Léogâne, HT 6210, Haiti
3
Department of Global Studies, Houghton College, Houghton, NY 14744, USA
*
Author to whom correspondence should be addressed.
Water 2020, 12(11), 3095; https://doi.org/10.3390/w12113095
Submission received: 15 September 2020 / Revised: 24 October 2020 / Accepted: 31 October 2020 / Published: 4 November 2020
(This article belongs to the Special Issue Point-of-Use Water Treatment)
Versions Notes

Abstract

:
Despite documented health benefits of household water treatment and storage (HWTS), achieving sustained use remains challenging. In prior evaluations of a long-term HWTS program in Haiti, multiple marketing interventions failed to increase use or had prohibitively high costs. Using mobile phones is a potentially cost-effective way to change HWTS behavior. We conducted a randomized experiment to evaluate the impact of sending short-message service (SMS) messages to promote household chlorination in this program in Haiti. Households (n = 1327) were randomly assigned to: One of four SMS frequencies; one of ten behavioral constructs; “cholera” or “disease” framing; and one or zero household visits from a sales agent. During the three-month campaign, there were no statistically significant relationships between the four outcomes related to chlorine purchases and any SMS frequency, any behavioral construct, or either “cholera” or “disease” framing. Receiving one visit increased the probability of purchasing a bottle of chlorine by 17.1 percentage points (p < 0.001) but did not affect subsequent purchase behavior. Costs of managing the SMS campaign were higher than expected. SMS campaigns may not be cost-effective behavior change interventions in certain contexts. If pursued, we recommend simple interventions, timed with the target behavior, and tailored to mobile phone usage patterns of the target population.

1. Introduction

Worldwide, approximately 844 million people lack access to a basic drinking water service, and an estimated 1.8 billion rely on contaminated water [1]. Household water treatment and storage (HWTS), which includes consumables such as chlorine and durables such as filters, can be a cost-effective means of improving drinking water quality [2] and reducing diarrheal disease [3,4,5,6,7,8,9]. Despite the health benefits of HWTS products, demand remains low. An estimated 33% of households in countries without reliable access to safe water self-report treating their drinking water, and use is lower among those in rural areas, lower socioeconomic groups [10], and those with access to poorer drinking water quality [11].
Some behavior change communication (BCC) strategies have been shown to increase demand for HWTS products, but there is no clear consensus on a single best approach [12]. Implementers must often decide between mass media campaigns (such as radio spots and billboards) and interpersonal communication (such as visits from community health workers), each of which has advantages and disadvantages. While mass media campaigns have been shown to increase awareness and other determinants of HWTS use, evidence for sustained behavior change at scale is mixed [13], and the most vulnerable populations often remain unreached. In contrast, frequent, personal contacts with a health promoter over a period of time have been considered “the most influential program factors associated with sustained adoption” of water, sanitation, and hygiene behaviors [14]. However, these strategies tend to be costly.
One HWTS program began in 2002 in rural Northwest Haiti and has been managed by the non-profit social enterprise Deep Springs International (DSI) since 2008. In rural Haiti, 4.5% of the population has water piped onto their premises [15], and 34.3% of DSI program participants used an improved source in 2010 [16]. Haitian staff manufactures and sells chlorine solution, branded Gadyen Dlo (“Water Guardian” in Haitian Creole). The chlorine dosage (concentration of chlorine added over contact time) used in this program was established via dosage testing in representative drinking waters in Haiti to ensure adequate chlorine residual over time without exceeding chlorine residual and disinfection byproduct standards, as described in Wilhelm et al. 2018 [17]. This dosage has been shown to lead to drinking water that meets World Health Organization standards for disinfection byproducts [18,19]. One bottle treats 185 gallons (700 L) of water and lasts a household 1–2 months. Community sales agents purchase chlorine at discounted rates, resell to users for 25 gourdes ($0.53 USD at the time of the study), conduct household visits, and collect data on correct use of chlorine.
A 2010 evaluation found that 56% of participants (vs. 10% of non-participants) had chlorine residual in their water during an unannounced visit, and children under 5 whose parents were participants had 59% reduced odds of caregiver-reported diarrhea [16]. However, the average household purchased less than one bottle of chlorine every three months, which was insufficient to ensure high levels of chlorine use and to achieve full cost recovery. Previous evaluations of BCC strategies in this program showed that household visits increase chlorine purchases, but their costs can be prohibitive [20], and that price subsidies are the most cost-effective way to increase use [21]. The combination of high prior exposure to BCC campaigns and a persistent gap between actual and ideal purchase rates makes this population unique to trial BCC strategies.
One BCC strategy that is becoming more widely used is mobile phones for health (mHealth). Mobile phone subscriptions have grown rapidly in recent years and now outnumber the global population [22]. An estimated 60.0% of Haitians have a mobile phone [23]. In systematic reviews of mHealth as a BCC strategy in developing countries, a positive impact was found in 21 of 26 studies on vaccination campaigns [24], 7 of 14 studies on physical activity [25], and 8 of 11 studies on healthy diets [25]. Despite the potential to be a cost-effective BCC strategy, to our knowledge mobile phones had not been used for changing HWTS behavior in Haiti at the time of our study.
It is plausible that mHealth can be a quick way to achieve frequent, targeted contact with HWTS consumers at a lower cost than interpersonal strategies like household visits from health agents, especially in rural areas where transportation costs are high. We evaluated an SMS campaign intended to increase chlorine purchase and use in the DSI program. The primary objective of this research was to compare the effectiveness (defined as chlorine purchase and use) of a less resource-intensive untested BCC strategy (sending SMSs) with a resource-intensive interpersonal communication strategy (household visits) in the context of a long-term program.

2. Materials and Methods

2.1. Treatment Groups

In 2016, in preparation for an SMS campaign, DSI-instructed sales agents spread throughout six communes in the Northwest and Upper Artibonite Departments to record telephone numbers of households on their quarterly reports. Sales agents collected phone numbers during sales transactions, household visits, and community meetings. In March 2017, we conducted an experiment with all households that provided a telephone number and consent to participate in the study (n = 1327), which was 22% of the total households in the database of chlorine users developed by DSI since 2002 (n = 6086).
We used a crosscut (factorial) randomization design to assign households to treatments along four dimensions: (1) Frequency of SMSs sent; (2) construct of SMSs; (3) “cholera” vs. “disease” framing; and (4) household visit from sales agent.
The first treatment dimension was the frequency of SMSs sent, which had four levels: (1) Weekly (12 SMSs in 3 months); (2) bi-weekly (6 SMSs); (3) monthly (3 SMSs); and (4) control group (0 SMSs).
The second treatment dimension was the constructs on which the content of SMSs were based. Prior research on household water treatment behavior in rural Haiti identified several determinants of chlorine purchases [26,27]. Key constructs from these studies and from behavioral theory were presented to DSI staff, who developed a set of messages they believed would increase chlorine purchases: (1) Health risks; (2) health benefits; (3) financial risks; (4) financial benefits; (5) productivity risks; (6) productivity benefits; (7) self-efficacy; (8) product accessibility; (9) social norms; and (10) maternal instincts. The first six constructs consisted of three themes (health, finances, and productivity) that were each framed both positively (“benefits” of treating one’s water) and negatively (“risks” of not treating water). For each of these and the other four constructs (total of ten constructs), DSI staff designed three messages in Haitian Creole that were less than 160 characters (Appendix A). These three versions used different language to communicate the same construct. Each version was scheduled to be sent once to the monthly SMS group, twice to the bi-weekly group, and four times to the weekly group. Thus, each household received all three messages for the construct to which they were randomized (sometimes more than once) but never received any messages from other constructs.
The third treatment dimension was the language used to frame health risks. One group received SMSs containing the word “cholera,” and the other received SMSs containing the word “disease.” Each SMS contained one occurrence of either “cholera” or “disease,” except for the SMSs for the health risk construct, which had three occurrences.
The fourth treatment dimension was a household visit by a sales agent. Each quarter, DSI supervisors assigned a fraction of households in the database for agents to visit once. The objectives were to promote the product, evaluate correct product use, and incentivize agents to sell more chlorine. All sales agents had previously received training on how to educate households about the importance of chlorinating their water, though they were not given a script for how to communicate during their visits. Sales agents were not instructed to visit households in the no-visit treatment, but they were instructed to record purchase data when any household came to purchase from them.

2.2. Randomization Process

To conduct the randomization, we separated all households with a phone number into blocks by sales agent. For each sales agent, we assigned each household an equal probability (25%) of receiving SMSs at each of the four frequencies (including the control group). Among households not in the control group, we assigned them equal probability (10%) of receiving one of ten constructs and an equal probability (50%) of receiving messages with one of two words (“cholera” or “disease”).
Within sales agent blocks, we assigned each household an equal probability of receiving or not receiving a household visit from their sales agent (Table 1). For 17 of 22 sales agents, each household was assigned a 50% probability of receiving a visit in order to balance the size of the visit and no visit groups. For 5 sales agents, we assigned 100% of households to receive a visit because of the constraints placed on the experiment by DSI’s management system (the number of households assigned to receive visits was determined by agents’ sales volume during the previous quarter). We accounted for this in our analysis by conducting additional analyses including only those households that had a 50% probability of being assigned to receive a visit.

2.3. Design and Implementation of SMS Campaign

We established a calendar such that households only receiving three SMSs would receive them on three different days of the week and at three different times of the day. Households that received SMSs more frequently also received SMSs on these days, plus additional days.
We used FrontlineSMS, a cloud-based platform, to send SMSs. Telephone numbers were uploaded and grouped based on the frequency and content of the intended SMSs, as well as which mobile carrier company the phone belonged to. We installed the FrontlineSMS app on two Android telephones that were used exclusively for communicating with study participants. One phone was dedicated to each of the two mobile carriers in Haiti, to reduce the costs of sending SMSs.
On scheduled SMS days, phone credit was added to the phones, and they were synced with the cloud via wireless internet connection at the DSI headquarters in Leogane, Haiti, which allowed data to flow in two directions. First, syncing downloaded the schedule of SMSs to the phones and executed the sending of SMSs to the intended numbers. Second, after the SMSs were sent and the telephone was synced again, data on which SMSs were successfully sent was uploaded to the cloud. We downloaded reports from the FrontlineSMS cloud-based platform for analysis of the SMSs sent.

2.4. Data Collection

The experiment took place from April to June 2017. DSI compensated sales agents at a defined rate based on the number of visits they completed, and sales agents turned in paper reports at the end of the three-month study. Forms for households that did not receive a visit included purchase date(s) and volume of chlorine purchased. Forms for households that received a visit included these variables, the visit date, and the chlorine residual test result. Agents tested samples of water that the household would drink for total chlorine residual (TCR) using a pool test kit with the indicator orthotolidine (OTO) and reported the presence or absence of TCR (Oreq Corporation, Temecula, CA, USA).
Sales agents turned in paper forms to DSI supervisors, who verified whether the total volume of chlorine sales shown in the agent reports matched with administrative records of chlorine volume purchased by the agent for resale. The research team entered and cleaned household data in Microsoft Excel (Redmond, WA, USA), checked for data quality, and analyzed using Stata 14.1 (StataCorp LP, College Station, TX, USA). Tufts University Institutional Review Board approved the study protocol (1605021) in conjunction with the Comité National de Bioéthique in Haiti (1516-53).

2.5. Analytic Approach

We assessed five outcomes: The probability of having purchased at least one, two, and three bottles; total bottles purchased; and probability of having a positive chlorine residual test. The probability of having purchased at least once measured purchases on the extensive margin and indicated whether or not households purchased at all during the experiment. The next two outcomes, the probability of having purchased two or three bottles, provided insight into whether the household likely had sufficient chlorine to treat all drinking water during the experiment. The total bottles purchased measured purchases on the intensive margin, and was an indicator of demand and was important for tracking program goals. Finally, the probability of having a positive chlorine residual test was a measure of product use, rather than purchase, among the sub-sample of households that received visits. With respect to all five outcomes, we tested for the impact of three types of treatments, described in separate sections below: (1) SMS frequency (Section 2.5.1); (2) constructs and framing (Section 2.5.2); and (3) household visits (Section 2.5.3).

2.5.1. Impact of SMS Frequency

To estimate the impact of sending SMSs on chlorine purchase and use, we regressed each of these five outcomes on the main treatment variable, SMS frequency. In Equations (1)–(3) below, we constructed linear probability models in which we replaced purchaseha with five dependent variables (four corresponding to rates of purchase, and one corresponding to the rate of use, as described above in Section 2.5). Our main specification included dummy variables for each frequency: 3 SMSs (weekly), 6 SMSs (bi-weekly), and 12 SMSs (monthly), with 0 SMSs (control) as the reference group. We included a dummy variable for being assigned to receive a visit (for all but the chlorine use outcome, for which we only had data for those that received a visit) and sales agent fixed effects:
purchaseha = α0 + β1 threeSMSh + β2 sixSMSh + β3 twelveSMSh + β4 visith + agenta + εha
To check for robustness and different types of relationships between SMSs and outcomes, we used two alternative formulations of the SMS frequency variable. First, we dichotomized it (at least one SMS vs. no SMS) to test whether SMSs had an impact at any frequency:
purchaseha = α0 + β1 atleastoneSMSh + β2 visith + agenta + εha
Second, we treated frequency as a continuous variable (total number of SMSs) to test for a linear relationship:
purchaseha = α0 + β1 totalSMSsh + β2 visith + agenta + εha
Our main approach used intention-to-treat (ITT) analysis to estimate these equations. To account for implementation challenges, we also presented treatment-on-treated (TOT) estimates. To obtain TOT estimates of Equation (1), we replaced threeSMS, sixSMS, and twelveSMS with dummy variables for the actual number of SMSs sent: (1) 1–5 SMSs; (2) 6–8 SMSs; and (3) >8 SMSs, respectively, with 0 SMSs sent as the reference group. To obtain TOT estimates of Equations (2) and (3), we substituted actual values for atleastoneSMS and totalSMS, respectively.

2.5.2. Impact of Constructs and Framing

To determine whether message constructs had an effect on purchases and use, we regressed each of our five outcomes on the message constructs, with a dummy variable for each of the ten constructs (with no SMSs as the reference group), controlling for visit treatment and sales agent fixed effects. Due to the multiple hypotheses being tested, we reported adjusted p-values for all coefficients using Holm’s method [28]. Given that we tested for five outcomes, this analysis with ten constructs meant that we were conducting 50 tests, and the Holm’s method adjusted for the fact that we were likely to find at least one p-value < 0.05, even if the null hypothesis was true.
Our messages also contained two types of framing. First, to test for a difference between the “cholera” and “disease” language (a treatment variable), we restricted our sample to households assigned to receive at least one SMS. We regressed our outcomes on a dummy variable for being assigned to receive cholera language, controlling for visit treatment and sales agent fixed effects.
Second, to test for a difference between positive and negative (a meta-construct embedded in the first six message groups), we restricted our sample to households assigned to one of the six groups that received either positive or negative messages. We regressed our outcomes on a dummy variable for the positive frame, controlling for visit treatment and sales agent fixed effects.

2.5.3. Impact of Household Visit

In all models described above, we included a dummy variable (visit) for whether the household was assigned to receive a household visit. The coefficient of visit gave an estimate of the effect of being assigned to receive a visit. We focused our estimate of visit impact on the coefficients of visit in Equations (1)–(3) above (for example, β4 in Equation (1)).
As described above, all households served by 5 of the 22 sales agents were assigned to receive a visit, as opposed to the 50% probability of being assigned to a visit among the other 17 agents. To account for potential bias and check for robustness of our results with respect to visit impact, we conducted an additional analysis excluding households that did not have a 50% probability of being assigned to receive a visit.

2.5.4. Multivariate Model

As a final test, we regressed each of the outcome variables on the full set of treatment variables, including dummy variables for each SMS frequency; for each of the ten constructs; for cholera language; and for household visits. We also included sales agent fixed effects. Given the multiple hypotheses being tested, we reported adjusted p-values for all coefficients using Holm’s method [28].

3. Results

We sent 7326 SMSs throughout the three-month experiment, which was 97% of the 7560 intended messages based on assigned SMS frequencies. The actual number of SMSs sent deviated by one SMS or less from the intended number of SMSs for 89.0% of households. When values of the four assigned frequencies were compared to values of four actual frequencies using the category definitions described above, all but 20 households (1.5%) belonged to the same corresponding group.

3.1. Checking Assumptions about Purchase and Use

During our three-month study, 56.5% of households purchased at least one bottle of chlorine; 29.7% purchased at least two bottles; and 2.5% purchased at least three bottles (Dataset provided in Supplementary Materials). The average number of purchases per household over the three months was 1.08. Of households that received a visit and for which a chlorine test result was recorded (n = 667), 86.2% of tests were positive.
To determine the relationship between purchase and use, we conducted a preliminary model of the probability of having a positive chlorine test as a function of chlorine purchases, among the subsample that received visits and for which chlorine test results were recorded by sales agents (Table 2). Households that purchased at least one bottle were 31.2 percentage points more likely to have a positive chlorine test than those that purchased less than one bottle (96.2% positive tests among purchasers vs. 65.0% positive tests among non-purchasers) (Table 2, Column 1). Similarly, those who purchased at least two bottles were 23.9 percentage points more likely to have a positive test than those that did not (Table 2, Column 2), and those who purchased at least three bottles were 26.3 percentage points more likely to have a positive test than those that did not (Table 2, Column 3). On average, for every bottle purchased, the probability of having a positive chlorine test increased by 11.2 percentage points (Table 2, Column 4). Note that households with positive tests that did not purchase may have had a supply of chlorine left over from before the experiment, or may have obtained chlorine from a source other than Gadyen Dlo sales agents who participated in the study.
Below, we present results for each of the three categories of treatment variables described in the three sections above: (1) Frequency of SMSs; (2) constructs and framing; and (3) household visits.

3.2. Impact of SMS Frequency

There was no statistically significant relationship between any of the assigned SMS frequencies and any of the four purchase-related outcomes (Table 3, Panel A, Columns 1–4). Among households that received visits, those assigned to receive SMSs at the lowest frequency were 3.7 percentage points more likely than the control group to have positive chlorine residual (p = 0.049; Table 3, Panel A, Column 5, Row 1), compared to a rate of 84.2% positive tests among control households (value of constant in Table 3, Panel A, Column 5). However, there was no statistically significant difference in the likelihood of the fifth outcome (having a positive chlorine test) between the control group and households assigned to receive SMSs at higher frequencies (Table 3, Panel A, Column 5).
In models with the dichotomous specification, being assigned to receive at least one SMS had no statistically significant relationship with any outcome (Table 3, Panel B), which aligns with the lack of consistent evidence for impact in the specification with dummy variables.
In models that posit a linear relationship between number of SMSs and outcomes, the intended number of SMSs had a slightly negative relationship with the probability of purchasing at least one bottle (p = 0.038; Table 3, Panel C, Column 1) and the total number of bottles purchased (p = 0.023; Table 3, Panel C, Column 4). This is the opposite direction than hypothesized, but the magnitudes are small. For example, for every additional SMS assigned, households purchased 0.015 bottles less (a 1.5% decrease) and were 0.5 percentage points less likely to have purchased at least one bottle, relative to a 50% mean probability of purchasing at least one bottle (Table 3, Panel C).
TOT models present a similar picture with respect to magnitudes and statistical significance of coefficients (Table 4). Considering all of these models together, we did not find evidence of a positive impact of our SMS treatments on purchases. We found minimal evidence of a small positive impact of low frequency SMSs on chlorine use, but not high frequency SMSs.

3.3. Impact of Constructs and Framing

None of the ten coefficients of message constructs had p-values less than p = 0.10 when adjusted for multiple comparisons using Holm’s method (Table 5). In unadjusted models (not shown here), there were two constructs that had relationships with at least one outcome that were significant at the p = 0.05 level (the health risk construct was positively associated with having purchased at least one bottle and the probability of having chlorine, and the mothers construct was negatively associated with having purchased at least two bottles). However, when taking into account the number of comparisons made, we cannot confidently reject the null hypothesis that none of the message constructs made an impact on chlorine purchase or use.
There was no statistically significant difference in any outcome between households that received “cholera” language and those that received “disease” language (Table 6, Panel A). There was no statistically significant difference in any outcome between households that received positive and negative messages (Table 6, Panel B).

3.4. Impact of Household Visits

Being assigned to receive household visits increased the number of bottles purchased by 33.9% (p = 0.002; Table 3, Panel A, Column 4), the probability of having purchased at least one bottle by 17.1 percentage points (p = 0.001; Table 3, Panel A, Column 1), and the probability of having purchased at least two bottles by 8.8 percentage points (p = 0.004; Table 3, Panel A, Column 2). Being assigned to receive visits had no statistically significant impact on the probability of having purchased three bottles (Table 3, Panel A, Column 3).
Upon further analysis, the increases in the number of bottles purchased and the probability of having purchased at least two bottles were driven almost entirely by the purchase of the first bottle. Among households that purchased at least one bottle, the difference in probability of purchasing a second bottle between those that received a visit and those that did not was small and was not statistically significant (2.4 percentage points, p = 0.209) (Table 7, Panel A, Column 2). Likewise, there were no statistically significant relationships at the p = 0.10 level between receiving a visit and the number of bottles purchased beyond the first bottle (Table 7, Column 3). All of these findings about visit impacts were robust to the exclusion of households that did not have a 50% probability of being assigned to receive a visit (Table 7, Columns 4–6).

3.5. Treatment Effects in Multivariate Models

Household visits were the only variable that was statistically significant in the multivariate model. Being assigned to receive household visits increased the number of bottles purchased by 0.319 (p = 0.039, Table 8, Column 4), the probability of purchasing by 17.1 percentage points (p = 0.010, Table 8, Column 1), and the probability of having purchased two bottles by 9.0 percentage points (p = 0.058, Table 8, Column 2). None of the variables for SMS frequency, constructs, or cholera framing were statistically significant at the p < 0.10 level. It is notable that this was true even without using Holm’s correction. In contrast, the three relationships described above with household visits were significant at the p < 0.01 level without using Holm’s correction.

4. Discussion

We hypothesized that an SMS campaign would be a cost-effective way to increase chlorine purchase and use in a population with prior exposure to information campaigns, even if the magnitude of impact was relatively low. However, we did not find evidence of a positive impact of sending SMSs on chlorine purchases in a long-term program in Haiti. Sending monthly SMSs increased the probability of having positive chlorine residual, but the effect was small (3.7 percentage points, p = 0.049, Table 3) and was not evident among those assigned to receive a higher frequency of SMSs. We did find that receiving one visit by a sales agent increased the probability of purchasing a bottle of chlorine by 17.1 percentage points (p < 0.001, Table 3), but a visit did not appear to affect purchase behavior after that purchase. Stakeholders considering using SMS as a BCC strategy in similar contexts should carefully address issues of mobile phone usage patterns of the target population, timing of the target behavior, simplicity of the intervention, and total costs of the campaign.
Some aspects of the way in which our target population uses mobile phones may have prevented decision-makers from receiving SMSs. Phone sharing is common, which can be viewed as an adaptation to the facts that 51% of the adult population is illiterate, school attendance is lower in rural than urban areas [29], and 83% of the rural population does not have access to electricity [30]. The impact of SMSs on household decisions would be muted if decision-makers were not in possession of the phone when the SMS was received, if the viewer ignored or deleted the SMS, or if an illiterate individual viewed the SMS. In addition, the decisions to purchase chlorine and add it to drinking water on a regular basis may be split within the household, which means that reading the SMS may have an impact on one behavior but not the other. These issues played a larger role than we anticipated, and future studies would benefit from verifying whether SMSs were read.
Even if there was a literate person responsible for purchasing chlorine and treating water that read the SMSs, our design might not have been timed in such a way that changed those behaviors. Jones et al. applied the stages of change theory to their experiment with health worker behavior and suggested that SMSs operated primarily by providing a prompt to action for behaviors that health workers know about but sometimes forget to do [31]. A successful prompt to action in our context would mean a reminder to purchase when the household runs out of chlorine (every 1–2 months) and a reminder to add chlorine to drinking water (several times per week). The frequencies we tested may not have occurred at the right timing in order to motivate them to purchase.
We found Krah and de Kruijf’s assertion that “the effectiveness of mHealth interventions depends on either the success of rare efforts to address circumstantial complications, or the possibility to avoid (many of) these complications through simple design and limited objective” [32] to be illustrative for interpreting our results. Our intervention seems to have fallen in between these two approaches. We used a complicated set of lengthy messages with the aim of changing behaviors that have been difficult to achieve in a population with prior exposure to more intense types of BCC campaigns, but our attempts to address circumstantial complications were not based on formal research about mobile phone usage patterns. In order to make an SMS campaign more effective, it is likely that we would need to either make substantial investments in understanding communication preferences and patterns (perhaps through qualitative and participatory research), or simplify the design to avoid lengthy messages and focus on a particular sub-population.
Our results were similar to others that found no evidence of additional impact by sending longer messages. One study found no difference in the impact on adherence to antiretroviral treatment in Kenya between individuals that received short (23 characters; “This is your reminder.” in Swahili and Dhouluo) vs. long messages (50 characters) [33]. Another found no impact on adherence to antimalarial treatment in Ghana of adding a phrase (79 characters) to a simple message (31 characters; “Please take your malaria drugs!”) [34]. The authors suggest that different message content and length can play different roles among different population subgroups. We chose relatively long messages (120–160 characters) and a complicated message scheme because of the study population’s prior exposure to information about HWTS. The length of our messages might have increased the frequency of recipients dismissing them and decreased their effectiveness among subgroups with lower literacy levels.
We note that our statistical power to detect differences between different constructs was limited for some of the hypotheses that we tested. We had sufficient power to detect a reasonable impact on the number of bottles purchased per household for our key comparisons: Between households that received SMSs at three different frequencies (n = 331, 335, and 336) relative to those that received no SMSs (n = 333). Under the least optimistic assumptions, these sample sizes were sufficient to detect a 30–40% increase in bottles purchased for each frequency relative to the control. However, we did not have sufficient power to a 50% increase in the share of households that purchased at least three bottles (3–4 percentage points relative to 5.7–6.4% baseline). While the complexity of the message scheme should not have prevented us from detecting an impact of sending SMSs in our main regressions, a simpler design with less constructs would have made it possible to capture the variation we wanted to measure and detect reasonable effect sizes in all dimensions of the study.
In addition to our failure to find positive impact, the total costs of the SMS campaign were higher than expected. We hypothesized that an SMS campaign would be more cost-effective than existing strategies, partly because sending SMSs is relatively inexpensive. Compensation paid to a sales agent to conduct one household visit was more than six times the price to send a weekly SMS for three months. However, the indirect costs of managing the SMS campaign were substantial. Collecting telephone numbers of program households required more than a year of work by sales agents, even within the context of a program that was already paying sales agents to visit participants and complete written reports. Despite this investment, we likely still did not obtain 100% of the numbers that were available. Sending SMSs required significant time on the part of the research team, especially coordinating the FrontlineSMS cloud-based platform with the SIM cards and troubleshooting separately for each of the two major Haitian mobile phone carriers. While we did not conduct a formal cost analysis, we suspect that the costs of our pilot were actually more expensive than other BCC strategies such as household visits when all costs are included.
Our findings on the impact of household visits suggest that the impact of the visit occurs during the visit itself and is not sustained beyond that interaction. Some mechanisms by which visits may impact purchase behavior that are consistent with our results are: By making it more convenient to purchase through decreased transportation costs, by providing a cue to purchase, or by persuading consumers to purchase while the agent is in the home through social pressure. Our results provide evidence against other potential mechanisms in which consumers’ beliefs or preferences are shifted, since we would expect this to lead to greater subsequent purchases after the visit. These results align with previous research on household visits with this population, which found that factors unique to certain agents are likely more important drivers of visit effectiveness than their information content [20]. Further study is warranted of the mechanisms by which personal contact affects demand for HWTS.
Several aspects of the way in which our sample was generated limit the application of our findings to other populations. First, the DSI household database contained all households served by the DSI program, but this is not necessarily a representative sample of all households in the region. Second, households that owned phones likely differed from program households that did not own phones (e.g., likely had higher socioeconomic status and lived closer to towns or main roads, given patterns of phone ownership in Haiti). Both of these factors would likely mean that our sample had a higher baseline rate of chlorine purchases than other populations in rural Haiti, which could have made it more difficult to detect a positive impact.
In addition, the set of households for which phone numbers were recorded was likely not fully representative of all program households that owned phones. Phone numbers were available for 22% of all households in the DSI database, which is less than the estimated national telephone ownership (60%) [23]. While it is likely that phone ownership in this rural, remote region is less than the national average, it is also likely that there were some households that owned a telephone, but these numbers were not recorded by a sales agent.
Our population is unique in that many households had high levels of prior exposure to HWTS promotion campaigns. All households in our sample had prior contact with a chlorine sales agent, through household visit(s) and/or community education sessions; many had purchased the product previously; and a few had been purchasing for up to 13 years. Populations with less exposure to the product of interest may not respond in similar ways. We suspect that our study population is less sensitive to marketing strategies, and that information campaigns may have larger effects among populations with less prior exposure.
We remain cautiously optimistic that future SMS campaigns that send simple, targeted messages might make a greater impact on chlorine purchase and use. However, we suspect that achieving impact would require a considerable amount of field-testing, refining, and targeting. Executing such a campaign well would likely require substantial resources, so we are no longer optimistic that such a campaign would be significantly more cost-effective than other marketing options. Considering the range of marketing options that have been evaluated with this program in Haiti, price reductions appear to be the most cost-effective way to increase chlorine purchase in this particular population [21]. Our results should be interpreted as a challenge to assumptions that SMS campaigns are simple and cost-effective BCC strategies for populations that are difficult to reach. Implementers should exercise caution before conducting mHealth interventions for behavior change and ensure that they consider mobile phone usage patterns, timing of messages, and substantial costs.

Supplementary Materials

The following are available online at https://www.mdpi.com/2073-4441/12/11/3095/s1, Dataset (Microsoft Excel) and statistical code (Microsoft Word) written for use with Stata 14.2.

Author Contributions

Conceptualization, M.R., E.C., C.V., R.-K.G., J.M.C., D.S.L.; methodology, M.R., D.S.L.; formal analysis, M.R.; writing—original draft preparation, M.R.; writing—review and editing, D.S.L.; project administration, E.C., R.-K.G.; funding acquisition, M.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research was partially funded by the National Science Foundation, IGERT grant 0966093.

Acknowledgments

The authors would like to thank Kelsey Jack and Clair Null for helpful input on analysis, and Samuel Roseme for data analysis.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Appendix A. List of SMS Messages by Construct

ConstructMessages (a–c; see note below about words underlined in italics)
(1) Health risks
  • Sickness can cost you your life. If you don’t use Gadyen Dlo, you can get sick. Protect yourself against disease with Gadyen Dlo!
  • Waterborne diseases are still here. When you don’t treat water with Gadyen Dlo, diseases will celebrate. Protect yourself against disease with Gadyen Dlo!
  • If you don’t treat water with Gadyen Dlo, you won’t have security. Gross diseases like typhoid and diarrhea can enter your house. Protect yourself against disease with Gadyen Dlo!
(2) Health benefits
  • Health gives you tons of advantages. If you use Gadyen Dlo, you have less chance to get sick. Stay healthy with Gadyen Dlo!
  • Waterborne diseases are still here. When you treat water with Gadyen Dlo, they won’t celebrate. Stay healthy with Gadyen Dlo!
  • Treating water with Gadyen Dlo puts your life in security. Gross diseases like typhoid and diarrhea will stay out of your house. Stay healthy with GD!
(3) Financial risks
  • When you get sick, you have to go to the doctor. If you don’t treat water with Gadyen Dlo, you will spend lots of money. Avoid hospital expenses with GD!
  • Hospital expenses are never small. When you don’t treat water with Gadyen Dlo and you get sick, the hospital will make a lot of money off of you. Avoid hospital expenses with Gadyen Dlo!
  • If you don’t treat water with Gadyen Dlo, your money won’t be safe. You could pay lots of money at the doctor because getting sick is expensive. Avoid hospital expenses with Gadyen Dlo!
(4) Financial benefits
  • When you’re healthy, you don’t have to go to the doctor for sicknesses. If you use Gadyen Dlo, you have less chance of spending money on a doctor. Invest in your family with Gadyen Dlo!
  • Hospital expenses are never small. But you can protect yourself against disease for only 25 gourdes. Invest in your family with Gadyen Dlo!
  • Treating water with Gadyen Dlo keeps your money safe. Disease and large hospital expenses won’t happen to you. Invest in your family with Gadyen Dlo!
(5) Productivity risks
  • When you get sick, you can’t do all your activities. If you don’t use Gadyen Dlo, you can’t do everything you have to do. Don’t lose a day of your activities with Gadyen Dlo!
  • Health is the greatest wealth. When you don’t treat water with Gadyen Dlo, time will pass you by while disease keeps you from your activities. Don’t lose a day of your activities with Gadyen Dlo!
  • If you don’t treat water with Gadyen Dlo, your activities won’t have security. Disease can take your workdays from you. Don’t lose a day of your activities with Gadyen Dlo!
(6) Productivity benefits
  • When you’re healthy, you can do all your activities. If you use Gadyen Dlo, disease won’t stop you from doing everything you have to do. Stay strong in your activities with Gadyen Dlo!
  • Health is the greatest wealth. When you treat water with Gadyen Dlo, you can do all your activities without disease stopping you. Stay strong in your activities with Gadyen Dlo!
  • Treating water with Gadyen Dlo places your activities in security. You can stay in your activities without sickness. Stay strong in your activities with GD!
(7) Self-efficacy
  • You have the capacity to kill disease with Gadyen Dlo. They are no match for you when you treat your water. It’s easy to take control of your health with GD!
  • Preventing disease with Gadyen Dlo is easy! You just add a capful of Gadyen Dlo to 5 gallons of water and wait 30 min. It’s easy to take control of your health with Gadyen Dlo!
  • Diseases can’t stop you when you treat your water with Gadyen Dlo. You can gain your independence from disease. It’s easy to take control of your health with GD!
(8) Product accessibility
  • You don’t have to get sick because Gadyen Dlo is available in your zone! Call the Gadyen Dlo supervisor at yyyy-xxxx to know how you can buy your GD.
  • It’s not hard to prevent disease because Gadyen Dlo is near you. Call the Gadyen Dlo Supervisor at yyyy-xxxx to know how you can buy your Gadyen Dlo.
  • Now there aren’t any problems for you to get Gadyen Dlo that prevents disease. Call the GD Supervisor at yyyy-xxxx to know how you can buy your GD.
(9) Social norms
  • Gadyen Dlo is selling fast in your zone because everyone is running to buy it. Many people here treat their water so they don’t get sick. All the neighborhood uses Gadyen Dlo—do like they do!
  • According to the recent follow-up Gadyen Dlo did in your zone, more than 80% of people treated their water well. We can cut off disease if we get to 100%. All of the neighborhood uses Gadyen Dlo—do like they do!
  • Many people in your zone use Gadyen Dlo so they don’t get sick. Don’t let your neighbors leave you behind. All of the neighborhood uses Gadyen Dlo—do like they do!
(10) Maternal instincts
  • Do you want to be a good mother? With Gadyen Dlo, your kids won’t get stomachaches. Treat your kids well with Gadyen Dlo!
  • If you want your kids to grow up well without disease, treat all the water they drink with Gadyen Dlo. That way they won’t miss any school days. Treat your kids well with Gadyen Dlo!
  • Did you know that waterborne diseases can prevent kids from realizing their dreams? Treat your kids well with Gadyen Dlo!
NotesEach construct had three messages (versions of that construct), labeled a, b, c; each household received message a, b, and c at least once.
Words underlined in italics replaced with “cholera” for the cholera framing.

References

  1. World Health Organization. Progress on Drinking Water, Sanitation, and Hygiene: 2017 Update and Sdg Baselines; World Health Organization: Geneva, Switzerland, 2017; Available online: http://www.who.int/water_sanitation_health/publications/jmp-2017/en/ (accessed on 2 November 2020).
  2. Clasen, T.; Haller, L. Water Quality Interventions to Prevent Diarrhoea: Cost and Cost-Effectiveness; World Health Organization: Geneva, Switzerland, 2008. [Google Scholar]
  3. Fewtrell, L.; Kaufmann, R.B.; Kay, D.; Enanoria, W.; Haller, L.; Colford, J.M. Water, sanitation, and hygiene interventions to reduce diarrhoea in less developed countries: A systematic review and meta-analysis. Lancet Infect. Dis. 2005, 5, 42–52. [Google Scholar] [CrossRef]
  4. Clasen, T.; Schmidt, W.P.; Rabie, T.; Roberts, I.; Cairncross, S. Interventions to improve water quality for preventing diarrhoea: Systematic review and meta-analysis. BMJ 2007, 334, 782. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Clasen, T. Household water treatment and safe storage to prevent diarrheal disease in developing countries. Curr. Environ. Health Rep. 2015, 2, 69–74. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Clasen, T.; Roberts, I.; Rabie, T.; Schmidt, W.P.; Cairncross, S. Interventions to improve water quality for preventing diarrhoea (review). Cochrane Libr. 2006. [Google Scholar] [CrossRef]
  7. Quick, R.; Kimura, A.; Thevos, A.K.; Tembo, M.; Shamputa, I.; Hutwagner, L.; Mintz, E. Diarrhea prevention through household-level water disinfection and safe storage in zambia. Am. J. Trop. Med. Hyg. 2002, 66, 584–589. [Google Scholar] [CrossRef] [PubMed]
  8. Quick, R.; Venczel, L.V.; Mintz, E.; Soleto, L.; Aparicio, J.; Gironaz, M.; Hutwagner, L.; Greene, K.D.; Bopp, C.; Maloney, K.; et al. Diarrhoea prevention in bolivia through point-of-use water treatment and safe storage: A promising new strategy. Epidemiol. Infect. 1999, 1222, 83–90. [Google Scholar] [CrossRef]
  9. Luby, S.; Agboatwalla, M.; Hoekstra, R.M.; Rahbar, M.H.; Billhimer, W.; Keswick, B. Delayed effectiveness of home-based interventions in reducing childhood diarrhea, karachi, pakistan. Am. J. Trop. Med. Hyg. 2004, 71, 420–427. [Google Scholar] [CrossRef] [Green Version]
  10. Rosa, G.; Clasen, T. Estimating the scope of household water treatment in low- and medium-income countries. Am. J. Trop. Med. Hyg. 2010, 82, 289–300. [Google Scholar] [CrossRef]
  11. WHO/UNICEF. Drinking Water: Equity, Safety, and Sustainability; World Health Organization: Geneva, Switzerland, 2011. [Google Scholar]
  12. Martin, N.A.; Hulland, K.R.S.; Dreibelbis, R.; Sultana, F.; Winch, P.J. Sustained adoption of water, sanitation, and hygiene interventions: Systematic review. Trop. Med. Int. Health 2017. [Google Scholar] [CrossRef] [Green Version]
  13. Evans, W.D.; Pattanayak, S.K.; Young, S.; Buszin, J.; Rai, S.; Bihm, J.W. Social marketing of water and sanitation products: A systematic review of peer-reviewed literature. Soc. Sci. Med. 2014, 110, 18–25. [Google Scholar] [CrossRef]
  14. Hulland, K.; Martin, N.; Dreibelbis, R.; DeBruicker Valliant, J.; Winch, P. What Factors Affect Sustained Adoption of Safe Water, Hygiene and Sanitation Technologies? A Systematic Review of Literature; ePPI Centre: London, UK, 2015. [Google Scholar]
  15. WHO/UNICEF. Progress on Sanitation and Drinking Water: 2015 Update and Mdg Assessment; World Health Organization: Geneva, Switzerland, 2015. [Google Scholar]
  16. Harshfield, E.; Lantagne, D.; Turbes, A.; Null, C. Evaluating the sustained health impact of household chlorination of drinking water in rural haiti. Am. J. Trop. Med. Hyg. 2012, 87, 786–795. [Google Scholar] [CrossRef] [PubMed]
  17. Lantagne, D.; Blanton, E.; Kaufmann, A.; Wilhelm, N. Sodium hypochlorite dosage for household and emergency water treatment: Updated recommendations. J. Water Health 2018, 16, 112–125. [Google Scholar] [CrossRef] [Green Version]
  18. Lantagne, D.S.; Blount, B.C.; Cardinali, F.; Quick, R. Disinfection by-product formation and mitigation strategies in point-of-use chlorination of turbid and non-turbid waters in western kenya. J. Water Health 2008, 6, 67–82. [Google Scholar] [CrossRef] [PubMed]
  19. Lantagne, D.S.; Cardinali, F.; Blount, B.C. Disinfection by-product formation and mitigation strategies in point-of-use chlorination with sodium dichloroisocyanurate in tanzania. Am. J. Trop. Med. Hyg. 2010, 83, 135–143. [Google Scholar] [CrossRef]
  20. Ritter, M. Incentivizing Demand for Household Chlorination in Rural Haiti: A Randomized Evaluation of Conditional in-Kind Transfers and Household Visits. 2017; Unpublished Working Paper. [Google Scholar]
  21. Ritter, M.; Camille, E.; Velcine, C.; Guillaume, R.K.; Lantagne, D. Optimizing household chlorination marketing strategies: A randomized controlled trial on the effect of price and promotion on adoption in haiti. Am. J. Trop. Med. Hyg. 2017, 97, 271–280. [Google Scholar] [CrossRef] [Green Version]
  22. Ericsson. Ericsson Mobility Report, June 2018. Available online: https://www.ericsson.com/assets/local/mobility-report/documents/2018/ericsson-mobility-report-june-2018.pdf (accessed on 17 June 2018).
  23. International Telecommunication Union. Mobile-Cellular Telephone Subscriptions, Haiti. 2016. Available online: https://www.itu.int/en/ITU-D/Statistics/Documents/statistics/2018/Mobile_cellular_2000-2016.xls (accessed on 2 November 2020).
  24. Kim, S.S.; Patel, M.; Hinman, A. Use of m-health in polio eradication and other immunization activities in developing countries. Vaccine 2017, 35, 1373–1379. [Google Scholar] [CrossRef] [Green Version]
  25. Muller, A.M.; Alley, S.; Schoeppe, S.; Vandelanotte, C. The effectiveness of e-& mhealth interventions to promote physical activity and healthy diets in developing countries: A systematic review. Int. J. Behav. Nutr. Phys. Act. 2016, 13, 109. [Google Scholar] [CrossRef] [Green Version]
  26. PSI. Enquete de suivi sur les determinants de l’utilisation des produits de traitement de l’eau au niveau des menages ayant des enfants de moins 5 ans en haiti. Wash. Dc Popul. Serv. Int. 2012. Unpublished data. [Google Scholar]
  27. Ritter, M. Determinants of Adoption of Household Water Treatment Products in Rural Haiti; Emory University: Atlanta, GA, USA, 2008. [Google Scholar]
  28. Holm, S. A simple sequentially rejective multiple test procedure. Scand. J. Stat. 1979, 6, 65–70. [Google Scholar]
  29. UNESCO/UIS. Literacy Rate by Age Group for Latest Year, Haiti. 2015. Available online: http://uis.unesco.org/en/country/ht (accessed on 2 November 2020).
  30. World Bank. Access to Electricity (% of Rural Population), Haiti. 2014. Available online: https://data.worldbank.org/indicator/Eg.Elc.Accs.Ru.Zs (accessed on 2 November 2020).
  31. Jones, C.O.; Wasunna, B.; Sudoi, R.; Githinji, S.; Snow, R.W.; Zurovac, D. “Even if you know everything you can forget”: Health worker perceptions of mobile phone text-messaging to improve malaria case-management in kenya. PLoS ONE 2012, 7, e38636. [Google Scholar] [CrossRef] [Green Version]
  32. Krah, E.F.M.; de Kruijf, J.G. Exploring the ambivalent evidence base of mobile health (mhealth): A systematic literature review on the use of mobile phones for the improvement of community health in africa. Digit. Health 2016, 2, 2055207616679264. [Google Scholar] [CrossRef] [Green Version]
  33. Pop-Eleches, C.; Thirumurthy, H.; Habyarimana, J.P.; Zivin, J.G.; Goldstein, M.P.; de Walque, D.; MacKeen, L.; Haberer, J.; Kimaiyo, S.; Sidle, J.; et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: A randomized controlled trial of text message reminders. AIDS 2011, 25, 825–834. [Google Scholar] [CrossRef] [Green Version]
  34. Raifman, J.R.; Lanthorn, H.E.; Rokicki, S.; Fink, G. The impact of text message reminders on adherence to antimalarial treatment in northern ghana: A randomized trial. PLoS ONE 2014, 9, e109032. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Table
Table 1. Number of households assigned to each treatment group.
Table 1. Number of households assigned to each treatment group.
With Household VisitWithout Household Visit
No SMSMonthlyBiweeklyWeeklyNo SMSMonthlyBiweeklyWeeklyTOTAL
Cholera
Health (positive) 10106 312950
Health (negative) 1077 157450
Activities (positive) 8810 781354
Activities (negative) 51113 4101356
Finances (positive) 71210 104750
Finances (negative) 6188 79957
Self-efficacy 8610 610646
Mothers 4128 351244
Accessibility 7714 99349
Social norms 1285 108851
Cholera subtotal07799910748284507
Disease
Health (positive) 439 67736
Health (negative) 6119 95949
Activities (positive) 847 5111348
Activities (negative) 12512 84950
Finances (positive) 557 99439
Finances (negative) 1667 841253
Self-efficacy 11911 136757
Mothers 17610 67248
Accessibility 996 1211451
Social norms 13178 56756
Disease subtotal010175860817074487
No SMS175000158000333
TOTAL1751781741771581551521581327
Table
Table 2. Relationship between chlorine purchase and chlorine use.
Table 2. Relationship between chlorine purchase and chlorine use.
(1)
Had Positive Chlorine Test		(2)
Had Positive Chlorine Test		(3)
Had Positive Chlorine Test		(4)
Had Positive Chlorine Test
Purchased at least 1 bottle0.312
(0.095) ***
Purchased at least 2 bottles 0.239
(0.103) **
Purchased at least 3 bottles 0.263
(0.090) **
Total bottles purchased 0.112
(0.031) ***
Constant0.650
(0.065) ***		0.776
(0.037) ***		0.855
(0.003) ***		0.716
(0.040) ***
R20.170.090.020.15
N667667667667
** p < 0.05; *** p < 0.01. Heteroskedasticity-robust standard errors; all include sales agent fixed effects.
Table
Table 3. Impact of different SMS frequencies (intention-to-treat (ITT) analysis).
Table 3. Impact of different SMS frequencies (intention-to-treat (ITT) analysis).
(1)(2)(3)(4)(5)
Purchased at Least
1 Bottle		Purchased at Least
2 Bottles		Purchased at Least
3 Bottles		Total Bottles PurchasedHad Positive Chlorine Test
Panel A: Assigned SMS frequency (reference group = no SMSs)
3 SMSs (monthly)0.0280.0030.0060.0540.037
(0.031)(0.025)(0.006)(0.055)(0.018) **
6 SMSs (bi-weekly)0.003−0.042−0.003−0.0310.028
(0.034)(0.031)(0.007)(0.077)(0.022)
12 SMSs (weekly)−0.050−0.052−0.009−0.1520.018
(0.032)(0.031)(0.016)(0.077) *(0.043)
Household visit0.1710.0880.0060.319
(0.042) ***(0.027) ***(0.005)(0.091) ***
Constant0.4790.2730.0240.9400.842
(0.025) ***(0.026) ***(0.006) ***(0.060) ***(0.016) ***
R20.040.020.000.030.00
N1335133513351335669
Panel B: Assigned to receive at least 1 SMS (reference group = 0 SMSs)
At least 1 SMS−0.006−0.030−0.002−0.0430.028
(0.022)(0.022)(0.007)(0.038)(0.021)
Household visit0.1710.0890.0060.319
(0.042) ***(0.028) ***(0.005)(0.091) ***
Constant0.4790.2730.0240.9400.841
(0.024) ***(0.026) ***(0.006) ***(0.060) ***(0.016) ***
R20.030.010.000.020.00
N1335133513351335667
Panel C: Assigned number of SMSs (as continuous variable)
# SMSs assigned−0.005−0.005−0.001−0.0150.001
(0.002) **(0.002) *(0.001)(0.006) **(0.004)
Household visit0.1710.0890.0060.319
(0.042) ***(0.028) ***(0.005)(0.091) ***
Constant0.5010.2760.0280.9860.858
(0.020) ***(0.020) ***(0.007) ***(0.056) ***(0.020) ***
R20.040.020.000.020.00
N1335133513351335669
* p < 0.10; ** p < 0.05; *** p < 0.01. Heteroskedasticity-robust standard errors; all models include sales agent fixed effects.
Table
Table 4. Impact of different SMS frequencies (treatment-on-treated (TOT) analysis).
Table 4. Impact of different SMS frequencies (treatment-on-treated (TOT) analysis).
(1)(2)(3)(4)(5)
Purchased at Least
1 Bottle		Purchased at Least
2 Bottles		Purchased at Least
3 Bottles		Total Bottles PurchasedHad Positive Chlorine Test
Panel A: Actual SMS frequency (reference group = no SMSs)
1–5 SMSs sent0.0240.0000.0080.0480.042
(0.029)(0.025)(0.005)(0.053)(0.017) **
6–8 SMSs sent0.003−0.036−0.002−0.0250.031
(0.033)(0.031)(0.007)(0.076)(0.023)
9+ SMSs sent−0.054−0.053−0.008−0.1570.022
(0.032)(0.031) *(0.016)(0.077) *(0.043)
Household visit0.1710.0880.0060.318
(0.042) ***(0.028) ***(0.005)(0.091) ***
Constant0.4810.2720.0230.9420.838
(0.024) ***(0.027) ***(0.006) ***(0.060) ***(0.016) ***
R20.040.020.000.030.00
N1335133513351335669
Panel B: Actually received at least 1 SMS (reference group = 0 SMSs)
At least 1 SMS−0.009−0.029−0.001−0.0440.032
(0.022)(0.023)(0.007)(0.039)(0.022)
Household visit0.1710.0890.0060.319
(0.042) ***(0.027) ***(0.005)(0.091) ***
Constant0.4810.2720.0230.9420.838
(0.024) ***(0.027) ***(0.006) ***(0.060) ***(0.017) ***
R20.030.010.000.020.00
N1335133513351,335669
Panel C: Actual SMSs sent (TOT) (as continuous variable)
# SMSs sent−0.005−0.006−0.001−0.0160.002
(0.002) **(0.002) **(0.001)(0.005) ***(0.004)
Household visit0.1710.0890.0060.319
(0.042) ***(0.027) ***(0.005)(0.090) ***
Constant0.5030.2810.0270.9940.853
(0.020) ***(0.022) ***(0.007) ***(0.057) ***(0.021) ***
R20.040.020.000.020.00
N1335133513351335667
* p < 0.10; ** p < 0.05; *** p < 0.01. Heteroskedasticity-robust standard errors; all models include sales agent fixed effects.
Table
Table 5. Impact of individual constructs.
Table 5. Impact of individual constructs.
(1)(2)(3)(4)(5)
Purchased at Least
1 Bottle		Purchased at Least
2 Bottles		Purchased at Least
3 Bottles		Total Bottles PurchasedHad Positive Chlorine Test
1. Positive health−0.024−0.0850.022−0.0720.040
(0.049)(0.053)(0.022)(0.158)(0.056)
2. Negative health0.081−0.015−0.0080.0300.065
(0.038)(0.047)(0.016)(0.105)(0.022)
3. Positive activities−0.0370.0040.003−0.0460.058
(0.055)(0.037)(0.023)(0.084)(0.040)
4. Negative activities−0.015−0.020−0.019−0.097−0.048
(0.044)(0.039)(0.010)(0.088)(0.078)
5. Positive finances−0.0300.017−0.024−0.048−0.017
(0.045)(0.049)(0.015)(0.112)(0.051)
6. Negative finances−0.000−0.0480.001−0.039−0.019
(0.056)(0.044)(0.012)(0.097)(0.044)
7. Self-efficacy−0.069−0.033−0.011−0.1340.052
(0.048)(0.033)(0.011)(0.080)(0.044)
8. Mothers0.008−0.1080.010−0.0670.051
(0.054)(0.039)(0.020)(0.126)(0.050)
9. Product access0.005−0.0140.0380.0930.063
(0.050)(0.044)(0.021)(0.120)(0.030)
10. Social norms−0.0090.010−0.019−0.0230.041
(0.038)(0.041)(0.010)(0.085)(0.043)
Household visit0.1710.0910.0060.321
(0.042) ***(0.028) **(0.005)(0.094) **
Constant0.4820.2700.0230.9370.842
(0.024) ***(0.026) ***(0.005) ***(0.062) ***(0.016) ***
R20.040.020.010.020.01
N1335133513351335669
** p < 0.05; *** p < 0.01, using Holm’s adjustment for multiple comparisons. Heteroskedasticity-robust standard errors; all models include sales agent fixed effects.
Table
Table 6. Impact of message framing.
Table 6. Impact of message framing.
(1)(2)(3)(4)(5)
Purchased at Least
1 Bottle		Purchased at Least
2 Bottles		Purchased at Least
3 Bottles		Total Bottles PurchasedHad Positive Chlorine Test
Panel A: Cholera vs. disease framing
Cholera framing−0.021−0.019−0.004−0.082−0.016
(0.030)(0.023)(0.006)(0.053)(0.028)
Household visit0.1730.0810.0020.320
(0.036) ***(0.023) ***(0.006)(0.070) ***
Constant0.4830.2550.0260.9380.879
(0.030) ***(0.017) ***(0.005) ***(0.047) ***(0.014) ***
R20.040.010.000.020.00
N1002100210021002503
Panel B: Positive vs. negative framing
Positive framing−0.0500.0060.008−0.0210.032
(0.038)(0.028)(0.007)(0.071)(0.044)
Household visit0.1600.0710.0050.284
(0.045) ***(0.028) **(0.011)(0.075) ***
Constant0.5040.2550.0150.9220.834
(0.023) ***(0.018) ***(0.007) **(0.048) ***(0.020) ***
R20.030.010.000.020.00
N592592592592282
** p < 0.05; *** p < 0.01, using Holm’s adjustment for multiple comparisons. Heteroskedasticity-robust standard errors; all models include sales agent fixed effects.
Table
Table 7. Differential impact of visits on different purchase indicators (ITT).
Table 7. Differential impact of visits on different purchase indicators (ITT).
(1)(2)(3)(4)(5)(6)
Excluding 5 Agents with Unequal Assignment
Purchased at Least 1 BottlePurchased at Least 2, Conditional on 1st PurchaseBottles Purchased after First PurchasePurchased at Least 1 BottlePurchased at Least 2, Conditional on 1st PurchaseBottles Purchased after First Purchase
Panel A: Assigned SMS frequency (reference group = no SMSs)
3 SMSs (monthly)0.028−0.036−0.0530.030−0.034−0.060
(0.031)(0.041)(0.083)(0.037)(0.047)(0.103)
6 SMSs (bi-weekly)0.003−0.077−0.078−0.007−0.089−0.090
(0.034)(0.042) *(0.066)(0.034)(0.051) *(0.081)
12 SMSs (weekly)−0.050−0.079−0.175−0.048−0.119−0.238
(0.032)(0.052)(0.107)(0.036)(0.055) **(0.125) *
Household visit0.1710.0240.0430.1650.0280.050
(0.042) ***(0.018)(0.053)(0.045) ***(0.021)(0.060)
Constant0.4790.5580.9570.4590.6281.084
(0.025) ***(0.035) ***(0.061) ***(0.026) ***(0.040) ***(0.072) ***
R20.040.010.010.040.010.01
N1,3357547541,136607607
Panel B: Assigned to receive at least 1 SMS (reference group = 0 SMSs)
At least 1 SMS−0.006−0.063−0.099−0.008−0.079−0.125
(0.022)(0.036) *(0.065)(0.025)(0.042) *(0.077)
Household visit0.1710.0230.0370.1650.0260.041
(0.042) ***(0.019)(0.057)(0.045) ***(0.022)(0.065)
Constant0.4790.5590.9610.4590.6291.088
(0.024) ***(0.035) ***(0.064) ***(0.026) ***(0.040) ***(0.075) ***
R20.030.010.000.030.010.00
N13357547541136607607
Panel C: Assigned number of SMSs (as continuous variable)
SMSs assigned−0.005−0.007−0.014−0.005−0.010−0.019
(0.002) **(0.004)(0.008)(0.003) *(0.004) **(0.010) *
Household visit0.1710.0250.0430.1650.0290.049
(0.042) ***(0.018)(0.055)(0.045) ***(0.020)(0.062)
Constant0.5010.5430.9560.4790.6201.089
(0.020) ***(0.028) ***(0.062) ***(0.020) ***(0.031) ***(0.072) ***
R20.040.010.010.040.010.01
N13357547541136607607
* p < 0.10; ** p < 0.05; *** p < 0.01. Heteroskedasticity-robust standard errors; all models include sales agent fixed effects.
Table
Table 8. Multivariate model.
Table 8. Multivariate model.
(1)(2)(3)(4)(5)
Purchased at Least:Total Bottles PurchasedHad Positive Chlorine Test
1 Bottle2 Bottles3 Bottles
3 SMSs (monthly)0.106−0.047−0.021−0.0330.036
(0.144)(0.084)(0.033)(0.364)(0.022)
6 SMSs (bi-weekly)0.084−0.089−0.031−0.1090.032
(0.161)(0.090)(0.036)(0.388)(0.030)
12 SMSs (weekly)0.033−0.100−0.036−0.2280.021
(0.144)(0.103)(0.046)(0.410)(0.051)
1. Positive health−0.0830.0040.0530.1010.017
(0.172)(0.110)(0.054)(0.473)(0.047)
2. Negative health0.0140.0670.0210.1800.041
(0.152)(0.100)(0.031)(0.368)(0.026)
3. Positive activities−0.0930.0930.0340.1310.034
(0.169)(0.102)(0.030)(0.396)(0.054)
4. Negative activities−0.0700.0690.0130.082−0.071
(0.148)(0.086)(0.033)(0.352)(0.068)
5. Positive finances−0.0930.1020.0070.112−0.039
(0.161)(0.100)(0.028)(0.374)(0.051)
6. Negative finances−0.0630.0370.0310.121−0.044
(0.178)(0.100)(0.045)(0.427)(0.047)
7. Self-efficacy−0.1340.0500.0190.0180.027
(0.159)(0.091)(0.032)(0.371)(0.033)
8. Mothers−0.055−0.0220.0400.0930.025
(0.160)(0.099)(0.032)(0.404)(0.036)
9. Product access−0.0630.0690.0670.2410.040
(0.164)(0.097)(0.048)(0.417)(0.029)
10. Social norms−0.0770.0920.0100.1230.014
(0.159)(0.095)(0.031)(0.380)(0.033)
Cholera framing−0.019−0.016−0.003−0.076−0.009
(0.029)(0.024)(0.006)(0.054)(0.028)
Household visit0.1710.0900.0060.319
(0.043) **(0.028) *(0.005)(0.093) **
Constant0.4790.2720.0240.9400.842
(0.025) ***(0.026) ***(0.006) ***(0.061) ***(0.016) ***
R20.040.020.010.030.01
N1335133513351335669
* p < 0.10; ** p < 0.05; *** p < 0.01, using Holm’s adjustment for multiple comparisons. Heteroskedasticity-robust standard errors; all models include sales agent fixed effects.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ritter, M.; Camille, E.; Velcine, C.; Guillaume, R.-K.; Casimir, J.M.; Lantagne, D.S. Impact Evaluation of an SMS Campaign to Promote Household Chlorination in Rural Haiti. Water 2020, 12, 3095. https://doi.org/10.3390/w12113095

AMA Style

Ritter M, Camille E, Velcine C, Guillaume R-K, Casimir JM, Lantagne DS. Impact Evaluation of an SMS Campaign to Promote Household Chlorination in Rural Haiti. Water. 2020; 12(11):3095. https://doi.org/10.3390/w12113095

Chicago/Turabian Style

Ritter, Michael, Eveline Camille, Christophe Velcine, Rose-Kerline Guillaume, Jean Marcel Casimir, and Daniele S. Lantagne. 2020. "Impact Evaluation of an SMS Campaign to Promote Household Chlorination in Rural Haiti" Water 12, no. 11: 3095. https://doi.org/10.3390/w12113095

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop