Next Article in Journal
The Effect of Therapeutic Hypothermia after Cardiac Arrest on the Neurological Outcome and Survival—A Systematic Review of RCTs Published between 2016 and 2020
Next Article in Special Issue
Factors Associated with Quit Interest and Quit Attempts among Young Adult JUUL Users
Previous Article in Journal
Multi-Angle Fusion-Based Safety Status Analysis of Construction Workers
Previous Article in Special Issue
Associations between Cognitive and Affective Responses to Tobacco Advertisements and Tobacco Use Incidence: A Four-Year Prospective Study among Adolescent Boys
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 18
Issue 22
10.3390/ijerph182211814
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Integrating Self-Report and Psychophysiological Measures in Waterpipe Tobacco Message Testing: A Novel Application of Multi-Attribute Decision Modeling

by
Elise M. Stevens
1,*,
Andrea C. Villanti
2,
Glenn Leshner
3,
Theodore L. Wagener
4,5,
Brittney Keller-Hamilton
4,5 and
Darren Mays
4,5
1
Department of Population and Quantitative Health Sciences, Division of Preventative and Behavioral Medicine, University of Massachusetts Medical School, Worcester, MA 01655, USA
2
Vermont Center on Behavior and Health, Department of Psychiatry, University of Vermont, Burlington, VT 05405, USA
3
Gaylord College of Journalism and Mass Communication, University of Oklahoma, Norman, OK 73019, USA
4
Center for Tobacco Research, The Ohio State University James Comprehensive Cancer Center, Columbus, OH 43214, USA
5
Department of Internal Medicine, The Ohio State University, Columbus, OH 43210, USA
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(22), 11814; https://doi.org/10.3390/ijerph182211814
Submission received: 9 September 2021 / Revised: 5 November 2021 / Accepted: 9 November 2021 / Published: 11 November 2021
(This article belongs to the Special Issue Behavioral Research in Regulatory Tobacco Science)
Review Reports Versions Notes

Abstract

:
Background: Waterpipe (i.e., hookah) tobacco smoking (WTS) is one of the most prevalent types of smoking among young people, yet there is little public education communicating the risks of WTS to the population. Using self-report and psychophysiological measures, this study proposes an innovative message testing and data integration approach to choose optimal content for health communication messaging focusing on WTS. Methods: In a two-part study, we tested 12 WTS risk messages. Using crowdsourcing, participants (N = 713) rated WTS messages based on self-reported receptivity, engagement, attitudes, and negative emotions. In an in-lab study, participants (N = 120) viewed the 12 WTS risk messages while being monitored for heart rate and eye-tracking, and then completed a recognition task. Using a multi-attribute decision-making (MADM) model, we integrated data from these two methods with scenarios assigning different weights to the self-report and laboratory data to identify optimal messages. Results: We identified different optimal messages when differently weighting the importance of specific attributes or data collection method (self-report, laboratory). Across all scenarios, five messages consistently ranked in the top half: four addressed harms content, both alone and with themes regarding social use and flavors and one addiction alone message. Discussion: Results showed that the self-report and psychophysiological data did not always have the same ranking and differed based on weighting of the two methods. These findings highlight the need to formatively test messages using multiple methods and use an integrated approach when selecting content.

1. Introduction

Waterpipe (i.e., hookah) tobacco smoking (WTS) is one of the most prevalent types of smoking among young people in the United States [1], and nearly 30% of young non-users are susceptible to use [2]. Because WTS produces harmful toxicants [3,4], which can cause cancer and other negative health outcomes [5,6], it is imperative to communicate the risks of use to the young adult population. Despite the need for risk communication and the proliferation of social media messaging promoting WTS smoking [7,8], there is little public education on WTS [2,9,10,11,12,13,14,15,16]. The 2016 Deeming Rule gave the U.S. Food and Drug Administration (FDA) regulatory authority over all tobacco products, including waterpipe tobacco [17]. Now, the FDA has the opportunity to disseminate public health education campaigns to effectively communicate the risks of WTS as they have with cigarettes [18] and e-cigarettes [19]. However, there is little research on how to effectively communicate the risks of WTS to young people [9].
The existing research base examining WTS messaging has relied heavily on self-report data assessed after message exposure [13,20], with few exceptions to that method [21]. Testing messages using only self-report data in formative work is the norm in tobacco prevention mass media campaign development [22]. For example, the FDA pre-tests campaign messages using self-report measures that include perceived message effectiveness, smoking-related beliefs, and attitudes after exposure to determine optimal content [23]. Because self-report measures can be limited by social desirability, response bias, and inaccurate recall, they can be complemented by psychophysiological assessments during message exposure to gain more insight into how individuals will react to messages in real time [24]. Psychophysiological measures such as heart rate combined with recognition tasks and eye-tracking provide measures of message processing and responses that are not directly accessible through self-report [25,26]. This includes cognitive resources allocated to message processing through heart rate measurement, encoding of the message through recognition tasks, and visual attention—processes that have been linked to changes in behavioral attitudes and intentions [27].
Psychophysiological measurements rely on the foundational premise of the Limited Capacity Model of Motivated Mediated Message Processing (LC4MP) [28], which addresses humans’ innate ability to respond to messaging emotionally and cognitively but with only a limited number of resources to encode, store, and retrieve the information later. Humans have two motivational systems that are activated when confronted with a message: appetitive/approach and aversive/avoid [28]. Psychophysiology allows researchers to collect these motivational reactions to complement self-report measures in order to understand which messages are encoded and stored at an increased rate in relation to other messages [28]. With only self-report data, researchers may not understand why some messages are leading to high aversion and low encoding and storage [29]. Heart rate, recognition tasks, and eye-tracking provide the ability to capture processes in real time during message exposure to ensure content is optimized for cognitive encoding and storage in the formative stage. For instance, heart rate deceleration indicates more cognitive resources allocated to message encoding [24]. Increased recognition indicates encoding of information [24], and eye-tracking ensures individuals are dedicating substantial visual attention to the message and captures attention to specific content (i.e., areas of interest) as well. The addition of psychophysiological data to campaign formative research can provide insight into message processing [24] and help optimize campaign content targeting health-related beliefs and behaviors. These psychophysiological methods coincide with cognitive and emotional responses that correlate with behavior change [24]. However, a significant limitation of including psychophysiological methods in campaign development has been the lack of methods to integrate psychophysiology data with the standard self-report data collected during the formative phase.
Multi-attribute decision-making (MADM) models provide a practical, transparent framework for considering data across multiple inputs (attributes), determining the relative importance of each attribute, and simultaneously evaluating detail at the attribute-level and in a summary measure across all attributes using aggregate data [30]. They have been widely used in business [31], economics [32], technology [33], and health care [34], but rarely in health communication science. Multi-attribute models are a decision analytic technique that allows for inclusion of qualitative and quantitative factors in decision-making [30] as is the case with multiple attributes for each message across multiple messages tested. The model evaluates each message by assigning each attribute a raw score obtained from data collection for each message, determining the relative importance (weight) of each attribute in the decision-making, creating a summary score for each message across attributes, and assigning an overall score or ranking to each message. Identifying the key attributes and their relative importance at the outset of decision-making allows for optimizing desired outcomes, using an algorithmic approach to combining data across sources, and conducting analyses to determine the impact of various attributes on overall rankings of the messages considered. This type of framework allows for explicit consideration and prioritization of specific message outcomes or specific types of data in determining optimal message content for a health communication campaign.
This study used a MADM approach to choose optimal WTS public health messages for young adults [30] by integrating traditional self-report measures and lab-based heart rate, eye-tracking, and recognition tasks. Rooted in LC4MP, a recent meta-analysis of self-report, behavioral, and psychophysiological responses to messages indicates self-report outcomes capture the largest effects, followed by behavioral and psychophysiological measures. Traditionally, these measures have been considered separately to inform message development. To our knowledge, they have not been systematically combined to optimize message content, and based on variability in effect size estimates [35], there has been no research to examine whether these methods and individual measures should be considered equally, or if certain methods or measures should be weighted more substantially, in making decisions about optimal messages. This has important practical implications for the development of public health campaigns, as practitioners who develop campaigns face decisions about the optimal message content to use relative to campaign resources. Integrating self-report measures of receptivity, engagement, attitudes, and emotions combined with psychophysiological measures of heart rate, eye-tracking, and a recognition task through an in-lab experiment, this study aimed to demonstrate the utility of this data collection approach and provide a roadmap for determining the optimal messaging content across multiple data sources and attributes using MADM.

2. Materials and Methods

2.1. Participants

Participants in both the crowdsourced testing and the in-lab study met the following criteria: (1) 18–30 years old, (2) self-reported WTS in the past month or had never smoked waterpipe tobacco but were deemed susceptible [36], and (3) within the United States. Participants were deemed as susceptible if they gave a response other than “No” to one or more questions about WTS soon, in the next year, sometime in the future, or if a best friend offered it [36].

2.2. Message Development and Selection

Using information from multiple contexts including research on health harms and addictiveness [37], health communication studies examining risk messaging [13,38], past WTS message testing studies [10,11,39], and young adults’ beliefs about WTS [40] and flavors [41,42], we created a bank of 18 candidate messages. Next, we consulted experts in the field using an iterative process until consensus was reached on the messages to use. Four key themes for effective hookah messaging emerged: (1) health harms, (2) addictiveness, (3) social use, and (4) flavoring. To reflect this, the current study resulted in a 2 (risk content: health harms or addiction) by 3 (message theme: risk alone; flavors; or social use) design. The 6 experimental conditions were: harms risk alone; harms, flavors; harms, social use; addiction risk alone; addiction, flavors; and addiction, social use. All messages were consistent in length (i.e., two sentences with one hashtag), layout, and structure, and only differed on their manipulation of risk content and message theme. Text content was accompanied by 1 of 4 counterbalanced images relevant to WTS use: a waterpipe; social WTS setting; a smoke cloud; and a ring of smoke. All messages were pre-tested for readability. See Table 1.

2.3. Crowdsourced Testing

2.3.1. Setting

In September 2018, participants (N = 713) were recruited using the crowdsourcing platform, Amazon Mechanical Turk (MTurk). We used data quality assurance measures including verification requirements to prevent automated responses from bots and prohibiting duplicate responses through manual review of the data. In addition, we used randomly generated completion codes [43,44].

2.3.2. Design

After meeting eligibility criteria and providing online consent, participants answered questions assessing demographics, WTS and other tobacco use, and intentions to engage in WTS. In a 2 (user status: current hookah user, susceptible non-user) by 2 (risk content: health harms or addiction) by 3 (message theme: harms/addiction risk alone, harms/addiction risk flavors, or harms/addiction risk social use) design, participants were randomly assigned to one of the six risk content by message theme categories and presented with a single message within that category. After viewing the candidate message for as long as they wanted, participants were asked questions pertaining to the study outcomes. Participants were compensated $2 and took on average 10 min to complete the survey. All procedures were approved by the host institution’s institutional review board. Supplemental Table S1 presents details on the messages tested. Procedures have been published elsewhere [20].

2.3.3. Measures

Receptivity: Message receptivity was assessed with 9 items, such as “The message grasped my attention” and “The message was convincing”. Each item was rated on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree; Cronbach’s α = 0.92) [38,45].
Engagement: Engagement was assessed with 3 items asking about how engaged, how involved the participant was in the message, and how much attention was paid to the message on a scale from 1 (none) to 7 (very; Cronbach’s α = 0.94).
Positive attitude: Positive attitude toward the message was assessed with 9 items on semantic differential scales from 1 to 7. Examples included “boring/exciting”; “not stimulating/stimulating”, with higher scores reflecting a more positive attitude toward the message (Cronbach’s α = 0.94) [36].
Negative emotions: Negative emotions were assessed with 4 items: frightened, anxious, nervous, and worried in response to the message on a scale from 1 (not at all) to 4 (extremely; (Cronbach’s α = 0.91) [46].

2.4. Psychophysiological Testing

2.4.1. Setting

Participants (N = 120) were recruited through mass emails and social media advertising at a large southwestern university. Eligible participants completed a one-hour in-person laboratory visit. Participant characteristics were similar in both studies. Demographics have been reported elsewhere [20,21].

2.4.2. Design

Once eligibility was determined, participants were scheduled for a lab visit. Upon arriving to the lab, written consent was obtained, baseline measures identical to those in the crowdsourcing survey were assessed, and then participants were randomly assigned to view 12 messages on combinations of each of the two risks with one of three themes: (a) health harms or addictiveness alone, (b) health harms or addictiveness with flavors, and (c) health harms or addictiveness with social use (Supplemental Table S1). Messages were presented in a random order. During the viewing of the messages, participants were monitored for heart rate and eye-tracking. After completing the viewing task, participants completed a recognition task where they indicated whether an excerpt of the message came from the messages they just saw or if they were foils. The full description of the psychophysiological testing is available elsewhere [21]. All procedures were approved by the host institution’s institutional review board.

2.4.3. Measures

Heart rate: Heart rate was measured to assess the cognitive resources allocated to the message [24]. Using four gelled sensors applied to the participants’ forearms (2) and left wrist (2), electrocardiogram (ECG) signals were recorded with a Shimmer 3 EXG module, while viewing the messages. Raw data were sampled at 512 Hz. Heart change scores were computed using beats per minute (BPM) and subtracting the last second of the baseline (BPM prior to message viewing) from scores for each second of message exposure for each participant. Heart rate scores were set to absolute values, but all reflect the amount of deceleration. Then, scores were averaged across the message, giving each message its own score. Greater values indicated greater deceleration in heart rate, signaling more cognitive resources allocated to the message [24].
Recognition accuracy: To assess recognition and the degree to which the message content was encoded into short-term memory, participants completed a recognition task. The task consisted of 48 fragments shown to participants in which they had to click “yes” or “no” as to whether it appeared in the messages they had just viewed. The fragments included 24 targets taken from the messages and 24 foils. Recognition accuracy was computed as a percentage of hits (i.e., responding “yes” to targets) [47].
Visual Attention: Visual attention was assessed using an eye-tracker connected to the base of the computer screen. For each message, three areas of interest (AOI) were identified—one for each of the two sentences and one for the hashtag included. All three AOIs were averaged for the number of milliseconds that a participant viewed the AOIs. Visual attention is reported in absolute values.

2.5. Data Integration and Analysis

Self-report scores for each construct (i.e., receptivity, engagement, positive attitude, and negative emotion) were averaged for each participant, and then averaged across participants to create a mean score for each message. Psychophysiological scores for each construct (i.e., heart rate deceleration, recognition accuracy, visual attention) were also averaged across participants to create a mean score for each message. Within each attribute, mean scores were ranked across messages, with the highest scores receiving the lowest ranking (i.e., “1”; most effective) and the lowest scores receiving the highest ranking (i.e., “12”; least effective).
Using the MADM framework [30,48], we created a decision matrix in which each row (e.g., alternative or message) represented a message and each column (attribute) represented a construct collected via the crowdsourced or psychophysiological study. For each study, the rank of each attribute was summed within each message to create a total rank for each message by study type (self-report, psychophysiological).
We used the attribute-level ranks for each message to compute summary scores and rankings under four weighting scenarios to identify the top-performing messages: (1) Equal attribute weights (4 self-report, 3 psychophysiological); (2) Equal study weights (Total self-report rank, total psychophysiological rank); (3) Prefer self-report (Individual self-report attribute ranks, total psychophysiological rank); and (4) Prefer psychophysiological (Total self-report rank, individual psychophysiological attribute ranks).

3. Results

3.1. Self-Report Data and Rankings

Table 1 shows mean scores for the crowdsourced self-report measures of receptivity, engagement, positive attitude, and negative emotion for each message. While ranking by self-report attribute differed across WTS messages, the summary rankings identified the top performing messages in the harms alone category (H1: “Many people think smoking hookah is safer than cigarettes. Truth is, hookah tobacco has the same health effects—breathing problems, cancer, lung and heart disease. #UnfollowHookah”), followed by harms/social category (HS2: “Consider skipping your turn. Sharing a hookah hose increases your risk of infections like herpes. #UnfollowHookah”), followed by harms/flavors category (HF2: “Don’t let flavored hookah tobacco smooth talk you. Flavored hookah tobacco masks the nasty chemicals and is manufactured the same way as cigarettes. #UnfollowHookah”), and followed by harms/social category (HS1: “Socializing at hookah bars isn’t all fun and games. Smoking hookah with friends exposes you to even more poisonous chemicals than cigarette smoking. #UnfollowHookah”).

3.2. Psychophysiological Data and Rankings

Mean heart rate deceleration, recognition accuracy, and visual attention scores are presented by message, with their attribute-level rankings in Table 1. Again, ranking by psychophysiological attribute differed from the summary rank for all psychophysiological measures combined. Summary rankings based on the psychophysiological data identified the top performing measures in the harms alone (H2: “The nicotine buzz comes with more than you bargained for. ‘Hookah sickness’ can be carbon monoxide poisoning. #UnfollowHookah”), addiction/flavor (AF1: “Mint, Blue Mist, Apple, Ambrosia. Flavors mask the bitter taste of nicotine and help get you hooked on smoking hookah. #UnfollowHookah”), and harms/flavor categories (HF2: “Don’t let flavored hookah tobacco smooth talk you. Flavored hookah tobacco masks the nasty chemicals and is manufactured the same way as cigarettes. #UnfollowHookah”).

3.3. Weighting Scenarios Integrating Self-Report and Psychophysiological Data

Table 2 presents the mean score and rank of each message under four weighting scenarios. In the equal attribute ranks scenario, each of the four self-report measures and 3 lab measures received equal weight in the mean score. Results from this method identified the top three messages in the harms/flavor (HF2), harms/social (HS2), and harms alone (H1) categories. The equal study weights averaged the two total ranks (crowdsourced, psychophysiological) for each message, selecting harms/flavor (HF2), harms alone (H2), and harms/social (HS2), and as the top performing messages. The third scenario provided greater weight to the self-report measures, using the four attribute-level ranks for the self-report measures and the total rank for the psychophysiological measures; the top three messages identified using this message were the harms/flavor (HF2), harms/social (HS2), and harms alone (H2). The final scenario provided greater weight to the psychophysiological measures, using the total rank for the self-report measures and the three attribute-level ranks for the psychophysiological measures. Similarly, this scenario identified harms/flavor (HF2) as the top performing message, followed by harms/social (HS2), and addiction/flavor (AF1).

4. Discussion

The current study tested a novel methodology to harness both self-report and lab-based data from formative message testing to optimize the selection of messages for a WTS education campaign. Summary findings identified different optimal messages when considering different scenarios for weighting the importance of specific attributes or study design. Across all scenarios, five messages consistently ranked in the top half: HF2 (“Don’t let flavored hookah tobacco smooth talk you. Flavored hookah tobacco masks the nasty chemicals and is manufactured the same way as cigarettes. #UnfollowHookah”), HS2 (“Consider skipping your turn. Sharing a hookah hose increases your risk of infections like herpes. #UnfollowHookah”), H2 (“The nicotine buzz come with more than you bargained for. ‘Hookah sickness’ can be carbon monoxide poisoning. #UnfollowHookah”), H1 (“Many people think smoking hookah is safer than cigarettes. Truth is, hookah tobacco has the same health effects—breathing problems, cancer, lung and heart disease”. #UnfollowHookah”), and A1 (“No one is invincible. Even smoking hookah just once or twice a month can lead to nicotine addiction. #UnfollowHookah”). Of these messages, four addressed harms content, both alone and with message themes regarding social and flavored use. Generally, the addiction content messages performed worse than the harms content messages, with the exception of the addiction alone message (A1) and one addiction/flavor message (AF1: “Mint, Blue Mist, Apple, Ambrosia. Flavors mask the bitter taste of nicotine and help get you hooked on smoking hookah. #UnfollowHookah”) that received a high ranking in “prefer lab” weighting scenario. The latter message (AF1) was ranked low (9th) in the self-report data, but performed well with respect to heart rate deceleration, recognition accuracy, and visual attention. These findings align with other work on self-reported measures for vaping prevention message testing that identified harms content as more effective than addiction content, though they differ with respect to the social and flavor themes [49]. Whereas rankings in the current study identified harms/social and harms/flavor messages for WTS as some of the most effective, our other research on vaping prevention messages identified them as potentially eroding the effectiveness of the harms messages [49]. With regard to the addiction findings, contrary to other addictive substances (i.e., heroin), addiction to WTS might not carry the same social stigma and therefore an addiction message might not be as impactful for this behavior. Future research could test messaging strategies such as the long-term financial burden of continued use to address WTS addiction. Finding ways to effectively communicate addiction risk is imperative for this young population.
Previous work assessing the effectiveness of marijuana prevention messages using self-report and psychophysiological data identified the latter as providing a more consistent picture of message processing [50]. However, few if any communication campaigns use these procedures in message development. In our current analyses, focusing on one method over the other (i.e., self-report versus psychophysiological) would have resulted in the selection of different messages for a WTS campaign. Our data suggest that a campaign derived from self-report data alone would have focused on harms content, while a campaign focused on psychophysiological data would have a greater focus on addiction content. This may partially explain difficulties in creating effective addiction content in substance use prevention campaigns, as reliance on self-reported outcomes in the formative stage may deter researchers from using addiction content.
Findings from our analyses using different weighing scenarios support the need to formatively test messages in multiple ways and use an integrated approach when selecting content for media campaigns. A recent meta-analysis of the LC4MP indicated that psychophysiological measurement can be low signal, high noise [24], but self-report and behavioral outcomes often provide larger effect sizes [35]. Because of this, an integrated approach using both self-report and psychophysiology is beneficial. The methods used in this study were easy to implement and allowed our team to explore a range of potential campaigns based on the range of data we collected during our message testing phase. Ultimately, we used the results from the equal attribute ranks scenario to select four messages (HF2, HS2, H1, A1) to include in a randomized controlled trial on WTS risk education (NCT04252014). Results from the trial are forthcoming. Based on our analyses, researchers and practitioners applying this multi-method approach could use a similar strategy to giving equal weights to self-report and psychophysiological attributes to inform message optimization decisions.
The limitations of this study include not testing this approach on actual effectiveness and behavior change. Research testing the various weighted scenarios with a behavioral outcome could further shed light on which weighted approach is predictive of behavior change. In addition, we used Mturk for crowdsourced data collection, which may not be representative of the target population for our messaging. However, past research has shown that Mturk samples used for tobacco research produce comparable experimental and correlational results to those of population-based samples [51]. The study also was not powered to detect how use of other nicotine and tobacco products impacted outcomes. Future studies should consider other tobacco and nicotine use status. Lastly, this study only tested WTS messages, and others should test this method with other health behavior messages. The MADM analytic approach we used does not test for statistical differences between messages, and in the future it will be prudent to pair this method with analyses that test hypotheses about message effects statistically [20,21]. Despite these limitations, this study was the first of its kind to use a MADM to determine optimal message content in the formative pre-testing of health communication messaging and makes an important contribution to the future of health communication research.

5. Conclusions

Integrating self-report and psychophysiological data using MADM is an innovative strategy to choose optimal content when designing health behavior messaging. By utilizing data from various inputs and following the decision-making model, messages chosen for a mass media campaign may induce a larger impact on public health targets. This strategy may be the most useful in formative work, when pre-testing messages to garner more information about how the campaign messages may fare among the target audience. Our analysis ultimately pointed to using equal attribute weights for the self-report and psychophysiological measures to identify optimal message content; however, future studies should continue to test and refine this approach. Future studies should continue to test and refine this approach in tobacco prevention messaging and in other health content areas. There is also a need for studies to compare the impact of self-report vs. psychophysiological data or integration of the two in selection of campaign messages on campaign-targeted beliefs and behaviors. These types of studies may shed light on how best to apply MADM in the development and evaluation of health communication campaigns.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/ijerph182211814/s1, Table S1: Message Examples.

Author Contributions

Conceptualization, E.M.S., A.C.V., G.L., and D.M.; methodology, E.M.S., A.C.V., G.L., and D.M.; formal analysis, E.M.S., A.C.V., G.L., and D.M.; investigation, E.M.S., A.C.V., G.L., T.L.W. and D.M.; resources, G.L, T.L.W., D.M.; data curation, E.M.S., A.C.V. and D.M.; writing—original draft preparation, E.M.S., A.C.V., and D.M.; writing—review and editing, E.M.S., A.C.V., G.L., T.L.W., B.K.-H., and D.M.; visualization, E.M.S. and A.C.V.; supervision, A.C.V., G.L., T.L.W., and D.M.; funding acquisition, D.M.. All authors have read and agreed to the published version of the manuscript.

Funding

Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health (NIH) and the Food and Drug Administration (FDA) Center for Tobacco Products (CTP) under grant number R01CA229082 and the National Institute on Drug Abuse (R00DA046563; Stevens, PI). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the FDA.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of the University of Oklahoma (protocol code 9135; 5 April 2019) and Georgetown University (2017-1378; 2 April 2018).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data available on request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Kasza, K.A.; Ambrose, B.K.; Conway, K.P.; Borek, N.; Taylor, K.; Goniewicz, M.L.; Cummings, K.M.; Sharma, E.; Pearson, J.L.; Green, V.R.; et al. Tobacco-Product Use by Adults and Youths in the United States in 2013 and 2014. N. Engl. J. Med. 2017, 376, 342–353. [Google Scholar] [CrossRef] [Green Version]
  2. Mays, D.; Arrazola, R.A.; Tworek, C.; Rolle, I.V.; Neff, L.J.; Portnoy, D.B. Openness to Using Non-cigarette Tobacco Products among U.S. Young Adults. Am. J. Prev. Med. 2016, 50, 528–534. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Cobb, C.O.; Shihadeh, A.; Weaver, M.F.; Eissenberg, T. Waterpipe tobacco smoking and cigarette smoking: A direct comparison of toxicant exposure and subjective effects. Nicotine Tob. Res. 2011, 13, 78–87. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Shihadeh, A.; Schubert, J.; Klaiany, J.; El Sabban, M.; Luch, A.; Saliba, N.A. Toxicant content, physical properties and biological activity of waterpipe tobacco smoke and its tobacco-free alternatives. Tob. Control 2015, 24, i22–i30. [Google Scholar] [CrossRef] [Green Version]
  5. El-Zaatari, Z.M.; Chami, H.A.; Zaatari, G.S. Health effects associated with waterpipe smoking. Tob. Control 2015, 24 (Suppl. S1), i31–i43. [Google Scholar] [CrossRef] [Green Version]
  6. Montazeri, Z.; Nyiraneza, C.; El-Katerji, H.; Little, J. Waterpipe smoking and cancer: Systematic review and meta-analysis. Tob. Control 2017, 26, 92–97. [Google Scholar] [CrossRef]
  7. Allem, J.P.; Chu, K.H.; Cruz, T.B.; Unger, J.B. Waterpipe promotion and use on instagram: #Hookah. Nicotine Tob. Res. 2017, 19, 1248–1252. [Google Scholar] [CrossRef] [PubMed]
  8. Ben Taleb, Z.; Laestadius, L.I.; Asfar, T.; Primack, B.A.; Maziak, W. #Hookahlife: The Rise of Waterpipe Promotion on Instagram. Health Educ. Behav. 2019, 46, 106–113. [Google Scholar] [CrossRef]
  9. Lopez, A.A.; Eissenberg, T.; Jaafar, M.; Afifi, R. Now is the time to advocate for interventions designed specifically to prevent and control waterpipe tobacco smoking. Addict. Behav. 2017, 66, 41–47. [Google Scholar] [CrossRef] [Green Version]
  10. Villanti, A.C.; Cobb, C.O.; Cohn, A.M.; Williams, V.F.; Rath, J.M. Correlates of hookah use and predictors of hookah trial in U.S. Young adults. Am. J. Prev. Med. 2015, 48, 742–746. [Google Scholar] [CrossRef]
  11. Villanti, A.C.; Johnson, A.L.; Rath, J.M.; Williams, V.; Vallone, D.M.; Abrams, D.B.; Hedeker, D.; Mermelstein, R.J. Identifying “social smoking” U.S. young adults using an empirically-driven approach. Addict. Behav. 2017, 70, 83–89. [Google Scholar] [CrossRef] [PubMed]
  12. Villanti, A.C.; Johnson, A.L.; Glasser, A.M.; Rose, S.W.; Ambrose, B.K.; Conway, K.P.; Cummings, K.M.; Stanton, C.A.; Edwards, K.C.; Delnevo, C.D.; et al. Association of Flavored Tobacco Use with Tobacco Initiation and Subsequent Use among US Youth and Adults, 2013–2015. JAMA Netw. Open 2019, 2, e1913804. [Google Scholar] [CrossRef] [PubMed]
  13. Mays, D.; Tercyak, K.P.; Lipkus, I.M. The effects of brief waterpipe tobacco use harm and addiction education messages among young adult waterpipe tobacco users. Nicotine Tob. Res. 2016, 18, 777–784. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Gardner, K.; Kearns, R.; Woodland, L.; Silveira, M.; Hua, M.; Katz, M.; Takas, K.; McDonald, J. A Scoping Review of the Evidence on Health Promotion Interventions for Reducing Waterpipe Smoking: Implications for Practice. Front. Public Health 2018, 6, 308. [Google Scholar] [CrossRef] [Green Version]
  15. Jawad, M.; Jawad, S.; Waziry, R.K.; Ballout, R.A.; Akl, E.A. Interventions for waterpipe tobacco smoking prevention and cessation: A systematic review. Sci. Rep. 2016, 6, 25872. [Google Scholar] [CrossRef] [Green Version]
  16. Maziak, W.; Jawad, M.; Jawad, S.; Ward, K.D.; Eissenberg, T.; Asfar, T. Interventions for waterpipe smoking cessation. Cochrane Database Syst. Rev. 2015, 7. [Google Scholar] [CrossRef] [Green Version]
  17. Food and Drug Administration; HHS Deeming Tobacco Products To Be Subject to the Federal Food, Drug, and Cosmetic Act, as Amended by the Family Smoking Prevention and Tobacco Control Act; Restrictions on the Sale and Distribution of Tobacco Products and Required Warning Statements for Toba. Fed. Regist. 2016, 81, 28973–29106.
  18. Farrelly, M.C.; Duke, J.C.; Nonnemaker, J.; MacMonegle, A.J.; Alexander, T.N.; Zhao, X.; Delahanty, J.C.; Rao, P.; Allen, J.A. Association Between The Real Cost Media Campaign and Smoking Initiation Among Youths—United States, 2014–2016. MMWR Morb. Mortal. Wkly. Rep. 2017, 66, 47–50. [Google Scholar] [CrossRef]
  19. Commissioner of the Press Announcements FDA to Expand Public Education Campaign to Focus on Prevention of Youth e-Cigarette Use. Available online: https://www.fda.gov/news-events/press-announcements/fda-expand-public-education-campaign-focus-prevention-youth-e-cigarette-use (accessed on 10 August 2018).
  20. Phan, L.; Villanti, A.C.; Leshner, G.; Wagener, T.L.; Stevens, E.M.; Johnson, A.C.; Mays, D. Development and pretesting of hookah tobacco public education messages for young adults. Int. J. Environ. Res. Public Health 2020, 17, 8752. [Google Scholar] [CrossRef]
  21. Leshner, G.; Phan, L.; Stevens, E.M.; Johnson, A.; Villanti, A.; Kim, N.; Kim, S.; Ma, H.; Seo, J.; Sun, F.; et al. Psychophysiological Responses to Hookah Tobacco Public Education Messages Among Young Adults. Health Psychol. in press.
  22. Crosby, K.; Santiago, S.; Talbert, E.C.; Roditis, M.L.; Resch, G. Bringing “The Real Cost” to Life Through Breakthrough, Evidence-Based Advertising. Am. J. Prev. Med. 2019, 56, S16–S23. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Zhao, X.; Alexander, T.N.; Hoffman, L.; Jones, C.; Delahanty, J.; Walker, M.; Berger, A.T.; Talbert, E. Youth Receptivity to FDA’s The Real Cost Tobacco Prevention Campaign: Evidence From Message Pretesting. J. Health Commun. 2016, 21, 1153–1160. [Google Scholar] [CrossRef] [Green Version]
  24. Potter, R.F.; Bolls, P.D. Psychophysiological Measurement and Meaning: Cognitive and Emotional Processing of Media; Routledge: New York, NY, USA, 2012; ISBN 0415994144. [Google Scholar]
  25. Cacioppo, J.T.; Gardner, W.L. Emotion. Annu. Rev. Psychol. 1999, 50, 191–214. [Google Scholar] [CrossRef] [PubMed]
  26. Larsen, J.T.; Berntson, G.G.; Poehlmann, K.M.; Ito, T.A.; Cacioppo, J.T. The psychophysiology of emotion. In Handbook of Emotions; Lewis, M., Haviland-Jones, J.M., Barrett, L.F., Eds.; The Guilford Press: New York, NY, USA, 2008; pp. 180–195. [Google Scholar]
  27. Atkin, C.K.; Rice, R.E. Advances In Public Communication Campaigns. In The International Encyclopedia of Media Studies; Blackwell Publishing Ltd.: Hoboken, NJ, USA, 2012. [Google Scholar]
  28. Lang, A. Using the limited capacity model of motivated mediated message processing to design effective cancer communication messages. J. Commun. 2006, 56, S57–S80. [Google Scholar] [CrossRef]
  29. Clayton, R.B.; Keene, J.R.; Leshner, G.; Lang, A.; Bailey, R.L. Smoking Status Matters: A Direct Comparison of Smokers’ and Nonsmokers’ Psychophysiological and Self-Report Responses to Secondhand Smoke Anti-Tobacco PSAs. Health Commun. 2020, 35, 925–934. [Google Scholar] [CrossRef]
  30. Yang, J.B.; Singh, M.G. An Evidential Reasoning Approach for Multiple-Attribute Decision Making with Uncertainty. IEEE Trans. Syst. Man Cybern. 1994, 24, 1–18. [Google Scholar] [CrossRef] [Green Version]
  31. Platts, K.W.; Probert, D.R.; Cáez, L. Make vs. buy decisions: A process incorporating multi-attribute decision-making. Int. J. Prod. Econ. 2002, 77, 247–257. [Google Scholar] [CrossRef]
  32. Tamosaitiene, J.; Turskis, Z.; Kazimieras, E.; Kaklauskas, A.; Tamošaitien, J. Multi-Attribute Decision-Making Model by Applying Grey Numbers. Informatica 2009, 20, 305–320. [Google Scholar]
  33. Medić, N.; Anišić, Z.; Lalic, B.; Marjanovic, U.; Brezocnik, M. Hybrid fuzzy multi-attribute decision making model for evaluation of advanced digital technologies in manufacturing: Industry 4.0 perspective. Adv. Prod. Eng. Manag. 2019, 14, 483–493. [Google Scholar] [CrossRef]
  34. Bohanec, M.; Zupan, B.; Rajkovič, V. Applications of qualitative multi-attribute decision models in health care. Int. J. Med. Inform. 2000, 58–59, 191–205. [Google Scholar] [CrossRef]
  35. Huskey, R.; Wilcox, S.; Clayton, R.B.; Keene, J.R. The limited capacity model of motivated mediated message processing: Meta-analytically summarizing two decades of research. Ann. Int. Commun. Assoc. 2020, 44, 322–349. [Google Scholar] [CrossRef]
  36. Lipkus, I.M.; Mays, D.; Tercyak, K.P. Characterizing young adults’ susceptibility to waterpipe tobacco use and their reactions to messages about product harms and addictiveness. Nicotine Tob. Res. 2017, 19, 1216–1223. [Google Scholar] [CrossRef] [PubMed]
  37. Ramôa, C.P.; Shihadeh, A.; Salman, R.; Eissenberg, T. Group waterpipe tobacco smoking increases smoke toxicant concentration. Nicotine Tob. Res. 2016, 18, 770–776. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  38. Johnson, A.C.; Lipkus, I.; Tercyak, K.P.; Luta, G.; Rehberg, K.; Phan, L.; Abroms, L.C.; Mays, D. Development and Pretesting of Risk-Based Mobile Multimedia Message Content for Young Adult Hookah Use. Health Educ. Behav. 2019, 46, 97–105. [Google Scholar] [CrossRef] [PubMed]
  39. Cobb, C.; Ward, K.D.; Maziak, W.; Shihadeh, A.L.; Eissenberg, T. Waterpipe tobacco smoking: An emerging health crisis in the United States. Am. J. Health Behav. 2010, 34, 275–285. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Leavens, E.L.S.; Meier, E.; Brett, E.I.; Stevens, E.M.; Tackett, A.P.; Villanti, A.C.; Wagener, T.L. Polytobacco use and risk perceptions among young adults: The potential role of habituation to risk. Addict. Behav. 2019, 90, 278–284. [Google Scholar] [CrossRef] [PubMed]
  41. Akl, E.A.; Ward, K.D.; Bteddini, D.; Khaliel, R.; Alexander, A.C.; Lotfi, T.; Alaouie, H.; Afifi, R.A. The allure of the waterpipe: A narrative review of factors affecting the epidemic rise in waterpipe smoking among young persons globally. Tob. Control 2015, 24, i13–i21. [Google Scholar] [CrossRef]
  42. Villanti, A.C.; Johnson, A.L.; Ambrose, B.K.; Cummings, K.M.; Stanton, C.A.; Rose, S.W.; Feirman, S.P.; Tworek, C.; Glasser, A.M.; Pearson, J.L.; et al. Flavored Tobacco Product Use in Youth and Adults: Findings From the First Wave of the PATH Study (2013–2014). Am. J. Prev. Med. 2017, 53, 139–151. [Google Scholar] [CrossRef]
  43. Chandler, J.; Shapiro, D. Conducting Clinical Research Using Crowdsourced Convenience Samples. Annu. Rev. Clin. Psychol. 2016, 12, 53–81. [Google Scholar] [CrossRef] [Green Version]
  44. Mason, W.; Suri, S. Conducting behavioral research on Amazon’s Mechanical Turk. Behav. Res. Methods 2012, 44, 1–23. [Google Scholar] [CrossRef]
  45. Zhao, X.; Nan, X. Influence of Self-Affirmation on Responses to Gain- Versus Loss-Framed Antismoking Messages. Hum. Commun. Res. 2010, 36, 493–511. [Google Scholar] [CrossRef]
  46. Dillard, J.P.; Shen, L. On the Nature of Reactance and its Role in Persuasive Health Communication. Commun. Monogr. 2005, 72, 144–168. [Google Scholar] [CrossRef]
  47. Macmillan, N.A.; Creelman, C.D. Detection Theory: A User’s Guide; Psychology Press: Hove, UK, 2004. [Google Scholar]
  48. Tzeng, G.H.; Huang, J.-J. Multiple Attribute Decision Making: Methods and Applications; Chapman and Hall: Boca Raton, FL, USA, 2011. [Google Scholar]
  49. Villanti, A.C.; LePine, S.E.; West, J.C.; Cruz, T.B.; Stevens, E.M.; Tetreault, H.J.; Unger, J.B.; Wackowski, O.A.; Mays, D. Identifying message content to reduce vaping: Results from online message testing trials in young adult tobacco users. Addict. Behav. 2021, 115, 106778. [Google Scholar] [CrossRef] [PubMed]
  50. Leshner, G.; Stevens, E.M.; Cohn, A.M.; Kim, S.; Kim, N.; Wagener, T.L.; Villanti, A.C. Cognitive and affective responses to marijuana prevention and educational messaging. Drug Alcohol Depend. 2021, 225, 108788. [Google Scholar] [CrossRef]
  51. Kraemer, J.D.; Strasser, A.A.; Lindblom, E.N.; Niaura, R.S.; Mays, D. Crowdsourced data collection for public health: A comparison with nationally representative, population tobacco use data. Prev. Med. 2017, 102, 93–99. [Google Scholar] [CrossRef]
Table
Table 1. Multi-Attribute Decision Matrix for Crowdsourced and Psychophysiology Waterpipe Tobacco Smoking Message Testing.
Table 1. Multi-Attribute Decision Matrix for Crowdsourced and Psychophysiology Waterpipe Tobacco Smoking Message Testing.
Crowdsourced (Self-Report)Psychophysiological (Lab)
ReceptivityEngagementAttitudeNegative EmotionTotalHeart Rate
Deceleration		Recognition
Accuracy		Visual AttentionTotal
MessageMean
(SE)		RankMean
(SE)		RankMean
(SE)		RankMean
(SE)		RankSum of
Attribute Ranks		RankMean (SD)RankMean (SD)RankMean (SD)RankSum of
Attribute Ranks		Rank
H15.29 (0.16)15.77 (0.14)44.86 (0.18)32.20 (0.11)21012.82 (0.50)30.871 (0.03)41220.72 (730.11)121812
H24.92 (0.19)65.94 (0.16)14.53 (0.20)72.04 (0.12)31752.87 (0.57)50.677 (0.03)31375.4 (779.43)1141
HS15.15 (0.19)25.74 (0.16)64.85 (0.20)42.04 (0.12)31542.45 (0.57)60.698 (0.03)61287.5 (786.94)102510
HS25.04 (0.17)45.78 (0.15)34.98 (0.19)22.23 (0.11)11012.16 (0.47)80.878 (0.02)81298.62 (763.33)6156
HF14.89 (0.18)75.74 (0.15)64.64 (0.19)51.99 (0.12)62472.24 (0.51)70.765 (0.03)71293.23 (765.70)8228
HF25.10 (0.19)35.77 (0.16)45.01 (0.20)12.01 (0.12)51332.60 (0.52)40.798 (0.03)51335.34 (796.33)3123
A15.01 (0.17)55.81 (0.14)24.54 (0.18)61.88 (0.11)82161.65 (0.51)100.769 (0.03)101317.9 (781.80)4204
A24.71 (0.16)85.63 (0.14)104.51 (0.18)91.99 (0.11)63381.98 (0.47)90.803 (0.03)91289.96 (781.45)9219
AS14.50 (0.18)115.56 (0.16)114.53 (0.20)71.81 (0.12)103991.64 (0.45)110.517 (0.04)111317.12 (764.58)5285
AS24.10 (0.17)125.73 (0.14)84.23 (0.18)121.73 (0.11)1244122.88 (0.51)20.643 (0.03)21295.01 (745.07)7207
AF14.57 (0.17)105.38 (0.15)124.47 (0.19)111.83 (0.12)942113.03 (0.52)10.798 (0.03)11344.93 (806.43)272
AF24.63 (0.18)95.64 (0.15)94.48 (0.20)101.74 (0.12)113991.47 (0.43)120.727 (0.03)121261.06 (752.70)113111
Note: Message receptivity was assessed on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree). Engagement was assessed on a scale from 1 (none) to 7 (very). Positive attitude toward the message was assessed on semantic differential scales from 1 to 7. Negative emotions were assessed on a scale from 1 (not at all) to 4 (extremely). Visual attention is measured in milliseconds.
Table
Table 2. Ranking of waterpipe tobacco smoking messages by weighting scenario.
Table 2. Ranking of waterpipe tobacco smoking messages by weighting scenario.
Equal Attribute Ranks (4 Self-Report, 3 Lab)Equal Study Weights (Total Self-Report Rank, Total Lab Rank)Prefer Self-Report (Individual Self-Report Attribute Ranks, Total Lab Rank)Prefer Lab (Total Self-Report Rank, Individual Lab Attribute Ranks)
MessageSummary ScoreRankSummary ScoreRankSummary ScoreRankSummary ScoreRank
H14.036.55 *4.444.754 *
H24.443.01 *3.634.754 *
HS15.757.07 *55 *7.257 *
HS23.61 *3.533.21 *4.002
HF16.677.596.477.257 *
HF23.61 *3.01 *3.21 *3.751
A15.965.0455 *6.506
A27.798.5108.487.257 *
AS19.6117.07 *8.89 *9.2511
AS29.1109.51110.2128.0010
AF17.086.55 *8.89 *4.503
AF210.01210.012101110.0012
Note: * denotes tied ranking.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Stevens, E.M.; Villanti, A.C.; Leshner, G.; Wagener, T.L.; Keller-Hamilton, B.; Mays, D. Integrating Self-Report and Psychophysiological Measures in Waterpipe Tobacco Message Testing: A Novel Application of Multi-Attribute Decision Modeling. Int. J. Environ. Res. Public Health 2021, 18, 11814. https://doi.org/10.3390/ijerph182211814

AMA Style

Stevens EM, Villanti AC, Leshner G, Wagener TL, Keller-Hamilton B, Mays D. Integrating Self-Report and Psychophysiological Measures in Waterpipe Tobacco Message Testing: A Novel Application of Multi-Attribute Decision Modeling. International Journal of Environmental Research and Public Health. 2021; 18(22):11814. https://doi.org/10.3390/ijerph182211814

Chicago/Turabian Style

Stevens, Elise M., Andrea C. Villanti, Glenn Leshner, Theodore L. Wagener, Brittney Keller-Hamilton, and Darren Mays. 2021. "Integrating Self-Report and Psychophysiological Measures in Waterpipe Tobacco Message Testing: A Novel Application of Multi-Attribute Decision Modeling" International Journal of Environmental Research and Public Health 18, no. 22: 11814. https://doi.org/10.3390/ijerph182211814

APA Style

Stevens, E. M., Villanti, A. C., Leshner, G., Wagener, T. L., Keller-Hamilton, B., & Mays, D. (2021). Integrating Self-Report and Psychophysiological Measures in Waterpipe Tobacco Message Testing: A Novel Application of Multi-Attribute Decision Modeling. International Journal of Environmental Research and Public Health, 18(22), 11814. https://doi.org/10.3390/ijerph182211814

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop