Next Article in Journal
Navigating Infodemics, Unlocking Social Capital and Maintaining Food Security during the COVID-19 First Wave in the UK: Older Adults’ Experiences
Previous Article in Journal
Effect of Chinese Corporate Average Fuel Consumption and New Energy Vehicle Dual-Credit Regulation on Passenger Cars Average Fuel Consumption Analysis
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 18
Issue 14
10.3390/ijerph18147219
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Prevalence of Missing Values and Protest Zeros in Contingent Valuation in Dental Medicine

by
Pedram Sendi
1,2,*,†,
Arta Ramadani
1,† and
Michael M. Bornstein
1
1
Department of Oral Health and Medicine, University Center for Dental Medicine Basel UZB, University of Basel, Mattenstrasse 40, 4058 Basel, Switzerland
2
Institute for Clinical Epidemiology, Basel University Hospital, Spitalstrasse 12, 4031 Basel, Switzerland
*
Author to whom correspondence should be addressed.
†
P.S. and A.R. contributed equally to this study.
Int. J. Environ. Res. Public Health 2021, 18(14), 7219; https://doi.org/10.3390/ijerph18147219
Submission received: 13 May 2021 / Revised: 30 June 2021 / Accepted: 2 July 2021 / Published: 6 July 2021
(This article belongs to the Section Oral Health)
Versions Notes

Abstract

:
Background: The number of contingent valuation (CV) studies in dental medicine using willingness-to-pay (WTP) methodology has substantially increased in recent years. Missing values due to absent information (i.e., missingness) or false information (i.e., protest zeros) are a common problem in WTP studies. The objective of this study is to evaluate the prevalence of missing values in CV studies in dental medicine, to assess how these have been dealt with, and to suggest recommendations for future research. Methods: We systematically searched electronic databases (MEDLINE, Web of Science, Cochrane Library, PROSPERO) on 8 June 2021, and hand-searched references of selected reviews. CV studies in clinical dentistry using WTP for valuing a good or service were included. Results: We included 49 WTP studies in our review. Out of these, 19 (38.8%) reported missing values due to absent information, and 28 (57.1%) reported zero values (i.e., WTP valued at zero). Zero values were further classified into true zeros (i.e., representing the underlying preference of the respondent) or protest zeros (i.e., false information as a protest behavior) in only 9 studies. Most studies used a complete case analysis to address missingness while only one study used multiple imputation. Conclusions: There is uncertainty in the dental literature on how to address missing values and zero values in CV studies. Zero values need to be classified as true zeros versus protest zeros with follow-up questions after the WTP elicitation procedure, and then need to be handled differently. Advanced statistical methods are available to address both missing values due to missingness and due to protest zeros but these are currently underused in dental medicine. Failing to appropriately address missing values in CV studies may lead to biased WTP estimates of dental interventions.

1. Introduction

In times of increasing pressures to contain healthcare resource consumption, there is an increasing interest in valuing dental healthcare goods and services using economic methods such as contingent valuation (CV) [1,2]. CV studies assess the value of a commodity by asking the respondent the maximum in monetary terms he is willing to pay (WTP) to receive a good or service, or alternatively, the minimum he is willing to accept (WTA) to forgo the good or service offered [3,4]. CV studies were first used in environmental economics for valuing goods and services for which no free real-world market exists, but have then gained more attention in healthcare, and more recently, also in dental medicine [1,2,5,6,7]. Other economic methods for valuing health states and interventions such as the time-trade-off method or standard-gamble method allow one to estimate health state utilities that are often used to compute composite outcomes such as quality-adjusted life-years (QALYs), or more oral health-related outcomes such as quality-adjusted tooth-years (QATYs) or quality-adjusted prosthesis-years (QAPYs) [8,9,10]. However, these outcomes do not allow a direct comparison of the benefit of a healthcare good or service with its associated costs. CV studies, on the other hand, ask the respondent to explicitly state their preference in monetary terms, and can therefore be used in cost-benefit analyses where the costs are directly compared with the benefits in monetary terms [11,12].
A common problem in many studies is missing values, usually because of absent information [13,14,15]. In CV studies, this may be due to the respondent not answering the WTP question because they may not have understood the question or because he or she is reluctant to answer the question. Pilot testing a WTP instrument may, therefore, prove to be helpful in identifying possible problems in the WTP elicitation procedure. Missing values are classified as missing completely at random (MCAR), missing at random (MAR), or missing not at random (MNAR) [13,14]. MCAR implies that the reason for missingness is unrelated to the data and caused by chance alone, i.e., the respondents with absent information do not differ from those who answered the WTP question. In that case, a complete case analysis is often performed, leaving out the subset of respondents with missing data [16]. Besides the assumption of MCAR being often unrealistic in practice, a complete case analysis leads to a loss of power [13,14,16]. MAR implies that the probability of missingness can be explained by the observed data. If, for example, younger respondents produce less missing values in WTP elicitation than older respondents, and MCAR within each age group is assumed, then we can use age group and statistical missing data methods, such as multiple imputation (MI), to replace missing WTP data with reasonable values [13,17,18]. MNAR implies that the reason for missingness cannot be explained by the observed data. MNAR is more complex and, in that case, more research is needed to elaborate on the reasons for missingness; usually sensitivity analyses need to be performed when missing data methods are used. However, when many variables with respect to patient characteristics are recorded in a study, usually MAR is assumed and methods such as MI are used to generate a reasonable dataset for further analysis [13,14,17].
A further problem specific to CV studies are zero values [7,19,20]. Respondents may value a good or service with zero WTP, implying that they are not willing to pay any amount of money for the good or service provided. This may be due to economic reasons, i.e., because the commodity has no value to them (i.e., true zeros), or due to non-economic reasons. Non-economic reasons may, for example, represent their opinion that the government or insurance should pay for the service [19,20]. Their zero WTP, therefore, does not reflect their true underlying valuation of the commodity, and is hence termed as a protest zero or protest answer. Protest zeros can be interpreted as missing values due to false information. Although several statistical methods such as the Heckman model have been suggested to handle zero values in CV studies, more recently, it has been proposed to replace protest zeros using missing data methods such as MI, assuming MAR, to generate a reasonable dataset for further analysis [19]. Since the literature in CV studies in dental medicine has been rapidly increasing in recent years, we believe that a review of the methods used to address missing data in the hitherto published studies would help to shed light on the current status quo.
The aim of the present study is, therefore, to (i) estimate the prevalence of missing and zero values in CV studies in dentistry, (ii) evaluate whether a distinction was made between true zeros and protest zeros in CV, (iii) evaluate how missing and zero values were dealt with in CV, and (iv) suggest recommendations for future research.

2. Materials and Methods

2.1. Systematic Literature Search

We searched the databases MEDLINE via PubMed and Web of Science on 8 June 2021 using a combination of Medical Subject Headings (MeSH in PubMed) and general search terms (PubMed and Web of Science). The search strategy for Web of Science included the query: (“Willingness-to-pay” OR WTP OR “Willingness to accept” OR WTA OR “Contingent valuation” OR “Conjoint analysis” OR “Cost benefit analysis” OR “Discrete choice experiment” OR “Monetary value”) AND Topic: (“dent*). The search strategy for PubMed included the query: (“Willingness-to-pay” OR WTP OR “Willingness to accept” OR WTA OR “Contingent valuation” OR “Conjoint analysis” OR “Cost benefit analysis” OR “Discrete choice experiment” OR “Monetary value”) AND (Dentistry (MeSH) or “dent”). In addition, we hand-searched the references of two published systematic reviews on WTP estimates for dental interventions [1,2]. We also searched the Cochrane Library (www.cochranelibrary.com, accessed 8 June 2021) and the PROSPERO database (www.crd.york.ac.uk/prospero, accessed 8 June 2021) for published or eventual ongoing studies that assessed WTP using CV as a stated preference method on 8 June 2021. We did not limit the time span for search in the respective databases.
After exclusion of duplicates, titles and abstracts were screened by two reviewers (A.R. and P.S.) and further considered using the following eligibility criteria: publication in English (considered as the scientific world language); empirical studies evaluating a clinical intervention or a dental health state; reporting WTP estimates using CV methodology for a dental service or good, and full-text available. If, from the abstract and titles eligibility was not clear, the full-text article was screened to assess eligibility. For each included article, we extracted the following study characteristics: first author, year of publication, country, dental good or service provided, sample size, CV elicitation method, reporting of missing values, how missing values were dealt with, reporting of zero values, whether a distinction was made between true zeros and protest zeros, and statistical methodology used. In order to minimize extraction errors, all data were extracted by two independent reviewers twice and then compared. Differences were resolved by discussion.

2.2. Analysis of Extracted Data

The proportion of missing values due to missingness (i.e., absent information) and zero values were collected from the included CV studies, and 95% confidence intervals were estimated for each statistic. In addition, the proportion of studies that made a distinction between true zeros and protest zeros (i.e., missing values due to false information) and the respective 95% confidence intervals were estimated. The statistical analyses were performed using R and RStudio (Version 1.2.1335, RStudio, Inc., Boston, MA, USA).

3. Results

Databases were searched on 8 June 2021 (MEDLINE, Web of Science, Cochrane Library and PROSPERO database). A total of 1705 records were initially identified through database searches (see Supplementary Material), with 1494 remaining after removal of duplicates for title and abstract screening. A total of 65 articles were included for full-text screening from database searches and 15 articles from citation searches, and of these, 49 remained for qualitative and quantitative analysis (see Figure S1 in Supplementary Material). Figure S1 displays the PRISMA 2020 (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.
Table 1 displays the descriptive characteristics of the studies included in this study. Missing values due to absent information (i.e., missingness) was reported in 19 out of 49 studies, corresponding to a prevalence of 38.8% (95% CI: 25.5–53.8%). Zero values in WTP estimates were reported in 28 out of 49 studies (Table 1), resulting in a prevalence of 57.1% (95% CI: 42.3–70.9%). Out of the 28 studies that reported zero WTP values, only 9 studies made a distinction between true zeros and protest zeros (32.1%, 95% CI: 16.6–52.4%) (Table 1). Only one study used MI to replace missing values due to missingness, whereas in the remainder of the studies, a complete case analysis was performed (Table 1). When zero values were included in the statistical analysis, one study used a Heckman selection model, whereas six studies applied a Tobit regression model to analyze the data (Table 1).

4. Discussion

In the present study, it was shown that a substantial proportion of CV studies (38.8%) reported missing values due to absent information, and even a larger proportion (57.1%) reported zero values. Zero values were further classified into true zeros (i.e., representing the underlying preference of the respondent) or protest zeros (i.e., false information as a protest behavior) only in less than one-third of the studies reporting zero values. Most studies used a complete case analysis to address missingness while only one study used MI [44].

4.1. Missing Values

Missing values are a common problem in medical and dental research [15,70], and they occur when information on a specific variable is absent and not recorded in the dataset. There may be different reasons for missing values in CV studies: (i) the respondent refuses to respond to the question, (ii) the respondent does not understand the WTP exercise and therefore does not provide a WTP value, or (iii) data may get lost due to investigator or computational error. If data are MCAR, then the dataset without missing values is a representative subsample of the complete dataset, and missingness does not depend on the observed as well as unobserved data [13,70]. If data are MAR, the probability of missing depends on the observed data, i.e., missing data can be explained by the observed variables and are independent of unobserved variables [13,14]. This is usually the most common assumption about the underlying mechanism of missing data. If missing data are neither MCAR or MAR, then they are MNAR [13]. MNAR implies that the probability of missing data cannot be explained by observed data and that the probability of missingness depends on the value of the missing data itself [13,14,15,70].
There are different approaches to handle missing data. The most commonly used method is complete case analysis where incomplete records are excluded from the analysis [16]. This approach, however, assumes MCAR and leads to a loss of power. Another approach is simple imputation such as mean value imputation [13]. Hereby the missing value is replaced with the mean value of those records where information is available. This approach, however, artificially reduces variance in that variable and the respective standard errors are thus deflated [13,16]. Furthermore, mean value imputation ignores the multivariate relationship between variables. A more advanced approach to handle missing data is conditional mean imputation [13]. In conditional mean imputation, a regression model is used to impute a single missing value conditional on the observed variables. A drawback of the conditional mean imputation method is that it amplifies the multivariate relationship between variables, and implies that after imputation, the missing value is known with certainty [13]. Finally, MI usually represents the most appropriate method to handle missing data as it respects that there is uncertainty with respect to the imputed value of a missing record [13,14,17,18]. In MI, multiple plausible values conditional on the observed data are generated for each missing record, resulting in multiple complete imputed datasets. The analysis is then performed for each of the imputed complete datasets and the results are thereafter pooled [13,14,17,18]. This allows the analysts to explicitly incorporate the uncertainty with respect to the true value of the missing record in the dataset. There are different statistical techniques to conduct MI such as MI by chained equations (MICE) or joint modeling (JM) [13,14,17,18]. From the 49 studies included, MICE to replace missing records has only been used by Stone et al. [44] and also in a reanalysis of the study by Sendi et al. [57,71]. Given the high prevalence of missing values of 38.8% due to absent information in the present study, this clearly shows that MI is underused in CV studies in dental medicine. It is certainly noteworthy that the present results are in line with other studies that evaluated how missing values were dealt with in medical research. For example, Vesin et al. reported that in clinical studies in intensive care, out of 44 published manuscripts, 45% reported missing values but only 2.5% used sophisticated methods to address this issue [15].

4.2. Zero Values

CV studies using WTP are a stated preference method where respondents directly express their valuation of a good or service, as opposed to other methods that assess the valuation of a commodity based on observed behavior using revealed preference methods such as discrete choice experiments [3,12]. CV methods were first used in environmental economics to value goods or services for which no market and, hence, price exists [72]. However, it has been recognized that CV is also a useful methodology for valuing a commodity in healthcare and its use in medicine, and more recently, also in dentistry which has substantially increased in recent years [1,2,71]. CV studies usually elicit the preference of respondents by asking them how much they are willing to pay for a good or service using a variety of different questioning methods such as open-ended questions, bidding games or payment cards [3,12,73]. WTP can also be interpreted as an alternative to other economic methods for valuing a health state such as the time-trade-off and standard gamble method. However, when using WTP, health benefits are directly expressed in monetary terms [10,11]. Nonetheless, there are also some potential biases associated with WTP elicitation methods such as strategic behavior, framing effects, response uncertainty and protest responses [19,20,74,75]. While respondents may express their protest by refusing to place a monetary value on a commodity, resulting in a missing value, some respondents may place a zero WTP on a service or good for non-economic reasons, such as their opinion that the government or insurance should cover the costs [19,20].
For example, insured individuals may value mandibular overdentures supported by two dental implants at zero WTP with the argument that they are not responsible for covering the costs (i.e., non-economic reason). This zero WTP valuation, however, does not reflect their true underlying preference and is hence termed as a protest zero. On the other hand, if non-insured individuals with otherwise no financial support value such a 2-implant mandibular overdenture with zero WTP, with the argument that the treatment is too expensive (i.e., economic reason), then the response is considered as a true zero and reflects the true underlying preference for the commodity. Protest zeros are a common problem in CV studies and may range from 50% up to 73% of all zero valuations and should be treated differently than true zeros [20]. In the present study, it was found that 57.1% of all WTP studies in dentistry reported zero values, and of those, only in 32.1% was a distinction made between true zeros and protest zeros. It has been suggested that in case of a zero valuation, the motivation for a zero valuation should be elicited in follow-up questions so that true zeros can be separated from protest zeros [19,20,21,75]. As some protest responses may also include a non-response in WTP elicitation, resulting in a missing value, it may be helpful to also evaluate the reason for a non-response using follow-up questions.
There is no common consensus on how zero values should be treated in analyzing the dataset [19,20]. However, methods to deal with protest responses can be separated into (i) those that deal with this issue in the data management phase (i.e., complete case analysis or imputation) and (ii) those that address zero values in the analysis phase (e.g., Heckman model, Tobit 2 model). The most commonly used approach is to remove protest zeros from the dataset [19,20]. However, this implies that those who protested are a representative subsample of those who did not, which is usually not true. Including protest zeros in the analysis leads to an underestimation of the mean WTP, and excluding protest zeros may lead to a distorted estimation of the mean WTP and reduced precision, i.e., a loss of statistical power. An alternative more advanced method is to treat protest responses as missing values and replace protest zeros with reasonable values using MI based on observed data such as socioeconomic variables and patient characteristics [19]. In a recent simulation study, it was shown that MI leads to less bias than addressing zero values in the analysis phase using alternative statistical models such as the Heckman model [19].
The Heckman selection model addresses non-response by adjusting for the probability of being a protester and, therefore, explicitly models that protesters may not be a representative subsample of the population analyzed. However, the Heckman model may be sensitive to other statistical issues such as deviations from the normality assumption, which is often encountered in practice [19]. In the present review, only one study used a Heckman model to address the problem of zero values in WTP elicitation. Other statistical models such as a Tobit 2 regression and the Double-Hurdle method have also been suggested to address non-responses [20,76]. However, these models do not make a distinction between protest zeros and true zeros per se; and their detailed discussion is beyond the scope of the present paper. The statistical methodology applied by the included studies is broad and also includes Tobit regression models, which account for the density of zero values (Table 1). Most of the included studies, however, used multivariate linear models (Table 1). Nevertheless, it should be noted that on a continuous linear scale, WTP values range from (true) zero to infinity and cannot be negative. The advantage of using MI for handling protest zeros in the data management phase is that it increases statistical power and does not predetermine the statistical model to be used when analyzing the data, so that any model can be used that best explains the data.

5. Conclusions

The present study shows that missing values and protest responses are common in CV studies in dental medicine, and that there is substantial uncertainty in addressing these issues. In cases of missing responses and zero WTP values, it is recommended to elicit additional information for the reason of such a response using follow-up questions. Zero WTP values should then be separated into true zeros due to economic reasons, and protest zeros based on non-economic reasoning. It is advisable to use multiple imputation to replace missing values due to missingness and protest zeros in the data management phase. This increases statistical power and does not predetermine the statistical model to be used in the analysis phase. The present study thus helps to clarify how missing values and protest zeros may be handled in future CV studies in dental medicine.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/ijerph18147219/s1.

Author Contributions

Conceptualization, P.S., A.R. and M.M.B.; methodology, P.S., A.R. and M.M.B.; formal analysis, P.S. and A.R.; data curation, P.S. and A.R.; writing—original draft preparation, A.R. and P.S.; writing—review and editing, M.M.B.; supervision, P.S. and M.M.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors thank Jennifer Hockley for the language editing of the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Tan, S.H.X.; Vernazza, C.R.; Nair, R. Critical review of willingness to pay for clinical oral health interventions. J. Dent. 2017, 64, 1–12. [Google Scholar] [CrossRef] [Green Version]
  2. Saadatfar, N.; Jadidfard, M.P. An overview of the methodological aspects and policy implications of willingness-to-pay studies in oral health: A scoping review of existing literature. BMC Oral Health 2020, 20, 1–12. [Google Scholar] [CrossRef]
  3. Gafni, A. Willingness to pay. What’s in a name? Pharmacoeconomics 1998, 14, 465–470. [Google Scholar] [CrossRef]
  4. O’brien, B.J.; Novosel, S.; Torrance, G.; Streiner, D. Assessing the Economic Value of a New Antidepressant. A willingness-to-pay approach. PharmacoEconomics 1995, 8, 34–45. [Google Scholar] [CrossRef]
  5. Tunçel, T.; Hammitt, J.K. A new meta-analysis on the WTP/WTA disparity. J. Environ. Econ. Manag. 2014, 68, 175–187. [Google Scholar] [CrossRef]
  6. Horowitz, J.K.; McConnell, K.E. A Review of WTA/WTP Studies. J. Environ. Econ. Manag. 2002, 44, 426–447. [Google Scholar] [CrossRef]
  7. Rotteveel, A.H.; Lambooij, M.S.; Zuithoff, N.P.A.; Van Exel, J.; Moons, K.G.M.; De Wit, G.A. Valuing Healthcare Goods and Services: A Systematic Review and Meta-Analysis on the WTA-WTP Disparity. PharmacoEconomics 2020, 38, 443–458. [Google Scholar] [CrossRef] [Green Version]
  8. Sendi, P.; Oppliger, N.; Chakroun, F.; Marinello, C.P.; Bucher, H.C.; Bornstein, M.M. Health State Utilities in Edentulous Patients: A Time Trade-off Approach. JDR Clin. Transl. Res. 2018, 3, 346–352. [Google Scholar] [CrossRef]
  9. Zitzmann, N.U.; Marinello, C.P.; Sendi, P. A cost-effectiveness analysis of implant overdentures. J. Dent. Res. 2006, 85, 717–721. [Google Scholar] [CrossRef]
  10. Birch, S.; Ismail, A. Patient preferences and the measurement of utilities in the evaluation of dental technologies. J. Dent. Res. 2002, 81, 446–450. [Google Scholar] [CrossRef] [Green Version]
  11. Gafni, A. Alternatives to the QALY measure for economic evaluations. Support Care Cancer 1997, 5, 105–111. [Google Scholar] [CrossRef] [PubMed]
  12. Gafni, A. Willingness-to-pay (WTP): The new-old kid on the economic evaluation block. Can. J. Nurs. Res. 2001, 33, 59–64. [Google Scholar]
  13. Austin, P.C.; White, I.R.; Lee, D.S.; van Buuren, S. Missing Data in Clinical Research: A Tutorial on Multiple Imputation. Can. J. Cardiol. 2020, 0828. [Google Scholar] [CrossRef]
  14. Donders, A.R.T.; van der Heijden, G.J.; Stijnen, T.; Moons, K.G. Review: A gentle introduction to imputation of missing values. J. Clin. Epidemiol. 2006, 59, 1087–1091. [Google Scholar] [CrossRef]
  15. Vesin, A.; Azoulay, E.; Ruckly, S.; Vignoud, L.; Rusinovà, K.; Benoit, D.; Soares, M.; Azeivedo-Maia, P.; Abroug, F.; Benbe-nishty, J.; et al. Reporting and handling missing values in clinical studies in intensive care units. Intensive Care Med. 2013, 39, 1396–1404. [Google Scholar] [CrossRef] [PubMed]
  16. van der Heijden, G.J.M.G.; Donders, A.R.T.; Stijnen, T.; Moons, K.G. Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: A clinical example. J. Clin. Epidemiol. 2006, 59, 1102–1109. [Google Scholar] [CrossRef] [PubMed]
  17. Azur, M.J.; Stuart, E.; Frangakis, C.; Leaf, P.J. Multiple imputation by chained equations: What is it and how does it work? Int. J. Methods Psychiatr. Res. 2011, 20, 40–49. [Google Scholar] [CrossRef]
  18. White, I.R.; Royston, P.; Wood, A.M. Multiple imputation using chained equations: Issues and guidance for practice. Stat. Med. 2010, 30, 377–399. [Google Scholar] [CrossRef]
  19. Pennington, M.; Gomes, M.; Donaldson, C. Handling Protest Responses in Contingent Valuation Surveys. Med. Decis. Mak. 2017, 37, 623–634. [Google Scholar] [CrossRef] [PubMed]
  20. Rankin, J.; Robinson, A. Accounting for Protest Zeros in Contingent Valuation Studies: A Review of Literature; HEG Working Paper, No. 18-01; University of East Anglia, Health Economics Group (HEG): Norwich, UK, 2018. [Google Scholar]
  21. Dixon, S.; Shackley, P. Estimating the benefits of community water fluoridation using the willingness-to-pay technique: Results of a pilot study. Community Dent. Oral Epidemiol. 1999, 27, 124–129. [Google Scholar] [CrossRef]
  22. Matthews, D.C.; Birch, S.; Gafni, A.; Di Censo, A.; Msc, D.P.S. Willingness to Pay for Periodontal Therapy: Development and Testing of an Instrument. J. Public Health Dent. 1999, 59, 44–51. [Google Scholar] [CrossRef] [PubMed]
  23. Matthews, D.; Rocchi, A.; Gafni, A. Putting your money where your mouth is: Willingness to pay for dental gel. PharmacoEconomics 2002, 20, 245–255. [Google Scholar] [CrossRef]
  24. Birch, S.; Sohn, W.; Ismail, A.I.; Lepkowski, J.M.; Belli, R.F. Willingness to pay for dentin regeneration in a sample of dentate adults. Community Dent. Oral Epidemiol. 2004, 32, 210–216. [Google Scholar] [CrossRef] [Green Version]
  25. Halvorsen, B.; Willumsen, T. Willingness to pay for dental fear treatment. Is supplying dental fear treatment socially beneficial? Eur. J. Health Econ. 2004, 5, 299–308. [Google Scholar] [CrossRef]
  26. Groot, W.; Van Merode, G. Willingness and ability of Bulgarian consumers to pay for improved public health care services. Appl. Econ. 2004, 36, 1117–1130. [Google Scholar] [CrossRef]
  27. Smith, A.S.A.; Cunningham, S.J. Which factors influence willingness-to-pay for orthognathic treatment? Eur. J. Orthod. 2004, 26, 499–506. [Google Scholar] [CrossRef]
  28. Tamaki, Y.; Nomura, Y.; Teraoka, K.; Nishikahara, F.; Motegi, M.; Tsurumoto, A.; Hanada, N. Characteristics and willingness of patients to pay for regular dental check-ups in Japan. J. Oral Sci. 2004, 46, 127–133. [Google Scholar] [CrossRef]
  29. Van Steenberghe, D.; Bercy, P.; De Boever, J.; Adriaens, P.; Geers, L.; Hendrickx, E.; Adriaenssen, C.; Rompen, E.; Malmenäs, M.; Ramsberg, J. Patient Evaluation of a Novel Non-Injectable Anesthetic Gel: A Multicenter Crossover Study Comparing the Gel to Infiltration Anesthesia During Scaling and Root Planing. J. Periodontol. 2004, 75, 1471–1478. [Google Scholar] [CrossRef]
  30. Atchison, K.A.; Gironda, M.W.; Black, E.E.; Schweitzer, S.; Der-Martirosian, C.; Felsenfeld, A.; Leathers, R.; Belin, T.R. Baseline Characteristics and Treatment Preferences of Oral Surgery Patients. J. Oral Maxillofac. Surg. 2007, 65, 2430–2437. [Google Scholar] [CrossRef] [Green Version]
  31. Balevi, B.; Shepperd, S. The management of an endodontically abscessed tooth: Patient health state utility, decision-tree and economic analysis. BMC Oral Health 2007, 7, 17. [Google Scholar] [CrossRef] [Green Version]
  32. Oscarson, N.; Lindholm, L.; Källestål, C. The value of caries preventive care among 19-year olds using the contingent valuation method within a cost-benefit approach. Community Dent. Oral Epidemiol. 2007, 35, 109–117. [Google Scholar] [CrossRef] [PubMed]
  33. Tianviwat, S.; Chongsuvivatwong, V.; Birch, S. Prevention versus cure: Measuring parental preferences for sealants and fillings as treatments for childhood caries in Southern Thailand. Health Policy 2008, 86, 64–71. [Google Scholar] [CrossRef] [PubMed]
  34. Tuominen, R. Evaluation of three methods assessing the relative value of a dental program. Acta Odontol. Scand. 2008, 66, 82–87. [Google Scholar] [CrossRef] [PubMed]
  35. Esfandiari, S.; Lund, J.P.; Penrod, J.R.; Savard, A.; Thomason, J.M.; Feine, J.S. Implant overdentures for edentulous elders: Study of patient preference. Gerodontology 2009, 26, 3–10. [Google Scholar] [CrossRef] [PubMed]
  36. Rosvall, M.D.; Fields, H.W.; Ziuchkovski, J.; Rosenstiel, S.F.; Johnston, W.M. Attractiveness, acceptability, and value of orthodontic appliances. Am. J. Orthod. Dentofac. Orthop. 2009, 135, 276.e1–276.e12. [Google Scholar] [CrossRef]
  37. Leung, K.C.M.; McGrath, C.P.J. Willingness to pay for implant therapy: A study of patient preference. Clin. Oral Implant. Res. 2010, 21, 789–793. [Google Scholar] [CrossRef]
  38. Al Garni, B.; Pani, S.C.; Almaaz, A.; Qeshtaini, A.E.; Abu-Haimed, H.; Sharif, A.K. Factors affecting the willingness to pay for implants: A study of patients in Riyadh, Saudi Arabia. Dent. Res. J. 2012, 9, 719–724. [Google Scholar]
  39. Ethier, M.-C.; Regier, D.A.; Tomlinson, D.; Judd, P.; Doyle, J.; Gassas, A.; Naqvi, A.; Sung, L. Perspectives toward oral mucositis prevention from parents and health care professionals in pediatric cancer. Support. Care Cancer 2011, 20, 1771–1777. [Google Scholar] [CrossRef]
  40. Feu, D.; Catharino, F.; Duplat, C.B.; Junior, J.C. Esthetic perception and economic value of orthodontic appliances by lay Brazilian adults. Dent. Press J. Orthod. 2012, 17, 102–114. [Google Scholar] [CrossRef]
  41. Vermaire, J.H.; van Exel, N.J.A.; van Loveren, C.; Brouwer, W. Putting your money where your mouth is: Parents’ valuation of good oral health of their children. Soc. Sci. Med. 2012, 75, 2200–2206. [Google Scholar] [CrossRef]
  42. Widström, E.; Seppälä, T. Willingness and ability to pay for unexpected dental expenses by Finnish adults. BMC Oral Health 2012, 12, 35. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Moshkelgosha, V.; Golkari, A. An evaluation of willingness to pay for orthodontic treatments in patients of Shiraz Dental School Clinic. J. Oral Health Oral Epidemiol. 2013, 2, 35–43. [Google Scholar]
  44. Stone, S.; McCracken, G.I.; Heasman, P.A.; Staines, K.S.; Pennington, M.W. Cost-effectiveness of personalized plaque control for managing the gingival manifestations of oral lichen planus: A randomized controlled study. J. Clin. Periodontol. 2013, 40, 859–867. [Google Scholar] [CrossRef]
  45. Augusti, D.; Augusti, G.; Re, D. Prosthetic restoration in the single-tooth gap: Patient preferences and analysis of the WTP index. Clin. Oral Implant. Res. 2013, 25, 1257–1264. [Google Scholar] [CrossRef] [PubMed]
  46. Srivastava, A.; Feine, J.S.; Esfandiari, S. Are people who still have their natural teeth willing to pay for mandibular two-implant overdentures? J. Investig. Clin. Dent. 2013, 5, 117–124. [Google Scholar] [CrossRef]
  47. Moshkelgosha, V.; Salahi, M.; Rostami, S. Evaluation of Perceived Acceptability, Beauty and Value of Different Orthodontic Brackets. J. Dent. Biomater. 2015, 2, 33–38. [Google Scholar]
  48. Re, D.; Augusti, G.; Battaglia, D.; Gianni’, A.; Augusti, D. Is a new sonic toothbrush more effective in plaque removal than a manual toothbrush? Eur. J. Paediatr. Dent. 2015, 16, 13–18. [Google Scholar] [PubMed]
  49. Vernazza, C.R.; Wildman, J.R.; Steele, J.G.; Whitworth, J.M.; Walls, A.W.; Perry, R.; Matthews, R.; Hahn, P.; Donaldson, C. Factors affecting patient valuations of caries prevention: Using and validating the willingness to pay method. J. Dent. 2015, 43, 981–988. [Google Scholar] [CrossRef] [Green Version]
  50. Vernazza, C.R.; Steele, J.G.; Whitworth, J.M.; Wildman, J.R.; Donaldson, C. Factors affecting direction and strength of patient preferences for treatment of molar teeth with nonvital pulps. Int. Endod. J. 2014, 48, 1137–1146. [Google Scholar] [CrossRef]
  51. Atanasov, N.; Stoyanova, R.; Alexandrova, M. Possibilities to Model Patients’ Preferences and Their Willingness to Pay for a Molar Treatment. Dent. Med Probl. 2016, 53, 41–49. [Google Scholar] [CrossRef] [Green Version]
  52. Farronato, G.; Re, D.; Augusti, G.; Butti, A.; Augusti, D. Biomimetic orthodontic treatments: Preferences of adult patients and analysis of the Willingness-To-Pay index. Dent. Cadmos 2016, 84, 408–417. [Google Scholar] [CrossRef]
  53. Fatani, E.J.; Al-Yousef, S.K. Willingness to pay for orthodontic treatment in Kingdom of Saudi Arabia, Riyadh Province. Saudi J. Oral Sci. 2016, 3, 104. [Google Scholar] [CrossRef]
  54. McKenna, G.; Tada, S.; Woods, N.; Hayes, M.; Damata, C.; Allen, P. Tooth replacement for partially dentate elders: A willingness-to-pay analysis. J. Dent. 2016, 53, 51–56. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  55. Nair, R.; Yee, R. Differences in willingness to pay for an extraction, a filling, and cleaning teeth at various levels of oral health-related quality of life, as measured by oral impacts on daily performance, among older adults in Singapore. Singap. Dent. J. 2016, 37, 2–8. [Google Scholar] [CrossRef] [PubMed]
  56. Re, D.; Ceci, C.; Cerutti, F.; Del Fabbro, M.; Corbella, S.; Taschieri, S. Natural tooth preservation versus extraction and implant placement: Patient preferences and analysis of the willingness to pay. Br. Dent. J. 2017, 222, 467–471. [Google Scholar] [CrossRef] [PubMed]
  57. Sendi, P.; Bertschinger, N.; Brand, C.; Marinello, C.P.; Bucher, H.C.; Bornstein, M.M. Measuring the Monetary Value of Dental Implants for Denture Retention: A Willingness to Pay Approach. Open Dent. J. 2017, 11, 498–502. [Google Scholar] [CrossRef] [Green Version]
  58. Berendsen, J.; Bonifacio, C.; van Gemert-Schriks, M.; van Loveren, C.; Verrips, E.; Duijster, D. Parents’ willingness to invest in their children’s oral health. J. Public Health Dent. 2018, 78, 69–77. [Google Scholar] [CrossRef]
  59. Nyamuryekung’E, K.K.; Lahti, S.; Tuominen, R.J. Patients’ willingness to pay for dental services in a population with limited restorative services. Community Dent. Health 2018, 35, 167–172. [Google Scholar]
  60. Re, D.; Del Fabbro, M.; Karanxha, L.; Augusti, G.; Augusti, D.; Fessi, S.; Taschieri, S. Minimally-invasive dental anesthesia: Patients’ preferences and analysis of the willingness-to-pay index. J. Investig. Clin. Dent. 2018, 9, e12275. [Google Scholar] [CrossRef]
  61. Vernazza, C.R.; Anderson, L.; Hunter, A.I.; Leck, H.C.; O’Connor, S.D.; Smith, G.R.; Stokes, R.J.; Rolland, S. The Value of Orthodontics: Do Parents’ Willingness-to-Pay Values Reflect the IOTN? JDR Clin. Transl. Res. 2018, 3, 141–149. [Google Scholar] [CrossRef]
  62. Christell, H.; Gullberg, J.; Nilsson, K.; Olofsson, S.H.; Lindh, C.; Davidson, T. Willingness to pay for osteoporosis risk assessment in primary dental care. Health Econ. Rev. 2019, 9, 1–8. [Google Scholar] [CrossRef] [Green Version]
  63. Emami, E.; Alesawy, A.; De Grandmont, P.; Cerutti-Kopplin, D.; Kodama, N.; Menassa, M.; Rompré, P.; Durand, R. A within-subject clinical trial on the conversion of mandibular two-implant to three-implant overdenture: Patient-centered outcomes and willingness to pay. Clin. Oral Implant. Res. 2019, 30, 218–228. [Google Scholar] [CrossRef] [Green Version]
  64. Walshaw, E.G.; Adam, N.; Palmeiro, M.L.; Neves, M.; Vernazza, C.R. Patients’ and Parents’ Valuation of Fluoride. Oral Health Prev. Dent. 2019, 17, 211–218. [Google Scholar]
  65. Harris, R.; Lowers, V.; Laverty, L.; Vernazza, C.; Burnside, G.; Brown, S.; Ternent, L. Comparing how patients value and respond to information on risk given in three different forms during dental check-ups: The PREFER randomised controlled trial. Trials 2020, 21, 21. [Google Scholar] [CrossRef]
  66. Sever, I.; Verbič, M.; Sever, E.K. Estimating Attribute-Specific Willingness-to-Pay Values from a Health Care Contingent Valuation Study: A Best–Worst Choice Approach. Appl. Health Econ. Health Policy 2019, 18, 97–107. [Google Scholar] [CrossRef]
  67. Srivastava, A.; Esfandiari, S.; Madathil, S.; Birch, S.; Feine, J. Willingness to Pay for Mandibular Overdentures: A Societal Perspective. JDR Clin. Transl. Res. 2019, 5, 30–39. [Google Scholar] [CrossRef] [PubMed]
  68. Saadatfar, N.; Jadidfard, M. Parents’ preferences for preventive and curative dental services: A comparison between fissure sealant and composite filling using willingness-to-pay method. Int. J. Paediatr. Dent. 2021. [Google Scholar] [CrossRef]
  69. Akuagwuagwu, C.; van der Pol, M.; Boyers, D. UK general population willingness to pay for scale and polish, and detailed and personalized oral hygiene advice. Community Dent. Oral Epidemiol. 2021. [Google Scholar] [CrossRef] [PubMed]
  70. Groenwold, R.H.H.; Donders, A.R.T.; Roes, K.C.B.; Harrell, F.E.; Moons, K.G.M. Dealing With Missing Outcome Data in Randomized Trials and Observational Studies. Am. J. Epidemiol. 2011, 175, 210–217. [Google Scholar] [CrossRef] [Green Version]
  71. Sendi, P.; Ramadani, A.; Zitzmann, N.U.; Bornstein, M.M. A Systematic Review of WTA-WTP Disparity for Dental Interventions and Implications for Cost-Effectiveness Analysis. Healthcare 2020, 8, 301. [Google Scholar] [CrossRef] [PubMed]
  72. Diener, A.; O’Brien, B.; Gafni, A. Health care contingent valuation studies: A review and classification of the literature. Health Econ. 1998, 7, 313–326. [Google Scholar] [CrossRef]
  73. Gafni, A. Willingness to pay in the context of an economic evaluation of healthcare programs: Theory and practice. Am. J. Manag. Care 1997, 3 (Suppl. 3), S21–S32. [Google Scholar] [PubMed]
  74. Meyerhoff, U.; Liebe, U. Protest beliefs in contingent valuation: Explaining their motivation. Ecol. Econ. 2006, 57, 583–594. [Google Scholar] [CrossRef]
  75. Frey, U.J.; Pirscher, F. Distinguishing protest responses in contingent valuation: A conceptualization of motivations and attitudes behind them. PLoS ONE 2019, 14, e0209872. [Google Scholar] [CrossRef] [Green Version]
  76. Halstead, J.M.; Lindsay, B.E.; Brown, C.M. Use of the tobit model in contingent valuation: Experimental evidence from the Pemigewasset Wilderness Area. J. Environ. Manag. 1991, 33, 79–89. [Google Scholar] [CrossRef]
Table
Table 1. Contingent valuation willingness-to-pay studies included in review (sorted by date of publication).
Table 1. Contingent valuation willingness-to-pay studies included in review (sorted by date of publication).
Reference
Year		CountryInterventionSample SizeElicitation MethodMissing ValuesZero Values
(True/Protest)		Distinction (True/Protest)
Statistical Methodology
Dixon and Shackley,
1999 [21]		UKWater fluoridation93Payment cardYes (n = 30)Yes (n = 12)Yes
Univariate analysis
Matthews et al.,
1999 [22]		CanadaPeriodontal treatment42Bidding gameYes (n = 2)Yes (n = 1)Yes (true zero)
ANOVA
Matthews et al.,
2002 [23]		CanadaAnesthetic gel 293Bidding gameNoYes
(13.3–27.3%)		No
Univariate analysis
Birch et al.,
2004 [24]		USADentin regeneration611Open-endedYes (1.7–3.2%)NoMultivariate linear regression
Halvorsen and Willumsen,
2004 [25]		NorwayDental fear treatment62Open-endedYes (n = 8)Yes (n = ?)No
Ordinary least squared corrected for heteroscedasticity
Pavlova et al.,
2004 [26]		BulgariaPublic healthcare services990Interval check-list and open-endedNoYes (n = ?)No
Tobit regression
Smith and Cunningham,
2004 [27]		UKOrthognathic treatment188Payment cardNoYes (>1)No
Multivariate linear regression
Tamaki et al.,
2004 [28]		JapanDental check-up5132Payment options with bid amountsYes (n = 241)NoANOVA and multivariate logistic regression
van Steenberghe et al.,
2004 [29]		BelgiumAnesthetic gel for hygiene treatment157Bidding gameYes (n = 1)Yes (n = 43)Yes (f-up question)
Tobit regression
Atchison et al.,
2007 [30]		USATreatment of mandibular fracture203Continuous payment scaleNoNoMultivariate linear regression
Balevi and Shepperd,
2007 [31]		CanadaManagement of an abscessed molar40Bidding gameNoYes (<20%)No (all included)
ANOVA and decision-tree model
Oscarson et al.,
2007 [32]		SwedenCaries preventive dental care82Double-bounded dichotomo-us choiceNoNoYes (but no zeros)
Multivariate linear regression
Tianviwat et al.,
2008 [33]		ThailandSealant and filling for children205Bidding game and open-endedNoNoMulti-level linear regression
Tuominen,
2008 [34]		FinlandCheck-up for 7-year-olds156Open-endedNoNoLogistic/Linear regression
Esfandiari et al.,
2009 [35]		CanadaMandibular 2-implant overdenture36Open-ended and dichotomo-us choiceNoYes (n = 4)No
Univariate analysis
Rosvall et al.,
2009 [36]		USAOrthodontic appliance50Payment scaleNoNoMultivariate linear mixed model (ANOVA)
Leung and McGrath,
2010 [37]		ChinaSingle tooth replacement with implant51Bidding gameNoNoMultivariate linear regression
Al Garni et al.,
2012 [38]		Saudi ArabiaTooth replacement with implant100Bidding gameNoNoBinomial logistic regression
Ethier et al.,
2012 [39]		CanadaPrevention of oral mucositis in children with cancer142Bidding gameNoYes (n = ?)No (all zeros as protesters and excluded)
Interval regression
Feu et al.,
2012 [40]		BrazilOrthodontic appliance252Payment scaleNoNoUnivariate analysis
Vermaire et al.,
2012 [41]		Nether-landsParental investment in good oral health for children290Closed-ended question with payment rangesNoYes (20.7%)No
Standardized linear regression coefficients/bivariate and multivariate correlations
Widström and Seppälä,
2012 [42]		FinlandReplacing lost filling987Open-endedYes (n = 283)NoOrdered logistic regression
Moshkelgosha and Golkari,
2013 [43]		IranOrthodontic treatment192Open-endedYes (n = 2)NoANOVA
Stone et al.,
2013 [44]		UKPersonalized plaque control39Payment cardYes (n = 3)
MI		NoMultivariate linear regression
Augusti et al.,
2014 [45]		ItalyImplant/crown versus FPD107Bidding gameNoNoMultivariate linear regression
Srivastava et al.,
2014 [46]		CanadaMandibular 2-implant overdenture39Payment scale with bid amountsNoYes (2.6–28.2%)No
Multivariate linear regression
Moshkelgosha et al.,
2015 [47]		IranOrthodontic treatment348Payment scaleNoNoANOVA
Re at al.,
2015 [48]		ItalySonic/manual toothbrush/
oral hygiene treatment		40Open-endedNoNoUnivariate analysis
Vernazza et al.,
2015 [49]		UK/GermanyCoating for root caries prevention112Bidding cardYes (n = 7)Yes (n = ?)No (all zeros set as true zeros, all missings set as protest)
Linear/logistic/multinomial regression
Vernazza et al.,
2015 [50]		UKMolar RCT and Crown/gap/RPD/FPD/Implant503Payment cardNoYes (n = 6)Yes (f-up question, all true zeros)
Logistic/multinomial/
Tobit regression/Heckman model
Atanasov et al.,
2016 [51]		BulgariaRCT/crown versus extraction/implant111Open-endedYes (n = 4)Yes (n = 1)No
Negative binomial regression
Farronato et al.,
2016 [52]		ItalyBiomimetic orthodontic treatment83Bidding gameNoNoMultivariate linear regression model
Fatani and Al-Yousef,
2016 [53]		Saudi ArabiaOrthodontic treatment171Dichoto-mous and open-endedYes (n = 3)NoUnivariate analysis
McKenna et al.,
2016 [54]		IrelandRPD versus SDA versus Implant55Payment cardNoYes (n = 4)No
Multivariate linear regression
Nair and Yee,
2016 [55]		SingaporeExtraction/filling/cleaning83Bidding game and open-endedNoYes (n = 16)No
Negative binomial regression
Re et al.,
2017 [56]		ItalyRCT and crown vs. extraction, implant and crown103Bidding gameNoNoUnivariate analysis
Sendi et al.,
2017 [57]		SwitzerlandTwo interforaminal dental implants for denture retention17Bidding gameYes (n = 1)NoUnivariate analysis
Berendsen et al.,
2018 [58]		Nether-landsChildren’s oral health630Closed-ended question with payment rangesYes (n = 23)Yes (n = 1.9%)No
Multivariate ordered logistic regression
Nyamuryekung’e et al.,
2018 [59]		TanzaniaTooth extraction and filling1522Open-endedYes (8.5–9.3%)Yes (2–5%)No
Multivariate linear regression with Ln(1 + WTP) as response
Re et al.,
2018 [60]		ItalyComputer-controlled anesthesia50Bidding gameNoYes (n = 21)No
Univariate analysis
Vernazza et al.,
2018 [61]		UKOrthodontic treatment401Payment cardYes (n = ?)Yes (n = ?)Yes (f-up questions)
Tobit regression
Christell et al.,
2019 [62]		SwedenOsteoporosis risk assessment144Payment scaleYes (n = 3)Yes (n = 12)Yes (f-up questions, included)
ANOVA
Emami et al.,
2019 [63]		CanadaThird midline implant for denture retention in addition to two interforaminal implants17Dichoto-mous and open-endedNoYes (53–70%)No
Univariate analysis
Walshaw et al.,
2019 [64]		Brazil/UKFluoride varnish200Bidding cardNoNoMultivariate linear and Tobit regression
Harris et al.,
2020 [65]		UKRisk-information during check-up412Payment card and double bounded dichoto-mous choiceYes (n = 21)Yes (n = ?)Yes (f-up questions, protesters excluded)
MANCOVA and Tobit regression
Sever et al.,
2020 [66]		CroatiaDental care delivery242Payment cardYes (n = 22)Yes (included in missings)No
Interval regression model
Srivastava et al.,
2020 [67]		CanadaMandibular 2-implant overdentures317Triple-bounded discrete choice with open-ended termination NoYes (1.3–7.3%)No (zeros included)
Lognormal regression models
Saadatfar and Jadidfard,
2021 [68]		IranFissure sealant and filling in molars290Payment card and open-endedNoYes (n = 3)Yes
Multivariate linear regression
Akuagwuag-wu et al., [69]UKScaling and polish1405Payment scaleNoYes (n = 117)Yes (protest zeros excluded)
Interval regression
Abbreviations: FPD denotes fixed partial denture, RPD denotes removable partial denture, SDA denotes shortened dental arch, RCT denotes root canal treatment, MI denotes multiple imputation, f-up denotes follow-up.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Sendi, P.; Ramadani, A.; Bornstein, M.M. Prevalence of Missing Values and Protest Zeros in Contingent Valuation in Dental Medicine. Int. J. Environ. Res. Public Health 2021, 18, 7219. https://doi.org/10.3390/ijerph18147219

AMA Style

Sendi P, Ramadani A, Bornstein MM. Prevalence of Missing Values and Protest Zeros in Contingent Valuation in Dental Medicine. International Journal of Environmental Research and Public Health. 2021; 18(14):7219. https://doi.org/10.3390/ijerph18147219

Chicago/Turabian Style

Sendi, Pedram, Arta Ramadani, and Michael M. Bornstein. 2021. "Prevalence of Missing Values and Protest Zeros in Contingent Valuation in Dental Medicine" International Journal of Environmental Research and Public Health 18, no. 14: 7219. https://doi.org/10.3390/ijerph18147219

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop