Instruments to Assess People’s Attitude and Behaviours towards Tolerance: A Systematic Review of Literature

Abstract

While tolerance is recognised as important, especially to diverse societies, understanding tolerance poses complexities, both theoretically and in practical application. Tolerance is relevant in different social contexts, yet these pose distinct challenges to measurement. Thus, understanding the measurement of tolerance is important. This study provides a systematic review of tolerance scales, with the procedure based on the PRISMA statement. The initial collection of over 1600 papers through the systematic review process yielded a set of 11 papers. These papers trace the development of tolerance scales to encompass distinct social contexts over time, including political, cultural, ethnic and racial, racial and religious, gender and sexual, and social. Correspondingly, the approaches reflect particular challenges relevant to these domains for understanding and measuring tolerance. In contrast, some recent contributions aim to develop broader, less context-specific means to assess tolerance. Notwithstanding the breadth of articles included, this systematic review yielded articles that covered only selected Western-centric geographies: this indicates the opportunity for further research to access and integrate non-English-language articles to broaden the geographical and cultural perspectives on tolerance.

1. Introduction

The rapid advancement of globalisation, accompanied by significant flows of people, information, and goods [1], makes it vital for us to respect and co-exist with individuals, groups, and communities with different beliefs, philosophies, behaviours, ethnicities, cultures, and historical backgrounds. At the same time, looking across the contemporary world, conflicts often arise due to “differences”, such as customs, religions, races, and politics—the differences per se are not problems but they incorporate a power imbalance [2,3,4,5]. Tolerance, an important moral and political value that respects diversity and heterogeneity and co-existing with people from different backgrounds, has become indispensable for the smooth functioning of democratic societal systems [6,7,8,9]. Individuals and societies are called upon to reconcile this tension between, on the one hand, achieving a sense of social unity while, on the other hand, respecting a growing diversity by practicing and promoting tolerance in communities [9], which is a key philosophical and political tension. The challenges associated with tolerance, hence, pose complexities both theoretically and in practical application.

The concept of tolerance has a long history, and it has been defined and discussed in different ways over time. For instance, historically, in medieval Europe, as different religious beliefs led to conflicts, tolerance was considered negative as an expression of weakness of faith; however, in addressing heretical beliefs, tolerance towards one’s neighbour was transformed in spirit towards an enabler of social tranquillity and charity [10]. Indeed, the importance of tolerance in particular domains can shift over time, such as in post-World War II United States, during which there was a shift towards political tolerance, meaning a willingness to recognise the civil liberties regarding the thoughts and expressions of individuals with differing opinions [7,11,12]. Nonetheless, tolerance is also pointed out as a core aspect of attitudes, behaviours, and society. During the Enlightenment era, the philosopher Voltaire asked, “What is tolerance? It is the consequence of humanity. We are all formed of frailty and error; let us pardon reciprocally each other’s folly—that is the first law of nature” [13]. He expressed tolerance as inherent at the core of human nature. In modern times, tolerance features in Article 1.1 of the United Nations Educational, Scientific, and Cultural Organization (UNESCO), which states, “…tolerance is respect, acceptance, and appreciation of the rich diversity of our world’s cultures, our forms of expression, and ways of being human… Tolerance is harmony in difference” [14]. Tolerance has been perceived as an important, even fundamental aspect of human being.

However, tolerance remains complex conceptually and in practice. Indeed, in 1976, Ferrar argued, “The concept of tolerance is in a state of disarray” [15]. Half a century later, however, no consensus has yet been reached on a comprehensive definition of tolerance [5,16], nor on a general method of measuring it [17]. Tolerance fundamentally concerns the state of otherness [5] and can be thought of as the ability and attitude to understand different cultures, religions, sex orientations, lifestyles, opinions, values, and behaviours, recognising that such differences permeate lived society. Tolerance does not necessarily entail agreement with and acceptance of diverse practices and values: tolerance does not mean to accept the immoral but may mean accepting a different moral perspective [9,18,19,20,21]. In this sense, toleration could be considered “an elusive virtue” [18], with neither tolerance inherently virtuous nor inherently vicious [22]. Considering the demand for tolerance towards different cultures, tolerance is considered as cultural capital [23,24]. In the context of the economy, tolerance has been positioned as a factor complementary to “technology” and “talent” to drive economic growth, representing an aspect of human capital [25]. Politically and socially, tolerance involves recognising the civil liberties of individuals who hold different opinions, allowing for their thoughts and expressions [26], and is a key enabler of the permission of others’ free political participation [11]. The extent to which there are related but different perspectives on tolerance and that tolerance is of relevance in diverse domains point to the importance of understanding the measurement of tolerance.

In the context of evolving perspectives over time and domains, there is value in a systematic analysis of literature organised to encompass the diverse scales developed to measure the construct of tolerance. The purpose of this paper, therefore, is to review the literature that has investigated and evaluated “tolerance” in different contexts. The aim is to understand how the focus of the research has shaped approaches to measurement, and the different approaches taken. Thus, the review would contribute to the development of more robust evaluation scales as well as highlight where there is need for further development. This is important not only to contribute to the discussion of tolerance in the academic realm but also to inform use of tolerance measures in practice, which is relevant across many societal domains.

The structure of this paper is as follows. Section 2 explains the methodology used and the steps taken to collect a broad sample of literature and identify papers to include in the systematic review: the procedure is based on the PRISMA statement [27]. In Section 3, the results of the systematic review are provided, which yielded a set of 11 papers for review. In Section 4, key themes of the set of papers are discussed, and Section 5 concludes the paper. To the best of our knowledge, this study represents the first effort to gather and assess evidence related to established tolerance scales in social settings. Consequently, the primary objective of this research is to pave the way for the future development of tolerance scales, fostering inclusiveness in contemporary society. This is achieved by identifying conceptual, empirical, and policy contributions and gaps in the existing literature, thereby providing valuable insights to guide and inform future research endeavours.

2. Materials and Methods

This section provides the criteria used to select articles, the information sources used to generate the initial set of papers, and the process flow that yielded the final sample of papers. This systematic approach is in accordance with the PRISMA guidelines.

2.1. Eligibility Criteria

The systematic review was based on several criteria to specify the articles to be included in the study. First, only published peer-reviewed articles were included. The reason is to ensure that the tolerance scales and measures were reviewed and approved by experts. Second, the tolerance scales and measures were applied only within several scopes of interest. Included were use of tolerance scales in a social context, such as related to gender, political views, and race; excluded, in contrast, was application in a medical or health contexts, such as in relation to pain, addiction, food, or allergy. Third, the selected articles are in English language. By only including articles written in English, we prevented misinterpretation in extracting the information from the articles. Fourth, the articles should provide a novel scale, such as in terms of the questions or composition of the scale, and not just apply an existing scale. Fifth, the articles should include the method used to quantify the scale. Last, the full-text version of the articles had to be accessible to the researchers, as the review and selection process required examination of the full text of the articles.

2.2. Information Sources

On 4 August 2023, three databases were searched for articles: Web of Science, Science Direct, and ProQuest. These databases were selected due to having a broad scope of coverage and since their search engines accept complex strings as the search input text. Complex strings are required to prevent repetitive searches and filter out duplication right from the beginning. The search strings were tailored to each database’s conditions, with the aim to surface all potentially relevant articles related to tolerance scales and/or measures.

Table 1. Search strings and categorisation that were used in the article search process.

Database	Search String	Categories
Web of Science	“tolerance scale*” OR “tolerance measure*” OR “measure* of tolerance” OR “scale* of tolerance”	Behavioural Sciences, Psychology, Paediatrics, Sociology, Social Issues, Anthropology, Family Studies, Ethnic Studies, Cultural Studies, Religion, Social Work
Science Direct	“tolerance scale” OR “tolerance measure” OR “scale of tolerance” OR “tolerance scales” OR “tolerance measures” OR “scales of tolerance” OR “tolerance measurement” OR “measurement of tolerance”	Psychology, Social Sciences
ProQuest	“tolerance scale*” OR “tolerance measure*” OR “measure* of tolerance” OR “scale* of tolerance”	Psychology, Political Science, Studies, Sociology, Religion, Social Psychology, Education, Tolerance, Politics, Personality, Mental Health, Research, Humans, Behaviour, Questionnaires, Attitudes, Emotions, Hypotheses, Male, Female, Minority and Ethnic Groups, Anxiety, Experiments, Families and Family Life, Students, College Students, Perceptions, Quantitative Psychology, Democracy, Population, Stress, Adult

The symbol * indicates a wild card in the search string.

contains the search strings and categories for each database. In addition, further articles from these databases were included through citation searching.

2.3. Search Strategy

To narrow the search results, the process of article collection involved several steps. First, conditional search was used to broaden the search in the specific context. Second, wild cards were combined into the string (if supported by the database), again to broaden the search. Third, broad coverage of relevant results was achieved through selecting relevant search result categories, reflecting each database’s categorisation. Three further articles were included through citation searching. This initial set of papers, net of duplicates, was then screened according to the Section 2.1 criteria to identify the sample for review. Two researchers independently performed the procedure and discussed the results until agreement was obtained. The flow of the process is illustrated in Figure 1.

3. Results

The search strategy led to collection of 1649 articles in total, which included 423 articles from Web of Science, 542 articles from Science Direct, and 684 articles from ProQuest. Removal of duplicates resulted in a set of 1500 articles. In addition, three articles were considered based on citation searching. The articles were screened according to the pre-agreed screening criteria for inclusion. As per Section 2.1, the criteria included that the articles must contain social context tolerance scale/measurements, and mention or cite the components or details of the scale/measurements. However, of note, the inclusion criteria did not depend on the articles’ study results.

The process of reviewing started with the title and abstract review: this yielded 41 articles relevant to our inclusion criteria plus 3 articles from citation searching. For these articles, we sought to access the full-text versions, with 11 articles not accessible. Of the 33 remaining articles, full-text review to apply the inclusion criteria led to 11 articles included in the final sample. The publication year of the 33 articles assed for eligibility spanned from 1962 to 2022, with the 11 selected articles spanning from 1979 to 2021. The main characteristics of these articles are summarised in Appendix A

Table A1. Main characteristics of the included articles.

Number	Title	Authors, Year	Study Characteristics	Study Location	Sample Characteristics	Sample Count	Aim/Dimension Explored	Tolerance Scale Measure
1	An Alternative Conceptualization of Political Tolerance: Illusory Increases 1950s–1970s	Sullivan et al., 1979 [28]	The study design in this paper involves conducting two independent surveys in Minneapolis and St. Paul, Minnesota, during the spring and summer of 1976. The surveys included two independent random samples of size 300 each, selected from the Twin Cities’ city directories. Interviews were completed with 200 persons using the old questions and with 198 persons using the new questions. The study design can be characterised as observational, cross-sectional, and comparative between the two sets of questions used in the surveys.	Minneapolis and St. Paul, Minnesota	The study population consisted of adults from Minneapolis and St. Paul, Minnesota, with two independent random samples of size 300 each being selected from the Twin Cities’ city directories in 1976.	398	To propose an alternative conceptualisation of political tolerance, introduce a new measurement strategy, challenge previous methods of measuring tolerance, and provide new findings based on this revised approach.	The tolerance scale measured in the study is a content-controlled measure where respondents selected the group they liked the least from a list of potentially unpopular groups and were then asked a series of questions testing their willingness to extend civil liberties to that self-selected group. This approach allowed for a more personalised and context-controlled measurement of tolerance.
2	The Sources of Political Tolerance: A Multivariate Analysis	Sullivan and John L, 1981 [29]	Observational study.	United States	Education Age Religion Sex, size of city, and region	1509	To address the problems in previous studies on political tolerance by using a content-controlled measure of tolerance and a more fully specified multivariate model.	The tolerance scale measured in the study is a six-item political tolerance scale, with scores ranging from 6 to 30 and a coefficient alpha of 0.78.
3	The Illusion of Political Tolerance: Social Desirability and Self-Reported Voting Preferences	Brown-Iannuzzi et al., 2019 [30]	Between-subjects design, nationally representative sample, random assignment to three conditions, traditional self-report, indirect measure using UCT.	United States	American adults Sample size of 3000 Data collected in June 2016 Data collected from U.S. residents	3000	To investigate the degree to which self-reported tolerance for stigmatised groups is overstated, estimate rates of political tolerance for various groups, and compare indirect estimates to direct self-reported tolerance. The study also utilises Bayesian hierarchical modelling for data analysis.	The tolerance scale measured in the study is self-reported willingness to vote for candidates from different stigmatised social groups, as well as indirectly measured tolerance using the unmatched count technique (UCT).
4	Development of a Scale to Measure Cross-Cultural Sensitivity in the Canadian Context	Pruegger et al., 1993 [31]	Observational study.	Canada	Undergraduate psychology students and geologists working in the International Geology Department of a large Calgary oil firm 55 undergraduate psychology students + 10 geologists + 49 female undergraduate psychology students + 22 male undergraduate psychology students.	136	To develop a measure of cross-cultural sensitivity in the Canadian context with acceptable levels of internal consistency and content validity, and to explore its validity through further research.	The tolerance scale measured in the study is the Cross-Cultural Sensitivity Scale, which was based on the response values for the “disagree-keyed” items, with higher scores indicating greater cross-cultural sensitivity.
5	Issues in Cross-Cultural Counseling—An Examination of the Meaning and Dimensions of Tolerance	Sutter and McCaul, 1993 [32]	Quantitative study.	Sweden	Professionals working with immigrants and refugees in Southern Sweden 65% social workers, 20% counsellors, 10% language teachers, 5% medical personnel. 64% female, 36% male Age ranged from 25 to 64, with 70% between 30 and 50. Majority from rural areas or small towns Relatively well educated with a mean of 3.6 years of education beyond high school.	123	Develop a measure of ethnic tolerance through fieldwork Assess the reliability and construct validity of the tolerance measure based on two models of ethnic tolerance. Investigate the relationship between factors like age, gender, and exposure to different cultures with scores on the tolerance measure.	The tolerance scale measured in the study is a 24-item ethnic tolerance measure that was found to be a unidimensional construct with a reliability coefficient of 0.80 and showed some evidence of construct validity through correlational analysis.
6	When the Rubber Meets the Road: Effects of Urban and Regional Residence on Principle and Implementation Measures of Racial Tolerance	Carter et al., 2005 [33]	Observational study.	United States	Individuals aged 18 years and older in the United States. Consideration of differences in racial tolerance based on urban and regional residence. Assessment of the influence of urban residency on attitudes, independent of socioeconomic status.	10,123	To revisit racial tolerance differences based on urban vs. non-urban residence, non-Southern vs. Southern residence, and changes over time, while incorporating both principle and implementation questions.	The tolerance scale measured in the study includes both principle questions focusing on endorsing broad principles of equal treatment regardless of race and implementation questions putting principles of equal treatment into effect by ending discrimination and enforced segregation. The study uses composite measures of tolerance based on responses to questions related to interracial marriage, voting for an African American president, neighbourhood segregation, government intervention in busing, and discriminatory practices of homeowners.
7	Race and Religion in the Bible Belt: Parental Attitudes Toward Interfaith Relationships	Sahl and Baston, 2011 [34]	Observational study.	United States	Majority white population in the Bible Belt region. Significant presence of Baptist religious affiliation in the survey area. Religious importance in the everyday lives of respondents. 78% white and 22% black respondents in the final sample. Historically racially segregated region in the South. African Americans more likely than whites to engage in religious activities.	412	To include examining racial and religious differences in parental attitudes toward interfaith relationships, determining differences in attitudes based on religion and race, exploring the increase in opposition as relationships become more intimate, and investigating variations in the association between religion and attitudes across racial groups.	The tolerance scale measured in the study ranges from 0 to 12, with different scores for race, religious affiliation, and religious importance. Whites had a mean tolerance score of 6.97, while black respondents had a score of 8.11. Baptists were less tolerant with a score of 6.96 compared to non-Baptists with a score of 7.72. Religious importance did not show significance in the tolerance scale analysis.
8	Between Homohysteria and Inclusivity: Tolerance towards Sexual Diversity in Sport	Piedra et al., 2017 [35]	The study design is a correlational study using a Likert scale self-report questionnaire, with data collected through different methods in Spain and the UK. The study did not involve randomisation, blinding, control groups, or placebos.	Spain and UK	The study population consisted of 879 men and women aged 16–78 from Spain and the United Kingdom who were actively participating in or following sports. 67.7% of the participants were involved in sports within a federation, while 32.2% were not. 75.3% of the participants had higher education backgrounds. The majority of participants (88.9%) identified as heterosexual, with smaller percentages identifying as lesbians (3.2%), gays (2.3%), bisexuals (2.5%), and others (0.6%)	879	To describe the level of tolerance towards sexual diversity in sport in Spain and the UK, measure and validate sportspeople’s attitudes in two dimensions (non-rejection and acceptance), identify metacognitive profiles based on tolerance dimensions, and contrast the samples from both countries to theorise the main differences.	The tolerance scale measured in the study includes two dimensions: non-rejection and acceptance of sexual diversity in sport, measured using a new instrument with high reliability, the Attitudes towards Lesbians and Gays (ATLG) scale, and the Attitude Scale towards Sexual Diversity in Sport.
9	Disapproved, but Tolerated: The Role of Respect in Outgroup Tolerance	Simon et al., 2019 [36]	Observational study	United States	Participants who self-identified as supporters of the Tea Party movement in the United States, with a mean age of 55 years, 60% male, 47% with a college or university degree, and a total household income ranging from up to USD 5000 to more than USD 250,000, with a mean of USD 68,026.	Total: 485 Tea Party supporters: 422 Undergraduate students in the experiment: 63	To test the hypothesis that respect for disapproved outgroups increases tolerance toward them, investigate the influence of respect for homosexuals and Muslims on Tea Party supporters’ tolerance, and conduct a larger longitudinal survey and an experiment with members of a majority as research participants and minorities as target groups.	The tolerance scale measured in the study includes individual tolerance towards specific groups (homosexuals and Muslims) as well as tolerance towards a specific student group in a university setting, measured using 5-point and 10-item scales, respectively.
10	A New Approach to the Study of Tolerance: Conceptualizing and Measuring Acceptance, Respect, and Appreciation of Difference	Hjerm et al., 2020 [37]	The study design is a cross-sectional observational study involving national and cross-national surveys to develop and validate measures of tolerance.	Sweden, Australia, Denmark, Great Britain, and the United States	The study includes a national sample from Sweden and a cross-national sample from Australia, Denmark, Great Britain, Sweden, and the United States. The Swedish sample has a slightly lower percentage of individuals with three or more years of tertiary education compared to the general population in 2016. The sample includes 11.1% foreign-born individuals, which is lower than the foreign-born percentage in the total population in 2016. The study does not provide specific demographic breakdowns by age, sex, race, ethnicity, or other characteristics.	6300	To advance research that distinguishes analytically between tolerance and prejudice, conceptualise tolerance as a value orientation towards difference, improve the measurement of tolerance, and assess the relationship between tolerance and prejudice.	The tolerance scales measured in the study are tolerance as respect for diversity and tolerance as appreciation for diversity
11	Measuring Tolerant Behavior	Liberati et al., 2021 [38]	Observational, cross-sectional study	Italy	University students at the University of Milan-Bicocca.	3389	To develop a multidimensional index for Likert-scale data to measure tolerant attitudes towards different social domains, determine the contribution of each dimension to overall tolerance, and apply this measure in a case study with Italian university students.	The tolerance scale measured in the study is a multidimensional index specifically designed for Likert-scale data, reflecting the intensity of tolerant attitudes towards different social domains and combining these dimensions using a weighted scheme. The study also discusses the potential generalisation of the index for non-Likert-scale data.

.

The set of studies revealed a variety of tolerance definitions (

Table 2. Characteristics of the tolerance scales for the included articles.

Number	Title	Authors, Year	Definition of Tolerance
1	An Alternative Conceptualization of Political Tolerance: Illusory Increases 1950s–1970s	Sullivan et al., 1979 [28]	The definition of tolerance refers to the willingness to permit the expression of those ideas or interests that one opposes.
2	The Sources of Political Tolerance: A Multivariate Analysis	Sullivan and John L, 1981 [29]	The “tolerance definition” refers to the measurement of political tolerance using a content-controlled measure, which allows respondents to specify the groups they most strongly oppose without contaminating the tolerance/intolerance dimension with their political beliefs. The paper emphasises the importance of using a content-controlled measure to avoid bias in assessing political tolerance.
3	The Illusion of Political Tolerance: Social Desirability and Self-Reported Voting Preferences	Brown-Iannuzzi et al., 2019 [30]	The definition of tolerance is that a tolerant person accepts the presence and participation of all kinds of people in society, regardless of personal thoughts or feelings about them.
4	Development of a Scale to Measure Cross-Cultural Sensitivity in the Canadian Context	Pruegger et al., 1993 [31]	Tolerance is defined as the willingness to permit the expression of ideas or interests that one opposes, implying a wide acceptance of challenging viewpoints in a political context. It arises when there is opposition or disagreement, and it is considered valuable for maintaining a stable democratic regime.
5	Issues in Cross-Cultural Counseling—An Examination of the Meaning and Dimensions of Tolerance	Sutter and McCaul, 1993 [32]	The tolerance definition is the presence of mutual respect, acceptance, and exchange of cultural beliefs between counsellor and client, and the absence of prejudicial attitudes and beliefs that interfere with accepting the reality of the individual. It also includes the concepts of ethnocentrism, stereotyping, and the influence of social, economic, and cultural threats on prejudicial attitudes.
6	When the Rubber Meets the Road: Effects of Urban and Regional Residence on Principle and Implementation Measures of Racial Tolerance	Carter et al., 2005 [33]	The tolerance definition is something that can be seen as merely putting up with people that are different. Rather than just passive acceptance of others, the paper envision tolerance from a more activist perspective, including favourable views and willingness to help others.
7	Race and Religion in the Bible Belt: Parental Attitudes Toward Interfaith Relationships	Sahl and Baston, 2011 [34]	The “tolerance definition” refers to the measurement of attitudes towards interfaith unions at different levels of intimacy, such as friendship, dating, and marriage, using a tolerance scale. This scale helps to explore racial and religious gaps in opposition towards interfaith relationships.
8	Between Homohysteria and Inclusivity: Tolerance towards Sexual Diversity in Sport	Piedra et al., 2017 [35]	Tolerance is defined as the acceptance regarding the participation of LGBT people in sport.
9	Disapproved, but Tolerated: The Role of Respect in Outgroup Tolerance	Simon et al., 2019 [36]	The tolerance definition refers to the valuation and acceptance of different cultures, particularly among dominant group members in Canada. It is a key aspect of cross-cultural sensitivity being measured in the study.
10	A New Approach to the Study of Tolerance: Conceptualizing and Measuring Acceptance, Respect, and Appreciation of Difference	Hjerm et al., 2020 [37]	The tolerance definition is a value orientation towards difference. It measures the response to the existence of diversity. It focuses on subjective reactions to difference; thus, this conceptualisation does not require dislike of or identification of potentially objectionable groups, ideas, or behaviour.
11	Measuring Tolerant Behavior	Liberati et al., 2021 [38]	The tolerance definition refers to a multidimensional view of tolerance, with attitudes along each dimension of tolerance contributing to overall tolerance.

) [28,29,30,31,32,33,34,35,36,37,38]. A key reason for the variation in definitions stems from differences in the social context in which tolerance was measured. Further, the definition of tolerance was grouped into types depending on the subject’s reaction to social differences. For seven of the studies, tolerance was defined around acceptance of differences [29,30,31,35,36,37,39], while four other studies defined tolerance more broadly to include potentially appreciation and willingness to act accordingly [32,33,34,38]. Therefore, even though the definition of tolerance shared the same basic definition, the introduction of social context potentially changed the specific definition of tolerance.

As the selected articles considered tolerance in different social domains and spanned four decades, keyword analysis was performed to identify patterns in word use across the papers. VOSviewer version 1.6.19 was used to build and visualise keyword maps [39], so as to identify multiple occurrences of keywords across the papers (i.e., keywords identified must be in at least two papers). For the included studies, 118 different keywords were identified. Of these, keywords that co-occurred with at least one other keyword were identified, which generated 13 keyword co-occurrences (Figure 2).

Not surprisingly, the main node was “tolerance”, which connected to almost all of the other keywords. The clusters were “united-states—us” with “cultural differences”, “sociology”, “toleration”, and “article”. The keyword “united-kingdom—uk” was in a separate cluster, with keywords “gender” and “multiculturalism and pluralism”. The other cluster comprised “polls and surveys”, “psychology”, and “homosexuality”. To consider the trends in keywords, the keyword network was identified based on the emergence over time of the keywords (Figure 3). The earlier time-based keywords to emerge were “united-states—us”, “sociology”, and “psychology”, then “article”, “gender”, and “multiculturalism and pluralism”, and more recently “united-kingdom—uk”, “polls and surveys”, and “homosexuality”. Thus, over time, there has been a broadening of geographies and of social domains focused on the perspective of tolerance.

Notwithstanding the differences in definitions and focus on diverse social domains, the articles all had scales based on surveys, with all but one using a response format based on Likert scales and one utilising the unmatched count technique (UTC). That said, the tolerance scales varied in number of questions from 3 to 32 questions, with a relatively high variation in number of questions: 4 scales had between 6 and 8 questions, 2 scales had 12 or 13 questions, and 2 scales had 24 questions (

Table 3. Characteristics of the tolerance scales for the included articles.

Number	Title	Authors, Year	Focus of Questionnaire	Number of Questions	Response Options
1	An Alternative Conceptualization of Political Tolerance: Illusory Increases 1950s–1970s	Sullivan et al., 1979 [28]	Political tolerance	Total: 12 questions	1 to 5 Likert scale 1 = Strongly disagree 5 = Strongly agree
2	The Sources of Political Tolerance: A Multivariate Analysis	Sullivan and John L, 1981 [29]	Political tolerance	Total: 6 questions	1 to 5 Likert scale 1 = Strongly disagree 5 = Strongly agree
3	The Illusion of Political Tolerance: Social Desirability and Self-Reported Voting Preferences	Brown-Iannuzzi et al., 2019 [30]	Political tolerance	Total: 8 questions	6 Candidate options Yes/No
4	Development of a Scale to Measure Cross-Cultural Sensitivity in the Canadian Context	Pruegger et al., 1993 [31]	Cultural tolerance	Total: 24 questions Agree-keyed: 13 Disagree-keyed: 11	1 to 6 Likert scale 1 = Strongly disagree 6 = Strongly agree
5	Issues in Cross-Cultural Counseling—An Examination of the Meaning and Dimensions of Tolerance	Sutter and McCaul, 1993 [32]	Ethnic and racial tolerance	Total: 24 questions Cultural subscale: 10 Economic subscale: 4 Social subscale: 4 Others: 6	1 to 5 Likert scale 1 = Yes, absolutely 5 = No, absolutely not
6	When the Rubber Meets the Road: Effects of Urban and Regional Residence on Principle and Implementation Measures of Racial Tolerance	Carter et al., 2005 [33]	Ethnic and racial tolerance	Total: 5 questions	1 to 4 Likert scale 1 = Strongly agree 4 = strongly disagree Yes/No
7	Race and Religion in the Bible Belt: Parental Attitudes Toward Interfaith Relationships	Sahl and Baston, 2011 [34]	Racial and religious tolerance	Total: 3 questions	0 to 4 Likert scale 0 = Yes, absolutely 4 = No, absolutely
8	Between Homohysteria and Inclusivity: Tolerance towards Sexual Diversity in Sport	Piedra et al., 2017 [35]	Gender and sexual tolerance	Total: 32 questions Acceptance questions: 20 Non-rejection questions: 12	1 to 5 Likert scale 1 = Totally agree 5 = Totally disagree
9	Disapproved, but Tolerated: The Role of Respect in Outgroup Tolerance	Simon et al., 2019 [36]	Social tolerance	Total: 6 questions	−3 to +3 Likert scale 0 to 5 acceptance scale
10	A New Approach to the Study of Tolerance: Conceptualizing and Measuring Acceptance, Respect, and Appreciation of Difference	Hjerm et al., 2020 [37]	Social tolerance	Total: 8 questions	1 to 5 Likert scale 1 = Completely disagree 5 = Completely agree
11	Measuring Tolerant Behavior	Liberati et al., 2021 [38]	Social tolerance	Total: 13 questions Interreligious dialog: 3 Women/religion relationship: 3 Death/religion relationship: 2 Multicultural society: 3 Homosexuality: 2	1 to 5 Likert scale 1 = Strongly disagree 5 = Strongly agree

). While with some differences in approaches, all the papers provided indications on the validity and reliability of the scales (

Table 4. Validity and reliability tests for the included articles.

Number	Authors, Year	Validity and Reliability Tests
1	Sullivan et al., 1979 [28]	The measurement scale has been tested and shown to have validity, as the groups selected as least liked by respondents aligned with their self-reported political ideology in an expected way.
2	Sullivan and John L, 1981 [29]	The measurement scale has been tested and found to be reliable and appropriate for measuring political tolerance, addressing the content bias of previous measures.
3	Brown-Iannuzzi et al., 2019 [30]	The measurement scale (the unmatched count technique) has been tested and used in prior research to measure sensitive or socially undesirable attitudes and behaviours, suggesting it is a validated approach for overcoming social desirability concerns.
4	Pruegger et al., 1993 [31]	The measurement scale has been tested and demonstrates reasonable levels of content validity and impressive internal consistency, but further testing is needed to fully evaluate the scale’s psychometric properties.
5	Sutter and McCaul, 1993 [32]	The measurement scale has been tested through a pilot study with faculty and graduate students and revised based on their feedback.
6	Carter et al., 2005 [33]	The measurement scale (the principle-based questions index) has been tested and found to be a similar and strong measure of racial tolerance and may even be more parsimonious and effective than the previous six-question index used in prior research.
7	Sahl and Baston, 2011 [34]	The measurement scale has been tested and found to have strong reliability, with a Cronbach’s alpha of 0.880.
8	Piedra et al., 2017 [35]	The measurement scale has been tested and found to have high reliability (Cronbach’s alpha > 0.95) and construct validity through factor analysis, demonstrating it is a reliable and valid measure of tolerance towards sexual diversity in sport.
9	Simon et al., 2019 [36]	The measurement scales used in the study had been tested and used in prior research, as indicated by the paper’s statement that the measures were “adapted from earlier work” and “the same measures were used at both time points”. The paper also reports on the reliability of some of the measures, such as Cronbach’s alpha for the tolerance measure in Study 2.
10	Hjerm et al., 2020 [37]	The measurement scale has been tested and validated through confirmatory factor analysis in a single country (Sweden) and across multiple countries (Australia, Denmark, Sweden, the UK, and the US), showing good model fit and metric invariance across countries.
11	Liberati et al., 2021 [38]	The measurement scale has been tested and found to have high internal consistency, as indicated by the relatively high Cronbach’s alpha values for the two submatrices of survey items.

). The characteristics of the included articles are highlighted in Table 3 and in the Section 3.1, Section 3.2, Section 3.3, Section 3.4, Section 3.5 and Section 3.6, organised by the domain of tolerance focused on in the paper and ordered based on recency of the studies and, consequently, of the domains focused on.

3.1. Political Tolerance Scale

The study conducted by Sullivan et al. (1979) [28] provided a critical analysis of the empirical research on political tolerance, emphasising the literature’s limited scope and referring to six major studies conducted over two decades. The paper proposes a new conceptualisation of political tolerance, a new measurement approach, and new findings based on this strategy, explaining that past attempts to measure tolerance were unsuccessful because respondents rated preselected groups, resulting in biased results. Furthermore, the study underlines the significance of understanding the relationship between tolerance and democratic politics, questioning the utopian image of democracy, and underlining the need to explore fundamental problems regarding this relationship highlighted by previous liberal thinkers.

Sullivan et al. (1981) [29] also investigated the sources of political tolerance, concentrating on the interaction between demographic, psychological, and political factors and how these affect individual tolerance levels. According to the study, education, social position, and psychological security all have a major impact on political ideology and tolerance, with education having a stronger relationship with target group selection than tolerance itself. Furthermore, the study emphasised the role of personality traits and political variables, such as general norms and perceived threats, in affecting people’s degrees of tolerance.

Brown-Iannuzzi et al. (2019) [30] studied the disparity in direct and indirect measures of political candidate preferences among different demographic groups in the United States. The study employed the unmatched count method (UCT) to indirectly measure the desire to vote for various targets, to avoid socially favourable responses and provide more accurate estimates of population prevalence. According to the findings, there was a convergence between direct and indirect measures of candidate preferences among highly educated participants, indicating that they are particularly skilled at suppressing prejudice. Furthermore, the study underscored the possibility of bias in self-reported tolerance for marginalised social groups, particularly those judged socially incorrect to stigmatise, and the need for more reliable measures of political tolerance to reflect genuine attitudes towards diversity in America.

3.2. Cultural Tolerance Scale

Pruegger and Rogers (1993) [31] described the construction of a Cross-Cultural Sensitivity Scale (CCSS) in the Canadian context, which included the generation of 140 items encompassing cultural knowledge, attitudes, beliefs, and lifestyles, as well as the subsequent selection of the 24 best items to form the CCSS. The study rigorously tested the scale for internal consistency, demonstrating a high coefficient alpha of 0.93, and developed parallel forms that met the criteria for rigorously parallel forms, indicating the scale’s reliability and validity for measuring cross-cultural sensitivity in the Canadian setting.

3.3. Ethnic and Racial Tolerance Scale

Sutter and McCaul (1993) [32] tackled the integration of cross-cultural counselling into professional interactions, emphasising the importance of cross-cultural competence training to help professionals. They emphasised the importance of cultural prejudices and failing to acknowledge cultural impacts on individuals, arguing that such oversights can result in ineffective counselling and ethical difficulties. The project investigated the creation and validation of a tolerance measure based on ethnocentrism and stereotyping theories. The measures were correlated to various demographic and experience variables. The study underlined the relationship between tolerance and the quality and quantity of contact with people from different cultural backgrounds, emphasising the importance of experiences and interactions in shaping tolerance levels.

Carter et al. (2005) [33] investigated the impact of urban and regional living on measures of racial tolerance. They covered the intricacies of question-wording and how this can influence responses, emphasising the contrast between principle and implementation questions in assessing racial prejudice. The study also investigated the disparities in discriminatory attitudes against African Americans between Southern and non-Southern residents. It further investigated the long-term impact of residential location on racial tolerance, as well as the relationship between urbanism and tolerance, such as the view that city life promotes tolerance owing to diversity and heterogeneity. The research underlined the necessity to evaluate how tolerance is measured and the possible differences in tolerance levels across different household settings.

3.4. Racial and Religious Tolerance Scale

Sahl and Batson (2011) [34] investigated the attitudes of parents in the United States Bible Belt region on interfaith relationships, with a focus on racial and religious differences. They used marriage market theory and third-party influence to explain parental attitudes, and a tolerance scale and logistic regression to predict opposition to interfaith marriages. According to the study, religious relevance is a key predictor of interfaith opposition, and as intimacy in the connection grow, so does opposition to interfaith unions. Furthermore, the study tied the findings to conservative beliefs in the region, particularly considering Southern socialisation centred on religion and aversion to change.

3.5. Gender and Sexual Tolerance Scale

The Piedra et al. (2017) [35] study analysed the dynamics of tolerance for sexual diversity in sports, with a special emphasis on attitudes and acceptance levels within the sporting community. It underlined the significance of evaluating and comprehending trends in attitudes regarding sexual diversity in sports, underlining the complexity and nuances involved. The study used a variety of ideas and concepts, such as inclusive masculinity and homohysteria, to create a complete framework for understanding the shifting levels of acceptance and antagonism in the sporting environment. It also examined the impact of cultural and socioeconomic variables on views regarding sexual minorities in sports, offering light on how these attitudes have evolved.

3.6. Social Tolerance Scale

Simon et al. (2019) [36] presented new data analyses and findings on the role of respect in outgroup tolerance, drawing on the disapproval–respect model of social tolerance. The findings provide significant empirical support for the outgroup respect–tolerance theory, emphasising the importance of treating others as equals in encouraging political tolerance and participation in diverse communities. The study also emphasised the importance of understanding the mechanisms underlying tolerance, specifically the interaction of disapproval and respect, as well as the effect of manipulation on individuals’ tolerance levels.

Hjerm et al. (2020) [37] provided a thorough analysis of tolerance as a value orientation towards difference. The study highlighted that tolerance should not be viewed as dependent on bias, and it investigated the many manifestations and levels of tolerance. The study investigated the definition and measurement of acceptance, respect, and appreciation for difference, emphasising the importance of a comprehensive understanding of tolerance.

Liberati et al. (2021) [38] evaluated tolerance using a multidimensional index for Likert-scale data, which reflected individuals’ views towards various social domains and weighted them accordingly. The index can be broken down by dimension to determine how each one contributes to overall tolerance. It was obtained through the geometric aggregate of tolerance dimensions. The paper also explored the feasibility of generalising the tolerance index to non-Likert-scale data.

4. Discussion

Starting from an initial collection of over 1600 papers, the systematic review process yielded 11 papers, which all put forward and explained a means to measure tolerance. The identified tolerance scales span diverse categories, in terms of the context in which tolerance is measured. The context addressed by the papers was political, for three papers; cultural, for one paper; ethnic and racial, for two papers; racial and religious, for one paper; gender and sexual, for one paper; social, for four papers. While all the papers shared a focus on tolerance, there was no common definition. Indeed, there was a differing emphasis on particular issues related to understanding tolerance, including direct versus indirect measurement [30], tolerance in principle versus in practice [33], non-rejection versus acceptance [35], understanding disapproval and respect in relation to tolerance [36], and distinguishing tolerance from prejudice [37]. Thus, the papers represent, over time, exploration of tolerance in diverse societal contexts. These differences and specificities of the social contexts in part inspire adaptations in approaches to measuring tolerance. Also, as exemplified by Sullivan et al. (1979) [28], tolerance scales need to be adapted to reflect the changing societal context, such as in what is asked and how. This would indicate, in turn, that as other divisions in society that potentially strain tolerance become salient, such as in how to address environmental challenges, further specific approaches to assessing tolerance may need to be developed.

While noting this tendency towards a broadening set of specific scales, two recent papers focused on social tolerance aimed at capturing a more general sense of tolerance, though with distinct approaches. Liberati et al. (2021) [38] constructed a multi-dimensional scale. This could be readily adapted to incorporate emergent dimensions. Also, the relevance of novel dimensions would likely be recognised due to the geometric weighting of dimensions. In contrast, Hjerm et al. (2020) [37] took, as a starting point, tolerance as relating to acceptance, respect, and/or appreciation of difference, which is operationalised with questions that do not relate to a specific social context or group of people. Thus, the set of papers represent approaches to measuring tolerance, spanning from potentially broad applicability to tailored to those that reflect a particular context.

The keyword analysis evidenced patterns of topics associated by geography and over time. For instance, one keyword association was “United States” and “cultural differences”, whereas another was the “United Kingdom” and “multiculturalism and pluralism”. Also, whereas “gender’ has been a longer-running keyword, “homosexuality” is more recently prominent. Further, tolerance has been addressed in connection with different disciplines as well as in diverse domains.

Nonetheless, all the papers identified through the systematic review were empirically situated in North America, Western Europe, or Australia. In contrast, there were no papers from other countries and continents. This could reflect underlying patterns in research on tolerance measures as well as the process of selecting articles. Clearly, the selection criteria requiring publication in English and with scales explained in English limits access to research published in other languages. The papers included some on measurement of tolerance in non-English-speaking Western Europe, but not from other regions. Thus, research on tolerance measures would seem to remain within a local language context: this could reflect that scales are language-dependent and, also considering the patterns in the papers reviewed, reflect evolving social issues that, in part, stem from the local context. That said, bearing in mind the general definitions of tolerance, such as by UNESCO [14], an interesting opportunity for further research is to access and integrate learning from research on tolerance scales set in other countries and cultures.

Also, the motivation to study tolerance is often rooted in its importance in society, which encompasses different social scales. The studies identified here focused on understanding tolerance for groups in particular contexts as well as tolerance at overall national or regional scales. If considering these as more micro- and macro-perspectives on tolerance, in contrast, a meso-perspective was not taken by any of the selected studies: for example, to assess tolerance for a whole community. However, themes highlighted by the selected papers point to this as of potential interest. Within a community, there could be multiple dimensions along which tolerance could be relevant. Also, there would likely be interaction across these dimensions due to social and other interconnections. A community-wide perspective could be valuable from a policy perspective, in particular with regard to implementation in practice, such as to track measures of tolerance over time.

Thus, an important practical implication is to deploy or develop tolerance scales at the appropriate level of aggregation. Further, the review evidenced the development of scales adapted to different social issues and with limited geographic coverage; thus, tolerance scales tended to be issue-specific and local. In turn, when developing a particular scale, this implies a relatively limited set of literature to draw on as dependent on issue and location/language. Thus, in practice, it would be worth considering tolerance scales on other social issues as well as accessing scales developed in other languages, so as to provide a broader set of approaches to draw upon. In terms of the format of the scales, the selected papers pointed to a common approach, Likert scales. There were varying numbers of questions, though with five of the eleven papers having less than ten questions. While this indicates the opportunity to explore alternative methodological approaches to measure tolerance, if pursuing a survey-based approach, based on current practice the systematic review suggests that a single-digit number of questions assessed on a Likert scale would be a reasonable starting point for developing a tolerance scale. In turn, this indicates that assessing tolerance is relatively feasible also as part of a broader survey; for instance, workplace surveys on employee engagement and so forth could also include an assessment of tolerance. Finally, from a training and development perspective, the systematic review pointed to the need to provide exposure to the variety of scales and the reasons for these.

In considering the findings and implications of the systematic review, certain limitations should be kept in mind. While the initial search aimed to be broad, encompassing three major databases, construction of search strings for broad coverage, and selection of results categories for broad coverage, there is nonetheless the risk of omission of certain studies. The screening down through review of the title and abstract may have excluded potentially relevant studies, which was mitigated by having two researchers perform the task independently. The risk of inappropriate inclusion is less relevant, considering the subsequent selection steps based on full-text review. Certain articles were not accessible in full text, which could reflect idiosyncratic access rights rather than a general lack of access. By design, the scope did not include non-English-language articles. Considering that the final results yielded papers covering selected geographies, as discussed, future research to access tolerance measures in other geographies would be of interest. Also, the review focused on the use of novel tolerance scales, and follow-on research could address the efficacy of the scales, though this review points to the challenge of varying definitions and the widening set of social contexts within which tolerance has been measured. To the best of our knowledge, this study is the first to collect and review the evidence of established tolerance scale in social settings. The aim is that such understanding contributes to attaining inclusivity in contemporary society.

5. Conclusions

While tolerance is recognised as important, especially to diverse societies, understanding tolerance poses complexities, both theoretically and in practical application. Tolerance is relevant in different social contexts, yet these pose distinct challenges to measurement, hence the importance of understanding the measurement of tolerance. This study provided a systematic review of tolerance scales. There has been a development of tolerance scales to encompass distinct social contexts over time, though some also focus on developing broader, less context-specific means to assess tolerance. The systematic review, however, yielded articles that covered only selected Western-centric geographies, indicating the future opportunity to broaden access to and integrate other geographical and cultural perspectives on tolerance.