A Meta-Analysis of Determinants of Success and Failure of Economic Sanctions ^†

Abstract

Political scientists and economists often assert that they understand how economic sanctions function as a foreign policy tool and claim to have backed their theories with compelling statistical evidence. The research puzzle that this article addresses is the observation that despite almost four decades of empirical research on economic sanctions, there is still no consensus on the direction and magnitude of the key variables that theoretically determine the success of economic sanctions. To address part of this research puzzle, we conducted a meta-analysis of 37 studies published between 1985 and 2018, focusing on three key determinants of sanction success: trade linkage, prior relations, and duration. Our analysis examines the factors contributing to the variation in findings reported by these primary studies. By constructing up to 27 moderator variables that capture the contexts in which researchers derive their estimates, we found that the differences across studies are primarily influenced by the data used, the variables controlled for in estimation methods, publication quality, and author characteristics. Our results reveal highly significant effects, indicating that sanctions are more likely to succeed when there is strong pre-sanction trade, when sanctions are implemented swiftly, and when they involve countries with better pre-sanction relationships. In our robustness checks, we consistently confirmed these core findings across different estimation techniques.

1. Introduction

The ‘unprecedented sanction packages’ against Russia on account of its war on Ukraine have put the topic of the potential contribution of economic sanctions on top of the research agenda. Evidence-based policy advice, however, is hampered by the fact that the empirical literature does not appear to reach a consensus on the efficacy and effectiveness of economic sanctions. This is especially troublesome given the enormous increase in the use of economic sanctions in the recent period (see Figure 1).

Research on sanctions already has a substantial history. Empirical research on economic sanctions began in 1985 with the landmark publication of Economic Sanctions Reconsidered by Hufbauer and Schott. This seminal work introduced one of the first large-N datasets, paving the way for numerous studies examining the determinants of economic sanctions and their success or failure. Over the past four decades, this initial dataset has undergone several revisions and extensions (see Hufbauer et al., 2007). Additionally, new databases, such as the Threats and Imposition of Economic Sanctions (TIES) (see Morgan et al., 2009, 2014) and the Global Sanction Database, have emerged to further enrich the field (see van Bergeijk, 2021 for a discussion). The research puzzle that this article addresses is the observation that despite almost four decades of empirical research on economic sanctions, no consensus has yet emerged on the sign and size of the impacts of the key variables that theoretically determine the success of economic sanctions. To solve this research puzzle, we have performed a meta-analysis of 37 primary studies published over the years 1985–2018 that aim to explain empirically whether an economic sanction succeeds.

Rather than focusing on only one variable of interest (as is typically done in meta-analyses), we decided to simultaneously meta-analyze three variables (pre-sanction trade linkage, duration of the sanction episode and prior relations). We believe that a credible assessment of the importance of determinants within a research field cannot rest on a meta-analysis of a single variable. By extending our analysis to three interrelated variables, we gain a broader perspective. This approach allows us to determine whether our findings reflect a general relevance of key determinants across the literature or if they are specific to a particular variable, such as trade linkage. The literature on economic sanctions provides a great many potential determinants of sanction success, including characteristics of the sanction goal (e.g., disruption of military activities, the removal from office of authoritarian leaders or the restoration of human rights), political characteristics (such as international cooperation with the sender or the target, e.g., Black Knighting) and economic characteristics (including trade linkage and costs). It is for practical reasons impossible to cover all these potential determinants in one article, so we have to be selective, and we will focus on economic determinants (again, this should be seen as a limitation of our study).

The selection of these three variables is grounded in three key factors: (i) the robustness of their relationship across various editions of the Hufbauer et al. (1985, 1990, 2007) dataset, which has been frequently utilized in empirical sanctions research and is often included in more recent datasets (van Bergeijk & Siddiquee, 2017); (ii) their clear theoretical foundation in international economics (Demena et al., 2021); and (iii) the observed pattern of estimated coefficients, which indicates ongoing disagreement among primary studies (Figure 2). Figure 2 plots on the vertical axis the estimated coefficients for our three variables of interest, as reported in the primary studies, and on the horizontal axis the year of publication of the primary study, thereby showing the development of the estimated parameters over time. Although the mainstream view holds that successful economic sanctions are linked to larger trade connections, shorter durations, and stronger prior relationships, the figure highlights significant heterogeneity in both the magnitude and direction of effects, with no clear convergence. In such a context, a meta-analysis proves to be a valuable tool for evaluating and synthesizing the existing research, especially in a field where findings continue to diverge.

Using a meta-analysis, our paper first examines the primary empirical studies performed to identify the factors that explain the divergence in findings of the reported coefficients. Next, our meta-analysis enables us to estimate the ‘genuine’ underlying meta-effects for three important sanction episode characteristics: (a) pre-sanction trade linkage between sanction sender and target (economic sanctions do not make much sense if the sanction target and sanction sender do not trade), (b) the duration of the sanction episode (sanctions probably need to be quick and unexpected to have a maximum impact) and (c) prior relations between sender and target (sanctions may work better against friends than foes).

Although Druckman (1994) was an early advocate for the use of meta-analysis in fields such as International Relations, International Political Economy, and Peace and Conflict Studies, many researchers in these areas remain unfamiliar with its advantages and limitations. Despite frequent citations of Druckman’s work, the methodology he promoted has not seen widespread adoption, leaving research synthesis in these disciplines methodologically underdeveloped. While meta-analysis has been employed in adjacent fields, its application within these specific areas has been limited. However, recent meta-analytic studies have begun to showcase its value for peace studies and international political science. Notable examples include investigations into the growth impact of military spending (Alptekin & Levine, 2012), the relationship between income and democracy (Broderstad, 2018), the tools of economic diplomacy (Moons, 2017), and the link between climate change and conflict (Sakaguchi et al., 2017).

Meta-analysis, however, remains a tool that is only used for peripherical questions so far. The next section, therefore, delves into the methodology of meta-analysis, with a focus on its application within this specific field. Section 3 reports on the process of data collection, in particular the structured review and the selection of primary studies to be included in the meta-analysis. The Section 4 provides the empirical analysis. The Section 5 concludes.

2. Why Meta-Analysis

Meta-analysis was developed in medicine, where it is was used to establish the true effect out of small-N samples of drug tests. The method quantitatively analyzes studies and statistically cleans for biases (Fonseca, 2022). Importantly, a meta-analysis allows us to objectively establish the central tendency in the empirical literature, and to investigate a potential explanation for the heterogeneity of findings in the literature, as documented in Figure 2 (Stanley & Doucouliagos, 2012). The meta-analysis both includes and goes beyond a traditional systematic review. The use of a meta-analysis adds value to a traditional review of the literature—a meta-analysis is less prone to subjective bias and more transparent than a traditional literature review because it systematically analyzes the sources of (quantitative) variation in earlier primary studies (Fonseca, 2022). While these strengths are important, meta-analysis also has an important weakness since it can only be applied to empirical literature, which also needs to be of a sufficient size in terms of studies and involved authors (Stanley & Doucouliagos, 2012). Indeed, meta-analysis has been labeled ‘unhelpfully positivist’ and biased against qualitative work, and even void of theory (Dacombe, 2018). It is therefore important to realize that meta-analysis is only one useful tool in the toolbox of research synthesis.

The first application of a systematic review with a meta-analysis in the area of Conflict Resolution, Peace Science and International Relations by Druckman (1994) occurred rather early in comparison to other social sciences, such as economics, where meta-analysis was at that time hardly ever applied. In economics, following a period of stasis, meta-analysis has now become an accepted way to synthesize research findings, to investigate publication bias and to analyze the impact of the research design and methodology of primary studies. We hope that our article can stimulate a similar development in Conflict Resolution, Peace Science and International Relations. It is encouraging that recently, several meta-analytic studies have been published that demonstrate the potential utility of meta-analysis for our field, including Alptekin and Levine (2012) on growth and military expenditures, Broderstad (2018) on democratization and income, and Sakaguchi et al. (2017) on climate wars. Our meta-analysis is the first to deal with determinants of economic sanctions (indeed, van Bergeijk et al. (2019) and Demena et al. (2021) investigated the issue of publication bias in the field of economic sanctions). This is a relevant topic for meta-analysis in view of the apparent disagreement on the success of sanctions, which does not only differ by ‘school’, but also over time shows significant fluctuation in policy discussions that often are strongly driven by recent high-profile cases. In this context, subjectivity can become problematic.

As specified above, we will focus on three key economic determinants of sanction impact. Underlying our analysis is the assumption that the costs and benefits of considered policies motivate decision-makers, and therefore we focus on economic welfare (aggregate utility) as a driver of behavior of the sanction target. We do of course not argue that economic costs and benefits are the only relevant variables, and recognize that many other non-economic variables can determine the outcome of concrete sanction episodes. An appendix with the mathematical modeling details is available upon request. Basically, the standard neoclassical model in the context of trade uncertainty can be used to motivate the choice of the variables of interest for the meta-analysis, and also to derive theoretical expectations from core economic theory on their signs. Our a priori expectations are that sanctions are more likely to succeed in the context of larger trade linkage, shorter duration, and better prior relations. Based on a standard economic model, we have strong priors regarding the sign and significance of the variables of interest, and this makes the findings of the meta-analysis even more challenging.

3. Identifying Eligible Studies and Meta-Data

3.1. Search Strategy and Selection Criteria

The dataset construction adhered strictly to the guidelines established by the Meta-Analysis in Economics Research Network (MAER-Net) (Stanley et al., 2013; Havránek et al., 2020). We conducted an extensive literature search, primarily using Google Scholar, supplemented by the Web of Science of the ISI. In addition, we reviewed reference lists from recent empirical studies and relevant reviews to ensure a comprehensive scope. Our search covered all potentially relevant published and unpublished primary empirical studies from 1985 through 2018 (for an extensive discussion including both peer-reviewed/published and unpublished studies, see Demena, 2024a).

To capture the breadth of research, we used various broad keyword combinations related to our three variables of interest, as shown in

Table 1. Search list of keywords, selected primary studies and collected observations.

Web Engine	Variable of Interest	Search Keywords	Results Returned (Selected)	No. of Reported Estimates
	trade linkage	economic sanctions, economic coercion, sanction threat, success*, fail*, work, sanction outcome*, episodes, determin*, cost and result.	430 (23)	174
Google Scholar and Web of Science (ISI Web of Knowledge)	prior relations	Economic sanctions, economic coercion, sanction*, episodes, determin*, success*, fail*, effect*, work, out-comes, result*, cost*, sender state, target state, foreign, *politic*, democratic*, autocrat*, *leader*, *stability, empirical analysis, sensitivity analysis, approach, econometric analysis, modelling	360 (19)	83
	duration	economic sanctions, sanctions*, success of economic sanctions, sanction*outcome*, sanction* duration, sanction time, sanctions episode*, sanctions imposition*, length sanction episode*	210 (13)	77

Source: Adapted from Demena et al. (2021), Table 6.1. p. 131 with a permission from Peter A. G. van Bergeijk (ed.) as editor of the Research Handbook on Economic Sanction (van Bergeijk, 2021).

. The studies were selected based on several criteria: they had to be written in English, provide empirical analyses focused on the success or failure of economic sanctions, and include at least one of our variables of interest, either as a key variable or a control. Furthermore, the studies had to report regression-based coefficients, sample sizes, t-statistics, or standard errors. Applying these selection criteria resulted in a final set of 33 studies on trade linkage, 15 studies on sanction duration, and 24 studies on prior relations, which were subsequently coded for further analysis.

The dataset was constructed by extending the research conducted by van Bergeijk et al. (2019) and Demena et al. (2021). The multi-stage search process was completed in February 2019. Prior to the data collection and coding for van Bergeijk et al. (2019) and Demena et al. (2021), the team leader, PB, developed a structured protocol, including a coding manual, inclusion criteria, and an Excel spreadsheet for data extraction, which was later expanded by BD for the current study. The coding process involved independent searches and eligibility assessments by multiple coders. Each coder systematically reviewed, read, and extracted data from the primary studies using three separate Excel spreadsheets, with each spreadsheet dedicated to a specific variable of interest. This dataset was later reviewed and expanded to include additional factors that could help explain the success and failure of economic sanctions. The three extended datasets, each corresponding to a specific variable of interest, were ultimately merged into a single comprehensive Excel spreadsheet. Using this structured data extraction template, the research team manually populated a total of 44,685 data cells.

One can distinguish three different kinds of meta-datasets (H. Doucouliagos & Paldam, 2008; Stanley & Doucouliagos, 2012). The best-set estimate is constructed using the ‘preferred specification’ of the authors of the primary studies. This preference is, however, not always explicitly reported in the empirical studies, but if reported, may introduce subjectivity, that is, selection bias based on the author’s preference. The average-set uses the average of all reported estimates in each primary study. The use of the best-set estimates or the average-set estimate has become common practice, in particular to avoid giving undue weight to a single study, but these two approaches can lead to an important loss of within-study information (Demena et al., 2024). Moreover, in this field the number of available studies is still relatively limited, and thus average-set or best-set estimates would result in numbers of observations that are too low to perform a meaningful analysis. Therefore, we use the all-set estimates, constructed through coding all relevant regressions in each of the studies (see a recent meta-analysis that commonly applies all-set estimates for, e.g., Afesorgbor et al., 2024; Demena, 2024b).

3.2. Meta-Data

The empirical primary studies included in our meta-data are in

Table 2. Empirical studies included in the meta-analysis.

Included Empirical Primary Studies
Allen (2008)	Hart (2000)	Major (2012)
Ang and Peksen (2007)	Hatipoglu (2010)	Michel et al. (2016)
Bapat and Kwon (2015)	Hufbauer et al. (1985)	Nooruddin (2002)
Bapat and Morgan (2009)	Hufbauer et al. (2007)	Peterson (2018)
Bonetti (1998)	Hull (2015)	Shagabutdinova and Berejikian (2007)
Chan (2009)	Jing et al. (2013)	van Bergeijk (1989)
Dashti-Gibson et al. (1997)	Kim (2009)	van Bergeijk (1994)
Dehejia and Wood (1992)	Kim (2013)	van Bergeijk (2009)
Drezner (2000)	Kleinberg (2018)	van Bergeijk and Siddiquee (2017)
Driscoll et al. (2011)	Lam (1990)	Whang et al. (2013)
Drury (1998)	Lektzian (2003)	Woo and Verdier (2014)
Early (2011)	Lektzian and Patterson (2015)
Elliott and Uimonen (1993)	Lektzian and Souva (2007)

. The dataset comprises 334 observations drawn from 37 studies, each of which provides the necessary data in the form of estimated coefficients. Of these 37 studies, only 2 focus on a single country, while the remaining studies are large-N analyses, examining multiple cases or countries. The timeframe of the studies included spans from 1985 to 2018, with the earliest study published in 1985 and the most recent in 2018. Notably, the median publication year is 2007, meaning that half of the primary studies have been published within the last decade, highlighting the ongoing relevance and timeliness of this research area.

The number of parameter estimates extracted from each primary study varies, with the median study contributing five coefficients. The range of observations per study also demonstrates considerable diversity, with the minimum number of regressions contributing just 1 coefficient, while the maximum is 48. On average, each study provides nine coefficients. This spread illustrates the variation in the level of detail and scope across the studies included in our dataset.

The descriptive statistics in

Table 3. Summary of reported effect size/coefficient according to the three variables of interest.

Effect Size	Mean	S.D.	Min.	Max.
Trade-linkage	−0.203	2.163	−10.67	3.05
Prior relations	0.400	0.498	−0.758	1.67
Duration	−0.440	0.682	−1.94	0.49

reveal interesting patterns across the three variables of interest. Trade-linkage effect size demonstrates a slight negative mean (−0.203), but with substantial variability (SD = 2.163) and the widest range (−10.67 to 3.05), suggesting inconsistent effects across observations. The prior relations effect size shows a moderate positive average (0.400) with relatively consistent measurements (SD = 0.498), ranging from −0.758 to 1.67, indicating generally favorable but occasionally negative effects. Duration size exhibits a moderate negative average (−0.440) with mid-range variability (SD = 0.682) and values spanning from −1.94 to 0.49, suggesting predominantly negative effects with some exceptions. Together, these statistics suggest that prior relations tend to have positive impacts, while trade and duration factors typically demonstrate negative effects, though with varying degrees of consistency.

The dataset includes 30 peer-reviewed journal articles, demonstrating that most primary studies in this analysis have been published in high-quality, peer-reviewed outlets. By March 2019, these studies had collectively gained 4523 citations on Google Scholar, reflecting the significant impact of this body of literature. Among these, the most highly cited work is Economic Sanctions Reconsidered: History and Current Policy (across all its editions), which has received 2065 citations. The second most cited study is Drezner’s 2000 article on bargaining, which has accumulated 310 citations.

Approximately 40% of the reported coefficients were drawn from articles published in A-level journals, underscoring the prominence of top-tier academic outlets in this field.1 However, despite the high quality of these publications, a clear consensus remains elusive. As shown in

Table 4. Qualitative findings from empirical research published in high-impact journals.

Author(s) and Year of Publication	The Journal That Publishes the Study	Empirical Results of the Three Variables of Interest
		Trade Linkage	Duration	Prior Relations
Peterson (2018)	Conflict Management and Peace Science	−
Kleinberg (2018)	Journal of Peace Research	*		−
Jing et al. (2003)				+
Drury (1998)		*		*
Drezner (2000)	International Organization	+		+
Chan (2009)	International Political Science Review	−	−
van Bergeijk and Siddiquee (2017)	International Interactions	*	−	+
Bapat and Kwon (2015)		+
Major (2012)		+
Nooruddin (2002)		*	*
Whang et al. (2013)	American Journal of Political Science	+
Dashti-Gibson et al. (1997)		*	*
Lektzian and Patterson (2015)	International Studies Quarterly	*	*	+
Bapat and Morgan (2009)		+	*	*
Lektzian and Souva (2007)	Journal of Conflict Resolution	*
Ang and Peksen (2007)	Political Research Quarterly	*	*	+
Hart (2000)		+		*

Notes: ‘+’ indicates positive and statistically significant at the 10% level or better; ‘−’ indicates negative and statistically significant at the 10% level or better, and ‘*’ indicates variable included but not significant. Blank cells signify variables not covered in the primary study. Source: Adapted from Demena et al. (2021), Table 6.2. p. 132 with a permission from Peter A. G. van Bergeijk (ed.) as editor of the Research Handbook on Economic Sanction (van Bergeijk, 2021).

, the qualitative findings from the empirical studies published in A-journals reveal considerable disagreement, with conflicting significant coefficients, and many cases where no significant relationship is found. This inconsistency highlights the ongoing debate and lack of uniformity in the literature, even among the most prestigious journals.

Success in economic sanctions is generally defined as the extent to which sanctions lead to full compliance by the target country, or at least a partial policy change that aligns with the stated objectives of the sanctioning country (Peksen, 2019). However, different studies employ varying methodologies to quantify and classify the success of economic sanctions, which results in a range of measurement approaches. One common approach to measuring the success of sanctions is through a binary classification system. In this method, sanctions are considered successful if they achieve the intended policy change and unsuccessful if they fail to do so. Success is coded as 1, while failure is coded as 0. Some studies further refine this binary classification by introducing a policy outcome scale ranging from 1 to 4. A score of 1 represents complete failure, 2 indicates partial failure, 3 reflects partial success, and 4 signifies full success. To simplify the measurement, outcomes scoring 3 or 4 are categorized as successful (coded as 1), while those scoring 1 or 2 are categorized as failures (coded as 0).

Another method used in the literature is an index-based success scoring system, which evaluates the effectiveness of sanctions on a broader scale. This approach calculates the total success score as the product of the policy outcome (ranging from 1 to 4) and the contribution of sanctions to the policy outcome (also ranging from 1 to 4). The resulting success score falls within a range of 1 to 16, where scores between 1 and 8 are classified as failures, and scores between 9 and 16 are considered successes. In some cases, this index is further converted into a binary scale, where a score of 9 or higher is labeled as success (coded as 1), and a score of 8 or lower is labeled as failure (coded as 0) (Hufbauer et al., 1990).

In addition to evaluating policy outcomes, some studies also measure the contribution of sanctions to the overall success of policy implementation. This contribution is often quantified on a scale from 1 to 4, where 1 indicates a negative contribution, 2 represents an insignificant contribution, 3 signals a partial positive contribution, and 4 reflects a strong positive contribution. For regression analysis using binary estimation models, contributions that positively influence policy outcomes are coded as 1, while negative or insignificant contributions are coded as 0.

Given the variety of measurement approaches, the evaluation of sanctions’ effectiveness remains a complex and debated issue. For further discussion on the challenges associated with measuring the success and failure of economic sanctions, see van Bergeijk and Siddiquee (2017).

4. Determinants of Success and Failure of Economic Sanctions

4.1. Potential Factors Explaining Success and Failure of Economic Sanctions

To gain deeper insights into the varying results associated with our variables of interest (depicted in Figure 2), we have gathered a range of contextual variables that capture the conditions under which the primary studies were conducted. The complete lists of these potential moderator variables are presented in

Table 5. Definition and descriptive statistics of determinant variables (trade linkage between sender and target).

Moderator Variables	Definition	Mean	S.D.	Min	Max
Outcome Characteristics
E	Trade effect size	−0.203	2.163	−10.67	3.05
SE	Standard error of effect size	0.510	1.143	0.001	11
	Data Characteristics
No. obs.	Logarithm of number of observations used by the study	5.282	0.926	2.944	8.205
HSE/O	=1 if data come from HSE or HSEO (reference category: mixed or other dataset sources not listed here)	0.529	0.5001
TIES	=1 if data come from TIES	0.253	0.436
	Estimation Characteristics
Probit	=1 if estimation method is Probit (reference category for this group of dummy variables: OLS, Censored Selection Method and a method not listed here)	0.345	0.477
Logit	=1 if estimation method is Logit	0.621	0.487
	Specification Characteristics
Sender Cost	=1 if cost of sanctions to sender is included	0.414	0.494
Target Cost	=1 if cost of sanction to target is included	0.402	0.492
US Sender	=1 if United State as sender is included	0.477	0.501
Political instability	=1 if political instability of target is included	0.443	0.498
Cooperation senders	=1 if external cooperation to sender is included (multilateral cooperation among senders)	0.443	0.498
Assistant target	=1 if international assistant to target is included	0.362	0.482
Relative power	=1 if relative economic senders’ power/size to target’s power is included	0.259	0.439
Militarized dispute	=1 if target is in militarized dispute with sender while sanction was imposed	0.339	0.475
Sanction length	=1 if sanction length episode is included	0.333	0.473
Democracy	=1 if target is democratic included	0.253	0.436
Institutional sanction	=1 if sanction is institution in sender coalition	0.155	0.363
Trade ratio	=1 if presanction trade is export + imports as % of target’s total trade (others as excluded category)	0.546	0.499
Trade flows	=1 if presanction trade is target’s export to senders and import from senders	0.328	0.471
	Publication Characteristics
Peer-reviewed	=1 if study is published in a peer-reviewed journal	0.655	0.477
Publication year	The publication year of the study (base, 1985)	24.57	8.526	0	33
Journal rank	High journal rank, 2019 ISI impact factor	0.523	0.501
Study citations	Study citations in Google Scholar per age of the study, till March 2019	3.382	4.235	0	17.25
	Author Characteristics
Co-authored	=1 if study is co-authored	0.236	0.426
Political scientist	=1 if a cited author is a political scientist (base, economist/others)	0.733	0.442
US affiliation	=1 a cited author is affiliated with US-based institution	0.684	0.466
Academic affiliation	=1 a cited author is affiliated with an academic institution	0.759	0.429

,

Table 6. Definition and descriptive statistics of determinant variables (prior relations between sender and target).

Moderator Variables	Definition	Mean	S.D.	Min	Max
Outcome Characteristics
E	Prior relations effect size	0.400	0.498	−0.758	1.67
SE	Standard error of effect size	0.364	0.3001	0.048	2.755
	Data Characteristics
No. obs.	Logarithm of number of observations used by the study	5.264	0.904	2.944	8.205
HSE/O	=1 if data come from HSE or HSEO (mixed or other dataset sources as excluded category)	0.687	0.467
TIES	=1 if data come from TIES (mixed or other dataset sources as excluded category)	0.096	0.297
	Estimation Characteristics
Probit	=1 if estimation method is Probit (OLS, Censored Selection Method and others as excluded category)	0.458	0.501
Logit	=1 if estimation method is Logit	0.518	0.503
	Specification Characteristics
Sender Cost	=1 if cost of sanctions to sender is included	0.530	0.502
Target Cost	=1 if cost of sanction to target is included	0.494	0.503
US Sender	=1 if United State as sender is included	0.470	0.502
Political instability	=1 if political instability of target is included	0.578	0.497
Cooperation senders	=1 if external cooperation to sender is included (multilateral cooperation among senders)	0.687	0.467
Assistant target	=1 if international assistant to target is included	0.482	0.503
Relative power	=1 if relative economic senders’ power/size to target’s power is included	0.337	0.476
Military force	=1 if senders used regular or quasi-military force	0.422	0.497
Sanction length	=1 if sanction length episode is included	0.590	0.495
Democracy	=1 if target is democratic included	0.361	0.483
Binary PR	=1 if prior relations between sender & target is dichotomic (friends or allies if not 0); weighted political affinity scores & others as excluded category)	0.181	0.387
Ordered PR	=1 if prior relations is ordered from antagonistic, neutral to cordial relationship between sender & target (weighted political affinity scores & others as excluded category)	0.687	0.467
Institutional sanction	=1 if sanction is institution in sender coalition	0.410	0.495
Policy change	=1 if sanction is destabilization or regime change	0.566	0.499
	Publication Characteristics
Peer-reviewed	=1 if study is published in a peer-reviewed journal	0.711	0.456
Publication year	The publication year of the study (base, 1985)	23.542	7.239	0	33
Journal rank	High journal rank, 2019 ISI impact factor	0.373	0.487
Study citations	Study citations in Google Scholar per age of the study, till March 2019	4.322	4.639	0	17
	Authors Characteristics
Co-authored	=1 if study is co-authored	0.361	0.483
Political scientist	=1 if a cited author is a political scientist (base, economist/others)	0.675	0.471
US affiliation	=1 a cited author is affiliated with US-based institution	0.675	0.471
Academic affiliation	=1 a cited author is affiliated with an academic institution	0.879	0.328

and

Table 7. Definition and descriptive statistics of determinant variables (duration of sanction episode).

Moderator Variables	Definition	Mean	S.D.	Min	Max
Outcome Characteristics
E	duration size	−0.440	0.682	−1.94	0.49
SE	Standard error of effect size	0.304	0.495	0.005	3.08
	Data Characteristics
No. obs.	Logarithm of number of observations used by the study	5.247	1.070	2.944	8.205
HSE/O	=1 if data come from HSE or HSEO (mixed or other dataset sources as excluded category)	0.844	0.365
TIES	=1 if data come from TIES (mixed or other dataset sources as excluded category)	0.052	0.223
	Estimation Characteristics
Probit	=1 if estimation method is Probit (OLS, Censored Selection Method and others as excluded category)	0.299	0.461
Logit	=1 if estimation method is Logit (OLS, Censored Selection Method and others as excluded category)	0.675	0.471
	Specification Characteristics
Sender Cost	=1 if cost of sanctions to sender is included	0.299	0.461
Target Cost	=1 if cost of sanction to target is included	0.390	0.491
US Sender	=1 if United State as sender is included	0.299	0.461
Political instability	=1 if political instability of target is included	0.610	0.491
Cooperation senders	=1 if external cooperation to sender is included (multilateral cooperation among senders)	0.506	0.503
Assistant target	=1 if international assistant to target is included	0.273	0.448
Relative power	=1 if relative economic senders’ power/size to target’s power is included	0.104	0.307
Militarized dispute	=1 if target is in militarized dispute with sender while sanction was imposed	0.325	0.471
Institutional sanction	=1 if sanction is institution in sender coalition	0.312	0.466
	Publication Characteristics
Peer-reviewed	=1 if study is published in a peer-reviewed journal	0.740	0.441
Publication year	The publication year of the study (base, 1985)	21.558	8.963	0	32
Journal rank	High journal rank, 2019 ISI impact factor	0.259	0.441
Study citations	Study citations in Google Scholar per age of the study, till March 2019	5.563	3.999	0	17
	Authors Characteristics
Co-authored	=1 if study is co-authored	0.273	0.448
Political scientist	=1 if a cited author is a political scientist (base, economist/others)	0.610	0.491
US affiliation	=1 a cited author is affiliated with US-based institution	0.442	0.499
Academic affiliation	=1 a cited author is affiliated with an academic institution	0.779	0.417

.

Data characteristics: To explore potential systematic differences between the included empirical studies with varying sample sizes, we include the number of observations reported for the regression coefficients. This allows us to test whether large sample sizes yield different results compared to smaller ones. Additionally, we account for the source of the data used in each study by introducing a dummy variable that indicates whether the study employed the HSE dataset. This (HSE) dataset, originally compiled by Hufbauer, Schott, and Elliott (Hufbauer et al., 1985, 1990), and later extended by Hufbauer, Schott, Elliott, and Oegg (Hufbauer et al., 2007), has been widely used in research on sanctions, and is referred to as HSEO. While just over half of the trade linkage data come from the HSE or HSEO datasets, most studies on sanction duration and prior relations rely heavily on these sources, with 84% and 69% of the data, respectively, drawn from them. Furthermore, we incorporate another prominent data source, the TIES dataset, which provides additional insights into sanctions dynamics. Since the majority of studies investigating the determinants of the success or failure of economic sanctions have relied on the HSE, HSEO, or TIES datasets, distinguishing between these sources allows us to assess whether the choice of data source contributes to the variation in study findings.

Estimation characteristics: Different estimation techniques can lead to varying outcomes, so controlling for these methods allows us to identify whether certain techniques contribute to divergent findings. The majority of the regression estimates in our dataset were generated using the logit estimation approach, which is commonly employed when the dependent variable is binary (e.g., success or failure of sanctions). For instance, in the case of trade linkage, approximately two-third adopted the logit model, whereas 52% and 68% for prior relations and duration, respectively. This is followed by the probit estimation approach, another widely used method for binary outcomes. However, some studies have employed alternative estimation techniques, such as ordinary least squares (OLS), which is typically used for continuous outcomes, or censored selection methods, which address potential biases in sample selection. Since these alternative techniques are used less frequently, we group them together as a reference category for comparison. By distinguishing between these estimation methods, we can investigate whether certain econometric approaches are systematically associated with different outcomes in the analysis of economic sanctions.

Specification characteristics: The included regression-based empirical studies use various variables to explain the success or failure of economic sanctions. In our meta-dataset, the following key explanatory variables include:

Costs of sanctions. Whether the cost of sanctions is accounted for in the regression models, both for the sender and the target country.

Sender characteristics. Whether the sanctions are imposed by the United States or involve multilateral cooperation among multiple sending countries.

Target country characteristics. Variables that indicate if the target country is politically unstable, democratic, or has received international assistance.

Relative economic power. The ratio of the sender’s Gross National Product (GNP) to the target’s GNP, capturing the relative economic strength of the sanctioning country compared to the target.

Militarized disputes. Whether the target country was already engaged in a militarized dispute with the sender at the time the sanctions were imposed.

In addition to these, we also consider various proxy measures used to capture our core variables of interest. For instance, trade linkage is primarily measured in one of two ways—either through trade flows or as a ratio of trade between the sender and target. Trade ratio accounts for 55% of the regressions in our dataset, while trade flows are used in roughly a third of the regressions. In the case of prior political relations, most empirical studies measure these relations using either a binary approach (assigning a value of 1 for allies or friends, and 0 otherwise) or an ordered scale that categorizes the relationship from antagonistic to neutral to cordial. By examining how these various specifications are included in the regression models, we aim to identify how these different variables influence the outcomes and contribute to the success or failure of economic sanctions.

Publication characteristics. We account for both the quality of the studies and the journals in which they are published to determine whether publication in peer-reviewed outlets is systematically linked to the reported regression-based determinants of economic sanctions. Peer review is generally expected to enhance the quality and reliability of the findings, ensuring more rigorous scrutiny (Demena, 2017). However, the peer-review process may also have a conservative bias, potentially avoiding the publication of results that challenge prevailing views or consensus in the field. To test this, we use a binary dummy variable, assigning a value of 1 if the study was published in a peer-reviewed journal. Notably, the majority of the reported coefficients (at least two-thirds) come from studies that underwent peer review. Additionally, we examine the quality of the publication outlets by incorporating the journal’s ISI impact factor. Some meta-analysts suggest that higher-ranked journals are more likely to publish studies employing superior methodologies, which in turn tend to yield more robust and reliable findings (Disdier & Head, 2008). This allows us to test whether studies published in journals with lower or higher impact factors are systematically associated with different patterns of success or failure in economic sanctions. High-quality journals (A-journals) are identified as those ranked in the top quartile of outlets based on the 2019 ISI impact factor.

Furthermore, we include author-level publication characteristics by calculating their total Google Scholar citations, adjusted for the age of the studies. This accounts for the impacts of both long-established works and newer publications, providing a measure of the author’s influence in the field. Finally, we control for the publication year of the primary studies, taking into consideration the evolution of research over time. In relation to the publication year of the studies, the research-cycle hypothesis formulated by Goldfarb (1995) examines whether the novelty and fashion of academic research play a role.

Author characteristics. We next examine how the characteristics of the authors behind each study might influence the reported success or failure of economic sanctions. This analysis starts by considering whether the study is authored by an individual or a team. Research teams are generally expected to provide more balanced results than single authors due to the internal peer review that occurs within the team. To account for this, we include a binary dummy variable that takes the value of 1 if the study is co-authored. Interestingly, approximately two-thirds of the reported regression coefficients come from single-authored studies, suggesting that solo authorship is quite common in this field. Another factor we consider is the academic discipline of the researchers. Differences in academic fields can lead to variations in reporting standards and methodologies. To test this, we introduce a binary dummy that takes the value of 1 if the author is a political scientist, as opposed to economists, sociologists, or other disciplines. Political scientists authored the majority of the estimates in our dataset, representing over two-thirds of the reported findings.

Building on the work of C. Doucouliagos and Paldam (2010), who highlighted the importance of researcher affiliation, we also assess whether the institution with which the author is affiliated plays a role. Following the framework proposed by Stanley (2005), we create two binary variables—one indicating if the author is affiliated with a US-based institution and another indicating if the author works at an academic institution. The former variable reveals that roughly two-thirds of the reported estimates (except in duration studies, where the figure is less than 50%) come from researchers based in the United States. The latter variable shows that nearly 80% of the observations are associated with authors working in academic settings. These affiliation data were collected from the institutional details provided by the authors at the time of their study’s publication.

By including these author-specific factors, we aim to identify whether characteristics such as team size, disciplinary background, and institutional affiliation influence the reported outcomes of economic sanctions research, thereby offering further insights into the sources of variability across studies.

4.2. Empirical Strategy

To investigate the determinants of the success and failure of economic sanctions, we start with a simple meta-regression, but with a powerful statistical approach known as meta-regression model, which is the basic for multivariate meta-regression analysis (MMRA). Our approach is informed by recent applications of multivariate MRA within the field of economics, drawing on studies such as those by Demena et al. (2024), Floridi et al. (2023), Iršová et al. (2023), Afesorgbor et al. (2024), and Demena (2024a).

$${effect}_{ij}={\beta }_0+{\beta }_1{SE}_{ij}+{\varepsilon }_{ij}$$

(1)

In Equation (1), we define the variable effect_ij as the ith regression coefficients reported in the jth included study, which quantifies the effect of specific determinants on the outcomes of economic sanctions. Accompanying this coefficient is the standard error SE, reflecting the uncertainty associated with the estimation of the coefficient, effect. The term β₁ represents the potential publication selection bias. This bias can occur when studies with statistically significant results are more likely to be published than those with non-significant findings. By accounting for this bias, we aim to enhance the validity of our results. The constant term ${\beta }_0$ denotes the underlying genuine effect size for each variable of interest, adjusted for the effect of potential publication bias. Finally, ɛ_ij represents the standard error term, which accounts for the unexplained variability in the model, reflecting other factors that may influence the relationship between the independent variables and the outcome measures.

However, Equation (1) possesses heteroskedasticity and should not be estimated by an OLS. Since the standard error is the independent variable of the reported coefficient or effect size, the latter have different estimated variances as researchers employ several empirical specifications, econometric designs, and sample sizes, and thus Equation (1) may suffer from heteroskedasticity (Demena, 2017; H. Doucouliagos & Laroche, 2009). To address this problem of heteroscedasticity, an OLS would be inappropriate, and we use weights using the inverse of the variance (Stanley & Doucouliagos, 2012). Applying the inverse variance weights, we derive the following weighted least squares (WLS) approach:

$$t_{ij}={\beta }_1+{\beta }_0({1/SE}_{ij})+e_{ij}$$

(2)

where $t_{ij}$ is the t-value measuring the statistical significance of the reported regressions coefficients collected from the primary empirical studies. Through testing the null hypothesis for the intercept in Equation (2), i.e., ${\beta }_1$= 0, and for the slope, i.e., ${\beta }_0$= 0, we assess the existence of publication bias and the underlying genuine effects of the three determinants of the sanction success or failure after accounting for potential bias, respectively. The former is known as the funnel asymmetry test (FAT) and the latter as the precision-effect test (PET) (Egger et al., 1997).

To investigate the factors of the determinants that contribute to the success or failure of economic sanctions empirically, we added the identified potential moderator variables that are presented in Table 5, Table 6 and Table 7 into Equation (2), deriving

$$t_{ij}={\beta }_1+{\beta }_0{(1/SE}_{ij})+{\displaystyle \frac{{\alpha }_k{X}_{kij}}{{SE}_{ij}}}+e_{ij}$$

(3)

where X denotes a vector of the moderator variables listed in Table 5, Table 6 and Table 7 and weighted by the inverse of the variance, ${\alpha }_k$ is the associated coefficient and K is the specific category of the identified potential moderator variable.

An important empirical concern in estimating Equation (3) is the issue of within-study dependence when ‘all-set’ estimates are used. In addition, in our case, between-study dependence is also an important concern, since multiple studies are published by the same authors (and thus unlikely to be statistically independent).2 Accordingly, we employ a multi-level model or hierarchical model to address the issue of between-study dependence while also controlling for the within-study dependence. This model enables us to account for dependence both within individual studies and between different studies by incorporating a random effect for each study, leading to its classification as a multilevel random effect model (Bateman & Jones, 2003; Ugur et al., 2020; Floridi et al., 2023). When we introduce moderator variables to control for these variations, the model evolves into a mixed-effects multilevel model (MEM), as it adjusts for both within-study and between-study differences.

More specifically, this model operates on two levels: the study level (capturing characteristics specific to each primary study) and the estimate level (focusing on the reported regression coefficients within each study). Thus, to incorporate these complexities, we extend Equation (3) to reflect the two-level structure as follows:

$$t_{ij}={\beta }_1+{\beta }_0{(1/SE}_{ij})+{\displaystyle \frac{{\alpha }_k{X}_{kij}}{{SE}_{ijc}}}+{\zeta }_j+e_{ij}$$

(4)

where ${\zeta }_j$ is the study-level random effects (random intercepts), and the others are the same as in Equation (3). In this modeling, estimates, which are level 1, are clustered and nested within studies, which are level 2. Our notations follow those of H. Doucouliagos and Stanley (2009) and Ugur et al. (2020). Following recent meta-analyses, therefore, estimates associated with determinants of economic sanctions that are reported by the empirical studies are nested within each study, and the estimates are modeled to differ between studies. The approach is widely applied in recent meta-analyses in economics (e.g., van Bergeijk et al., 2019; Afesorgbor et al., 2024; Demena et al., 2024).

Furthermore, in its general form, estimating Equation (4) presents a potential issue of multicollinearity due to the large number of moderator variables listed in Table 5, Table 6 and Table 7. Beyond the risk of multicollinearity, previous studies by Floridi et al. (2023), Afesorgbor et al. (2024), and Demena et al. (2024) have highlighted that including all these moderator variables, especially given that many are binary, can significantly reduce the degrees of freedom in the analysis. To address this issue of model uncertainty in multivariate MRA, we follow the MEAR-Net reporting guidelines (Stanley et al., 2013; Havránek et al., 2020) by applying the general-to-specific (G-to-S) technique. This approach begins with a comprehensive specification of Equation (4), where all potential moderator variables are initially included. Then, variables that are statistically insignificant are removed one at a time, progressively refining the model until only those with significant influence remain in the final specification. This method ensures that the model remains parsimonious while accounting for the most relevant factors, mitigating the risks of overfitting and multicollinearity (Abdullah et al., 2015; Stanley & Doucouliagos, 2012; Mekasha & Tarp, 2013).

Applying this procedure, we observed that almost half of the moderator variables, included in the general MRA, are statistically not significant, and that they are not equally important in describing the potential source of heterogeneity. For trade linkage3, during the G-to-S procedure, we note that 16 of the 26 moderators included in Equation (4) are not statistically significant. The joint insignificance of these variables yields F_{(16, 146)} = 0.61 (p-value = 0.8702), suggesting that they are not only individually but also jointly equal to zero. The joint test of the other 10 included moderators of trade-linkage rejects the null hypothesis of a zero joint effect with F_{(10, 146)} = 2.66 (p-value = 0.0051). The G-to-S procedure is widely applied in recent meta-analyses (see e.g., van Bergeijk et al., 2019; Floridi et al., 2021, 2023).

4.3. Results

Table 8. Multivariate MRA for source of heterogeneity: reduced model.

	Trade-Linkage		Duration		Prior Relations
Moderator Variables	(1) GTS	(2) MEM	(3) GTS	(4) MEM	(5) GTS	(6) MEM
Constant—FAT	0.101	0.354 **	−0.344	−0.344 *	1.370 ***	1.370 ***
	(0.202)	(0.177)	(0.210)	(0.190)	(0.335)	(0.291)
Precision—PET	0.198 ***	0.169 ***	−0.948 ***	−0.948 ***	0.305 ***	0.305 ***
	(0.036)	(0.042)	(0.117)	(0.106)	(0.039)	(0.031)
Data Characteristics
No. obs.					−0.322 ***	−0.322 ***
					(0.067)	(0.058)
HSE/O	−0.070 ***	−0.056 ***	−0.232 ***	−0.232 ***	−0.379 **	−0.379 **
	(0.015)	(0.017)	(0.058)	(0.053)	(0.180)	(0.156)
TIES	−0.260 ***	−0.244 ***	−0.412 ***	−0.412 ***	−2.047 ***	−2.047 ***
	(0.079)	(0.088)	(0.107)	(0.097)	(0.311)	(0.271)
Estimation Characteristics
Probit					2.801 ***	2.801 ***
					(0.743)	(0.647)
Logit			−0.093 **	−0.093 **	2.403 ***	2.403 ***
			(0.039)	(0.035)	(0.710)	(0.619)
Specification Characteristics
Sender cost					0.351 ***	0.351 ***
					(0.111)	(0.097)
Target cost	−0.060 ***	−0.051 ***
	(0.010)	(0.013)
Political instability					−0.432 ***	−0.432 ***
					(0.112)	(0.097)
Cooperation senders	−0.013 *	−0.011			−0.219 ***	−0.219 ***
	(0.008)	(0.009)			(0.056)	(0.049)
Assistant target			0.134 ***	0.134 ***	0.457 ***	0.457 ***
			(0.039)	(0.036)	(0.143)	(0.124)
Militarized dispute	−0.060 ***	−0.052 **	−0.021 ***	−0.021 ***
	(0.018)	(0.021)	(0.005)	(0.005)
Institutional sanction	0.053 ***	0.049 ***
	(0.014)	(0.016)
Trade ratio	−0.110 ***	−0.101 ***
	(0.022)	(0.026)
Trade flows	−0.095 ***	−0.083 ***
	(0.022)	(0.025)
Binary PR					0.712 ***	0.712 ***
					(0.151)	(0.131)
Ordered PR					1.072 ***	1.072 ***
					(0.188)	(0.164)
Publication Characteristics
Peer-reviewed			−0.282 ***	−0.282 ***	−1.103 ***	−1.103 ***
			(0.090)	(0.082)	(0.203)	(0.177)
Journal rank	−0.043 ***	−0.035 ***	−0.431 ***	−0.431 ***	−1.234 ***	−1.234 ***
	(0.009)	(0.011)	(0.054)	(0.028)	(0.172)	(0.150)
Study citations			−0.033 ***	−0.033 ***	0.052 ***	0.052 ***
			(0.004)	(0.004)	(0.012)	(0.011)
Author Characteristics
Co-authored			0.485 ***	0.485 ***	1.873 ***	1.873 ***
			(0.055)	(0.050)	(0.178)	(0.183)
Political scientist			0.436 ***	0.436 ***	1.189 ***	1.189 ***
			(0.053)	(0.048)	(0.178)	(0.155)
US affiliation	0.051 ***	0.041 ***	0.334 ***	0.334 ***	0.836 ***	0.836 ***
	(0.013)	(0.015)	(0.056)	(0.051)	(0.161)	(0.140)
Academic affiliation			1.034 ***	1.034 ***	2.210 ***	2.210 ***
			(0.096)	(0.087)	(0.335)	(0.283)
Constant	0.101	0.354	−0.344	−0.344 *	1.370 ***	1.370 ***
	(0.202)	(0.277)	(0.210)	(0.190)	(0.335)	(0.291)
Observations	174	174	77	77	83	83

Notes: ***, ** and * stand for 1%, 5% and 10% levels of statistical significance. All estimates use the inverse variance as the weights and standard errors reported in parentheses are clustered at the study level. Columns 1, 3 and 5 report the specific model derived from the G-to-S for trade linkage, duration and prior relation, respectively; columns 2, 4, and 6 (MEM) are multilevel mixed effects estimated through the restricted maximum likelihood, addressing the issue of between-study dependence while also controlling for the within-study dependence.

(columns 1, 3, and 5) reports the results of the reduced multivariate MRA using G-to-S modeling. As in Equation (4), this reduced model (column 1 for trade linkage; column 3 for duration and finally column 5 for prior relation) is then re-estimated using the preferred MEM model, as reported in columns 2, 4, and 6, respectively.

4.3.1. Publication Bias and Genuine Effect

The first two sets of results focus on publication bias (FAT) and the underlying genuine effect (PET). Concerning bas, our analysis reveals a statistically significant publication bias, with significance levels at 10% or better, indicating that the effects of the variables of interest are consistently overstated. The magnitude of this bias ranges from 0.344 to 1.370 in absolute terms, suggesting a notable degree of distortion. According to C. Doucouliagos and Stanley (2013), selectivity bias can be categorized as ‘little to modest’ if the bias is insignificant or less than 1, ‘substantial’ if it falls between 1 and 2, and ‘severe’ if it exceeds 2. Based on this framework, the publication bias in our findings varies from modest to substantial, underscoring the need for caution when interpreting the reported effects in this literature.

Regarding the precision (PET), testing our main hypothesis, we found that economic sanctions do make much sense if the sanction-target and sanction-sender do trade (i.e., sanction success needs a lot of pre-trade linkage, reflecting the economic ties that existed between the sanctioning-sender and -target prior to the imposition of sanctions, column 2). Sanctions indeed need to be quick and unexpected to have a maximum impact (i.e., a shorter duration is likely to generate sanction success, capturing the length of time the sanctions are in effect, column 4). Sanctions could work better against friends than against foes (i.e., better prior relation indeed enhances the success of economic sanctions, encompassing the historical, political or economic relationships that existed between the two parties before the sanctions were enacted, column 6).

Next, we investigate the potential sources and determinants of the moderator variables influencing the success or failure of economic sanctions.

4.3.2. Data Characteristics

Focusing on the data characteristics of the included studies, the results in columns 2, 4, and 6 reveal that the number of observations (for prior relations only) and the data source for all variable of interest significantly impact both the direction and magnitude of the reported estimates for sanction success. For prior relations, we observe that a larger sample size, measured by the number of observations, has a negative and statistically significant effect on the success of economic sanctions. This finding suggests that studies with smaller sample sizes may tend to report higher levels of success for economic sanctions. However, this relationship between sample size and reported success does not hold for the other two variables—trade linkage and duration—where no statistically significant effects are found based on the number of observations. This indicates that sample size plays a role in shaping the outcomes for prior relations, but is less relevant for trade linkage and duration in determining sanction success.

Contrary to Stanley and Doucouliagos (2012) claim that different data sources do not significantly influence reported estimates, our findings indicate that the source of data has a notable impact on the estimated effects. Specifically, we observe that when the HSE or the extended HSEO datasets are used, as well as when data are drawn from TIES, as opposed to mixed or alternative sources, the effect of the magnitude of the three determinants on the success of economic sanctions is significantly lower. In fact, when comparing these two primary data sources across the three variables of interest, the results from TIES consistently show a substantially lower magnitude of success for economic sanctions compared to those derived from HSE/HSEO data. This difference is not only consistent, but also statistically significant, highlighting the important role that the choice of data source plays in shaping the reported outcomes of economic sanctions studies.

4.3.3. Estimation Characteristics

Turning to the estimation characteristics, we find that the magnitude of the estimated effect on economic sanction success or failure is sensitive to whether the coefficients are derived from a probit or logit estimation approach as opposed to any other methods (for instance, OLS or censored selection method). Specifically, duration studies that apply the logit estimation approach are likely to report lower economic sanction success. The probit model of sanction success associated with duration analysis did not provide any evidence of success or failure of economic sanctions. In contrast, findings of the prior relations studies that are estimated with the probit or logit technique are equally more likely to report substantially higher economic sanction success. Contrary to this, regardless of the estimation technique employed, all the studies that apply either the probit or logit estimation approaches show no significant impact of trade linkage on economic sanction success or failure.

4.3.4. Specification Characteristics

Our analysis also highlights several other moderator variables that are systematically linked to the success or failure of economic sanctions, though the significance of these relationships varies depending on the specific specification used. When a specification accounts for factors such as the cost of sanctions to the target, the political instability of the target, multilateral cooperation among the senders, and any military disputes between the sender and the target while the sanction was imposed, they tend to show a lower likelihood of sanctions succeeding. Conversely, specifications that include factors like the cost of sanctions to the sender, international assistance provided to the target4, and the institutionalization of sanctions within the sender coalition tend to demonstrate a higher success rate for economic sanctions compared to models that exclude these variables. This suggests that certain economic and political conditions play a critical role in determining the outcomes of sanctions, with some factors enhancing their effectiveness while others reduce their chances of success.

Additionally, the way in which trade linkage is measured significantly influences the estimated success of economic sanctions. When trade linkage is represented by trade flows (i.e., the actual volume of exports and imports between the sender and target countries) or as a trade ratio (the proportion of exports and imports relative to total trade), the results tend to show a lower magnitude of sanctions’ success compared to other measures. However, the impact is notably stronger when trade linkage is estimated using the trade ratio, implying that this method may provide a stronger indicator of the relationship between trade and sanctions’ effectiveness.

Similarly, the way prior relations between the sender and target are quantified also affects the estimates of sanction success. When prior relations are measured on an ordered scale—ranging from antagonistic to neutral or cordial associations—or using simpler dichotomous measures, such as whether the sender and target are classified as friends or allies, the estimates tend to show a larger magnitude of sanction success compared to other measures, such as weighted political affinity scores. Furthermore, the effect size is substantially greater when using the ordered measure of prior relations compared to the simpler binary measure, suggesting that capturing the nuanced degrees of political relationships may offer a more accurate reflection of how prior relations influence sanctions’ outcomes.

4.3.5. Publication Characteristics

Our analysis of publication characteristics reveals that factors such as peer-review, study citations, and journal ranking have significant and negative effects on the reported success of economic sanctions. Specifically, studies focusing on the duration of sanctions and prior relations, which have undergone the peer review process, tend to report a significantly lower magnitude of sanction success. This suggests that peer-reviewed studies may offer more cautious or conservative estimates compared to non-peer-reviewed research. However, when examining economic sanctions in relation to trade linkage, we find no significant effect of peer review on the reported outcomes. This finding may be somewhat disappointing given the rigorous efforts of referees to enhance study quality, but it also implies that the peer review process alone does not directly influence the success or failure of economic sanctions (Demena et al., 2021).

Furthermore, our findings indicate that studies published in high-ranking journals and those that have generated significant citations tend to report a lower magnitude of economic sanctions’ effectiveness (except for prior relations, the opposite holds for citation counts). This observation may align with the research cycle hypothesis, which suggests that as a field matures, newer studies may yield different results compared to seminal works. In fact, this pattern is illustrated in Figure 2, particularly concerning trade linkage and duration. However, we find compelling reasons to reject Goldfarb’s research cycle hypothesis for all the variables of interest in our study. Specifically, our analysis reveals no systematic differences between seminal studies and more recent research. Thus, while high journal rankings and citation counts are associated with lower estimates of sanction effectiveness, the anticipated changes over time as outlined by the research cycle hypothesis do not hold in this context.

4.3.6. Authors Characteristics

Shifting our focus to the characteristics of the authors, we observe that the magnitude of the estimated coefficients is significantly influenced by several moderators within this category. One of the most notable findings is that studies authored by individuals affiliated with institutions in the United States tend to report a more pronounced effect of the three key variables related to economic sanctions. This suggests that the geographical and institutional context of the researchers can play a crucial role in shaping the outcomes of their studies on sanctions. Additionally, we identify three other moderator variables that specifically impact the duration of sanctions and prior relations between the sender and target. For instance, studies that are co-authored—indicating collaborative efforts among researchers—tend to yield larger reported estimates of the effectiveness of economic sanctions. This highlights the potential benefits of teamwork in enhancing the robustness of findings.

Moreover, the academic field of the authors also contributes to variations in reporting standards. Authors with a background in political science, as opposed to those from other disciplines such as economics or sociology, tend to produce higher estimates for the success of economic sanctions. This discrepancy could stem from differing theoretical frameworks and methodological approaches inherent in various academic fields. Finally, the academic affiliation of the authors also plays a significant role; studies conducted by researchers associated with academic institutions are more likely to adhere to rigorous reporting standards, which could result in higher reported outcomes for economic sanctions. Together, these factors underline the importance of considering author characteristics when interpreting the findings on the effectiveness of economic sanctions.

4.4. Further Investigation: Robustness Checks and Implied Effect

4.4.1. Robustness Checks

To gauge the robustness of our findings, we employ two meta-regression models, as follows: (i) the clustered ordinary least squares (CDA) and (ii) the wild bootstrapped standard errors model. We first apply a CDA (

Table 9. Robustness check—Multivariate MRA for source of heterogeneity: reduced model.

	Trade-Linkage		Duration		Prior Relations
Moderator Variables	(1) CDA	(2) Bootstrapped	(3) CDA	(4) Bootstrapped	(5) CDA	(6) Bootstrapped
Constant—FAT	0.101	0.101	−0.344 *	−0.343 *	1.370 ***	1.369 *
	(0.376)	0.766	(0.172)	0.088	(0.610)	0.062
Precision—PET	0.376 ***	0.766 ***	0.172 ***	0.188 ***	0.610 ***	0.062 ***
	(0.035)	0.000	(0.088)	0.002	(0.016)	0.002
Data Characteristics
No. obs.					−0.322 ***	−0.322 ***
					(0.031)	0.002
HSE	−0.070 ***	−0.069 ***	−0.232 ***	−0.232 ***	−0.379 **	−0.379 **
	(0.012)	0.002	(0.031)	0.001	(0.147)	0.018
TIES	−0.260 ***	−0.260 ***	−0.412 ***	−0.412 ***	−2.047 ***	−2.047 ***
	(0.070)	0.002	(0.039)	0.002	(0.183)	0.000
Estimation Characteristics
Probit					2.801 ***	2.801 ***
					(0.547)	0.000
Logit			−0.093 ***	−0.093 **	2.403 ***	2.403 ***
			(0.018)	0.002	(0.500)	0.000
Specification Characteristics
Sender cost					0.351 ***	0.351 ***
					(0.072)	0.000
Target cost	−0.060 ***	−0.059 ***
	(0.010)	0.002
Political instability					−0.432 ***	−0.432 ***
					(0.077)	0.002
Cooperation senders	−0.013 *	−0.131			−0.219 ***	−0.219 ***
	(0.006)	0.200			(0.041)	0.002
Assistant target			0.134 ***	0.134 ***	0.457 ***	0.457 ***
			(0.019)	0.000	(0.111)	0.000
Militarized dispute	−0.060 ***	−0.059 **	−0.021 ***	−0.021 ***
	(0.013)	0.002	(0.001)	0.002
Institutional sanction	0.053 ***	0.053 ***
	(0.014)	0.000
Trade ratio	−0.110 ***	−0.109 ***
	(0.020)	0.002
Trade flows	−0.095 ***	−0.095 ***
	(0.018)	0.002
Binary PR					0.712 ***	0.712 ***
					(0.083)	0.000
Ordered PR					1.072 ***	1.072 ***
					(0.1167)	0.000
Publication Characteristics
Peer-reviewed			−0.282 ***	−0.282 **	−1.103 ***	−1.103 ***
			(0.069)	0.014	(0.129)	0.001
Journal rank	−0.043 ***	−0.043 ***	−0.431 ***	−0.431 ***	−1.234 ***	−1.234 ***
	(0.009)	0.002	(0.028)	0.002	(0.120)	0.002
Study citations			−0.033 ***	−0.033 ***	0.052 ***	0.052 ***
			(0.003)	0.002	(0.008)	0.000
Author Characteristics
Co-authored			0.485 ***	0.485 ***	1.873 ***	1.873 ***
			(0.037)	0.000	(0.134)	0.000
Political scientist			0.436 ***	0.436 ***	1.189 ***	1.189 ***
			(0.033)	0.000	(0.095)	0.000
US affiliation	0.051 ***	0.051 ***	0.334 ***	0.334 ***	0.836 ***	0.836 ***
	(0.010)	0.000	(0.033)	0.000	(0.161)	0.000
Academic affiliation			1.034 ***	1.034 ***	2.210 ***	2.210 ***
			(0.069)	0.000	(0.220)	0.000
No. Obs.	174	174	77	77	83	83

Notes: ***, ** and * stand for 1%, 5% and 10% levels of statistical significance. All estimates use the inverse variance as weights and standard errors reported in parentheses are clustered at the study level. Columns 1, 3 and 5 are CDA (clustered data analysis, also known know as WLS) estimated via the study level clustered robust standard errors; Columns 2, 4, and 6 (wild bootstrapped) concern regression bootstrapping the standard error (a non-standard cluster adjustment) reported with p-values.

, Columns 1, 3 and 5) using study-level clustered analysis, followed by a wild bootstrapped estimation (Columns 2, 4 and 6). We find that the identified effects tend to be comparable and stable in magnitude and statistical significance.5 Next, several studies have shown that when the number of studies/clusters is small, standard adjustment for clustering may potentially produce biased result, and these studies adopted the use of non-standard cluster adjustments, such as the wild bootstrap (Floridi et al., 2023). We also follow this wild bootstrap approach—a non-standard cluster adjustment recommended by Cameron et al. (2008). We continue to observe similar results, with the main exception that the peer-reviewed indicator for duration and the bias for prior relations reduce statistical significance from 1% to 5% and 10%, respectively.6

4.4.2. Implied Effect

In Section 4.3, we discussed the initial genuine effect (PET), controlling for observable drivers of heterogeneity and publication bias. However, there are many potential genuine heterogeneity effects that measure the underlying effect, like the case related to a single PET. Instead of relying on the selection of baseline studies in an attempt to create synthetic studies that would employ a given approach to estimate the determinants of economic sanctions, we follow recent meta-analyses to derive the ‘best practice’ genuine effect from the multivariate MRA conditional on the identified moderator variables used to capture the heterogeneity (Floridi et al., 2023; Afesorgbor et al., 2024). The approach is labeled as best practice as it potentially alleviates omitted variable bias and endogeneity problems on top of accounting for publication selection bias.

We specify the best practice approach conditional on the moderator variables that are most frequently used by the primary studies included in our analysis. In doing so, we include estimates that utilize data from HSE or the extended HSEO, with the logit approach as the chosen estimation method. The specifications account for several key factors, including the cost of sanctions to the target (trade linkage), the cost to the sender (prior relations), political instability, multilateral cooperation among sanction senders, international assistance to the target, and (for trade linkage) pre-sanction trade, where the target’s exports to and imports from senders are examined. Prior relations are categorized on an ordered scale, ranging from antagonistic to neutral to cordial. Additionally, we include two controls for study quality—whether the study was published in peer-reviewed journals and whether it appeared in high-impact or top-ranked journals. Furthermore, we account for two author-related variables—whether the study was co-authored and the authors’ academic affiliations.

The procedure yields the predicted genuine underlying effect conditional on the identified heterogeneity for trade linkage, prior relation and duration as 0.111, 0.695, −0.012, respectively, these values being statistically significant at the 99% confidence level. In testing our main hypothesis, in line with Table 8 and Table 9, the best practice, implied effect reaffirm that economic sanctions are most effective when there is significant pre-sanction trade between the target and the sender, as strong trade linkages increase the likelihood of success. Additionally, sanctions tend to be more effective when imposed on allies or partners rather than adversaries, highlighting that better prior relations enhance their success. Moreover, sanctions need to be swift and unexpected to maximize their impact, with shorter durations showing a higher likelihood of success.

5. Concluding Remarks

Despite nearly four decades of empirical research on economic sanctions, no clear consensus has emerged regarding the sign and magnitude of the key determinants of their success. While early studies suggested agreement on the importance of pre-sanction trade linkage, sanction duration, and prior relations between sender and target, more recent research presents conflicting findings. Our meta-analysis of 37 studies published between 1985 and 2018 highlights this growing divergence, particularly after 2005 (as illustrated in Figure 2). The primary objective of this study is to investigate the sources of this non-convergence. Then, we estimate the implied genuine effects of these three determinants on the success of economic sanctions controlling for observable drivers of heterogeneity and publication bias.

Our analysis identifies several key moderators that significantly influence the success of economic sanctions. These moderators highlight the contextual factors that influence how researchers derive their estimated parameters.

First, in examining the data employed across the three determinants of economic sanctions’ success, we find that the source of data significantly impacts the estimated effects. This contradicts Stanley and Doucouliagos (2012) claim that data source variation does not noticeably affect reported estimates. As noted by Peksen (2019), the empirical literature relies heavily on three main data sources, with large-N studies predominantly using the HSE/HSEO database, the TIES database, and their derivatives. While large-N studies are the dominant approach in the field, the sample remains relatively small, and researchers often reuse existing datasets. Comparing the two principal and competing data sources across the three variables of interest, findings from the TIES database consistently produce a significantly lower magnitude of economic sanction success compared to those derived from the HSE/HSEO dataset.

Second, our findings indicate that the estimated impact of economic sanctions varies by estimation method. Studies using a logit approach for duration analysis tend to report lower sanction success, while probit models show no clear effect. In contrast, prior relations studies using probit or logit methods are more likely to report higher success rates. However, for trade linkage, the estimation technique, whether probit or logit, does not significantly influence reported sanction outcomes. Regarding the specifications of the empirical studies, factors such as sanctions’ cost to the target, political instability, multilateral cooperation, and military disputes reduce the likelihood of success, while cost to the sender, international assistance to the target, and institutionalized sanctions increase it. Additionally, how trade linkage and prior relations are measured significantly affects estimated outcomes in that trade ratios and ordered scales of political relations provide stronger indicators of sanction effectiveness than simpler measures. These findings highlight the importance of accounting for economic and political contexts in assessing sanction success.

Third, peer review is often expected to improve the quality and reliability of the reported results, but it could also be conservative and avoid results that challenge consensus. Specifically, duration and prior relations studies that underwent the peer-review process are likely to significantly reduce the magnitude of sanction success. In contrast, we find no significant impact of peer review when the analysis of economic sanctions is associated with trade linkage. In a sense, this is disappointing, given the efforts of the referees, but on the positive side, it also indicates that it is not the peer review process per se that creates the success or failure of economic sanctions (Demena et al., 2021). Moreover, high-ranked journals and highly cited studies tend to report a lower magnitude of economic sanctions’ effectiveness. While these publication characteristics generally decrease the reported success of sanctions, the authors’ characteristics influence the relationship between sanctions findings and the three key variables in the opposite direction. Specifically, academic field (studies authored by political scientists), academic affiliations (researchers from academic institutions), co-authored papers, and those affiliated with US-based institutions are more likely to produce findings that show a higher success rate for economic sanctions. These factors suggest that differences in academic background, affiliation, and collaboration may affect reporting standards, leading to varying outcomes on sanction effectiveness.

Next, using a best-practice meta-analysis approach to control for heterogeneity and publication bias, we find that trade linkage (0.111) and prior relations (0.695) have a positive and statistically significant effect on sanction success, while duration (−0.012) has a small impact. In support of our prior expectations, strong pre-sanction trade ties increase the likelihood of success, as does imposing sanctions on allies rather than adversaries. Additionally, sanctions are more effective when implemented swiftly and unexpectedly, reinforcing the importance of short durations for maximizing impact.

Finally, we recommend that future research should carefully examine the role and patterns of the identified moderator variables, particularly concerning the specification of empirical models. Moreover, we emphasize that policymakers and researchers relying on empirical findings must consider the relevance of these moderators, especially in how they influence sanctions and the three key determinants we have highlighted. Furthermore, while the genuine impact of the three variables of interest is significant and aligns with our prior expectations, the analysis reveals considerable publication bias, ranging from modest to substantial. This highlights the need for caution when interpreting the effects reported in this body of literature.