Explaining Asylum Law Using Qualitative Comparative Analysis

Abstract

This article demonstrates how Qualitative Comparative Analysis (QCA) can be applied to the study of case law, with an emphasis on the granular analysis of jurisprudence. This article’s empirical focus is a study of asylum decisions issued by the US Circuit Courts. Prior research, using statistical methods, has observed disparities in asylum case outcomes that are partly explained by sociopolitical factors such as the partisan affiliation, gender, and home-state politics of the judiciary. This article uses QCA to revisit these findings; incorporating an analysis of jurisprudential criteria alongside the sociopolitical factors that have been identified by prior studies. All of the Circuit Court decisions for the cases included in the QCA analysis were issued during the first year of the Trump presidency; a time at which asylum-seekers at the US–Mexico border were becoming a focal point both for immigration enforcement and a polarized national debate over immigration policy. Despite the charged political context for these decisions, the QCA findings show that the two most decisive factors for Circuit Court decision-making on these cases were their rulings on nexus and patterns of decision-making that were specific to each court. The closing discussion cautions the reader against generalizing these findings to all appellate-level asylum decisions out of consideration for the epistemological orientation of QCA. Hence, the findings from this study should not be taken as conclusive evidence that sociopolitical factors are of little causal value for research on the appellate courts. Nevertheless, the findings do indicate that more attention should be paid to the explanatory power of jurisprudence. The concluding discussion also highlights the potential that QCA holds for building out a logic-based theory of legal decision making that can account for jurisprudence in tandem with sociopolitical factors and localized cultures of decision-making that help to explain disparate applications of the law.

1. Introduction

Qualitative Comparative Analysis (QCA) is often introduced to readers as a new methodology, but it has been around for close to forty years (Ragin 1987). It also bears noting that the comparative logic of QCA is derived from innovations in Boolean algebra—namely the Quine–McClusky algorithm—which date to the 1950s (Thiem 2014, p. 20).

One of the attractions of QCA is that it allows researchers to examine complex causal relationships among small groups of cases with greater precision than would be possible using available statistical methods, conventional qualitative research, or even mixed-methods approaches. For example, statistics such as Chi Square and Gamma, which are suitable for the analyses of groups of less than thirty cases, cannot do much more than provide a univariate measure of association between paired observations.1 In contrast, the sorting and coding methods used in qualitative field and interview-based research can be used, to great effect, to capture the causal complexities of small groups of cases, but they cannot be used to generate comparable estimates of the causal power of different hypotheses that are applied to the same data set. Mixed-methods research, on the other hand, allows researchers to use qualitative and quantitative data to supplement one another; but the ability of mixed-methods research to advance causal arguments is limited by the unreconciled epistemological suppositions of its diverse data sources (Gobo 2023). These are among the reasons why many researchers have found QCA to be a viable alternative to conventional quantitative and qualitative research methods.

Since the early 1980s, the number of published QCA studies has increased exponentially in tandem with the diversification of QCA’s disciplinary foci (Rihoux et al. 2013). The single largest area of concentration for QCA research is macro-scale political sociology (including welfare state studies, labor studies, and theories of regime change), with management studies being the next largest area of concentration (Rihoux et al. 2013, p. 177).

Rihoux et al. have identified legal studies as a relatively new foci of QCA research that falls within a heterogenous cluster of studies that places third, after management studies. But on reflection, it could be argued that legal studies, broadly defined, are thematically relevant to most QCA studies that are examining matters of government regulation and/or policy change; insofar as the law is implicated in the explanation of these changes (see, for example, Li and Ma 2019; Marx and Soares 2016). Furthermore, when QCA is more explicitly focused on the law, researchers still tend to conceptualize causal conditions in a way that is comparable to the political sociology orientation that is prevalent within QCA research. Some examples include research on legal developments that are explained in light of watershed (before and after) changes in the national legal and/or political landscape (Monteiro et al. 2019); cycles of social movement activity and changing configurations of political power at the Parliamentary level or in the field of international relations (Laux 2015); or the salience that a given assemblage of rights-based arguments holds for currents of public sentiment in the national and international civic sphere (Ingrams 2018; Rittberger and Schimmelfennig 2005).

The research described above builds on the established strengths of QCA research. But, as I aim to show in this article, QCA is also suited to a more granular analysis of case law. In particular, there are issues raised by research on asylum law that highlight the need for this kind of analysis.

Several studies have shown that there is significant variability in asylum decisions that cannot be explained by the merits of a case, case characteristics, or collateral factors like differences in case processing time and case volume (Collins et al. 2010; Gould et al. 2010; Soennecken 2008; Stobb 2018; Yarnold 1994). These disparities have been documented in several national contexts and across all levels of adjudication, though some of the most dramatic disparities have been observed at the level of immigration judge (IJ) decisions in the US (Ramji-Nogales et al. 2007; GAO 2016). Some IJs, for example, grant relief to most of the petitioners who come before them, and other judges deny relief on a similar scale (TRAC 2023).

In the US context, IJ decisions are where disparities in decision-making are most pronounced, and also where the least amount of information is available on the legal reasoning process. There is no publicly available resource that archives the text of IJ decisions, and there is also no federal requirement for immigration judges to issue written decisions.2 In contrast, the appellate courts normally issue written decisions, and although they do not exhibit the same idiosyncratic extremes as IJ decisions, appellate-level decisions still exhibit disparities that have attracted the attention of researchers (Stobb 2018; Yarnold 1994). The Circuit Courts have also seen an increase in asylum cases over the past two decades due to restrictions on the discretionary authority of the Board of Immigration Appeals (BIA) which have increased the load of asylum appeals for the Circuit Courts (Kerwin 2005, p. 4). Hence, the Circuit Courts provide a venue in which both jurisprudential criteria and sociopolitical factors can be considered together and scoped in a way that is attuned to the judiciary that presided over each case.

The next section discusses the epistemological premises of QCA and explains how they can be applied to research on case law, using asylum law as a point of illustration.

2. Applying QCA to Case Law

2.1. The Confluences of Boolean Logic for QCA and Legal Reasoning

Logical propositions are central to the analysis of causal conditions for QCA and also for the application of jurisprudence, which is one reason why QCA holds great potential for the analysis of case law. QCA research and legal scholarship on logical reasoning also share a common heritage in Boolean algebra. As noted above, QCA is heavily indebted to innovations in Boolean algebra. Moreover, as Samuel Damren (1998) has explained, the influence of Boolean algebra on legal reasoning can be traced to the early 20th century (long before the advent of QCA), as part of an intellectual movement that displaced Aristotelian logic to become the new paradigm.3

One of the foundational elements of Boolean algebra is its use of a binary code that registers the presence or absence of a factor. For this reason, Boolean algebra could be regarded as a cousin of logistic regression, which codes on the basis of ordinal distinctions (which can also take the form of a binary). But unlike regression analysis, Boolean algebraic logic does not rely on formulae that are premised on the normal distribution of data.

If statistics is about describing the variable qualities of populations with normally distributed characteristics, Boolean algebra has more to do with mapping logical relationships between a series of propositions (i.e., if “A” is present then “Y” follows, or if “A and B” are present but not “C”, then “X” follows, etc.) This concatenation of relationships is not a variable population characteristic, but a causal logic that is endemic to the conditions it describes. Hence, Boolean logic describes causal pathways that are “always true” (always producing the expected result) so long as all of the specified conditions are met, which is a very different epistemological supposition than that of statistical models, which describe causal outcomes that are variably true for a given population, within a given margin of error.

The point I have just made about the epistemological difference between QCA and conventional statistical analyses also applies to the legal reasoning process. For example, findings from regression analyses are frequently submitted as supporting evidence in court proceedings, to the point where some District Courts have found it necessary to issue guidance on how these analyses should be submitted and how they will be evaluated by the judiciary (IMS Legal Strategies 2014). Even so, the jurisprudence that is actually used to decide a case does not operate in the same epistemological register as a regression analysis.

When it comes to judicial deliberations, cases are best conceptualized as singularities which have the potential to surface arguments that could set precedents for many bodies of jurisprudence. This is why a case cannot be reduced to an assortment of variable demographic qualities concerning the gender, socioeconomic status, or nationality of the petitioner, or even the details of the violations that brought the person before the court. Each case also sits at the intersection of several histories of legal argumentation, and, especially when it comes to appellate-level decisions, the case is decided with an eye for how plausibly and accurately these arguments were applied by lower courts.

To boil things down a little more, the appellate court is not re-deciding the petitioner’s case on its merits. It is determining whether the law has been correctly applied to the petitioner’s case. This standard of review underscores the salience of jurisprudence for legal deliberations, with a nod to the utility of logical reasoning, as a means of reconciling the divergent criteria introduced by different bodies of jurisprudence. As many jurists have argued, there is more to the law than logic alone (see Damren 1998, pp. 64–71), but it is also hard to deny that logical reasoning is, at least, a component of appellate-level deliberations. If this much is admitted, it stands to reason that a research method, like QCA, that relies on Boolean logic, has the potential to capture nuances in the causal conditions that explain judicial decisions that could be missed with a regression analysis; especially if you are examining small numbers of cases.

2.2. Constructing Case Populations

It also bears emphasizing that QCA’s suitability for small N research has to be situated epistemologically. The reader may have noticed that, when discussing case groups for QCA, I have avoided references to “samples” and “sampling”. Rutten and Rubinson (2022) have gone a step further, recommending the use of the term “case population” to describe case groups for QCA.

If you conceptualize your cases as a population, it follows that the goal of the research is not to generate findings that can be generalized to an even larger group of cases. The concept of generalization follows from an epistemological premise about the variable nature of data, which are conceived as incomplete renderings of randomly distributed properties that are endemic to a larger population. Because these data can never provide a completely accurate account of the population from which they have been extracted, you need to sample enough data, and sample it in just the right way, to ensure that your findings are a reasonably accurate representation of this larger population.

When critics fault QCA research findings for being too sensitive to small changes in the composition of case groups, they are appealing to the epistemological premises that were just described (for a summary discussion, see Thiem 2014, pp. 21–23). The expectation is that, if QCA case groups were normally distributed in their composition (and also large enough in size) that QCA research findings would not be sensitive to these small omissions (or additions) of cases.

But if the case group is understood as a population—rather than a sample—then one has to fashion an explanation that accounts for its irregular composition, rather than implement sampling techniques that are designed to smooth out these irregularities. So it could be that, in a group of twenty-five cases, there is a cluster of two or three that exert an outsized influence on the hypothesis that optimally explains outcomes for the entire group. It is also common for QCA findings to manifest two or three causal recipes that are derived from partly overlapping (or in some cases, completely separate) subgroups of cases; with each recipe making a qualitatively distinct contribution to the optimal hypothesis that is fit to the entire case group.

It should be apparent that QCA case groups require a non-stochastic method of explanation. Instead of training the analysis on quanta with variable properties, QCA is attuned to explaining invariant relationships; understanding all data sets to be composed of durable patterns of co-occurring relationships that are latent to the structure of the data.

Given this premise, the selection criteria for case group membership are of paramount importance. The researcher should be able to show that the case group that has been assembled for the analysis is sufficiently comprehensive for the conditions they are planning to explain, which is also to say that the case group should exhibit the qualities of a population. This is one reason why Charles Ragin (2000, pp. 53–57) has emphasized that there must be a theoretical justification for case selection.

It also bears noting that the study of jurisprudence is well suited to small, carefully selected case populations. There are entire histories of federal jurisprudence (spanning the 18th and 20th centuries) that amount to no more than 100 decisions (Kretsedemas 2018). It is also possible to construct smaller groups of five to ten cases, which describe the entire population of District, Circuit, and Supreme Court decisions for a specific complaint (for some examples see Braaten and Braaten 2023, pp. 21–25). There is also a rich history of legal scholarship that revolves around the comparative analysis of a handful of precedent decisions that are widely regarded as watershed moments in U.S. (and international) legal history, which could be accompanied by a QCA analysis; and the list can go on. All of these case groups can be constituted as populations, so long as the selection criteria (and the relevance of these criteria for legal scholarship and social science research) is adequately explained.

Some critics have taken this emphasis on theoretically informed selection criteria to mean that QCA is, necessarily, a deductive method. This premise has been used to raise questions about the validity of QCA research that takes an inductive approach to hypothesis testing (Hug 2013). This is both an insightful and pernicious critique. It puts its finger on an apparent contradiction that is common to many QCA studies, which begin by scoping a theoretically relevant case population and then proceed, through a process of induction, to iteratively test the combination of causal conditions that best explain the case group. The critique cited above inserts a wedge between these deductive and inductive features of QCA, which are, in fact, epistemologically related.

The inductive moment in QCA is much more conducive to Burawoy’s (1988) extended case method than Glaser and Strauss’ (1967) grounded theory. Induction is used to refine a theoretically informed hypothesis which is being used to explain a case population that has also been carefully scoped for its theoretical relevance. Due to the non-stochastic qualities of the case population (which, it bears emphasizing, are epistemologically related to the composition of the case group as a population) the initial hypothesis is often rejected. But if the goal of the analysis is to identify causal conditions that maximally explain the population, then it is necessary to refine and retest the hypothesis, until an optimal explanation is reached.

Thiem has summed up this process as “the identification of minimally necessary disjunctions of minimally sufficient conjunctions of conditions with respect to one or several outcomes” (Thiem 2014, pp. 20–21). Thiem also notes that this summary is his distillation of the logical operations of the Quine–McClusky algorithm that anchors most QCA analyses (taking care to distinguish the QMC from the widely recited limitations of the Millsian inductive method, which is sometimes attributed to QCA).

The key point is that QCA induction is not dictated by the subjective whims of the researcher. Assessments of the relative strength of a given set of causal recipes are always informed by a substrate of Boolean logical operations (built into QCA software) which are used to determine the factors that will be included in subsequent rounds of hypothesis testing.

It also bears noting that theoretically informed decisions are not just salient for the “front end” of QCA (i.e., for the composition of the case population). These decisions are an ongoing feature of the research, as illustrated by the process of data calibration.

2.3. Calibrating Data

In order to render the causal properties of the case population as accurately as possible, it is necessary for the researcher to make decisions about how to code for the presence or absence of causal conditions. In QCA lexicon, the process I have just described is called data calibration.

For example, you may be analyzing a series of asylum case decisions and need to determine whether a petitioner has established that they are likely to suffer persecution if returned to their home nation. You could code all of these cases on the basis of whether the judge determined that the petitioner had established a well-founded fear of future persecution (assigning a value of 0 to a case in which this was not established and a value of 1 in cases where it was established). But you may also encounter cases in which the judge found that the petitioner had established a subjective fear of persecution but had questions about the objective basis of these fears and issued a final decision without clearly stating whether the petitioner had established a well-founded fear of future persecution. Furthermore, the judge could go on to issue a final decision that weighs other factors while declining to reach a conclusion about the likelihood of future persecution.

For a situation like this, you may decide to use a tripartite coding schema: 1 for cases in which the judge determined a well-founded fear of future persecution, 0.5 for cases in which no clear decision was reached on future persecution, and 0 for cases in which the judge clearly ruled that the petitioner had not established a well-founded fear of persecution.

You also may come across cases in which a judge decides that it was “more likely than not” that a petitioner would experience persecution on return to their home nation (which is the highest evidentiary standard for future persecution in asylum law). To account for these cases, you may decide to use a four-part coding schema: 0 for cases in which a well-founded fear was not established, 0.5 for cases in which no clear decision was reached, 0.75 for cases that meet the bar for well-founded fear, and 1 for cases that meet the “more likely than not”, standard.

I have just described a process that began with a conventional (a.k.a. crisp set) approach to Boolean coding, with is limited to the values of 1 and 0, and transitioned to a fuzzy set coding schema that allows for a broader range of values (see Ragin 2000, 2008).

It is important to emphasize, however, that all values for a fuzzy set analysis have to correspond with breakpoints in the data that the researcher has determined are meaningful indicators of the presence or absence of a factor. Perhaps not surprisingly, the apparent vagaries of this process have invited critical attention. Some researchers have questioned the criteria used by QCA researchers to convert raw, continuous data into theoretically relevant, numerical breakpoints that also effectively function like continuous data (Tanner 2014, pp. 292–93). Consider the following example: you are calibrating historical data on unemployment, with the goal of organizing these data into tiered segments for low, moderate, and high unemployment rates (using the values of 0.25, 0.50, and 0.75, respectively). So, how can you reliably determine theoretically relevant breakpoints in your raw data that align with these calibration values, especially since the interval between “high”, “average”, and “low” unemployment in your raw data may amount to no more than a few percentage points? A 4 percent unemployment rate is generally considered “low”, and unemployment rates of 10 percent or higher are generally considered “high”, but it could be more difficult to determine how to code for a “moderate” (or normal) rate. Furthermore, the theoretical salience of what constitutes “high” or “low” unemployment can vary by historical epochs and also by more temporal electoral and economic cycles.

QCA researchers have found ways of answering all of these questions. But it also bears noting that the calibration of jurisprudential criteria is a more straightforward matter because the jurisprudence supplies its own criteria for determining whether a given legal construct is relevant or not to the petitioner’s case. The researcher’s job is simply to determine whether the court found that the criterion was met, or whether, perhaps, the court failed to reach a decision on the matter. All of these coding decisions can be verified through reference to the text of the court opinion. There may also be occasions in which the researcher uses their discretion to simplify the hierarchical organization of a legal construct, so that higher and lower thresholds for determining its application are flattened into a binary (i.e., did the petitioner meet the criterion or not, no matter the threshold?). But unless the researcher has an exceptional and compelling theoretical argument, there should be no reason to add a layer of complexity to the coding of a case that cannot be justified through reference to jurisprudential criteria that appears in the court opinion.

3. A QCA Analysis of Appellate Court Asylum Decisions

3.1. Constructing the Case Population

This section expands on my discussion of QCA and the law with a case study of asylum decisions reached by the US appellate courts. The information I am going to share is excerpted from an analysis I conducted of asylum cases that were appealed to the Circuit Courts in 2017.

The overarching goal of this study was twofold: first to measure the extent to which the asylum decisions of the appellate courts were determined by the kinds of sociopolitical factors that have been identified by other studies (cited in the introduction), and also to gauge the explanatory power of these sociopolitical factors, relative to jurisprudential criteria which have not been factored into the explanatory models of prior studies. In the initial phase of this study, I deliberately constructed a case group that was larger than the one that I intended to select for my case population.

The initial group of cases (N = 319) included petitioners who were seeking asylum on the basis of all possible protected grounds (race/ethnicity, political opinion, religion, etc.). My aim, for this phase of the analysis, was to review all of the asylum cases heard by the appellate courts in 2017, in order to identify cases that would be suitable for a smaller and more carefully scoped case population. Following the guidance of other researchers, I aimed to create a group of ten to fifty cases, which is considered to be of optimal size for QCA (Ide and Mello 2022, p. 9). The selection criteria for this smaller group were determined by a specific protected ground that was being claimed by the petitioners. The next section provides more context on the legal and sociological significance of this protected ground.

3.2. Family-Membership as a Grounds for Seeking Relief from Persecution

The protected ground that delimited my case population is a subcategory of particular group membership. The particular social group is a legal construct that can be traced to the 1951 Refugee Convention. It functions as a broadly defined placeholder for forms of persecution that were not explicitly named in the convention (Lobo 2012). The US Board of Immigration Appeals (BIA) issued its first precedent opinion on particular social group membership with its 1985 decision, Matter of Acosta (BIA 1985). In Matter of Acosta, and in subsequent decisions, the BIA opined that particular social group membership should be defined by immutable traits (that a person either cannot change or should not be reasonably expected to change), as well as traits that describe a socially distinct group membership (BIA 1996, 2007, 2014a, 2014b). From Matter of Acosta onward, the BIA used persecution based on kinship ties as an archetypal example of traits that meet its criteria for membership in a particular social group.

In more recent years, the legal and sociopolitical visibility of this kinship-based interpretation of particular social group membership has been elevated due to the unrelenting escalation of global displacement and the politics of the US–Mexico border. As some refugee scholars have noted, the displacements that have defined the refugee experience from the late 20th century do not conform to the definition of persecution that were established in the early post-War era (Betts 2010; Jenkins and Schmeidl 1995). Instead of being persecuted by their home governments, these people are more likely to suffer from a vacuum of state power which is exploited by a host of para-state and non-state predatory actors. These desperate conditions have contributed to the growing numbers of asylum-seekers at the US–Mexico border.

Kinship-based definitions of particular social group membership are one of the few protected grounds that are salient for many of these people, because it allows for forms of persecution that are carried out by non-state actors. Kinship is often cited as a pretext for persecution resulting in domestic violence (in the context of societies with high rates of femicide and mainstream institutions that either tacitly endorse or tolerate gender violence) or as a pretext for being targeted by gangs (in the context of societies in which criminal cartels rival the power of the government). These are among the reasons why kinship-based definitions of particular social group membership are playing an important role in keeping asylum jurisprudence up to date with the reasons why people seek refuge today (Hill 2012; Musalo 2010); but for this same reason, they have become a site of legal and political contestation.

The persecutory conditions that were described above are especially acute in Central American nations. In the late 2010s, Central American migrants and asylum-seekers were also becoming more visible, for federal enforcement and in the public eye, as symbols of a new “immigration problem”. In 2019, barely one year after I completed this study, the Trump administration rolled out a Zero Tolerance Border policy that, among other things, restricted asylum-seeking along the US–Mexico border and was also accompanied by a legal opinion, issued by the Attorney General (AG 2018), which eviscerated asylum jurisprudence for victims of non-state persecutors, with special reference to victims of gang and domestic violence. This jurisprudence was restored to its pre-2018 status by a new Attorney General (AG 2019, 2021) after the transition to the Biden administration. Nevertheless, this sequence of decisions demonstrated that kinship-based definitions of particular social group membership had become one of the hot spots of the national immigration debate, which was being closely watched by Executive appointees of the Republican and Democratic parties.

All of these conditions underscore why kinship-based definitions of particular group membership were theoretically relevant to the goals of my study. They described a protected ground that was being used to legitimize experiences of persecution that had become a distinguishing feature of the contemporary refugee experience and which also resonated with the polarized political currents of the national immigration debate. These were optimal qualities for a study that sought to explain the relative causal power that sociopolitical factors and jurisprudential criteria exerted on appellate level asylum case decisions.

3.3. Describing the Case Population

Of the initial group of 319 appellate-level asylum cases that I analyzed, 35 involved a kinship-based definition of particular group membership. This group of 35 cases became my case population (the focus of the QCA analysis). It also bears emphasizing that, on the basis of my analysis, this group makes up all asylum cases involving a kinship-based protected ground that were heard by the Circuit Courts in the first year of the Trump presidency.

On initial review of the Circuit Court decisions, it seemed petitioners in this case population were more likely to receive a favorable ruling than the typical asylum-seeker. I was able to corroborate this observation by comparing outcomes for cases in the initial group (N = 319) to outcomes for my case population (N = 35). (see

Table 1. Remand rates for kinship cases compared to all asylum cases.

Circuit Court	Remand Rate for All Asylum Cases (N = 319)	Remand Rate for Kinship Cases (N = 35)	Kinship Cases Remanded	Kinship Cases Denied	Democratic Appointees
2nd	29%	100%	5	0	64%
4th	17%	100%	2	0	59%
1st	12.5%	100%	1	0	50%
3rd	32%	100%	1	0	43%
9th	18%	50%	2	2	65%
6th	8%	33%	2	6	35%
7th	22%	n/a *	0	0	25%
10th	0%	0%	0	1	53%
8th	0%	0%	0	1	12.5%
5th	4%	0%	0	6	41%
11th	6%	0%	0	6	50%

* The remand rate for family cases for the 7th circuit is listed as “n/a” because this court did not hear any cases that met the criteria used to select the final group of family cases.

).

Before proceeding further, I should note that, even if it is sympathetic to the petitioner’s argument, it is rare for a Circuit Court to grant relief. In these cases, a favorable case outcome typically involves the Circuit Court remanding a decision to the BIA for further review. On the other hand, if the Circuit Court does not remand, it is effectively letting the BIA decision stand, which usually means the petitioner will be issued an order of removal.

Table 1 shows that, across most of the Circuit Courts, the remand rate for cases with a kinship-based protected ground was higher than the remand rate for all cases. If there was a difference between these two remand rates, the “kinship case” remand rate was usually the higher of the two (the only exceptions being the 5th and 11th Circuit Courts).

Kinship cases also seemed to have a polarizing effect on Circuit Court decisions. Courts that were inclined to issue few, or no, favorable decisions did not find kinship cases any more meritorious than any other asylum case (the only exception being the 6th Circuit; see Table 1). But courts that were already more inclined to issue favorable decisions for all asylum cases, were even more likely to issue favorable decisions for kinship cases (see the top five courts in Table 1). For these courts, the remand rate for kinship cases ranged between 2.5- to 8-times their remand rate for the larger group of cases.

Table 1 also includes a description (reported as a percentage) of the partisan affiliation of the appointing President for the judiciary of each court. Courts with a majority of Democratic appointees were generally more inclined to grant favorable decisions to asylum seekers; but there were exceptions. The 11th circuit, for example, rarely issued remand decisions, and it was evenly split between Democratic and Republican appointees, and the 3rd Circuit, which issued remand decisions for all the family cases that it heard, was mostly composed of Republican appointees. There was also the 10th Circuit, which had a majority of Democratic appointees but declined to issue remand decisions for any of the asylum cases that it heard.

3.4. Explaining Decisions on Cases with a Kinship-Based Protected Ground

I began the analysis of the 35 cases with a kinship-based protected ground by compiling a list of factors, derived from my reading of the court opinion on each case and my knowledge of the relevant legal studies literature. I grouped these factors into several categories. See

Table 2. Explanatory factors.

Sociopolitical Factors
Partisan Composition of the Judiciary (PartisanCourt)	Proportion of Democratic appointees in each court.
Partisan Composition of Case Panel (PartisanCase)	Proportion of Democratic appointees in the panel of judges that heard each case.
Gender Composition of the Judiciary (Gender Court)	Proportion of female judges in each court.
Gender of Case Panel (GenderPanel)	Proportion of females in the panel of judges that heard each case.
Immigrant Constituency (Immigration)	Size of the foreign-born population in states in which each judge from the case panel resides.
State Electoral Politics (StatePolitics)	Partisan voting patterns in the state in which each judge from the case panel resides.
Patterns of Decision Making
Court Specific Patterns of Decision-Making (CourtPattern)	Proportion of cases remanded by each court in the large case group (N = 319).
Case Attributes
Domestic Violence (Domestic)	Did the case involve a claim of persecution involving domestic violence?
Gang Violence (Gang)	Did the case involve a claim of persecution involving gang violence?
Jurisprudential Factors
Credibility	Was the testimony of the petitioner deemed credible by the court?
Nexus	Did the court decide there was a plausible relationship between the persecution suffered and the family-based protected ground?
Future Persecution	Did the court determine that the petitioner will likely suffer future persecution if forced to return?
Past Persecution	Did the court determine that the petitioner had a plausible claim of past persecution?
Jurisprudential Criteria Specific to Kinship-Based Persecution
Kinship PSG Legally Cognizable (PSGCognizable)	Did the court issue statements in defense of the legal cognizability of the petitioner’s kinship-based PSG?
Immediate Family (Immediate)	Did the definition of particular group membership involve immediate family relations?
Nuclear Family (Nuclear)	Did the definition of the particular group membership involve nuclear family relations?
Attenuation of Kinship Ties (Attenuate)	Degree to which particular group membership involved extended kinship/family members.

(below) for a description of each factor.

Table 2 includes factors that have been used to explain disparities in appellate court or IJ decisions by other researchers (see sociopolitical factors), factors that are well-established features of asylum jurisprudence (see jurisprudential factors), factors derived from the attributes of the cases I was examining (see case attributes) and factors that are specific the jurisprudence on kinship ties as a type of particular group membership or are features of established asylum jurisprudence that have been applied to the jurisprudence on kinship and particular group membership (see final group of factors in Table 2).

There are also some factors on this list that are unique constructs. In the group of sociopolitical factors, for example, I introduced a distinction between the panel composition of the judicial panel and composition of the entire judiciary from each court, which I have not seen used in prior studies.

I also introduced a factor, “Court Specific Patterns of Decision-Making” (CourtPattern), which fell into a category all its own. This factor provided an empirical description of each court’s pattern of decision making for the large group of cases (N = 319). I added this factor to the analysis to see if patterns of decision making for the larger group of cases helped to explain decision-making for the group of 35 kinship cases. I constructed this factor by separating the courts into two groups: one group that was most likely to grant a favorable decision to asylum seekers and another group of courts that was least likely to grant a favorable decision.4 In the remainder of this article, I refer to these court-groups as those with a more liberal and a more conservative pattern of decision making, but it is important to keep in mind that these labels are derived from the decision-making pattern itself, which is not always aligned with the partisan leanings of the court (as noted above, in the discussion of Table 1).

Following the guidance I offered at the end of the prior section, I coded all of these factors using a schema of minimal complexity, which relied on a minimal degree of interpretation on the part of the researcher to determine breakpoints in the raw data. I used a three-tiered coding schema, which is the most rudimentary form of fuzzy-set coding (see Ragin 2008, p. 31). For jurisprudential criteria, I coded on the basis of whether a given criterion had been explicitly rejected or affirmed by the court in the text of its opinion (assigning a value of 0 or 1, respectively). My coding schema included one intermediary value (0.5) which was reserved for instances in which the court either did not address the criterion in its decision or may have briefly discussed the criterion but did not issue a conclusive decision. I also assigned a value of 0.5 for factors based on quantitative estimates of the partisan or gender composition of the judiciary in which the gender or partisan balance was exactly 50 percent. But there were also a number of binary factors for which a value of 0.5 did not apply (for example, whether or not a claim involved immediate family relations or whether the petitioner was claiming persecution due to domestic violence).

After the data were calibrated, I ran an analysis for necessary conditions, to identify factors that should be included in my starting hypothesis. Tests for necessity are used to assess the causal power of each factor. These tests, like all Boolean measures, assess set–subset relationships between factors

In this study, for example, set Y (the QCA equivalent of the dependent variable or outcome) can be defined as all cases that received a remand decision, and set X (the casual conditions, which are the QCA equivalent of independent variables) would be any of the factors listed in Table 2. The necessity test assesses the extent to which the outcome is a subset of a causal condition; e.g., are all family cases with remand decisions a subset of all cases in which past persecution is present, as a factor, in the judicial panel’s ruling?

It bears noting that the Boolean reliance on set–subset relationships is one of its main points of distinction from the causal logic of statistical reasoning, which relies on the normal distribution of data (and, consequently, would require the inverse of a causal relationship to also be true, so that X→Y cannot be true without ~X→~Y also being true). For a QCA analysis, on the other hand, if Y is shown to be a subset of X, it does not automatically follow that “not Y” is a subset of “not X” (i.e., that in cases where Y is not present, X is also not present). The “positive” set–subset relationship and the inverse of this relationship are treated as two qualitatively distinct causal relationships that are not contingent on each other.

The value that is calculated for tests for necessity is a consistency score, which ranges from 0 to 1. The closer to 1, the greater the strength of the set–subset relationship between the factors in question. Charles Ragin has advised that an acceptable consistency score should be no lower than 0.75, noting that “When observed consistency scores are below 0.75, maintaining on substantive grounds, that a set relation exists … becomes increasingly difficult” (Ragin 2008, p. 46). Many QCA researchers have taken this to mean that 0.75 can be used as a minimally acceptable threshold for determining whether a factor can be deemed necessary for explaining an outcome (Emmenegger et al. 2014; Ingrams 2018; Sendra-Pons et al. 2022; Thiem 2017). But other highly respected researchers (Schneider 2018) have advised a minimally acceptable threshold of 0.9, which aligns with Ragin’s guidance that “consistency scores should be as close to 1.0 … as possible” (Ragin 2008, p. 46). But it is also important to note that both standards (0.75 and 0.9) are still being used by QCA researchers, and there is also published work that provides a rationale for the value of hypothesis testing even when observed necessity scores fall below 0.75 (Li and Ma 2019, pp. 613–15).5

Being mindful of what I have shared above, I selected factors for my initial hypothesis on the basis of observed consistency scores that exceeded 0.75 and also on the basis of their conceptual relevance for the goals of my analysis (see below).6

Case Outcome = State Politics + Court Pattern of Decision Making + Credibility + Nexus + Future Persecution + Past Persecution

After determining the composition of my initial hypothesis, I proceeded to conduct a test for sufficiency. Factors that may be necessary to explain an outcome are not usually sufficient in and of themselves to provide the best explanation. Whereas the test for necessary conditions is used to vet the individual factors that will be included in a hypothesis, the test for sufficient conditions is used to assess the explanatory power of the hypothesis itself, with the goal of identifying the combination of factors that provide the most viable explanation (but as I will explain shortly, the analysis of sufficient conditions can involve combinations composed of a single factor).

Tests for sufficiency are also distinguished by their measures. The test for necessity relies only on consistency scores, which are indices of explanatory strength. The analysis of sufficiency, in contrast, relies on both consistency scores (for explanatory strength) and coverage scores (for explanatory scope). For example, a hypothesis may weakly explain a larger number of cases (low consistency and high coverage) or provide a very strong explanation of a small number of cases (high consistency and low coverage). The goal is to arrive at a solution that exhibits high consistency and high coverage.

Table 3. Causal recipes generated by the initial hypothesis.

Hypothesis: Case Outcome = State Politics * CourtPattern * Credibility * Nexus * FuturePersecution * PastPersecution
	Raw Coverage	Unique Coverage	Consistency
Recipe 1
Nexus * Credibility * FuturePersecution *	0.438462	0.438462	0.857142
PastPersecution * CourtPattern * StatePolitics
Recipe 2
Nexus * Credibility * FuturePersecution *	0.0730769	0.073077	0.826087
~PastPersecution * CourtPattern * StatePolitics
Solution Coverage	0.511539
Solution Consistency	0.858064

presents the results for the hypothesis that I introduced above which were generated using the software fsQCA. The solution coverage score provided at the bottom of the table can be read as a proportionate estimate of the number of cases explained by the entire hypothesis (0.511539 can be translated to mean that outcomes for approximately 51 percent of cases in the group of family cases were explained). The solution consistency score (0.858064) assesses the strength of the explanation (with 1 being the upper limit of this range).

The casual recipes7 presented in the top half of the table are closely related subsets of factors that contributed to the overall explanation. I allowed fsQCA to test for combinations of both present and absent factors; so that absent factors could be included in these recipes if they contributed to the explanation of case outcomes (see ~Past Persecution in recipe 2).

Two types of coverage scores are displayed for each recipe. The raw scores assess the total proportion of cases explained by the recipe (allowing for overlap with the explanatory scope of other recipes) whereas unique coverage scores assess the proportion of cases uniquely explained by each recipe. The fact that the raw and unique scores for both recipes were the same indicates that there was no overlap in their explanatory scope. So, each recipe explained outcomes for an entirely different cluster of cases within the group of family cases.

Next, I treated each recipe as a separate hypothesis and re-ran the analysis. The results of this second stage revealed several smaller sets of causal recipes that had greater explanatory power than my first hypothesis. I separately tested all combinations of this more powerful and more granular group of factors, treating each combination as a separate hypothesis. The results of this analysis are presented in

Table 4. Roster of the strongest causal recipes.

Hypothesis	Remand Decisions Explained (Remand Coverage)	Solution Coverage (for All Case Outcomes)	Solution Consistency
Nexus * Credibility * CourtPattern	0.69	0.742308	0.897674
Nexus * Past Persecution * CourtPattern	0.69	0.684615	0.890000
Nexus * Credibility	0.77	0.811538	0.871901
Nexus * Past Persecution	0.77	0.823077	0.795539
Nexus * CourtPattern	0.77	0.742308	0.897674
Nexus	0.85	0.880769	0.773648
Court Pattern	0.77	0.768924	0.742308

below.

These hypotheses were so granular that there were no subgroups of causal recipes that could be distinguished from the larger hypothesis. The last two solutions in the table, for example, are composed of just one factor, CourtPattern and Nexus. Of these two lone-factor solutions, Nexus was the most robust. It explained approximately 85 percent (solution coverage of 0.85) of all remand decisions, as well as explaining case outcomes for about 88 percent of all cases (solution coverage of 0.880769) So, the explanatory scope of Nexus was greater than any other hypothesis (or causal recipe) listed in Table 4, but its explanatory power (exhibited by its consistency score of 0.773648) was weaker than all of the other hypotheses (except for CourtPattern which had a consistency score of 0.742308.

But when these two factors were combined into the same solution (Nexus * CourtPattern) they provided the most well-rounded explanation of all the hypotheses that I tested. The explanatory scope of Nexus * Court Pattern (its solution coverage) was not as great as several other hypotheses, but its explanatory power was unmatched by any other hypothesis.

According to this solution, appellate court decisions for kinship cases could be explained by each court’s assessment of whether there was a plausible relationship between persecution suffered and the definition of the protected ground (Nexus: also described in Table 2) and the court’s pattern of decision-making for the large group of asylum cases (CourtPattern).The only solution of equal power to Nexus * CourtPattern was Nexus * Credibility * CourtPattern which had exactly the same coverage and consistency scores as Nexus * Court Pattern but slightly weaker coverage for remand decisions (0.69 compared to 0.77).

The results for these two hypotheses showed that Credibility (of the petitioner) did not add any explanatory power to Nexus and CourtPattern. The combination of CourtPattern and Nexus also had greater explanatory power than Nexus on its own. Meanwhile, CourtPattern on its own had weaker coverage and consistency scores than all of the hypotheses listed in Table 4, which is why it does not appear on the table.

To sum up, Nexus * CourtPattern was the most viable causal solution due to its explanatory power (solution consistency) and also because its explanatory scope (solution coverage) outperformed the only other solution of comparable explanatory power (Nexus * Credibility * Court Pattern).

4. Discussion

The main goal of this article was to showcase an application of QCA for legal studies that takes a more granular approach to the scoping of causal conditions than previously published research (Laux 2015; Ingrams 2018; Monteiro et al. 2019; Rittberger and Schimmelfennig 2005). Instead of looking at the relationship between court decisions and changes in the broader political landscape, I explained case outcomes using jurisprudential criteria and sociopolitical factors that were all scoped at the case-level.

My QCA analysis produced a final hypothesis composed of two causal conditions (Nexus * CourtPattern) that explained most of the decisions in my case population and at considerable explanatory power (with coverage and consistency scores of 0.742308 and 0.897674, respectively). This hypothesis indicates that decisions about whether there was a viable nexus between persecution suffered and the protected group, combined with patterns of decision-making for asylum cases that were more or less specific to each appellate court, explained most of the outcomes for asylum cases involving kinship-based definitions of particular group membership. These findings are of potentially great significance for research on asylum jurisprudence, and they may also provide a glimpse into the explanatory potential that QCA holds for the analysis of other types of jurisprudence.

One of the main accomplishments of the analysis is the identification of Nexus as the most causally decisive jurisprudential criterion. Of the eight jurisprudential criteria that were initially considered (see Table 2), Nexus is the only one that made it into the final hypothesis (in addition to outperforming all of the sociopolitical factors). Moreover, of all the causal conditions included in the initial hypothesis, Nexus had the highest consistency score (of 0.8807, see footnote 6).

This finding lends support to legal scholarship that has identified nexus decisions as a pivotal feature of asylum jurisprudence (Gupta 2015; Musalo 2002). This scholarship presumes that it is possible to refine the categorical logic of nexus decisions in a way that can be uniformly applied by all judges. This premise is complicated, however, by the research I cited in the introduction to this article that has called attention to the role that partisanship and other sociopolitical factors can play in appellate-level asylum decisions (Collins et al. 2010; Yarnold 1994). But it bears noting that none of these studies have examined the explanatory power of sociopolitical factors relative to that of jurisprudence, as I have realized here.

I also want to emphasize that my analysis did not prove that politics is irrelevant so much as it put the influence of politics in context. On one hand, none of the sociopolitical factors from Table 2 made their way into the solutions that are listed in Table 4. But when I shifted the focus of the analysis, to explain jurisprudential criteria, some of these factors became more salient. When I ran necessity tests that assessed the set–subset relationship between my sociopolitical factors and jurisprudential criteria, I found that the partisan composition of the judiciary and the home-state politics of the judiciary helped to explain their application of jurisprudential criteria,8 despite the fact that the causal power of these political factors was not strong enough to directly explain the final decision.

So, measures of partisan politics exhibited some explanatory power, contingent on how the goals of the analysis were framed. But it also bears emphasizing that—at least as it concerns this study—political factors did not exert a causal power, of any consequence, that was independent of jurisprudence. Instead, they became more causally salient when they were used to explain jurisprudence. Meanwhile, the presence of Nexus in the final hypothesis indicates that nexus decisions had an effect on case outcomes, in excess of any influence that political factors may also have had on these decisions. This is why the findings of this study lend support to legal scholarship that is focused on refining the categorical logic of nexus. The findings show that jurisprudence has explanatory power in its own right.

There are a number of questions that follow from these findings. For example: is the pivotal role that I observed for nexus decisions unique to cases involving kinship-based definitions of particular group membership? Could the salience of nexus decisions apply broadly to all (or most) asylum cases? And does this finding only hold for appellate-level decisions, or could the analysis of nexus help to explain IJ decisions?

Before heading further down this path, please keep in mind that QCA research findings are not conducive to conventional notions about generalizability. As I explained previously, this is why the concept of the case population is a better fit for QCA than that of the sample (see Rutten and Rubinson 2022). Each case population is more or less unique in its composition, and the causal solutions generated via QCA are designed to optimally explain the singular qualities of these populations. Consequently, you should not assume that you can generalize the explanatory power of a specific causal factor like Nexus (i.e., its consistency and coverage scores) to other case populations. Instead of taking this approach, it is more instructive to treat QCA as a method for identifying causal conditions (including combinations of these conditions) that could be useful for explaining other case populations. The explanatory power of Nexus * CourtPattern is one example.

Another example is provided by the causal condition StatePolitics, which was included in my initial hypothesis. StatePolitics is a measure derived from the partisan leanings of the majority of voters in each judge’s home state. This factor did not end up in my strongest causal solutions (see Table 4), but it was the only sociopolitical factor, from Table 2, which was included in my initial hypothesis (see note 5). It so happens that Braaten and Braaten included a county-level measure of electoral politics (based on percentage of Democratic voters) into a logistical regression analysis of over 12,000 IJ-level asylum cases. This county-level measure outperformed all other independent variables, with an odds ratio that was up to 100-times greater than all other political variables (including whether a case was decided in the Trump era, the IJ’s partisan affiliation, and the IJ’s ideology (Braaten and Braaten 2023, pp. 69–70)).9

Of course, there are many differences between both studies. For example, because they were analyzing IJ-level decisions, it was not possible for Braaten and Braaten to integrate a granular analysis of jurisprudence into their model. Nevertheless, both studies turned up a similar finding, which is that local estimates of partisan electoral leanings provide a better indicator of the influence of politics on asylum decisions than other measures of politics (like partisan affiliation of the judge, partisan affiliation of the appointing President, or the partisan politics of the sitting administration). But my main point is that a QCA analysis of only 35 cases was able to register the salience of a causal condition that was also identified by a much-larger-scale statistical analysis. This finding indicates that QCA analyses that focus on small, carefully scoped case populations can be a very efficient means of excavating causal conditions that could be explored further by larger-scale studies, whether this involves an integration with conventional statistical analyses (Meuer and Rupietta 2017) or large N QCA research (Rutten 2022).

But it is also important not to lose sight of the salience of small N research, which has always been near and dear to the epistemological foundations of QCA. When you translate this small N worldview for legal studies research it means, among other things, that we should resist the temptation to try and explain “everything all at once”.

Instead of looking for universal laws of legal reasoning, it could be more instructive to conceptualize the law as being composed of a multitude of overlapping but discrete causal fields. Although there may be rules of logic and jurisprudential criteria that are common to all of these fields, the reasoning process that explains how they are interpreted and applied could be more or less field specific. So, it could be that these rules and criteria are calibrated in ways that are specific to the contexts in which they are applied, which may include the deliberative norms of a given court, the composition of the judiciary deciding a case, or the novelty and legal significance of a particular case.

The explanatory power of CourtPattern—the other half of the Nexus * CourtPattern solution—is an apt example. The salience of CourtPattern suggests that there is a way of interpreting and applying jurisprudence that is more or less specific to each Circuit Court.

There is a quality to CourtPattern that is reminiscent of the theory of law famously advanced by Oliver Wendell Holmes in The Common Law (Holmes [1881] 1991). It evokes a terrain, similar to the one described by Holmes, in which localized traditions, histories of governance, and bodies of opinion coalesce into more or less distinct cultures of law making. There are also directions in contemporary social theory—epitomized by relational theory (Emirbayer 1997; Tilly 2000)—that provide a comparable, but more abstractly rendered, depiction of a social world composed of autopoietic networks that are each a world unto themselves. A QCA approach to case law is uniquely suited to explaining how decisions get made in this kind of milieu. But there is more to this story than meets the eye.

In his historiography of Boolean logic and legal reasoning, Samuel Damren notes (1992, pp. 63–64) that Holmes was one of the leaders of the intellectual movement that was challenging the stranglehold of Aristotelian logic on 19th century legal theory. Damren also points out that George Boole was trying to do the same, but the significance of Boolean algebra, as an alternative to the Aristotelian syllogism, was not recognized by the legal community until much later in the 20th century. Consequently, the legal debates of the early 20th century were often mired in a distorted binary of logic versus custom, or logic versus experience, or logic versus morality (i.e., logic versus something else that was “not logic”). Meanwhile, Boole’s singular contribution, which was to show that Aristotelian logic could be replaced by a better logic, was shunted to the periphery of these debates.

As one of the standard bearers of Boolean logic for contemporary social science research, QCA is in a position to revisit and reframe these debates about the role of logic in legal theory. A granular analysis of case law, as proposed by this article, is well suited to this task. It should also be apparent that QCA can do much more than provide a counterargument to Holme’s vision of the law. As I have just noted, QCA is uniquely suited to exploring and explaining the variegated legal terrain described by Holmes’ theory. But instead of railing against logic, QCA can render more clearly the trajectories of legal reasoning that explain how this terrain has been constituted, which is another reason why QCA is a very good fit for the analysis of jurisprudence.