Explainable Machine Learning for Fallout Prediction in the Mortgage Pipeline

Abstract

This study examines mortgage loan fallout using data provided by a leading financial institution. By accurately predicting mortgage loan fallout, lenders can protect their bottom line, maintain financial stability, and contribute to a healthier economy. The paper employs various machine learning models to predict mortgage fallout based on loan, market, property, and borrower characteristics. A large dataset of locked mortgage applications from a major U.S. lender was analyzed. The random forest model demonstrated superior predictive efficiency and stability. To understand the factors influencing mortgage fallout, the SHAP method, along with empirical analysis with logistic regression, was utilized to identify key determinants. The paper discusses the implications of these findings for mortgage lenders and future research.

1. Introduction

One of the main challenges that mortgage lenders face in their daily operations is how to effectively manage their pipeline of loans that have been locked in by borrowers at a certain interest rate but have not yet closed and funded. This pipeline represents the lender’s expected future revenue, but it is also subject to a high degree of uncertainty due to the possibility of fallout, which occurs when a locked loan fails to close and fund for various reasons. Fallout can be caused by factors that are related to the borrower, such as withdrawal, credit denial, or inability to provide sufficient documentation, or factors that are related to the market, such as interest rate changes, appraisal issues, or regulatory changes (Hakim et al. 1999). Accurately predicting the fallout of a mortgage application is crucial for optimal hedging and capital allocation decisions, as it affects the lender’s profitability and liquidity.

Mortgage fallout, which refers to the failure of mortgage applications to close and fund, has significant implications for lenders and the broader financial system. Accurate prediction of fallout helps to mitigate lender losses and optimize pricing decisions. Lenders can better manage their risk exposure by anticipating potential cancellations (Morckel 2015). This is particularly crucial in volatile market conditions or economic downturns like we currently face due to higher inflation rates, geopolitical tensions, interest rate hikes, and elevated debt levels. Understanding fallout rates allows lenders to adjust their pricing strategies, ensuring they remain competitive while accounting for potential losses (Weiner et al. 2010). Moreover, predicting fallout enables lenders to allocate resources and improve mortgage pipeline management. By anticipating which loans are more likely to fallout, lenders can allocate their staff and resources more efficiently (Agarwal et al. 2020). Accurate forecasts help in managing the loan pipeline, ensuring smoother operations and better cash flow management (Weiner et al. 2010). On a broader scale, fallout prediction can also contribute to overall market stability. By helping lenders avoid overexposure to risky loans, fallout prediction can contribute to preventing large-scale financial crises, such as the 2008 global financial crisis (Chen et al. 2016). Additionally, regulators and policymakers can use fallout predictions to make informed decisions about mortgage market regulations and interventions. By improving risk management, operational efficiency, and market stability, accurate prediction of mortgage fallout plays a crucial role in maintaining a healthy and sustainable mortgage industry. In summary, mortgage loan fallout can lead to significant financial damage for individuals, financial institutions, and the broader economy. This can also result in losses for financial institutions, which can weaken their stability and reduce lending. A widespread increase in mortgage loan fallout can indicate an economic downturn, as reduced credit availability, falling home prices, and declining consumer spending can slow growth and lead to job losses. In extreme cases, a wave of fallouts can pose a systemic risk to the financial system, similar to what happened during the Great Recession.

In this paper, we propose various machine learning models for predicting the fallout of a mortgage application based on a set of explanatory variables that capture the characteristics of the loan, property characteristics, market conditions, and borrower characteristics. These machine learning models employ different statistical techniques for modeling binary outcomes of fallout or non-fallout of a mortgage application by estimating the probability of an event occurring as a function of one or more explanatory variables. For example, unlike linear regression, logistic regression does not assume a linear relationship between the response variable of binary fallout and the predictor variables and can handle nonlinear effects and interactions among the predictors (Sperandei 2014). Moreover, logistic regression provides interpretable coefficients that measure the impact of each predictor variable on the log odds of the outcome variable.

We applied the logistic regression model along with other popular machine learning models to a large dataset of mortgage portfolios recorded by one of the national mortgage lenders in the United States. The dataset contained information on various loan, borrower, and property attributes, such as loan amount, loan-to-value (LTV) ratio, credit score, debt ratio, property occupation, loan type, and lock duration, as well as market variables, such as interest rate movements. We also used information on the loan stage, such as application date, lock date, approval date, and closing date, to estimate the fallout probability. We used various performance metrics, such as accuracy, precision, recall, area under the receiver operating characteristic (ROC) curve, and F1 score, to evaluate the predictive power of our model. We found that our random forest model performs best in predicting the fallout of a mortgage application in comparison to other machine learning models, providing valuable insights for lenders and researchers.

The rest of the paper is organized as follows: Section 2 provides a literature review of previous studies that have addressed the problem of mortgage fallout prediction using different methods and data sources. Section 3 describes the data and the variables used in our empirical analysis. Section 4 presents the methodology and the results of our machine learning models. Section 5 discusses the performance metrics with Section 6 describing the model results. Section 7 includes methodology and results of an empirical analysis using logit model. Section 8 has discussion related to implications and limitations of our findings for mortgage lenders and researchers. Section 9 concludes the paper and offers directions for future research.

2. Literature Review

The problem of mortgage fallout prediction has been addressed by previous studies using different methods and data sources. In this section, we review some of the relevant literature and highlight the contributions and limitations of each study.

To accurately predict fallout rates in the mortgage pipeline, it is crucial to consider various factors that influence mortgage behavior and risks. One key aspect is the prediction of prompt refinancing behavior, which is closely linked to low mortgage rates (Campbell 2006). This connection underscores the importance of understanding how interest rates impact borrower decisions within the mortgage market. Additionally, the measurement of fallout risk in the mortgage pipeline is crucial, as it provides insights into the potential for prepayment of loans and the associated risks (Hakim et al. 1999).

By assessing fallout risk, lenders can better anticipate and manage the challenges posed by prepayment behavior. Moreover, the role of mortgage debt and commitment in homeownership and entrepreneurship is significant (Bracke et al. 2014). Understanding how mortgage debt influences individuals’ decisions regarding borrowing and entrepreneurship can shed light on the dynamics within the mortgage market. Factors such as LTV ratios play a crucial role in determining mortgage default rates (Harrison et al. 2004). This highlights the importance of assessing risk factors such as LTV ratios when predicting fallout rates in the mortgage pipeline. Furthermore, the choice of mortgage contracts, such as fixed-rate mortgages (FRMs) and adjustable-rate mortgages (ARMs), is influenced by inflation risks and interest rate expectations (Coulibaly and Li 2009). This suggests that economic conditions and market expectations are key drivers of mortgage decisions, which in turn can impact fallout rates. Different borrower segments may exhibit varying behaviors that can affect mortgage outcomes. Environmental hazards, such as pipeline incidents, can also impact mortgage credit risk (Xu and Xu 2017). Lenders’ concerns about environmental risks associated with properties can influence lending decisions and ultimately affect fallout rates in the mortgage pipeline. Understanding how environmental factors intersect with mortgage credit risk is essential for a comprehensive risk assessment in the mortgage market. Additionally, the impact of credit ratings on security prices in the subprime mortgage-backed securities (MBS) market highlights the interconnectedness of credit risk and market dynamics (Ashcraft et al. 2011). Assessing credit ratings and their implications can provide valuable insights into potential fallout risks within the mortgage market. Moreover, the effects of prepayment regulations on the termination of subprime mortgages demonstrate how policy interventions can shape mortgage market outcomes (Steinbuks 2015). Regulatory measures can influence borrower behavior and, consequently, impact fallout rates in the mortgage pipeline. Additionally, the use of bankruptcy to reduce foreclosures raises questions about the potential effects of policy changes on mortgage credit supply (Li et al. 2014). Understanding the implications of regulatory interventions on mortgage credit availability is crucial for predicting fallout rates and managing associated risks. In conclusion, predicting fallout rates in the mortgage pipeline requires a comprehensive analysis of various factors, including borrower behavior, economic conditions, risk factors, regulatory influences, and environmental considerations. By integrating insights from research on mortgage behavior, credit risk, regulatory dynamics, and market conditions, stakeholders can enhance their ability to forecast fallout rates accurately and proactively manage risks in the mortgage market.

The closely related papers on the prediction of fallout risk in the mortgage pipeline include (Rosenblatt and Vanderhoff 1992; McMurray and Thomson 1997; Hakim et al. 1999; Wang et al. 2019). Rosenblatt and Vanderhoff (1992) investigated the factors influencing the closing rates of residential mortgage commitments, focusing on the differences between fixed-rate mortgages (FRMs) and adjustable-rate mortgages (ARMs). Their work highlights how mortgage banks offer options to borrowers, allowing them to secure specific interest rates for a limited time, which can significantly impact their decision to close on a mortgage. The model outlines that borrowers’ decisions to exercise their options on mortgage commitments are influenced by various factors, including the value of the option, the costs associated with finding alternative financing, and the borrower’s financial situation during the lock period. It emphasizes that the value of the option is more sensitive to interest rate changes for FRMs compared to ARMs, leading to the expectation that closing rates for FRMs will be more affected by interest rate fluctuations. The analysis also indicates that the costs of obtaining a different mortgage commitment are generally higher for retail loans compared to brokered loans, which can affect the probability of closing. Retail customers deal directly with the loan officers of the lender and hence have the relationship premium. The model estimates different parameters for retail (physical branch) and direct marketing (call center) loans, taking into account the value of the commitment, the borrower’s financial capacity, and the purpose of the mortgage acquisition.

McMurray and Thomson (1997) underscored the importance of considering a combination of factors, including applicant characteristics, property attributes, and economic conditions, in assessing the closing probability of residential mortgage applications. The authors utilized a dataset of mortgage applications to examine the impact of different variables on the closing probability, such as applicant and loan characteristics, property attributes, and changes in economic conditions during the lock period. One of the key findings of the research is that the credit history of the applicant plays a significant role in determining the closing probability of a mortgage application. Applicants with a strong credit history are more likely to have their applications approved and closed successfully. Additionally, the type of property being mortgaged also influences the closing probability. Properties occupied as primary residences have a higher likelihood of closing compared to the ones that are second homes or investment properties. Economic factors, such as interest rates and the overall economic climate, are identified as important determinants of the closing probability. Fluctuations in interest rates or economic instability can impact the approval rates of mortgage applications.

Hakim et al. (1999) examined the measurement of fallout risk in the mortgage pipeline, emphasizing the factors that influence mortgage closing rates. The study analyzed the timing and likelihood of loan closings, revealing that elements such as interest rate volatility, loan type, and borrower characteristics play significant roles in determining closing rates. The key findings indicate that rising interest rates tend to accelerate closings, while a widening spread between fixed- and adjustable-rate mortgages can slow them down. Additionally, the article addresses the risks associated with mortgage rate commitments, particularly focusing on price risk and fallout risk. The authors proposed a hazard model to analyze the closing process of mortgage loans, drawing from a dataset of fixed-rate mortgage applications from 1988 and 1989, and highlighted the importance of understanding these factors for effective risk management in mortgage banking.

A more recent study by (Wang et al. 2019) applied a deep neural network (DNN) to predict the loan rate for mortgage data based on various loan attributes and market variables. This article analyzed the behavior of mortgage borrowers using an unprecedented dataset of origination and monthly performance records for over 120 million mortgages originated across the United States between 1995 and 2014. They found that the DNN outperformed the robust regression models in terms of accuracy, sensitivity, specificity, and area under the ROC curve.

The recent literature on fallout prediction in the mortgage pipeline has seen significant advancements, with a strong focus on machine learning techniques and behavioral economics. For example, (Devine et al. 2023) highlighted that interest rate expectations significantly influence the choice between fixed-rate and adjustable-rate mortgages. Higher levels of education and financial knowledge increase the likelihood of households making accurate predictions about future interest rates. A probit model is used to estimate the effect of various household characteristics and behavioral influences on mortgage product selection. From a psychological standpoint, (Lee et al. 2023) examined home buyers’ decision-making during the Global Financial Crisis, particularly their coping mechanisms and risk aversion. High-income buyers exhibit higher risk aversion (1.74 to 1.99) compared to low-income buyers (0.60 to 0.62). Low-income buyers are more likely to endure unfavorable situations due to fear of negative outcomes, aligning with passive coping behavior. The study highlighted the influence of psychological factors like passive coping and cognitive dissonance on homebuyers’ decisions. More recently, (Zhang 2024) explored the application of machine learning techniques in residential real estate, focusing on automated valuation models (AVMs), residential property price indices (RPPIs), and land development analysis. Machine learning is used to develop AVMs to estimate property prices with minimal human intervention, addressing challenges like missing data. The thesis investigated whether accurate price predictions from AVMs can improve the RPPI, which tracks market value trends. Machine learning techniques were applied to evaluate land development options, considering factors like property prices and construction costs. These studies collectively highlight the complex, multifaceted nature of the mortgage pipeline and the ongoing efforts to improve accuracy through advanced analytics, behavioral insights, and technological innovations.

Our study differs from the previous studies in several aspects. First, we focus on predicting the fallout of a mortgage application rather than the loan termination probability or the mortgage rate. Fallout is a distinct phenomenon from prepayment or default, as it occurs before the loan is closed and funded. Fallout can be influenced by factors that are not relevant for prepayment or default, such as loan stage, lock duration, and approval status. Second, we use various popular machine learning models to predict the fallout rate rather than a single model. Third, we use a large dataset from the actual mortgage pipeline recorded by one of the national mortgage lenders in the United States rather than a sample of loans originated by a single lender or obtained from online sources. Our dataset contains information on various loan characteristics, market conditions, and loan stages that are relevant for fallout prediction. In this way, we capture accurate variations in market movements and borrower behavior and their impact on the fallout rate of a mortgage application. Moreover, we provide an empirical analysis of the relationship between the explanatory variables and the mortgage fallout, which, to our knowledge, has been less studied in previous works. Additionally, the existing methods in the literature analyzed and compared the capabilities of a limited number of machine learning models. In contrast, we comprehensively compared eight commonly used machine learning models and found that the random forest model has the best overall performance due to its anti-overfitting and automatic feature selection. To explain our prediction model, we applied the SHAP method and empirical logistic regression to figure out key factors in predicting the fallout risk of a mortgage application, and these factors are also less discussed in the literature.

3. Data and Variables

In this section, we describe the data and the variables used in our analysis. The data consist of a large dataset from the mortgage pipeline recorded by one of the national mortgage lenders in the United States. The dataset contains information on 84,000 loans that were locked in by borrowers between January 2013 and July 2016. The dataset also includes information on whether the loans were closed and funded or not, as well as the date of closing or fallout. We define the fallout rate as the proportion of loans that did not close and fund among the total number of locked loans.

There are 11 characteristics that describe a mortgage application. These variables were provided by the mortgage lender corresponding to each mortgage application. We used the following types of variables as predictors in our machine learning models: borrower, property and loan characteristics, market conditions, and loan stage.

The loan characteristics include the following variables:

• Loan amount: The amount of money borrowed by the borrower.
• Loan-to-value ratio (LTV): The ratio of the loan amount to the appraised value of the property.
• Loan product: The various loan products, such as government or conventional products, are captured by this field, which includes the maturity of the loans into 15-year and 30-year buckets.
• Loan purpose: This field collects the type of loan (whether the loan is a purchase or a refinance). If the loan is refinanced, then it is a term refinance or cash-out loan.
• Division: This field captures the origination channel of the loan (whether the loan originated in a retail branch or through a direct-to-consumer call center).

The borrower characteristics include the following variables:

• Credit score: The numerical score that represents the borrower’s creditworthiness based on their credit history.
• Debt ratio: The debt divided by the gross income of the borrower.

The market conditions include the following variables:

• Moneyness: We use a moneyness indicator, which indicates the market move from the day of the locking of the loan. High moneyness indicates interest rates have been decreasing since the lock period, thereby reducing the incentive for the borrower to potentially go to a competitor for a new mortgage application with another bank.
• W2 bucket: This variable measures the interest rate move in the first 2 weeks of the interest rate lock. After the passing of 2 weeks, the borrower is closely tied to the lender since the purchase and sale document provides the borrower with limited time to be able to process and obtain loan approval from a different lender.

The property characteristics include the following variables:

• Property occupied: This variable considers whether the borrower occupies the property as a primary residence, secondary residence, or investment property.

The loan stage includes the below stages. A mortgage application can fail at any stage in the mortgage pipeline.

• Application date: When the borrower starts the application.
• Package sent date: When the package is sent to the borrower to provide more information about the loan application.
• Package received: When the borrower provides all the needed documentation related to the loan application.
• Subbed: When the loan application is submitted to underwriting by the loan officer.
• Lock date: The date when the borrower locked in the interest rate with the lender.
• Approval date: The date when the lender approves the loan application.
• Closing date: The date when the loan was closed and funded (if applicable).
• Fallout date: The date when final action is taken on the loan that did not close and fund (if applicable).

The stage of the application is captured through the FirstStage field. FirstStage represents the stage in the mortgage pipeline where the interest rate is locked. All variables in our analysis are categorical in the model except mortgage amount. These categorical variables are created by dividing the data into specific subsets. The subsets were established based on their impact on the mortgage rate and traditional bucketing in the industry. The categories corresponding to each variable except mortgage amount are as follows:

Credit Score Buckets:

• <600
• 600–620
• 620–640
• 640–680
• 680–720
• 720+

LTV Buckets:

• <80
• 80–90
• 90–95
• >95

DTI Buckets:

• <35
• 35–40
• 40–45
• >45

Product:

• Cov30
• Cov15
• Govt30
• Govt15
• ARM
• Conv15

First Stage:

• Package Sent
• Package Received
• Approved
• Application
• Subbed
• Reviewed
• Set to Close
• Closed
• Denied

Division:

• Retail
• Direct
• Correspondent

Property Occupation:

• Owner Occupied
• Investment

Loan Purpose:

• No Cash Out
• Purchase
• Cash Out Refinance

Moneyness:

• M < −3
• M > −3 and <−2
• M > −2 and <−1
• M > −1 and <0
• M > 0 and <1
• M > 1 and <2
• M > 2 and <3
• M > 3

W2 Buckets:

• <−0.5
• −0.5 to −0.375
• −0.375 to −0.25
• −0.25 to −0.125
• −0.125 to 0
• 0 to 0.125
• 0.125 to 0.25
• 0.25 to 0.375
• 0.375 to 0.5
• >0.5

The most frequent FirstStage, Division, FICO Bucket, LTV Bucket, DTI Bucket, Loan Purpose, Moneyness, Product, Property Occupation, and W2 Bucket is Package Sent, Direct, 720+, 80–100, <35, Cash Out Refinance, 0 < M < 1, Govt30, Owner Occupied, and −0.125 to 0, respectively. We represent the categorical variables in the machine learning models using a combination of binary variables via one-hot encoding. Next, we will provide a summary of the only numerical field, i.e., the mortgage amount field. The field was subject to min–max normalization prior to the application of machine learning models. Figure 1 shows a Kernel Density Estimation (KDE) plot of mortgage amounts. The distribution is strongly right-skewed (positively skewed), with a long tail extending to the right. There is a prominent peak around 100,000 to 150,000, indicating this is the most common range for mortgage amounts. The x-axis extends from 0 to about 1.4 million, suggesting mortgages in the dataset range from very small amounts up to at least 1.4 million. The y-axis shows density, with the highest density (about 5 × 10⁻⁶) corresponding to the peak of the distribution. There is a long, thin tail extending to the right, indicating a small number of very high-value mortgages. There appears to be a smaller, secondary peak around 400,000, suggesting a possible bimodal distribution. This could indicate two distinct groups of borrowers or property types. The extended right tail suggests the presence of outliers—very high-value mortgages that are significantly above the typical range. This distribution indicates that while most mortgages are clustered around a lower range, there is significant variability, with some mortgages reaching much higher values. This information could be valuable for risk assessment, market segmentation, and understanding the overall mortgage landscape in the dataset.

Next, we will discuss the important notion of class imbalance in the target variable of mortgage fallout or not. After removing the missing data, we have a total of 60,206 observations. Around 10% of the entire dataset is reserved for testing, thus leading to a total of 54,184 observations for training. The following graph (Figure 2) illustrates the distribution of the target variable for mortgage applications, categorized as either fallout (“FO”) or not fallout (“C”). The dataset shows a significant imbalance between fallout and non-fallout cases. This imbalance could impact predictive modeling, potentially requiring techniques like oversampling, undersampling, or using class weights to address it. Most mortgage applications (about 70–72%) do not result in fallout, suggesting that fallout is a minority event. The fallout rate appears to be around 28–30% of total applications, which is a substantial portion and likely an important concern for lenders. While fallouts are less common, their occurrence in about 3 out of 10 applications could have significant financial and operational implications for mortgage lenders. Given this distribution, any predictive model would need to be carefully evaluated not just on overall accuracy but also on its ability to correctly identify the minority fallout class, possibly using metrics like precision, recall, and F1-score. This distribution provides valuable context for understanding the mortgage fallout problem and would be crucial in developing strategies to predict and mitigate fallout risk. The distribution highlights the substantial financial and operational impact fallouts could have on lenders, underscoring the importance of developing strategies to predict and mitigate fallout risk in the mortgage industry.

We used the Synthetic Minority Oversampling Technique (SMOTE) to correct class imbalance (Chawla et al. 2002). Note that we only applied SMOTE to the training data, not the test data. SMOTE creates synthetic examples of the minority class, so it increases dataset size. In this paper, we present the results with the class imbalance correction. Utilizing SMOTE, we obtained results for the eight employed algorithms.

The machine learning models employed in this study predict mortgage fallout by meticulously analyzing a multitude of borrower characteristics. Moreover, the model offers valuable insights into the relative importance of these variables in driving fallout risk. Building upon this foundation, we conducted in-depth empirical analyses to further elucidate the relationship between key factors and mortgage fallout outcomes. We used the above variables to construct our machine learning models for predicting the fallout of a mortgage application. In the next section, we present the methodology and the results of our model.

4. Methodology and Results

In this section, we comprehensively compare eight commonly used machine learning algorithms and choose the model with the best performance to assess mortgage fallout. These methods are k-nearest neighbors (kNN), logistic regression (Logistic), Gaussian naive Bayes (GNB), decision tree (DT), random forest (RF), multi-layer perceptron (MLP), extreme gradient boosting (XGBoost), and support vector machine (SVM). We briefly review these methods below. These machine learning algorithms were employed to predict the fallout of loan applications in the mortgage pipeline. The eight algorithms were selected based on their suitability for binary classification tasks and their previous success in similar applications.

• k-Nearest Neighbors: The method k-nearest neighbors (kNN) is a simple yet effective classification algorithm that predicts the class of a data point based on the classes of its k-nearest neighbors. It works by calculating the distance between the new data point and all training examples, selecting the K closest points, and assigning the new point to the class that is most frequent among its neighbors (Peterson 2009).
• Logistic Regression: Logistic regression is a statistical model used to predict binary outcomes (e.g., yes/no, true/false) based on one or more predictor variables. It estimates the probability of an event occurring as a function of these predictors, assuming a nonlinear relationship between the response variable and the predictor variables (Sperandei 2014).
• Gaussian Naive Bayes: Gaussian naive Bayes (GNB) assumes feature independence within each class and models feature distributions as Gaussian. By applying Bayes’ theorem, GNB calculates posterior probabilities for classification (Kamel et al. 2019).
• Decision Trees: Decision trees are another popular machine learning algorithm used in loan application fallout prediction. Decision trees recursively split the dataset based on different features to create a tree-like structure, enabling the classification of loan applications into various outcomes (Song and Ying 2015). Decision trees are interpretable and provide insights into the most critical features influencing mortgage fallout. However, decision trees can suffer from overfitting and lack robustness when dealing with complex datasets.
• Random Forest: A random forest is an ensemble learning algorithm that combines multiple decision trees to make predictions (Breiman 2001). It is known for its ability to handle complex interactions between features and provide robust predictions. The number of trees and other hyperparameters were optimized using techniques such as grid search or random search.
• Multi-Layer Perceptron (MLP): An MLP classifier is a supervised learning algorithm that implements a multi-layer perceptron (MLP), a building block of a neural network (Popescu et al. 2009). It is a neural network model capable of learning complex non-linear relationships in data. It uses backpropagation for training the network and relies on various parameters like the number of layers, neurons per layer, activation functions, solver, and learning rate for customization.
• Gradient Boosting: Gradient boosting is another ensemble learning algorithm that builds a strong predictive model by iteratively combining weak models. It sequentially adds decision trees to correct the errors made by previous models (Natekin and Knoll 2013). Hyperparameters, such as the learning rate and the number of boosting iterations, were tuned to achieve optimal performance.
• Support Vector Machine (SVM): A support vector machine is a powerful algorithm that separates data points in a high-dimensional space using hyperplanes. It aims to find the best hyperplane that maximally separates the two classes (Hearst et al. 1998). Different kernel functions, such as linear, polynomial, or radial basis functions (RBFs), could be explored to find the best decision boundary.

We used the scikit-learn package in Python to perform the data analysis. We used the default parameter settings for the corresponding algorithms in scikit-learn (Pedregosa et al. (2011)). We understand that parameter tuning might lead to different results and reserve the same for future research. In the next section, we describe the performance metrics used to evaluate the model. We then report the results of the different machine learning model estimates, followed by the empirical analysis using the logistic regression model.

5. Performance Metrics

To evaluate the performance of our machine learning models, we used various metrics that measure how well the model predicts the fallout of a mortgage application (Jordan and Mitchell 2015). We split our data into two subsets: a training set and a testing set. The training set contains 90% of the data, and the rest of the data are reserved for testing. We used the following metrics to evaluate our model:

• Accuracy: The proportion of correct predictions among all predictions. It is calculated as follows:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t} \mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}}{\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}}}$$

It measures how often the model predicts the correct outcome for an observation. A higher accuracy indicates better model performance.

2.

Recall: The proportion of correct predictions among all actual fallout cases. It is calculated as follows:

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{t}\mathrm{r}\mathrm{u}\mathrm{e}\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{s}}{\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{t}\mathrm{r}\mathrm{u}\mathrm{e} \mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}}}$$

Recall measures how well the model identifies the observations that actually experienced fallout. A higher sensitivity indicates better model performance.

3.

Precision: The proportion of positive predictions that are actually correct. It is calculated as follows:

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{t}\mathrm{r}\mathrm{u}\mathrm{e} \mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{s}}{\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{s}}}$$

Precision helps ensure that the model’s positive predictions are trustworthy and actionable. A higher specificity indicates better model performance. High precision indicates that when the model predicts a positive class, it is highly likely to be correct.

4.

F1-Score: The F1-score provides a single metric to measure model performance and is expressed as the harmonic mean of precision and recall. It is calculated as follows:

$$\mathrm{F}1 \mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}={\displaystyle \frac{2\ast \mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\ast \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}}$$

An F1-score is a comprehensive metric that helps assess a model’s overall performance, making it a valuable tool for model selection and comparison.

5.

Area under the ROC curve (AUC): The area under the receiver operating characteristic (ROC) curve plots the true positive rate (sensitivity) against the false positive rate (1-specificity) for different cutoff values of predicted probability. AUC measures how well the model discriminates between fallout and non-fallout cases. A higher AUC indicates better model performance.

The choice of performance metrics like accuracy, precision, recall, F1-score, and AUC for predicting mortgage fallout depends on the specific goals and priorities of the decision maker. Accuracy is a good general-purpose metric, but it can be misleading in imbalanced datasets (where one class is significantly more frequent than the other). Recall is important when it is crucial to minimize false negatives (missing cases of fallout), for example, if the goal is to identify as many potential fallouts as possible to take preventive measures. Precision is important when it is crucial to minimize false positives (incorrectly predicting fallout), for example, if the goal is to avoid unnecessary actions based on false alarms. F1-score is a balanced metric that considers both precision and recall. It is useful when there is a trade-off between minimizing false positives and false negatives. AUC is particularly useful when the goal is to evaluate a model’s performance across different classification thresholds. It is often used in conjunction with other metrics.

In the context of mortgage fallout prediction, the choice of metrics might also depend on business objectives, cost of misclassification, and data imbalance. If the primary goal is to minimize losses due to fallout, recall might be prioritized. If false positives (incorrectly predicting fallout) lead to significant costs, precision might be emphasized. If the dataset is imbalanced (e.g., with many more non-fallouts than fallouts), metrics like F1-score and AUC can be more informative than accuracy.

In the next section, we present the results of the eight machine learning models for the binary classification task of the mortgage fallout.

6. Model Performance

As discussed earlier in Section 3, to reduce the impact of the imbalance in the number of “fallout” samples and “closed” samples, we adopted the Synthetic Minority Oversampling Technique (SMOTE). SMOTE is an oversampling technique used to address imbalanced datasets in machine learning. It focuses on the minority class and generates synthetic samples to balance the class distribution. Instead of simply duplicating existing minority instances, SMOTE creates new data points by interpolating between existing minority class samples in the feature space.

The performance evaluation results of these models are shown in

Table 1. Summary of the results of the eight machine learning models.

Algorithm	Mean_Accuracy	Mean_Precision	Mean_Recall	Mean_F1_Score	Mean_ROC_AUC	Total_Time
kNN	0.714004	0.721239	0.714003	0.711859	0.780903	95.2
Logistic_Regression	0.585734	0.587115	0.585733	0.584047	0.620301	18.5
Naive_Bayes	0.602934	0.60869	0.602934	0.597109	0.644394	2.2
Decision_Tree	0.667325	0.668371	0.667324	0.666859	0.667745	7.8
Random_Forest	0.744714	0.746993	0.744713	0.744191	0.819282	242.9
MLP	0.659559	0.661594	0.659558	0.65845	0.721224	832.6
Gradient_Boosting	0.653698	0.655483	0.653697	0.65272	0.712721	121.5
SVM	0.585386	0.597841	0.585386	0.571227	0.622987	2576.8

. The table presents a comparative analysis of eight machine learning algorithms applied to a binary classification task for mortgage fallout prediction. The metrics evaluated include accuracy, precision, recall, F1-score, ROC AUC, and total computation time. These presented metrics represent the mean scores after 10-fold cross-validation using a 90–10 train–test split. Based on the mean accuracy, random forest emerges as the top-performing model with a score of 0.7447, suggesting it correctly predicts mortgage fallout outcomes 74.47% of the time. kNN follows closely with an accuracy of 0.7140. At the lower end, logistic regression and Gaussian naive Bayes show relatively lower accuracy scores. Precision and recall offer insights into the model’s ability to correctly predict positive (fallout) cases. While random forest maintains a high precision of 0.7469, indicating a low rate of false positives, kNN also exhibits strong precision (0.7212). However, to comprehensively assess model performance, considering recall is crucial. In this case, kNN and random forest demonstrate similar recall values, suggesting comparable abilities to correctly identify actual fallout cases. The F1-score, a harmonic mean of precision and recall, provides a balanced evaluation. Random forest again excels with an F1-score of 0.7441, followed closely by kNN. This indicates that random forest effectively balances precision and recall in predicting mortgage fallout. ROC AUC measures the model’s ability to distinguish between positive and negative classes. Random forest and kNN once again demonstrate superior performance with ROC AUC scores of 0.8192 and 0.7809, respectively. We also present the total run time of the 10-fold cross-validation in the column Total_time. Total computation time reveals significant differences in terms of computational efficiency among models. Gaussian naive Bayes is the fastest, followed by logistic regression and a decision tree. On the other hand, an SVM incurs the highest computational cost, potentially limiting its applicability in real-time scenarios.

The analysis indicates that random forest is a strong contender for mortgage fallout prediction due to its high accuracy, precision, recall, F1-score, and ROC AUC. However, its computational cost might be a slight concern for real-time applications. kNN also demonstrates promising performance across multiple metrics while being computationally efficient. Further exploration of hyperparameter tuning and feature engineering could potentially improve the performance of all models. Overall, the random forest (RF) model demonstrates superior predictive performance. Its random nature helps mitigate overfitting, and it is robust to noise and outliers in the data. By building multiple decision trees on various data subsets, RF effectively prevents overfitting. This ensemble approach captures diverse patterns within the data, resulting in more accurate predictions compared to a single decision tree.

Next, we present an explanation of the random forest (RF) model using SHAP (SHapley Additive exPlanations) values (Fernández et al. 2018). Figure 3 shows a bar plot of mean SHAP values for different features in a random forest model related to mortgage fallout prediction. SHAP values indicate the importance and impact of each feature on the model’s predictions.

FirstStage is the most influential feature with a mean SHAP value of 0.48, suggesting it has the strongest impact on the model’s predictions. Division is the second most important feature (0.36), followed closely by FICO_Bucket (0.32). DTI_Bucket (0.25) and Moneyness (0.24) have similar levels of importance, ranking fourth and fifth, respectively. Amount (0.21), Loan_Purpose (0.20), and LTV_Bucket (0.15) show moderate influence on the model’s output. Product has the least individual impact among the named features, with a SHAP value of 0.14. All provided features have positive mean SHAP values, suggesting they all contribute to increasing the model’s prediction (i.e., the likelihood of mortgage fallout). There is a clear hierarchy of feature importance, with the top 3–4 features having substantially higher impact than the others.

This analysis suggests that the model relies heavily on the FirstStage, Division, and FICO_Bucket features for making predictions, while factors like Product and LTV_Bucket play smaller roles. This information could be valuable for understanding the model’s decision-making process and for potential feature selection or model refinement.

In addition, Figure 4 shows the SHAP values of all samples, where the abscissa represents the SHAP value, the ordinate lists all features, and each point corresponds to a sample.

Figure 4 represents a summary plot of SHAP (SHapley Additive exPlanations) values for a random forest model predicting mortgage fallout. The features are ordered by importance from top to bottom. FirstStage appears to be the most influential, followed by Division and FICO_Bucket. Red dots indicate high feature values, while blue dots represent low values. For FirstStage, high values (red) tend to push predictions to the right (higher likelihood of fallout), while low values (blue) push left. Division shows a mixed impact, with both high and low values pushing predictions in both directions. FICO_Bucket has a wide distribution of impacts, suggesting it is important, but its effect varies significantly across instances. DTI_Bucket and LTV_Bucket show some extreme positive impacts for certain instances. Many features (e.g., Moneyness and Amount) show non-linear relationships, as evidenced by the spread of impacts across their value ranges. Loan_Purpose and Product show distinct clusters, indicating categorical variables with different impacts for each category. W2_Bucket and Prop_Occ have narrower distributions of SHAP values, suggesting they have less overall impact on predictions. Several features (e.g., LTV_Bucket and Product) have outlier instances with extreme positive impacts. The SHAP values range roughly from −3 to +3, indicating the model’s predictions can be significantly swayed by individual features.

This analysis suggests that the model considers a complex interplay of factors when predicting mortgage fallout, with FirstStage, Division, and FICO_Bucket being particularly influential. The non-linear and varied impacts across features indicate the model captures nuanced relationships in the data. Understanding these patterns could help in refining the model or in developing strategies to mitigate mortgage fallout risk.

In the next section, we present the methodology and the results of an empirical analysis using a logit model for predicting the fallout of a mortgage application. We first describe the underlying logistic regression model.

7. Empirical Analysis

As mentioned previously, the studied machine learning models predict mortgage fallout by analyzing multiple variables. In the meantime, the models can also provide the variable’s importance. Based on this information, we conducted further empirical tests using logistic regression.

Logistic regression is a statistical technique for modeling binary outcomes, such as fallout or non-fallout, by estimating the probability of an event occurring as a function of one or more explanatory variables. The logistic regression model can be expressed as follows:

$$\log \left({\displaystyle \frac{p}{1-p}}\right)={\beta }_0+{\beta }_1{X}_1+{\mathsf{\beta }}_2{X}_2+\dots +{\mathsf{\beta }}_p{X}_p$$

where:

-

$p$ is the probability of fallout

-

$X_j$ is the $j$th predictor variable

-

${\mathsf{\beta }}_j$ is the coefficient estimate for the $j$th predictor variable

-

$\log \left({\displaystyle \frac{p}{1-p}}\right)$ is the log odds of fallout

The logistic regression model can be interpreted as follows: for a one-unit increase in the predictor variable $X_j$, holding all other variables constant, the log odds of fallout increase by ${\beta }_j$ units. Alternatively, we can exponentiate both sides of the equation and obtain the following:

$${\displaystyle \frac{p}{1-p}}=e^{{\mathsf{\beta }}_0+{\mathsf{\beta }}_1{X}_1+{\mathsf{\beta }}_2{X}_2+\dots +{\mathsf{\beta }}_p{X}_p}$$

This can be interpreted as follows: for a one-unit increase in the predictor variable $X_j$, holding all other variables constant, the odds of fallout are multiplied by $e^{{\mathsf{\beta }}_j}$. The logistic regression model can be estimated using a method known as maximum likelihood estimation, which finds the values of the coefficients that maximize the likelihood function and then measures how well the model fits the data. The likelihood function for a binary outcome can be written as follows:

$$L\left(\mathsf{\beta }\right)=\prod _{i=1}^n{p}_i^{y_i}{\left(1-p_i\right)}^{1-y_i}$$

where:

-

$L\left(\beta \right)$ is the likelihood function

-

n is the number of observations

-

$p_i$ is the predicted probability of fallout for the $i$th observation

-

$y_i$ is the actual outcome of fallout for the $i$th observation (1 for fallout, 0 for non-fallout)

The likelihood function can be interpreted as follows: it is the product of the predicted probabilities of fallout for all observations that actually experienced fallout and the predicted probabilities of non-fallout for all observations that did not actually experience fallout. The higher the likelihood function, the better the model fits the data.

The main logistic regression results for the binary classification task of mortgage fallout are shown in

Table 2. Results of the logistic regression model.

Variables	Coefficient	Standard Error	z	p-Value	[0.025	0.975]
Intercept	−0.526	0.069	−7.578	0	−0.663	−0.390
Amount	−1.49 × 10−6	1.06 × 10−7	−14.095	0	−1.69 × 10−6	−1.28 × 10−6
FirstStage	−0.020	0.005	−4.464	0	−0.030	−0.012
Division	−0.375	0.023	−16.019	0	−0.421	−0.329
FICO_Bucket	−0.190	0.009	−21.051	0	−0.208	−0.172
LTV_Bucket	0.163	0.019	8.498	0	0.126	0.201
DTI_Bucket	0.227	0.010	1.707	0	0.207	0.248
Loan_Purpose	−0.107	0.015	−7.124	0	−0.137	−0.078
Moneyness	0.031	0.006	5.412	0	0.020	0.043
Product	0.071	0.010	7.211	0	0.052	0.091
Prop_Occ	0.233	0.024	9.719	0	0.186	0.280
W2_Bucket	0.002	0.003	0.691	0.489	−0.004	0.009

.

The logistic regression results provide insights into predicting mortgage fallout based on various factors. The model uses FOStatus as the dependent variable, with 54,184 observations. It is a logit model estimated using maximum likelihood estimation, with a Pseudo R-squared of 0.64224, indicating it explains about 64.22% of the variance in fallout status.

The model converged successfully and is statistically significant overall, as evidenced by the p-value of 0.000. All variables except W2_Bucket are statistically significant at the 0.05 level. Most variables are highly significant (p < 0.001), except for W2_Bucket (p = 0.489). The coefficients reveal that higher mortgage amounts, earlier stages, certain divisions, and higher FICO scores are associated with lower fallout risk. Conversely, higher loan-to-value ratios, higher debt-to-income ratios, certain loan purposes, moneyness, some products, and property occupancy status are linked to increased fallout risk.

The key influential variables (based on z-scores) are FICO_Bucket (z = −21.051), Division (z = −16.019), Amount (z = −14.095), and Prop_Occ (z = 9.719) The strongest predictors, based on absolute z-values, are FICO_Bucket, Division, and Amount. However, the model’s relatively lower Pseudo R-squared suggests limited explanatory power, indicating there may be other important factors not included or that the relationship might be more complex than a logistic regression can capture.

The direction of effects can be interpreted as follows: Negative of the fields Amount, FirstStage, Division, FICO_Bucket, and Loan_Purpose are associated with decreasing fallout probability. Whereas the positive coefficients linked with increasing fallout probability are exhibited in fields like LTV_Bucket, DTI_Bucket, Moneyness, Product, and Prop_Occ. This signifies that an increase in DTI_Bucket is associated with a higher fallout probability. Also, higher FICO scores (FICO_Bucket) are associated with a lower fallout probability. Moreover, larger loan amounts slightly decrease fallout probability.

In summary, this empirical analysis using a logistic regression model identified several significant factors influencing mortgage fallout, with FICO score, division, and loan amount being particularly important. However, the model’s overall explanatory power is somewhat lower, indicating that mortgage fallout prediction is complex and may require additional variables or more sophisticated modeling approaches.

8. Discussion

In this section, we discuss the implications and limitations of our findings for mortgage lenders and researchers. We also provide some suggestions for future research directions.

8.1. Implications

Our findings have several implications for mortgage lenders who want to predict the fallout of their mortgage applications and optimize their hedging and capital allocation decisions in the mortgage pipeline. First, our model can help lenders identify the key factors that affect the fallout rate and quantify their impact on the log odds of fallout. For example, lenders can use our model to estimate how much the fallout rate would change if they adjusted the interest rate, the lock duration, or the loan type for a given loan. They can also use our model to compare the effects of different factors and determine which ones are more important or influential. This can help lenders design and implement more effective strategies to reduce the fallout rate and increase the pull-through rate. Second, our models can help lenders improve their accuracy and specificity in predicting the fallout rate compared to other machine learning models. This can help lenders reduce the risk of over-hedging or under-hedging their pipeline and increase their profitability and liquidity. Over-hedging occurs when lenders hedge more than the expected amount of loans that will close and fund, which exposes them to unnecessary costs and losses. Under-hedging occurs when lenders hedge less than the expected amount of loans that will close and fund, which exposes them to market risk and volatility. By using our empirical analysis with logistic regression, lenders can better estimate the expected amount of loans that will close and fund and adjust their hedge ratio accordingly. Third, our machine learning models can help lenders monitor and update their fallout rate prediction as new information becomes available. For example, lenders can use our model to update their prediction when they receive new market data, such as interest rate changes or volatility. They can also use our model to update their predictions when they receive new loan applications or when they process existing loans and obtain new approval or closing dates. This can help lenders capture the dynamic nature of the fallout rate and respond quickly and appropriately to changing market conditions and borrower behavior.

Accurate models for predicting mortgage fallout can provide significant value to both financial institutions and regulatory bodies. For financial institutions, these models offer crucial benefits in risk management, allowing them to identify potential fallouts early and take proactive steps to mitigate risk. This can lead to improved profitability and enhanced financial stability. Additionally, these predictive models can inform decision-making processes related to loan origination, pricing, and portfolio management while also helping institutions comply with regulatory requirements related to risk management and consumer protection.

From the perspective of regulatory bodies, mortgage fallout prediction models serve as powerful tools for assessing systemic risks in the mortgage market and identifying potential vulnerabilities. The insights gained from these models can inform the development of effective regulatory policies to protect consumers and maintain financial stability. Furthermore, these models can provide early warning signs of potential financial crises, allowing regulators to take proactive measures to prevent or mitigate their impact. By identifying potential risks to borrowers, regulatory bodies can also take steps to protect consumers from predatory lending practices and financial hardship. In summary, mortgage fallout prediction models offer a valuable tool for both financial institutions and regulatory bodies to enhance risk management, improve decision-making, and protect the financial system, ultimately promoting a more stable and resilient mortgage market.

Our study illustrates the potential insights and benefits that machine learning models with the SHAP method can bring to banks, lending institutions, and regulatory bodies, leading to more informed risk management systems. An explainable machine learning method facilitates the understanding of the model’s decision-making process, which can provide banks and lending institutions with a clearer basis for the loan application process while improving customer satisfaction. In addition, for regulatory bodies, our findings can provide more suggestions about regulatory evaluation standards, helping to enhance financial stability.

8.2. Limitations

Our findings also have some limitations that should be acknowledged and addressed in future research. First, our model assumes that the fallout probability is constant over time for given loan characteristics, which may not be realistic in practice. Fallout probability may vary over time depending on the market conditions and the borrower’s behavior. For example, fallout probability as a function of interest rate might increase or decrease, or the lock duration impact might change depending on the regulation regime that is in place. This may cause our model to underestimate or overestimate the fallout rate for some loans, especially those with long lock durations or volatile interest rates. A possible way to address this limitation is to use a time-varying model that allows the coefficients to change over time according to some functions of time or other variables. This can help capture the time-dependent nature of the fallout probability and improve the prediction accuracy.

Second, our machine learning models do not account for censoring, which occurs when some loans are still in the pipeline at the end of the study period and their final outcome is unknown. Censoring may introduce bias and uncertainty in the estimation and prediction of the fallout rate. For example, if some loans that are more likely than the model prediction to fallout are censored, our model may overestimate the pull-through rate and underestimate the fallout rate. Conversely, if some loans that are less likely than our model prediction to close and fund are censored, our machine learning models may underestimate the pull-through rate and overestimate the fallout rate. A possible way to address this limitation is to use a survival analysis model that can handle censoring properly. Survival analysis can account for both right censoring (when some loans are still in the pipeline at the end of the study period) and left censoring (when some loans enter the pipeline before the start of the study period). Survival analysis can also estimate hazard functions, which measure how the risk of fallout changes over time for a given loan.

Third, our machine learning model does not account for unobserved heterogeneity among borrowers, which may affect their propensity to fall out of the pipeline. Unobserved heterogeneity may arise from factors that are not captured by our predictor variables, such as borrower preferences, expectations, or satisfaction. For example, some borrowers may be more sensitive to interest rate changes or more loyal to their lenders than others, which may affect their likelihood of falling out or pulling through. Unobserved heterogeneity may cause our model to miss some important sources of variation in the fallout probability and reduce its explanatory power. A possible way to address this limitation is to use a mixed-effects model that can incorporate random effects for each borrower. Random effects are unobserved variables that capture the individual differences among borrowers and their impact on the fallout probability. By using a mixed-effects model, we can account for both the fixed effects of the predictor variables and the random effects of the borrowers and improve the model fit and prediction.

In summary, our paper contributes to the literature on mortgage fallout prediction by providing effective techniques that can help lenders optimize their hedging and capital allocation decisions. Our paper also provides some suggestions for future research directions, such as using a time-varying logistic regression model (Zhang et al. 2018), a survival analysis model (Wang et al. 2019), or a mixed-effects logistic regression model (Vermunt 2005) to address some of the limitations of our current model.

9. Conclusions

This study aimed to predict mortgage loan fallout using a comprehensive set of eight machine learning algorithms applied to data from a major mortgage lending institution. Our findings demonstrate the superior performance of the random forest model, which achieved an accuracy rate of 74.47% in predicting mortgage fallout. This outcome underscores the potential of machine learning techniques to enhance risk assessment and decision-making in the mortgage lending industry.

To gain deeper insights into the factors driving mortgage fallout, we employed the SHAP method to identify the most influential variables and their relative importance. Our analysis revealed that the stage of the loan, originating division, and FICO score of the customer were among the primary determinants of loan performance. These findings provide valuable guidance for lenders in evaluating loan applications and mitigating risk.

Predicting mortgage loan fallout is also crucial for maintaining financial stability, informing economic policy, protecting consumers, and promoting economic growth. Accurate predictions can serve as an early warning sign of potential economic downturns, allowing financial institutions to manage their risk exposure and governments to implement appropriate measures.

By understanding the factors that contribute to mortgage fallout, policymakers, financial institutions, and individuals can take proactive steps to mitigate risks and ensure a more resilient economy. This includes implementing measures to prevent foreclosure, providing support to struggling homeowners, and maintaining market confidence.