2. Methodology
In this section, the machine learning algorithms used as individual predictors are briefly introduced, along with the stacking method employed to aggregate these algorithms.
2.1. k-Nearest Neighbor Algorithm (k-NN)
The k-nearest neighbor (k-NN) algorithm is a non-parametric, instance-based learning method widely utilized for regression tasks. This algorithm operates on the premise that similar data points yield similar target values. The methodology of k-NN regression involves storing the entire training dataset, encompassing both feature vectors and their corresponding target values. To predict the target value for a new input data point, the algorithm calculates the distance between this new point and all points in the training dataset. Common distance metrics include Euclidean, Manhattan, and Minkowski distances. Once distances are calculated, the algorithm identifies the k-nearest neighbors to the new data point. Upon identifying the nearest neighbors, k-NN regression aggregates their target values to produce a final prediction. Typically, this aggregation involves computing the mean of the neighbors’ target values, although methods such as the median can be used to mitigate the influence of outliers. This approach allows the algorithm to make predictions based on the local structure of the data.
The primary advantages of k-NN regression include its simplicity and ease of implementation. While the prediction phase can be computationally expensive for large datasets due to the need to calculate distances to all training points, it does not require a computationally intensive training phase, as it is a lazy learning algorithm. Despite these computational challenges, the k-NN algorithm remains an effective and practical choice for regression tasks, providing accurate predictions by leveraging the similarity between data points. For studies on applying the algorithm to mass appraisal problems, please refer to [
34,
35].
2.2. Decision Tree Algorithm (Regression Tree)
The regression tree algorithm builds a binary tree where each decision node splits the dataset based on specific feature values to reduce the variance of the target variable within the resulting subsets. This recursive partitioning process continues until a stopping criterion, such as a minimum number of samples per leaf or a maximum tree depth, is reached. To make predictions, the algorithm traverses the tree from the root to a leaf node, with the final prediction being the mean target value of the samples within the leaf node.
In [
36], it is highlighted that decision trees and their ensemble models (random forest (RF), gradient boosting, XGBoost, CatBoost, and AdaBoost) are particularly suitable for real estate appraisal problems. Firstly, these methods effectively handle categorical variables with many levels. In contrast to multiple regression or neural networks, which require a large number of parameters to accommodate qualitative variables, leading to overfitting, tree-based methods manage these variables more efficiently. Secondly, they perform well with missing data. When data is missing for an observation, the prediction is made using the existing structure of the tree, eliminating the need to exclude observations or impute missing values. Thirdly, predictions for new observations fall within the range of existing observations, which helps prevent significant overestimation or underestimation of real estate values.
Our dataset includes all real estate transactions in Jeju Island, South Korea, from 2012 to 2021, totaling 63,306 transactions. Among these, several variables are categorical (such as heating system and structure). Also, if records with missing values are removed, only 12,967 transactions remain, resulting in a significant loss of information. Therefore, as mentioned in the literature, decision tree-based algorithms are appropriate for this analysis. Consequently, we employed decision tree-based algorithms in our study.
2.3. Random Forest Algorithm
Random forest (RF) regression is an ensemble learning technique that enhances prediction accuracy by aggregating the outputs of multiple decision trees. This method was initially introduced by [
37], who demonstrated that combining tree-based predictors using hyperplane splits can improve predictive performance and prevent overfitting. In RF regression, multiple decision trees are independently constructed using different bootstrapped samples of the original dataset. Each tree is grown to its maximum extent without pruning. Unlike traditional decision trees, which use the best possible split at each node based on all available features, RF trees use a random subset of features for each split. This randomness increases the diversity of the trees in the ensemble. After generating a substantial number of trees, the RF model makes predictions by averaging the outputs of all individual trees. In [
38], it was noted that “this somewhat counterintuitive strategy turns out to perform very well compared to many other classifiers, including discriminant analysis, support vector machines, and neural networks, and is robust against overfitting”. The robustness and high predictive performance of the RF algorithm have led to its widespread application in real estate valuation [
9,
36]. The method effectively handles large numbers of input variables, copes well with missing data, and provides insights into variable importance.
2.4. Gradient Boosting Algorithm
Gradient boosting improves the predictive performance of models by sequentially combining the outputs of several weak learners. Initially introduced by [
39], it builds an ensemble of models in an additive manner, where each new model aims to correct the errors made by the previous ones, thereby incrementally improving performance. The algorithm constructs new base learners that align closely with the negative gradient of the loss function, which measures how well the model fits the data. This iterative process continues until a predefined number of iterations is reached or the model’s performance stabilizes.
A significant distinction of gradient boosting is its flexibility in handling various loss functions, which can be tailored to specific applications. For instance, in real estate valuation, the loss function typically gauges the difference between actual and predicted property prices. The algorithm employs a gradient descent procedure to minimize the loss function as new trees are added, ensuring that each successive tree effectively reduces the overall prediction error. This method enhances the model’s ability to capture complex, non-linear relationships within the data.
Gradient boosting offers several mathematical advantages, including the reduction of bias and variance through iterative refinement. The algorithm’s capacity to focus on the most challenging data points and its robust handling of outliers make it a powerful tool for regression tasks. By incrementally improving the model’s predictions, gradient boosting creates a strong predictive model that generalizes well to new data. These advantages have led to the development and expansion of various techniques based on the principles of gradient boosting, such as XGBoost, CatBoost, and AdaBoost, each incorporating enhancements to improve performance and scalability.
2.5. XGBoost Algorithm
XGBoost, short for extreme gradient boosting, is a highly efficient and scalable implementation of the gradient boosting framework introduced by [
40]. This algorithm enhances traditional gradient boosting principles with several advancements, making it particularly effective for regression tasks. One of the primary improvements is the integration of advanced regularization techniques, specifically L1 (Lasso) and L2 (Ridge) regularization, which help prevent overfitting and improve model generalization by penalizing model complexity and encouraging simpler, more robust trees.
XGBoost also excels in handling missing data and sparse features. The algorithm automatically determines the best way to manage missing values, maintaining predictive accuracy without the need for explicit imputation. It employs a sparsity-aware approach that efficiently processes datasets with high proportions of missing or zero values. This capability is crucial for real-world applications where incomplete data is common and traditional gradient boosting methods might struggle.
Additionally, XGBoost introduces significant improvements in computational efficiency and scalability. It optimizes memory usage and training speed through parallel processing and advanced data structures like column blocks and cache-aware prefetching. These enhancements substantially reduce training time, enabling XGBoost to manage large-scale datasets more effectively. This efficiency supports rapid experimentation and model iteration, which is essential for dynamic fields such as real estate valuation. For a comprehensive understanding of XGBoost, refer to the seminal paper by [
40], which details its theoretical foundations and practical applications.
2.6. CatBoost Algorithm
CatBoost is an advanced implementation of the gradient boosting algorithm developed by [
41], designed specifically to handle categorical data more effectively. CatBoost incorporates several key advancements that distinguish it from traditional gradient boosting methods, particularly in its approach to processing categorical features and preventing overfitting. Traditional gradient boosting methods often struggle with categorical data, necessitating transformation into numerical formats, which can increase dimensionality and potentially lead to information loss. CatBoost addresses this by employing a technique called “ordered target statistics”, which converts categorical features into numerical representations using statistical averages. This method preserves the natural ordering and relationships within the data, maintaining the integrity of the original information.
The algorithm also utilizes a unique approach called ordered boosting to combat overfitting and enhance model generalization. In standard gradient boosting, each new model is trained on the residuals of the entire training set, which can lead to overfitting, especially with smaller datasets. CatBoost mitigates this issue by building each tree using a permutation of the dataset, ensuring that each model learns only from past observations rather than future ones. This approach maintains the causal structure of the data and produces more robust models, reducing the risk of overfitting.
Additionally, CatBoost incorporates a symmetric tree structure, where splits at each level are determined simultaneously rather than sequentially. This symmetry simplifies the tree-building process and accelerates both training and prediction phases. Optimized for efficient computation on both CPUs and GPUs, CatBoost can scale effectively and handle large datasets, making it suitable for various applications.
2.7. AdaBoost Algorithm
AdaBoost, short for adaptive boosting, is an ensemble learning algorithm developed by [
42]. Although it shares foundational principles with gradient boosting, such as combining weak learners to form a strong learner, AdaBoost introduces unique mechanisms that set it apart. It is particularly effective for both classification and regression tasks and is known for its simplicity and robustness.
One of the primary distinctions of AdaBoost is its adaptive reweighting mechanism. In AdaBoost, simple decision trees are often used as weak learners. Initially, all training instances are assigned equal weights. After each weak learner is trained, AdaBoost increases the weights of incorrectly predicted instances, forcing subsequent learners to focus more on these difficult cases. This differs from gradient boosting, where the focus is on minimizing the residual errors of previous models through gradient descent.
Another significant feature of AdaBoost is how it forms its final model. AdaBoost does not rely on gradient descent to minimize a loss function. Instead, it aggregates the predictions from all weak learners by taking a weighted sum, where the weights are determined by the accuracy of each learner. This method ensures that the final model is a robust combination of all learners, each contributing based on its performance. While this approach effectively reduces both bias and variance, it can be sensitive to noisy data and outliers, as the adaptive weighting may amplify the influence of these instances. Despite its sensitivity to noise, AdaBoost remains a popular choice due to its effectiveness and straightforward implementation.
2.8. Stacked Ensemble Model
Stacking, or stacked generalization, is an advanced ensemble learning technique introduced by Wolpert in 1992 [
43]. It aims to improve predictive performance by combining multiple base models, leveraging their individual strengths through a meta-learner to produce a more accurate and robust final prediction. This approach is particularly effective for complex regression tasks where capturing diverse patterns and relationships in the data is crucial.
The stacking process begins with training several base models, often referred to as level 0 models, on the same dataset. In our study, the base models correspond to the algorithms mentioned above. Each base model generates predictions, which are then used as inputs for the meta-learner, also known as the level 1 model. The meta-learner is typically a simple model, such as linear regression, which is trained on the outputs of the base models to make the final prediction. Mathematically, let
represent the set of base models, where
denotes the number of these base models and
denotes the set of input features. Each base model
produces a prediction
. The meta-learner
takes these predictions as input and produces the final prediction. Formally, the stacked model can be represented as:
The meta-learner is trained to minimize the prediction error on the validation or training set, optimizing the combination of base models to reduce overall prediction error. This differentiates stacking from other ensemble methods like bagging, which reduces variance by averaging predictions from multiple resampled datasets, and boosting, which sequentially builds models to correct residual errors from previous ones. Stacking’s ability to integrate diverse models is crucial for its effectiveness as it allows the meta-learner to exploit the strengths of each base model while mitigating their individual weaknesses.
The primary advantage of stacking is its flexibility to combine heterogeneous models. By allowing various algorithms to serve as base models, stacking can capture a broader range of patterns and relationships within the data, leveraging the strengths of different algorithms. This diversity helps improve the overall generalization capability of the ensemble model. Additionally, stacking can help mitigate overfitting. While individual models may overfit specific parts of the data, the meta-learner in stacking can learn to down-weight the influence of these overfitted models, resulting in a more robust final prediction. This is particularly beneficial when dealing with complex and noisy datasets, as the meta-learner can effectively balance the contributions of each base model. From a practical standpoint, stacking simplifies the model selection process for practitioners. The combined model presented in this study evaluates the performance of individual algorithms based on the features of the data. When estimating the price of real estate, the model selects and applies the most appropriate techniques based on the learned performance of these algorithms, considering the specific characteristics of the property. This approach eliminates the need to search for a single best predictive model, as the stacking model assigns higher weights to the most suitable models. Models that perform poorly on specific data are given low weights or excluded, ensuring they have a minimal effect on the overall predictive performance. This strategy allows practitioners to confidently utilize various machine learning models with reduced concern about selecting the optimal one. Furthermore, because stacking combines various machine learning techniques, it can be effectively applied not only to the real estate market in a specific region but also to property valuations in other regions. This adaptability makes it a versatile tool in real estate appraisal, ensuring reliable predictions across diverse geographic locations.
As mentioned earlier, while the stacked model can automatically identify the most suitable model regardless of the base algorithms used, this study primarily employs decision tree models as the base algorithms. Decision tree algorithms and their ensemble models, such as random forest (RF), gradient boosting, XGBoost, CatBoost, and AdaBoost, are particularly well-suited for real estate valuation tasks, as discussed in [
36]. Tree-based algorithms efficiently manage categorical variables with numerous levels. Unlike multiple regression or neural networks, which may require complex parameterization to incorporate qualitative data—thereby increasing the risk of overfitting—tree-based models naturally accommodate these variables through data splitting, avoiding excessive complexity. This feature-based data splitting can also be advantageous for accurately evaluating property values by effectively considering the location of real estate. Note that the latitude and longitude of properties are used as features in this study. Other algorithms typically represent a property’s locational characteristics through factors such as accessibility to public transportation or proximity to elementary schools, assuming a linear (or directly proportional or inversely proportional) relationship with property value. However, when observing the actual real estate market in Korea, property values may be influenced non-linearly by variables representing specific locational characteristics. In such cases, decision tree techniques, which are based on data splitting, can identify specific grids with different price distributions based on latitude and longitude, allowing for a more accurate estimation of property values. Since the ensemble models used in this study incorporate decision trees, they are particularly effective at capturing the non-linear effects of location on property value. In addition, the models handle missing data effectively by using the existing tree structure to make predictions even when some information is missing, eliminating the need to discard observations or impute missing values. Our dataset comprises 63,306 real estate transactions, but removing records with missing data reduces this number to 12,967 transactions, resulting in substantial information loss. Thus, this approach preserves the integrity of the analysis and ensures that all available data is utilized.
We utilize linear regression, specifically ridge regression, as the meta-learner in our study. Given that the predictions from our base models serve as inputs, the meta-learner employs these seven predictions to generate the final output. This approach is akin to training a model on a dataset comprising seven features, indicating that algorithms designed for managing complex structures, such as neural networks, may be less suitable for this specific context. Linear regression is advantageous due to its computational efficiency and simplicity of implementation, making it particularly effective for large datasets and facilitating rapid model deployment. By assigning appropriate weights to each base model, linear regression effectively synthesizes their strengths, ensuring that contributions from models excelling in different scenarios are balanced to enhance overall performance. Additionally, in cases where all individual algorithms exhibit similar biases, the aggregation process via linear regression can adjust for these biases through the intercept term. The linear combination of predictions from multiple models also helps to mitigate overfitting, as linear regression introduces minimal complexity and typically generalizes more effectively compared to more intricate aggregation methods. Moreover, ridge regression is particularly well-suited when independent variables are highly correlated. Given the potential for correlation among the predictions generated by different algorithms, employing ridge regression as the meta-learner is an appropriate choice. This method helps to stabilize the model by effectively managing the multicollinearity among the input predictions, ensuring a more robust and reliable final output.
3. Data Set and Basic Statistics
Jeju Island, also known as Jeju-do, is the largest island off the coast of the Korean Peninsula and functions as a self-governing province of South Korea. With a population of approximately 700,000 and an area of 1846 square kilometers, Jeju Island is notable for its unique geographical and cultural features (
Figure 1). While South Korea is known for its competitive real estate market, property prices on Jeju Island have risen significantly due to its growing popularity as both a tourist destination and a residential area. The average housing price in Jeju has increased substantially, driven by investments from both local and foreign sources. In recent years, real estate on Jeju Island has become highly desirable, particularly for its scenic beauty and relatively relaxed living conditions compared to the mainland.
Over the past decade, Jeju Island has experienced a significant number of real estate transactions, reflecting its appeal for both tourism and permanent residence. We collected 63,306 samples from apartment transaction records on Jeju Island from 2012 to 2021, provided by South Korea’s Ministry of Land, Infrastructure, and Transport (MOLIT). This dataset covers all transactions on Jeju Island during the period, offering a comprehensive overview of market trends and property values. Note that the objective of our study is to develop an automated valuation model for residential properties using the stacking method. Thus, our target variable is the price of the properties, and basic information about the features necessary for estimating the value is summarized in
Table 1 and
Table 2.
The features used to estimate apartment prices are categorized into three groups based on their characteristics. This classification follows the framework presented in [
33]. The attributes of a property include its inherent characteristics, such as the floor area, the property’s age, and various other factors. Previous research has shown a positive correlation between property prices and attributes like the floor area [
10]. Ref. [
12] noted that a property’s age, or the number of years since its construction, can negatively impact its value. In our study, we utilize variables such as the occupancy approval date and transaction date to consider the property’s age. Additionally, the net floor area refers to the actual usable space within the apartment, whereas the gross floor area includes shared facilities such as gyms and swimming pools. In South Korean apartments, a significantly larger gross floor area compared to the net floor area indicates a higher likelihood that the apartment complex includes a variety of amenities. Both variables are considered in this study. The land share of an apartment represents the actual land area owned by each unit. As buildings age and their value depreciates over time, the importance of the land share increases. In the context of redevelopment or reconstruction, a larger land share can lead to higher compensation and an increase in the per-square-meter price. Additionally, variables such as floor level, heating system, the direction of the living room windows, and the presence of additional balconies, duplex layout, private gardens, and attics are used to construct the model as property attributes.
Apartment attributes refer to the characteristics that are common across all units within an apartment complex. For example, two properties within the same apartment complex may differ in size or the presence of an additional balcony (property attributes), but variables such as the total number of parking spots in the complex will have the same value for all properties within that complex. In South Korea, multi-family housing is categorized into three types based on the number of floors and the total floor area. Multiplex housing refers to buildings with four or fewer floors, whereas those with five or more floors are classified as apartments. Multiplex housing with a total floor area exceeding 660 square meters is referred to as “yeonlip” (Type 1), while those with a total floor area of 660 square meters or less are called “dasedae” (Type 2). Additionally, the construction method of the apartment is considered in the structure variable. To reflect the convenience of the apartment’s parking facilities, we use the total number of parking spots as a variable. Generally, apartment complexes with a larger number of households tend to have higher prices. To account for this, we included the number of buildings and households in the complex as variables. The floor area ratio (FAR) is determined by dividing the total gross floor area by the land area, while the building coverage ratio (BCR) is calculated by dividing the building area by the land area.
Locational attributes pertain to the characteristics associated with a property’s geographic location. According to references [
9,
44], these attributes encompass all external factors linked to a house’s location, including accessibility to the central business district (CBD), public transportation systems, schools, and proximity to environmental amenities. To integrate geographic location into our models, we use the property’s latitude and longitude coordinates and the nearest school information. Note that the nearest elementary, middle, and high schools are treated as categorical variables. In South Korea, children are assigned to schools based on the proximity to their residence. Given the high emphasis on children’s education in Korea, the specific school to which a property is zoned is crucial and can significantly impact the property’s value. Zoning districts represent the most fundamental classification of land use, ensuring that all land across the nation is designated without overlap. These zones regulate land use to ensure its alignment with specific purposes and apply various restrictions based on the area’s characteristics. According to Article 6 of the National Land Planning and Utilization Act, land in South Korea is categorized into four types of zoning districts: urban areas, which require systematic development and management due to high or anticipated population and industrial density; control areas, which need urban-level management or conservation similar to agricultural and forest zones; agricultural and forest areas, designated for the promotion and conservation of agriculture and forestry outside urban zones; and natural environment conservation areas, essential for conserving natural environments, water resources, ecosystems, and cultural properties. Each category is further subdivided, with specific restrictions applied based on the detailed zoning designations. These subdivisions impose various limitations and regulations to ensure appropriate and efficient land use, tailored to the specific needs and characteristics of each zone. For example, the type 1 exclusive residential zone is restricted to a building coverage ratio of 50% and a floor area ratio of 100% or less. The types of buildings permitted in this zone are limited to single-family homes (excluding multi-family homes) and certain neighborhood facilities, such as community centers, communal workshops, and supermarkets. For specific restrictions related to detailed subdivisions, please refer to [
45].
The basic statistics for the numerical variables in the dataset are presented in
Table 2. Most variables have no missing values; however, approximately 25% of the data for apartment parking spaces is missing. There are also significant missing values for categorical variables, with about 70% of the data for the heating system being incomplete. Removing records with missing values would result in retaining only 13,065 out of 63,306 transactions, leading to a substantial loss of data. Therefore, instead of removing records with missing values, we utilize tree-based algorithms to handle the missing data. As mentioned earlier, tree-based algorithms are particularly well-suited for analyzing datasets with missing values. These algorithms can naturally handle missing data by using surrogate splits, which allow them to make the best possible decision even when some data is missing. This feature ensures that the integrity of the analysis is maintained without discarding a significant portion of the data.
4. Results and Post Analysis
In this section, we compare the performance of individual algorithm predictors (i.e., k-NN, decision tree, random forest, gradient boosting, XGBoost, CatBoost, and Adaboost) with that of a stacked ensemble model. To construct the ensemble model, we utilized the aforementioned algorithms as the base learners and employed linear regression (ridge regression) as the meta-learner. Linear regression, being computationally efficient and straightforward to implement, is advantageous for large datasets and facilitating rapid model deployment. By assigning appropriate weights to each base model, linear regression effectively amalgamates their strengths. This method ensures that if one model performs well in certain contexts and another excels in different scenarios, their contributions are balanced to enhance overall performance. Moreover, if all individual algorithms exhibit similar biased predictions, the aggregation process via linear regression can adjust these biases through the intercept term. Additionally, the linear combination of predictions from multiple models helps mitigate overfitting, as linear regression introduces minimal complexity and typically generalizes better compared to more complex aggregation methods.
The performance comparison was conducted using 5-fold cross-validation to ensure a robust and reliable evaluation after optimizing the hyperparameters of the individual algorithms. In 5-fold cross-validation, the dataset is divided into five equal parts. Each part is used as a test set once, while the remaining four parts serve as the training set. This process is repeated five times, with each fold being used as the test set exactly once. The results from each fold are then averaged to provide a comprehensive assessment of the model’s performance. During this process, we fine-tuned the hyperparameters of each algorithm to identify the configuration that yielded the best performance. The algorithms with the highest performance were then selected to construct an ensemble model, leveraging their strengths to further enhance predictive accuracy. To compare the performance of the combined model using algorithms and stacking methods, results for various performance metrics were derived. The performance metrics used for comparison included root mean squared error (
), mean absolute error (
), mean absolute percentage error (
), coefficient of determination (
), and coefficient of variation of the root mean squared error (
). The number of samples in the test set is defined as
, the actual value of the
-th sample in the test set is denoted as
, the predicted value of the dependent variable through the trained algorithm is denoted as
, and the mean of
is represented as
. The performance metrics are defined as follows:
The results for each performance metric are presented in
Table 3.
The stacked ensemble model demonstrated the best performance across all metrics, achieving the lowest RMSE of 23,041,799.68, MAE of 13,604,466.61, MAPE of 0.0840, and CVRMSE of 11.2157, as well as the highest R
2 of 0.9725. This indicates the smallest average error, highest precision, and best relative performance. XGBoost and AdaBoost also showed strong performance, with XGBoost achieving an RMSE of 24,175,049.55, MAE of 14,447,068.81, MAPE of 0.0894, R
2 of 0.9697, and CVRMSE of 11.7673. AdaBoost followed closely with an RMSE of 25,515,035.69, MAE of 13,696,174.92, MAPE of 0.0847, R
2 of 0.9663, and CVRMSE of 12.4195. In contrast, k-NN and decision tree exhibited higher error metrics and lower R
2 values, indicating less accurate predictions. Specifically, k-NN had an RMSE of 60,906,942.70, MAE of 33,667,069.83, MAPE of 0.2029, R
2 of 0.8079, and CVRMSE of 29.6468, while decision tree had an RMSE of 58,479,005.01, MAE of 34,265,276.34, MAPE of 0.2058, R
2 of 0.8230, and CVRMSE of 28.4649. The results suggest that the stacked ensemble model effectively combines the strengths of individual algorithms, leading to superior predictive performance.
Figure 2 shows the scatter plot between the actual transaction prices and the predicted values.
Additionally, the variance of errors in the stacked ensemble model is smaller compared to other models. The maximum and standard deviation (STDEV) of both the absolute errors (AEs) and the absolute percentage errors (APEs) for each technique are presented in the
Table 4. In the most extreme case (the maximum APE), the k-NN algorithm’s absolute percentage error is 11.6407, indicating that the prediction error is more than 11 times the actual transaction price. In contrast, the stacked ensemble model has the smallest maximum absolute percentage error of 3.8315 among all the algorithms. Also, the standard deviation of both the absolute error and absolute percentage error are the lowest among the algorithms, implying that the stacked ensemble model provides more stable and reliable predictions.
Table 5,
Table 6,
Table 7,
Table 8 and
Table 9 provide the probability that the model corresponding to the row has a higher score than the model corresponding to the column. The interpretation of a higher score varies depending on the metric; for metrics such as R
2, a higher score indicates a better model, while for other metrics, a higher score indicates a worse model. If negligible difference is enabled, the smaller number below indicates the probability that the difference between the pair is negligible. This test is based on the Bayesian interpretation of the
t-test. The results indicate that, for most performance measures, the stacked ensemble model is likely to outperform the other models.
To evaluate the contribution of each feature to the prediction of the stacked model, the sensitivity analysis can be conducted [
46]. Here, we utilized the permutation feature importance technique. In this method, the values of a specific feature are permuted, disrupting the relationship between that feature and the target variable. After the permutation, the change in the model’s performance metric is measured. A significant decrease in performance indicates that the feature was important to the model’s predictions, while a minimal change suggests that the feature had little influence.
Figure 3 presents the results of the feature importance analysis. The left side shows how the model’s R-squared value changes when each variable is permuted, while the right side illustrates the changes in the mean absolute percentage error (MAPE). The results consistently indicate that the variables gross floor area, approval date, transaction date, net floor area, number of buildings in a complex, and number of households in a complex are the most important, in that order. This may be because, in the case of Korean real estate, many properties tend to have similar external appearances and internal layouts. Therefore, once the area (gross floor area, net floor area) and construction year are determined, the appearance and structure of the property and complex are likely to be similar. The transaction date might reflect the economic conditions or real estate policies of that particular year. Subsequently, variables such as latitude and longitude, which reflect locational value, appear in the order of importance.
From a practical standpoint, understanding the conditions and factors that lead to inaccuracies is crucial for improving model reliability and performance. By identifying these sources of error, practitioners can make better decisions by appropriately determining when to use the models.
Figure 4 is a scatter diagram illustrating the percentage error by variable. Note that the percentage error (PE) is defined as the difference between the actual transaction price and the predicted value, divided by the actual transaction price. Through scatter diagrams, various insights can be obtained regarding percentage errors. Firstly, the errors show a similar distribution of overestimation and underestimation. Out of a total of 63,306 transactions, 33,015 cases had a percentage error of 0 or higher (underestimation), while 30,291 cases had a percentage error of less than 0 (overestimation). However, extreme predictions tend to occur with overestimations. In cases where the prediction differs from the actual transaction by more than 100%, no underestimations were observed (no cases with PE > 1). In contrast, there were 190 cases of overestimation (PE < −1). Secondly, the percentage error varies depending on the feature. The top left scatter diagram in
Figure 3 depicts the percentage error according to the number of households. As shown, extreme predictions tend to occur with overestimations and are more likely to happen when the number of households in a complex is small. This can be interpreted in several ways. When the number of households is small, the frequency of transactions is generally low, leading to insufficient data for accurate model training. Additionally, small household complexes may include both luxury villas and low-cost housing, which can result in diverse property characteristics being grouped into the same node after decision tree splitting. This, in turn, may lower prediction accuracy. Floor level can be similarly interpreted. Typically, high-rise apartments consist of many units, so properties located on higher floors are likely to be in complexes with a large number of households. Therefore, when appraising properties in apartment complexes with a small number of households, it is advisable to exercise caution in using the model for valuation. Conversely, when estimating the value of properties in complexes with a sufficient number of households, the model can provide relatively stable performance. This information is useful in determining the appropriate conditions for using the model, such as evaluating the value of specific types of properties. The right-hand diagram shows that the percentage error is relatively higher for low-cost, small properties. This indicates that the appraisal model is better suited for predicting properties above a certain size. Additionally, since the actual transaction price is used in the denominator of the percentage error calculation, the same error value has a greater impact on small, low-cost apartments.
To address the aforementioned issue, we used the decision tree method to analyze under which conditions specific features provide more accurate results or less reliable outcomes. The decision tree technique was particularly useful in this context because it allows us to explicitly observe the patterns where errors occur, rather than treating the model as a black box. We constructed a decision tree to classify cases where the absolute percentage error exceeds a certain threshold. For example, if the absolute percentage error is greater than 20%, the category variable is set to 1; otherwise (if the error is less than 20%), it is set to 0. The features used for this classification were the same as those used in the appraisal model, with the category variables serving as the outcome variable. The decision tree used to classify cases where the percentage error is 20% or higher is presented in
Figure 5. For ease of interpretation, the depth of the decision tree was limited to three.
Out of the total 63,306 transactions, 5613 transactions have a prediction error of 20% or higher (8.9%, the top node). However, the occurrence of errors varies depending on whether the apartment has an elevator. In cases where there is an elevator (right node), 2115 out of 41,735 transactions (5.1%) have errors of 20% or higher. In contrast, for apartments without an elevator, 3498 out of 21,571 transactions (16.2%) have errors of 20% or higher. Following the leftmost node down the decision tree reveals the conditions under which errors are most significant. When there is no elevator and the apartment building has two or fewer units, 2740 out of 13,651 transactions (20.1%) exhibit errors of 20% or higher. This is considerably higher than the overall rate of 8.9% for transactions with errors exceeding 20%. An interesting aspect of decision tree analysis is that the ranking of features identified as important by the permutation feature importance method does not always perfectly align with the features that appear at the top nodes of the tree. For instance, in
Figure 5, neither gross floor area nor net floor area is presented, which suggests that while the size of the property is important for price prediction, it does not significantly contribute to prediction errors once the area is accounted for. It is important to note that there are no missing values for gross floor area and net floor area. This indicates that the model performs well across properties of all sizes. However, analysis of the decision tree reveals that the size of the apartment complex (number of units or buildings) and the age of the building significantly impact predictive performance. Variables such as the number of apartment buildings in a complex, apartment parking spaces, and the number of households in a complex are all related to the size of the apartment complex. It can be observed that the smaller the apartment complex, the lower the prediction accuracy. At each branching point, the segments corresponding to smaller complexes show a higher proportion of errors exceeding 20% compared to other segments. Additionally, it was found that the older the approval date, the lower the predictive accuracy. Interestingly, the presence of an elevator encompasses both these attributes. In apartment complexes without elevators, the absence of an elevator may be due to the buildings being too old or the complex being small (comprising low-rise buildings). This analysis using the decision tree method provides valuable insights into when the appraisal model may yield higher errors, thereby informing better usage of the model.
Figure 6 presents the decision tree used to classify cases with a percentage error of 50% or higher. Similar to the previous example, we can analyze the decision tree to identify the conditions of features that lead to significant percentage errors. Out of the total 63,306 transactions, 1121 transactions have a prediction error of 50% or higher (1.8%, the top node). For transactions involving properties with an approval date prior to 21 August 1970, 978 out of 29,036 transactions exhibited a prediction percentage error exceeding 50%. In contrast, properties approved after 21 August 1970, showed only a 0.4% (143 out of 34,270) prediction percentage error in transactions. This indicates that properties with a more recent approval date tend to have relatively smaller prediction errors. Following the left node of the decision tree, for properties with fewer units (fewer than 90 units in the apartment complex), 4.6% of the transactions (738 out of 16,072) had an absolute percentage error exceeding 50%. On the other hand, following the right node, for properties approved after 21 August 1970, and equipped with elevators, only 94 out of 30,789 transactions (0.3%) had an absolute percentage error exceeding 50%.
This decision tree model provides valuable insights into when predictive techniques can be reliably applied. While efforts to improve prediction accuracy are always necessary, from a practical perspective, if the prediction accuracy is low under certain conditions, it may be difficult to trust these predictive techniques. In such cases, additional analysis through on-site inspections by experts or alternative methods could be warranted. Conversely, if the prediction’s results are sufficiently reliable under specific conditions, an automated valuation model could be applied. For example, among properties approved after 21 August 1970, with elevators and more than 19 units, only 53 out of 23,981 transactions (0.2%) exhibited a percentage prediction error exceeding 50%. Under these conditions, it could be reasonably concluded that an automated valuation model would be sufficiently trustworthy.
While this approach allows practitioners to identify scenarios where an automated model can be trusted, thereby streamlining the appraisal process and reducing the need for costly manual assessments, it is important to recognize the limitations of automated models, particularly in situations where data is scarce or where complex, non-quantifiable factors play a significant role in property valuation. Observing our decision tree model, we notice that significant errors tend to occur under conditions of limited data. For instance, in the lower nodes of
Figure 6, the total number of transactions associated with certain conditions is limited, resulting in larger errors. This suggests that insufficient data may hinder the model’s ability to learn effectively, thereby reducing prediction accuracy. In such cases, automated models may need to be supplemented with expert judgment or alternative methodologies to ensure a comprehensive and accurate assessment.
5. Conclusions
Based on the analysis presented in this study, it can be concluded that the application of a stacked ensemble model can significantly enhances the predictive accuracy of automated real estate valuation models. The research demonstrates that by integrating various machine learning algorithms through the stacking approach, with ridge regression as the meta-learner, it is possible to achieve superior performance across multiple metrics, including RMSE, MAE, MAPE, R2, and CVRMSE. This ensemble model not only minimizes prediction errors but also ensures greater stability and reliability, as evidenced by the lower variance in both absolute errors and absolute percentage errors. The advantage of the stacking method is its ability to integrate a diverse set of models, effectively leveraging the strengths of various algorithms to identify a broader range of patterns and relationships within the data. This diversity enhances the ensemble model’s ability to generalize to new data. From a practical perspective, this advantage can simplify model selection by assessing the performance of individual algorithms based on the characteristics of the data. When applied to real estate price estimation, the model automatically chooses the most appropriate techniques, assigning greater weight to algorithms that perform best for the given data set. This approach eliminates the need to search for a single best predictive model, as the stacked ensemble model naturally prioritizes the most effective models and reduces the influence of those that perform poorly on certain data points. Consequently, this method allows practitioners to confidently use a variety of machine learning models, thereby reducing uncertainty and improving overall predictive performance.
We also proposed a decision tree analysis method to identify the conditions under which certain features lower prediction accuracy. By employing a decision tree technique to classify instances where the absolute percentage error exceeds a defined threshold, we can determine the specific scenarios in which the model’s predictive accuracy diminishes. The analysis revealed that the size and age of an apartment are crucial factors, with smaller and older complexes showing reduced accuracy and higher error rates. This finding provides valuable insights into practical considerations when applying the model in real-world scenarios. For instance, the case study suggests that additional efforts may be required for more accurate valuations in apartments with fewer units or those that are older. Conversely, apartments with a sufficient number of units or those that are less aged may yield more accurate predictions than the performance metrics discussed in the paper.
Our research can be extended in several directions. First, the incorporation of a broader range of machine learning techniques could enhance the construction of ensemble models. In this study, the majority of predictors used were decision tree-based techniques. The predictive power of such a model might improve by integrating algorithms grounded in diverse principles. For example, incorporating an artificial neural network could be a promising approach. As our data contains a significant number of missing values, often due to the difficulty in collecting certain information when constructing features for real estate appraisal models, one potential direction for future research would be to investigate various methods for handling missing data. This research could explore how different imputation techniques impact the predictive performance of the models. By examining the effectiveness of these techniques in dealing with missing data, we can enhance the robustness and accuracy of real estate appraisal models in situations where complete data collection is challenging. Additionally, exploring various methods of combining predictors and applying the stacking method alongside the decision tree technique for identifying error patterns to different regions presents another potential direction for future research. Another direction for future research is the development of a reliable housing price index using machine learning techniques. In [
47], a price index for free-market transactions of mixed-use income properties was developed to analyze the impact of speculative bubbles on property prices. By assuming the existence of a speculative bubble during the property boom of 1985–1990 and subsequently removing this hypothetical bubble from the price index, the study demonstrated that the proportion of explainable price variation increased when the bubble term was excluded. A similar methodology could be employed to analyze the effects of various economic conditions on the real estate market. In [
48], it is noted that the performance of traditional valuation methods can decline under certain economic conditions. Enhancing the model’s performance by incorporating variables that better capture these economic conditions could be a promising direction for future research. Lastly, as proposed in [
49], research could be extended to focus on making appropriate investment decisions from the perspective of investors and estimating the associated risks.