Optimal Parameter Selection and Indicator Design for Technical Analysis Strategies by Computer Software: An Empirical Analysis of the Taiwan Futures Market ^†

Abstract

In algorithmic trading, “overfitting” often arises during parameter optimization. To avoid scenarios where in-sample performance is excellent but out-of-sample performance fails to meet expectations, appropriate parameter combinations are crucial for enhancing the robustness of a strategy. To find the appropriate parameter combinations, we identified “parameter plateaus” in the three-dimensional space generated by strategy performance metrics. These plateaus represent parameter combinations where the surrounding performance metrics are relatively similar, reducing the risk of drastic performance drops. Utilizing the four parameter selection methods designed in this study (weighted selection, Standard Deviation Selection, Island Area Selection, and Island Volume Selection), we selected parameter combinations in-sample and validated them out-of-sample, complemented by “rolling window analysis” for long-term profitability stability. We used historical backtesting data from the Taiwan Stock Index Futures, covering the period from 1 January 2000 to 31 December 2022. The data were paired with trading strategies developed based on the moving average technical indicator. Through the four parameter selection methods and the system backtesting approach using rolling windows, we identified parameter combinations in-sample and then validated them out-of-sample. The results showed that the performance metrics improved by more than 50% over those generated using traditional optimal point selection methods, demonstrating the superiority of the parameter selection methods proposed in this study.

1. Introduction

In algorithmic trading, when optimizing parameters for trading strategies, it is common to divide historical price data into two sets: in-sample and out-of-sample. The performance of a strategy applied to trading conditions often exhibits a phenomenon where the in-sample performance looks impressive, but the out-of-sample performance is significantly worse. This situation is referred to as “overfitting” in parameter optimization. Overfitting occurs due to various reasons such as excessive parameter optimization, insufficient sample size, or instability in the performance of parameter combinations.

To avoid overfitting in strategy development, it is crucial to choose appropriate parameter combinations. In a three-dimensional space created by parameter combinations and their performance, the performance of that combination differs significantly from those around it if the selected parameter combination is isolated. This leads to unstable profitability in real trading. Therefore, parameter plateaus or stability regions through appropriate parameter selection methods must be found.

There has been a lack of research proposing an objective strategy parameter selection method. Most studies have relied on systematic backtesting to determine the appropriate strategies for a particular asset. For example, Yan (2017) [1], Cheng (2018) [2], Lee (2019) [3], and Lin (2020) [4] have used plateau search algorithms to identify parameter plateaus and assess the strategy’s applicability to assets. To address the issue of overfitting in parameter optimization, we proposed to use four parameter selection methods: “Weighted Parameter Combination Selection”, “Standard Deviation Parameter Combination Selection”, “Island Area Parameter Combination Selection”, and “Island Volume Parameter Combination Selection”. The purpose of using these methods is to avoid parameter isolation and identify parameter combinations within the in-sample dataset that belong to parameter plateaus. Each of these parameter selection methods is designed to cater to different characteristics of strategies and assets, aiming to find suitable methods for specific scenarios.

2. Literature Review

2.1. Parameter Optimization

In a strategy model, overfitting refers to the phenomenon where the parameters used in the strategy excessively memorize historical data, resulting in the poor generalization ability of the strategy [5]. In representing the performance indicators and parameters as a three-dimensional graph, high performance may be observed in certain parameter combinations where the performance significantly deviates from the surrounding parameters, resembling an isolated island. This is not desirable for a good strategy because, in real trading, the information about the traded asset is not historical but continuously updated. Therefore, a robust strategy must be used to avoid overfitting and strive to find stable parameter combinations (similar to plateaus) though they may not have the best performance and are not significantly different from neighboring parameter combinations [6]. To prevent the issue of overfitting, it is essential to avoid finding parameter combinations that form parameter islands and strive to discover what are known as parameter plateaus. At this point, it is challenging to define these parameter plateaus. In earlier research, the majority relied on visual judgment, meaning they subjectively determined whether a parameter combination constituted a plateau. However, Lee (2015) [7] introduced the concept of using the Regional Growth Algorithm to identify parameter plateaus. Subsequently, researchers such as Yan (2017) [1], Cheng (2018) [2], Lee (2019) [3], and Lin (2020) [4] also applied this algorithm to various technical indicators and assets. In this study, we used four parameter selection methods based on the literature mentioned above.

2.2. Walk Forward Analysis (Moving Window)

In validating the robustness of a trading strategy, it is common to split the backtesting period into in-sample and out-of-sample data sets based on a certain proportion. Assuming that the out-of-sample data represent unknown real market data, the optimization of parameter combinations is performed using historical data within the in-sample period. These optimized parameters are then tested on the out-of-sample data, simulating the performance of using the optimized parameter combination in a real market scenario.

The concept of “Walk Forward Analysis” was initially introduced by Robert Pardo [8]. This method involves dividing the backtesting period into multiple windows to examine how the optimized parameter combinations from the in-sample data perform in the out-of-sample period. If the strategy continues to yield profits in the out-of-sample period and consistently across different windows, the strategy possesses robustness. Conversely, if the strategy fails to perform well in the out-of-sample period, the strategy lacks robustness. To measure the robustness of a strategy in Walk Forward Analysis, the Walk Forward Efficiency (WFE) indicator is often used. This indicator is used to compare the performance of in-sample and out-of-sample data for each window by dividing the out-of-sample data by the in-sample data [6,8]. Through Walk Forward Analysis, it is assessed whether a strategy model is susceptible to overfitting. This method improves the strategy model or changes in parameter selection methods to avoid overfitting of strategy parameters [6].

3. Research Methodology

In this study, Taiwan Stock Index Futures (FITXN) was used as the trading asset, and a moving average-based trading strategy [9] was employed. Python’s BackTrader external library was used for initial backtesting, while subsequent parameter search and strategy analysis were automated using Python. The BackTrader package in Python was used for optimization and backtesting. It generates daily profit and loss performance reports for all parameter combinations. Subsequently, Python was used for data processing and analysis, setting the in-sample and out-of-sample window sizes. Three performance indicators were evaluated using the four parameter selection methods designed in this study: “Weighted Parameter Combination Selection”, “Standard Deviation Parameter Combination Selection”, “Island Area Parameter Combination Selection”, and “Island Volume Parameter Combination Selection”. It was determined whether these methods outperform the “Single-Point Parameter Combination Selection” method in terms of out-of-sample performance. This assessment was used to determine the suitability of each parameter selection method for different combinations of assets and strategies.

• Trading Backtest Asset and Data Source: The asset used for trading backtesting in this study was the Taiwan Stock Index Futures (FITXN), with data sourced from XQ Cai Pan Gao Shou’s near-month futures daily data. The data from 1 January 2000 to 31 December 2021 comprised 2703 data points.
• Trading Strategy Design (Moving Average Trading Strategy)

  (1)

  Indicator Introduction: The moving average is the arithmetic mean of the closing price over ‘n’ days.

  (2)

  Name Definitions: Two moving averages are defined, namely MA1 and MA2.

  (3)

  Entry Signals

  • Long Entry: When MA1 crosses above MA2 from below, and the position is flat (0), enter long at the next day’s opening price.
  • Short Entry: When MA1 crosses below MA2 from above, and the position is flat (0), enter short at the next day’s opening price.

• Exit Signals
  • Long Exit: When MA1 crosses below MA2 from above, and the position is long (1), exit at the next day’s opening price.
  • Short Exit: When MA1 crosses above MA2 from below, and the position is short (−1), exit at the next day’s opening price.
  • Indicator Parameters: MA1 and MA2 have lengths ranging from 1 to 60 with an interval of 1, resulting in 3600 parameter combinations.
• Trading Basic Settings: The trading size for both entry and exit was fixed at 1 contract. The transaction cost was set at 50 units with a contract value of 200 units per point. The trading margin was 184,000 units.
• Definition of Trading Performance Metrics

  (1)

  Net Profit: The total return is generated by the trading strategy over the entire backtesting period.

  $$\mathrm{Net}\mathrm{profit}=\mathrm{Gross}\mathrm{profit}-\mathrm{Gross}\mathrm{loss},$$

  (1)

  (2)

  Maxdrawdown (MDD): The MDD is calculated as a risk of the trading strategy over the entire backtesting period.

  $$\mathrm{m}\mathrm{a}\mathrm{x}({DD}_1,{DD}_n),$$

  (2)

  (3)

  Minimum Acceptable Return (MAR): The MAR index is a performance metric used to measure the risk-adjusted return of a trading strategy. It can be interpreted as the average annual return per unit of maximum risk. The formula for MAR includes a simple interest formula and a compound interest formula. If the number of contracts traded varies with the size of the capital, the compound interest formula is used. In this study, the number of contracts traded was fixed, so the simple interest formula is employed as (3).

  $$\text{MAR}=(\mathrm{Net}\mathrm{Profit}/\mathrm{Number}\mathrm{of}\mathrm{Years})/\text{MDD},$$

  (3)

  (4)

  Walk Forward Efficiency (WFE): It is the annualized out-of-sample net profit divided by the annualized in-sample net profit, and this performance metric is used to measure the robustness of the trading system.

  $${\displaystyle \frac{(\mathrm{O}\mathrm{u}\mathrm{t}\text{-}\mathrm{o}\mathrm{f}\text{-}\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{N}\mathrm{e}\mathrm{t}\mathrm{P}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{i}\mathrm{t}-\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{Y}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{s})}{(\mathrm{I}\mathrm{n}\text{-}\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{N}\mathrm{e}\mathrm{t}\mathrm{P}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{i}\mathrm{t}-\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{Y}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{s})}},$$

  (4)

Figure 1 depicts a three-dimensional plot with the x-axis representing Parameter 1, the y-axis representing Parameter 2, and the z-axis representing the performance metric net profit. In order to avoid finding parameter combinations that may lead to performance peaks (where neighboring parameter combinations may not perform well, resembling isolated islands), parameter combinations that may not have the highest central performance but are robust (where neighboring parameter combinations also perform well, resembling plateaus) need to be identified. We used the four parameter selection methods to address this issue.

In constructing the three-dimensional plot of parameters, we used net profit as the z-axis and two other performance metrics (MDD and MAR) separately to construct three-dimensional plots. We determined which performance metrics resulted in the most robust performance when combined with one of the parameter selection methods. If net profit is chosen as the filtering criterion, higher values are better. However, if the filtering criterion is MDD, since MDD is a risk indicator, lower values are preferable. If the chosen filtering criterion is MAR, higher values indicate better performance because MAR is the result of net profit divided by MDD.

The following is an explanation of the four parameter selection methods designed in this study.

3.1. Weighted Parameter Combination Selection Method

By taking a weighted average of the performance of each parameter combination and its neighboring parameter combinations, where neighboring parameter combinations are defined as those within one to three circles of expansion. The design of the weighting ratios adopts the concept of “inverse distance weighting”, using the reciprocal of the distance between parameter combinations as weights (Equation (5)). The weight of the central parameter combination is 1. After weighting, parameter combinations that are farther away have a smaller influence.

$$w_i={\displaystyle \frac{1}{\sqrt{{(x_i-x_{center})}^2+{(y_i-y_{center})}^2}}},$$

(5)

$w_i$: Weight of parameter combination $i$

$x_i,y_i$: Position of parameter combination $i$

$x_{center},y_{center}$: Position of the central parameter combination

$x_i\ne x_{center}and{y}_i\ne y_{center}$

If $x_i$, $y_i$ are the central points, the formula’s result would be infinite; in this study, the weight for this case is set to 1.

3.2. Standard Deviation Parameter Selection Method

The Standard Deviation Parameter Selection Method is used to identify parameter combinations within the in-sample dataset that exhibit relatively stable performance compared to their neighboring parameter combinations. This method begins by filtering out parameter combinations for each period in the in-sample dataset where the net profit is greater than the average net profit of all parameter combinations. It then calculates the standard deviation for each parameter combination concerning its neighboring parameter combinations. The standard deviation indicates the level of variation between the parameter combination and its neighbors; a higher standard deviation implies greater variation, while a lower standard deviation implies less variation. Finally, the method selects the parameter combination with the lowest standard deviation from all parameter combinations as the recommended parameter and evaluates its performance on the out-of-sample data. The formula for standard deviation is given by Equation (6).

$${\sigma }_i=\sqrt{{\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{({Ind}_i-\overline{Ind})}^2},$$

(6)

${\sigma }_i$: The standard deviation of the performance (net profit, MDD, MAR) of parameter combination $i$ and its neighbors within two circles.

${Ind}_i$: The performance value of parameter combination $i$ (net profit, MDD, MAR).

$\overline{Ind}$: The mean of parameter combination $i$ and its neighboring parameter combinations.

$N$: The number of parameter combinations $i$ and its neighbors.

3.3. Island Area Parameter Selection Method

This method can be likened to comparing benchmark performance to sea level as illustrated in Figure 2. It examines the areas occupied by different islands located at sea level and selects the parameter point with the best performance from the largest island as the parameter choice. Regarding benchmark performance, we arranged the performance of all parameter combinations within the in-sample dataset from smallest to largest for each period. Subsequently, benchmark performance was determined by percentile ranks (PR). In this study, PR values were set at 90, 95, and 99.

3.4. Island Volume Parameter Selection Method

As shown in Figure 2, the approach involves examining the volumes (sum of performance) occupied by different islands located at sea level and selecting the parameter point with the best performance from the largest island as the parameter choice. Regarding benchmark performance, we arranged the performance of all parameter combinations within the in-sample dataset from smallest to largest for each period. Subsequently, benchmark performance was determined by percentile ranks (PR). As previously mentioned, PR values in this method were set at 90, 95, and 99.

4. Results

4.1. Weighted Parameter Selection Method

Table 1. Out-of-sample performance for the weighted parameter selection method.

Window Proportion	Filtering Indicator		NetProfit				MDD				MAR
		Circles	0	1	2	3	0	1	2	3	0	1	2	3
	Benchmark
11	NetProfit		68.16	62.13	58.32	−14.94	−2.24	−18.3	88.94	71.36	107.83	14	43.58	−7.98
	MDD		99.46	130.45	86.38	123.92	73.56	60.64	50.36	38.27	49.38	111.32	78.46	107.58
	MAR		0.033	0.023	0.032	−0.006	−0.001	−0.014	0.084	0.089	0.104	0.006	0.026	−0.004
21	NetProfit		10.8	−21.38	−43.62	−18.47	−34.23	−0.39	18.8	−16.38	−36.84	−76.8	−49.38	−18.4
	MDD		78.79	100.26	74.02	77.3	62.98	70.51	85.26	65.32	76.9	104.44	86.19	73.88
	MAR		0.007	−0.011	−0.029	−0.012	−0.027	0	0.011	−0.013	−0.024	−0.037	−0.029	−0.012
31	NetProfit		26.64	101.4	47.18	−0.98	16.34	−27.8	−34.82	−44.42	−93.81	5.46	75.72	−11.1
	MDD		153.29	84.54	96.09	104.72	48.33	76.46	102.55	98.2	119.66	63.26	70.25	90.55
	MAR		0.009	0.063	0.026	0	0.018	−0.019	−0.018	−0.024	−0.041	0.005	0.057	−0.006
41	NetProfit		10.89	168.6	165.46	81.19	−31.14	−15.58	11.85	47.13	−34.9	81.3	85.59	99.68
	MDD		154.26	70.96	74.78	46.14	51.23	75.88	37.9	40.04	70.92	66.32	46.14	46.14
	MAR		0.004	0.132	0.123	0.098	−0.034	−0.011	0.017	0.065	−0.027	0.068	0.103	0.12
51	NetProfit		133.34	217.8	183.62	165.8	−14.23	−18.96	25.02	42.59	−102.56	126.7	193.66	98.06
	MDD		112.62	73.17	57.29	61.82	54.38	34.36	36.8	30.2	159.68	64.51	59.84	62.5
	MAR		0.07	0.175	0.189	0.158	−0.015	−0.032	0.04	0.083	−0.038	0.116	0.19

presents the out-of-sample performance for the Weighted Parameter Selection Method. When filtering by net profit, 28 performance metrics outperformed the Best Point Selection Method, accounting for 62% of the total. Among these, one was statistically significant, and none were statistically inferior. When filtering by MDD, 28 performance metrics outperformed the Best Point Selection Method, making up 62% of the total. One was statistically significant, and none were statistically inferior. When filtering by MAR, 30 performance metrics outperformed the Best Point Selection Method, accounting for 67% of the total. Among these, seven were statistically significant, and two were statistically inferior.

Table 2. Comparison of in-sample and out-sample performance for the weighted selection method.

Window Proportion	Filtering Indicator		NetProfit				MDD				MAR
		PR	0	1	2	3	0	1	2	3	0	1	2	3
	Benchmark
11	ISNetProfit		1080.6	902.75	862.75	778.66	353.4	261.16	240.64	234.4	738.98	727.06	786.62	707.7
	WFE		0.063	0.069	0.068	−0.019	−0.006	−0.07	0.37	0.304	0.146	0.019	0.055	−0.011
21	ISNetProfit		1454.39	1184.47	1114.48	983.08	436.78	410.06	412.34	422.86	1261.32	1088.49	1032.11	1023.56
	WFE		0.014	−0.036	−0.078	−0.038	−0.156	−0.002	0.092	−0.078	−0.058	−0.142	−0.096	−0.036
31	ISNetProfit		1623.42	1421	1305.52	1193.8	550.28	488.21	473.56	475.6	1492.98	1207.53	1248.66	1035.88
	WFE		0.048	0.213	0.108	−0.003	0.09	−0.171	−0.222	−0.279	−0.189	0.015	0.183	−0.033
41	ISNetProfit		1819.73	1595.12	1420.3	1285.49	694.26	550.04	460.5	453.86	1538.33	1372.39	1266.86	1156.65
	WFE		0.024	0.424	0.464	0.252	−0.18	−0.112	0.104	0.416	−0.092	0.236	0.272	0.344
51	ISNetProfit		1989.32	1806.24	1611.88	1531.26	777.12	491.86	460.86	458.26	1799.66	1726.03	1689.72	1599.44
	WFE		0.335	0.605	0.57	0.54	−0.09	−0.195	0.27	0.465	−0.285	0.365	0.575	0.305

presents a comparison of in-sample and out-of-sample performance for the weighted selection method. When using net profit as the selection criterion, 10 WFE values were higher using the Best Point Selection Method, accounting for 67% of the total. When using MDD as the selection criterion, 10 WFE values were better than using the Best Point Selection Method, also accounting for 67% of the total. Finally, when using MAR as the selection criterion, 10 WFE values were better than using the Best Point Selection Method, constituting 67% of the total. As shown in Table 1 and Table 2, the performance indicator ISNetProfit was measured in ten thousand units, and the shaded areas indicated that the performance of this indicator is superior to the “Best Point Parameter Selection Method”.

4.2. Standard Deviation Selection Method

The out-of-sample performance of the Standard Deviation Selection Method, using net profit as the selection criterion, showed that 34 performance indicators outperformed the Best Point Selection Method, accounting for 76% of the total. Among these, 11 indicators exhibited statistically significant outperformance, while none showed significant underperformance. When MDD was used as the selection criterion, 19 performance indicators outperformed the Best Point Selection Method, making up 42% of the total. Only one of these indicators was statistically significantly better, while five indicators were significantly worse. When using MAR as the selection criterion, 38 performance indicators with the Standard Deviation Selection Method were better, representing 84% of the total. Out of the 38 indicators, 12 indicators exhibited statistically significant outperformance, and 3 showed significant underperformance. Therefore, when comparing in-sample and out-of-sample performance for the Standard Deviation Selection Method, 12 WFE values that the Best Point Selection Method based on net profit, constituting 80% of the total. For MDD-based selection, seven WFE values were better than the Best Point Selection Method, accounting for 47%. Finally, when selecting based on MAR, 14 WFE values outperformed the Best Point Selection Method, making up 93% of the total.

4.3. Island Area Selection Method

The out-of-sample performance for the Island Area Selection Method with net profit as the filtering criterion showed that 30 performance indicators outperformed the Best Point Selection Method, accounting for 67% of the total. Two were statistically significant improvements, and none exhibited significant deterioration. When using MDD as the filtering criterion, 19 performance indicators outperformed the Best Point Selection Method, constituting 42% of the total. Two showed statistically significant improvements, while one exhibited significant deterioration. With MAR as the filtering criterion, 38 performance indicators outperformed the Best Point Selection Method, representing 84% of the total. Among these, eight exhibited statistically significant improvements, and none showed significant deterioration. Furthermore, in the comparison of in-sample and out-of-sample performance for the Island Area Selection Method, 10 WFE outperformed the Best Point Selection Method when using net profit as the filtering criterion, accounting for 67% of the total. When using MDD as the filtering criterion, 12 WFE outperformed the Best Point Selection Method, constituting 80% of the total. Lastly, when using MAR as the filtering criterion, 10 WFE outperformed the Best Point Selection Method, representing 67% of the total.

4.4. Island Volume Parameter Selection Method

The out-of-sample performance for the Island Volume Selection Method with net profit as the filtering criterion showed that 32 performance indicators outperformed the Best Point Selection Method, accounting for 71% of the total. Among them, four were statistically significant improvements, and one exhibited significant deterioration. When using MDD as the filtering criterion, 24 performance indicators outperformed the Best Point Selection Method, constituting 53% of the total. One showed statistically significant improvements, and none exhibited significant deterioration. With MAR as the filtering criterion, 30 performance indicators outperformed the Best Point Selection Method, representing 67% of the total. Nine exhibited statistically significant improvements, and one showed significant deterioration.

When comparing in-sample and out-of-sample performance for the Island Volume Selection Method, 11 WFE outperformed the Best Point Selection Method, accounting for 73% of the total when using net profit as the filtering criterion. When using MDD as the filtering criterion, six WFE outperformed the Best Point Selection Method, constituting 40% of the total. Lastly, when using MAR as the filtering criterion, 10 WFE outperformed the Best Point Selection Method, representing 67% of the total.

5. Conclusions

We proposed using the Selection Method, Standard Deviation Selection Method, Island Area Selection Method, and Island Volume Selection Method in the Walk-Forward system backtesting approach. These methods identified appropriate parameter combinations within the in-sample data. These combinations were validated for the out-of-sample data using a trading strategy constructed based on the moving average indicator and applied to Taiwan Stock Index Futures from 1 January 2000 to 31 December 2022. The performance of the proposed parameter selection methods was compared with that of the traditional single-point selection method. Furthermore, the relationship between various parameters within each selection method (window ratio, extension cycles, or benchmark PR) and the MAR performance indicator was analyzed.

The performance indicators of the proposed four parameter selection methods exceeded those of the traditional single-point selection method by more than 50%. Therefore, compared to the traditional single-point selection method, the parameter selection methods proposed are effective in selecting robust out-of-sample strategy parameter combinations. Based on the relationship between the various parameters (window ratio, extension cycles, or benchmark PR) of each selection method and the MAR performance indicator, it was observed that superior performance tended to cluster around specific windows or extension cycles when combined with specific parameter selection methods. This phenomenon suggests that users can consider appropriately expanding the parameters (window ratio, extension cycles, or benchmark PR) of the selection methods in future research.