Forecasting Stock Market Dynamics using Market Cap Time Series of Firms and Fluctuating Selection ^†

Abstract

Evolutionary economics has been instrumental in explaining the nature of innovation processes and providing valuable heuristics for applied research. However, quantitative tests in this field remain scarce. A significant challenge is accurately estimating the fitness of companies. We propose the estimation of the financial fitness of a company by its market capitalization (MC) time series using Malthusian fitness and the selection equation of evolutionary biology. This definition of fitness implies that all companies, regardless of their industry, compete for investors’ money through their stocks. The resulting fluctuating selection from market capitalization (FSMC) formula allows forecasting companies’ shares of total MC through this selection equation. We validate the method using the daily MC of public-owned Fortune 100 companies over the period 2000–2021.

1. Introduction

Evolutionary economics is a branch of economics that applies principles from evolutionary biology to understand economic systems and their dynamics [1]. It focuses on the processes of variation, selection, and adaptation within economic systems, viewing them as evolving and dynamic entities. Evolutionary economists emphasize the roles of innovation, technological change, institutional evolution, and learning mechanisms in shaping economic behavior and outcomes over time [2]. This approach challenges traditional neoclassical economics by highlighting the importance of historical context, path dependence, and non-equilibrium dynamics in economic analysis. Evolutionary economics has provided valuable insights for studying significant qualitative issues. For instance, it was used to explore how complex socio-economic interactions shape evolving preferences and habit formation [3], the types of institutional structures that can best support evolutionary change [3,4], and the conditions necessary for economic activities to promote long-term economic prosperity [5]. Evolutionary economics embraces the biological notion of fitness, which reflects the relative competitiveness of a company compared to other companies and, in turn, determines its probability of growth and survival.

A major challenge has been estimating this fitness. Fitness is often discussed synonymously with production-related performance, i.e., assessing how efficiently a company manages its resources, processes, and production activities to deliver goods or services. This canonical standpoint relates to firms producing a homogeneous good but with different costs, determining their fitness in competing for market shares [6,7]. The shares of each firm’s product in the total output of the competitor population are used to describe the population structure at each time. As the market evolves, the market shares of less fit firms decrease, while companies with greater fitness capture larger market shares [8]. However, using market shares to assess the competitive position of firms poses different problems [9], particularly in rapidly evolving industries and technology-driven markets [10]. In fact, market shares can sometimes be negatively related to profitability [11,12]. Moreover, market shares do not always provide a complete or accurate picture of a company’s performance [13].

In this article, to fill this gap, we present a new deterministic selection-driven dynamic method [6,7] to model the evolutionary dynamics of companies with fluctuating fitnesses estimated from empirical time-series data. The method is based on an alternative way to measure a company’s fitness that focuses on financial performance rather than traditional industrial performance, aligning more closely with the Malthusian fitness concept used in biology [14,15]. This new fitness definition, inspired by ecology and evolution, is based on market capitalization. We validate the method using the daily market capitalization data of public Fortune 100 companies from 2000 to 2021 [16]. Our results indicate that the fluctuating selection from market caps (FSMC) method produces reasonably accurate forecasts of the proportions or shares of market capitalization among companies.

2. Materials and Methods

2.1. Data

The dataset we use here is the same as in [17], and is based on the Fortune 100 list, which includes the top 100 companies by revenue, both public and privately held, in the United States, as published by Fortune magazine [16]. From these 100 companies, we selected the 78 publicly owned companies that reported annual revenue and market capitalization from 1 January 2000 to 31 December 2021 (see

Table 1. The set of 78 companies considered in this study ordered by their market value, as of 31 December 2021.

Company	Ticker	Market Cap. (USD Bill)	Rank	Industry
Apple	AAPL	2902	1	Consumer Electronics
Microsoft	MSFT	2522	2	Software–Infrastructure
Amazon	AMZN	1697	3	Internet Retail
Berkshire Hathaway	BRK	662.63	4	Insurance
JP Morgan	JPM	472.51	5	Banks
United Health Group	UNH	466.21	6	Healthcare Plans
Johnson & Johnson	JNJ	450.36	7	Drug Manufacturers
Home Depot	HD	433.37	8	Home Retail
Walmart	WMT	401.35	9	Discount Stores
P&G	PG	392.11	10	Household
Bank of America	BAC	359.38	11	Banks
Pfizer Inc.	PFE	331.86	12	Drug Manufacturers
The Walt Disney Company	DIS	281.54	13	Entertainment
Cisco Systems, Inc.	CSCO	267.27	14	Comm. Equipment
Nike	NKE	263.55	15	Footwear and Access.
Thermo Fisher Scientific Inc.	TMO	263.18	16	Diagnosis and Research
Exxon Mobil	XOM	259.38	17	Oil and Gas
The Coca-Cola Company	KO	256.09	18	Beverages
Costco	COST	251.74	19	Discount Stores
Abbott Laboratories	ABT	248.28	20	Medical Devices
PepsiCo, Inc.	PEP	240.24	21	Beverages
Oracle	ORCL	232.89	22	Software–Infrastructure
Comcast	CMCSA	228.16	23	Telecom Services
Chevron	CVX	226.46	24	Oil and Gas
Verizon	VZ	218.12	25	Telecom Services
Intel Corporation	INTC	209.6	26	Semiconductors
QUALCOMM Incorporated	QCOM	205.73	27	Semiconductors
Merck & Co., Inc.	MRK	193.72	28	Drug Manufacturers
Wells Fargo	WFC	186.44	29	Banks
Anthem	UPS	186.41	30	Integrated Freight and Logistics
Lowe’s	LOW	174.15	31	Home Retail
Morgan Stanley	MS	173.96	32	Banks
Honeywell International Inc.	HON	142.79	33	Conglomerates
CVS Caremark	CVS	136.38	34	Healthcare Plans
Bristol-Myers Squibb Company	BMY	134.24	35	Drug Manufacturers
AT&T	T	132.58	36	Telecom Services
Raytheon Technologies Corp.	RTX	128.51	37	Aerospace and Defense
The Goldman Sachs Group, Inc.	GS	127.61	38	Banks
American Express Company	AXP	124.5	39	Credit Services
IBM	IBM	120.04	40	Information Tech. Serv.
Citigroup	C	119.84	41	Banks
Boeing	BA	118.56	42	Aerospace and Defense
Target	TGT	110.89	43	Discount Stores
Caterpillar Inc.	CAT	110.79	44	Farm and Heavy Constr.
Deere & Company	DE	105.68	45	Farm and Heavy Constr.
General electrics	GE	103.83	46	Specialty Industry Machinery
3M Company	MMM	101.58	47	Conglomerates
Lockheed Martin Corporation	LMT	96.32	48	Aerospace and Defense
ConocoPhillips	COP	94.01	49	Oil and Gas
Phillips 66	TJX	90.56	50	Oil and Gas
Ford Motors	F	85.59	51	Auto Manufacturers
Cigna Corporation	CI	74.16	52	Healthcare Plans
FedEx Corporation	FDX	68.53	53	Integrated Freight and Logistics
Northrop Grumman Corp.	NOC	60.49	54	Aerospace and Defense
Capital One Financial Corp.	COF	60.05	55	Credit Services
The Progressive Corporation	PGR	59.99	56	Insurance
Humana Inc.	HUM	59.75	57	Healthcare Plans
General Dynamics	GD	57.88	58	Aerospace and Defense
Enterprise Products Partners L.P.	EPD	47.79	59	Oil and Gas
AIG	AIG	47.21	60	Insurance
Walgreens Boots Alliance	WBA	45.03	61	Pharmaceutical Retailers
HP Inc.	HPQ	40.79	62	Computer Hardware
Exelon Corporation	EXC	40.34	63	Utilities-Regulated Electrics
Sysco Corporation	SYY	40.27	64	Food Distribution
Archer-Daniels-Midland Comp.	ADM	37.85	65	Farm Products
The Travelers Companies, Inc.	TRV	37.73	66	Insurance
McKesson Corp.	MCK	37.24	67	Medical Distribution
The Kroger Co.	KR	33.28	68	Grocery Stores
The Allstate Corporation	ALL	33.06	69	Insurance
Tyson Foods, Inc.	TSN	31.65	70	Farm Products
Nucor Corporation	NUE	31.1	71	Steel
Valero Energy	VLO	30.73	72	Oil and Gas
AmerisourceBergen	ABC	27.78	73	Medical Distribution
Best Buy Co., Inc.	BBY	24.44	74	Specialty Retail
Cardinal Health	CAH	14.26	75	Medical Distribution
Arrow Electronics, Inc.	ARW	9.14	76	Electronics Distribution
Fannie Mae	FNMA	0.95	77	Mortgage Finance
Chico’s FAS, Inc.	CHS	0.66	78	Apparel Retail

). The resulting dataset consists of the time series of daily closing market values for each company i, v_i(t), with t measured in days, for these 78 companies spanning T = 5536 days [18].

2.2. Modeling

2.2.1. Fluctuating Selection from Market Caps (FSMC) Method

As we previously pointed out, selection involves considering the differential rates of expansion (that is, the ‘fitness’) among the competing, interacting members of a population. To estimate the fitness of firms, we start with the market capitalization of each company i at time t, as denoted by v_i(t). This quantity plays the role played by the biomass or number of individuals of a genotype or phenotype in biology [15,19]. Consequently, the proportion or share of total market cap of each company (analogous to the frequency of a genotype or phenotype in a population) is given by the following:

$$x_i(t)\equiv \frac{v_i(t)}{{\displaystyle \sum _{j=1}^N{v}_j(t)}},\hspace{1em}i=1,2,\dots ,N.$$

(1)

Note that, in Equation (1), the number of companies is denoted by N, and the same is true for all the equations of this section for the generality of presentation. In the Section 3, we always take N = 78.

The Malthusian fitness, f_i, of each company, which is identical to its growth rate, is thus defined as

$$f_i(t)\equiv \frac{\frac{\partial v_i}{\partial t}}{v_i},\hspace{1em}i=1, 2, \dots , N$$

(2)

Incidentally, note that the fitnesses, as defined by Equation (2), is in general time-dependent (we stress this by writing it explicitly as a function of t). In fact, this dependence on time reflects the Schumpeterian view that permeates evolutionary economics, i.e., winners and losers emerge from an ongoing process of disequilibrium [20]. This implies a fluctuating selection [21,22] within a variable environment characterized by fitnesses that are not constant. This is why the method is called fluctuating selection from market capitalization (FSMC).

By deriving Equation (1) with respect to t, it is straightforward to obtain the following identity:

$$\frac{d{x}_i}{dt}=x_i(f_i(t)-\varphi (t))\hspace{1em}\hspace{1em}\hspace{1em}i=1, 2, \dots , N,$$

(3)

where ϕ(t) is the weighted average fitness, i.e.,

$$\varphi (t)={\displaystyle \sum _{i=1}^N{x}_i(t)f_i(t)}.$$

(4)

Identity (3) is the so-called selection equation [19,23].

2.2.2. Forecasting with the FSMC Method

In order to forecast future shares of the total market cap of companies, we firstly rewrite the selection equation in terms of discrete time (measured in days). By rearranging it, we obtain

$$x_i(t+1)=x_i(t)+x_i(t)\left[f_i(t)-\varphi (t)\right],\hspace{1em}i=1,2,\dots ,N,$$

(5)

with

$$f_i(t)\equiv \frac{v_i(t)}{v_i(t-1)}-1,\hspace{1em}i=1,2,\dots ,N.$$

(6)

Nevertheless, directly obtaining the fitness values from Equation (6) presents the issue of rapid daily variations in the data, leading to very noisy fitness estimates. Additionally, since the selection process is unlikely to be instantaneous, the concept of fitness must exhibit some degree of constancy over time [24], reflecting the firm’s behavioral continuity [1]. The method overcomes this problem by averaging f_i(t) over a training period of length T_T. Hence, a smoothed fitness for day t₀ is obtained as follows:

$$f_i^{\mathrm{smooth}}(t_0)=\frac{{\displaystyle \sum _{\tau =t-T_{\mathrm{T}}+1}^{t_0}{f}_i}(\tau )}{T_{\mathrm{T}}}.$$

(7)

Figure 1 compares the ‘instantaneous’ fitness, given by Equation (6), and the smoothed one. Using this smoothed fitness in Equation (5), we finally arrive to the FSMC forecasting equation:

$$\boxed{\begin{array}{l}{x}_i^{\mathrm{FSMC}}(t+1)=x_i^{\mathrm{FSMC}}(t)\left[1+f_i^{\mathrm{smooth}}(t_0)-{\displaystyle \sum _{j=1}^N{x}_j^{\mathrm{FSMC}}(t)f_j^{\mathrm{smooth}}(t_0)}\right],\\ i=1,2,\dots ,N,\hspace{1em}t=t_0,\hspace{0.17em}{t}_0+1,\dots \\ \mathrm{starting} \mathrm{at} t\hspace{0.17em}=t_0\mathrm{with} x_i^{\mathrm{FSMC}}(t_0)=x_i(t_0)\end{array}}$$

(8)

where the superscript FSMC is to distinguish the FSMC predictions for total capital market shares from the observed ones, which are just denoted by x(t).

Two remarks about the FSMC forecasting Equation (8):

• It is no longer an identity. This is because we have replaced the instantaneous fitness with the smoothed one evaluated at day t₀ in selection Equation (5).
• Furthermore, we kept this value of the smoothed fitness at day t₀ fixed over the validation period, t > t₀, shown in Figure 1 as a red-dotted line. Notice that the smoothed fitness generated by Equation (7) across the validation period (thick gray line) slightly departs from this constant fitness value, thus supporting the assumption of the constant fitness of the FSMC method over the validation period.

A rule of thumb of series forecasting is to take the validation period T_V as less or equal to the training period T_T [25]. Here, we always take T_V = T_T (taking T_V < T_T does not introduce qualitative differences). We used T_T = 21, 63, 126, and 252 market days, corresponding, respectively, to a month, a quarter, two quarters, and a year. The first prediction of the FSMC method starts at t = T_T and the last one at t = T − T_V = 5536-T_V. Therefore, to validate the FSMC method, we consider 5536 − 2T_V validation instances of length T_V.

2.2.3. Assessing the Performance of the FSMC Method

To evaluate the quantitative forecasting performance of the FSMC method, we computed the errors of its predictions of shares of total market capital for each firm i. Specifically, we computed for each given day t and for each validation instance α, both the absolute error $\epsilon {}_i{}^{(\alpha )}(t)=\left|x{}_i{}^{\mathrm{FSMC}(\alpha )}(t)-x_i(t)\right|$ and the percentage error $\delta {}_i{}^{(\alpha )}(t)$, defined as

$$\delta {}_i{}^{(\alpha )}(t)=100\times \frac{\epsilon {}_i{}^{(\alpha )}(t)}{x_i(t)},\hspace{1em}\begin{array}{c}\begin{array}{l}i=1,2,\cdots ,N\\ t=1,2,\cdots ,T_{\mathrm{V}}\end{array}\\ \alpha =1,2,\cdots ,5536-2T_{\mathrm{V}}\end{array}.$$

(9)

Then, to measure the forecast accuracy, we used the following metrics obtained in terms of {$\epsilon {}_i{}^{(\alpha )}(t)$} and {$\delta {}_i{}^{(\alpha )}(t)$}:

i.

Their averages over the 5536 − 2T_V validation instances:

$${\epsilon }_i^{\mathrm{av}}(t)=\frac{{\displaystyle \sum _{\alpha =1}^{5536-2T_{\mathrm{V}}}{\epsilon }_i^{(\alpha )}(t)}}{5536-2T_{\mathrm{V}}},\hspace{1em}i=1,2,\cdots ,N$$

(10)

$${\delta }_i^{\mathrm{av}}(t)=\frac{{\displaystyle \sum _{\alpha =1}^{5536-2T_{\mathrm{V}}}{\delta }_i^{(\alpha )}(t)}}{5536-2T_{\mathrm{V}}},\hspace{1em}i=1,2,\cdots ,N$$

(11)

ii.

The mean absolute error (MAE) for firm i, which is given by

$${\mathrm{MAE}}_i(T_{\mathrm{V}})=\frac{{\displaystyle \sum _{t=1}^{T_{\mathrm{V}}}{\epsilon }_i^{\mathrm{av}}(t)}}{T_{\mathrm{V}}},\hspace{1em}i=1,2,\cdots ,N,$$

(12)

where the T_V within parentheses is to highlight the dependence of this metric on the validation period. Specifically, it quantifies the error of the predicted “trajectory” followed by the share x_i of company i with respect to the real trajectory over T_V.

iii.

The mean absolute percentage error (MAPE) for firm i, given by

$${\mathrm{MAPE}}_i(T_{\mathrm{V}})=\frac{{\displaystyle \sum _{t=1}^{T_{\mathrm{V}}}{\delta }_i^{\mathrm{av}}(t)}}{T_{\mathrm{V}}},\hspace{1em}i=1,2,\cdots ,N,$$

(13)

iv.

A grand MAE and a grand MAPE, denoted, respectively, as GMAE and GMAPE, obtained as averages of MAE_i and MAPE_i across all firms:

$$\mathrm{GMAE}\left(T_{\mathrm{V}}\right)={\displaystyle \sum _{i=1}^N\frac{{\mathrm{MAE}}_i(T_{\mathrm{V}})}{N}}=\frac{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^{T_v}{\epsilon }_i^{\mathrm{av}}(t)}}}{N{T}_{\mathrm{V}}}.$$

(14)

$$\mathrm{GMAPE}\left(T_{\mathrm{V}}\right)={\displaystyle \sum _{i=1}^N\frac{{\mathrm{MAPE}}_i(T_{\mathrm{V}})}{N}}=\frac{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^{T_v}{\delta }_i^{\mathrm{av}}(t)}}}{N{T}_{\mathrm{V}}}.$$

(15)

The GMAE and GMAPE offer a comprehensive evaluation of the method’s accuracy: the smaller the global metric, the more accurate the method overall.

3. Results

Figure 2 shows the averaged absolute error ${\epsilon }_i^{\mathrm{av}}(t)$ produced by Equation (10) across the T_V = 252 days of one year for the 78 companies. We can see the following:

(a)

Most of these errors are < the mean of fractions {x_i(t)} of all companies and days (=0.0025). Indeed, for the first month (day 21), they are all much smaller.

(b)

They tend to increase with the forecasted day.

(c)

For firms #60, AIG, and #77, Fannie Mae, these errors reach very large values.

Now, let us consider the other extreme of the forecasts, namely T_V = 21 days. Figure 3 shows the averaged percentual absolute error ${\delta }_i^{\mathrm{av}}(t)$ produced by Equation (11) over T_V = 21 days for the 78 companies.

We can see the following from Figure 3:

(a)

Most of these errors are <5%.

(b)

Consistently with the absolute errors, they tend to increase with the forecasted day.

(c)

Also in agreement with what we found for the absolute errors, the relative errors for AIG and Fannie Mae reach very large values (>20%)

Likewise, as shown in Figure 4, the MAPE_i (Equation (11)) over T_V = 21 days for most of the companies is <5%, while only for two companies, AIG and Fannie Mae, it is >15%.

The large errors of AIG and Fannie Mae can be understood from their singular behavior shown in Figure 5. We can see that the market cap of both companies plummeted in 2008 during the Great Recession. Additionally, in 2011, the market cap of AIG skyrocketed. These drastic sudden variations are of course very difficult to capture through most forecasting methods. The FSMC is able to capture the trend, but not its intensity (see inset).

The MAE and MAPE increase monotonically with T_V. In fact, for T_V = 252 days, the MAPEs of most of the companies are between 10% and 15%, while for AIG and Fannie Mae, they are >100%.

Likewise, the GMAE and GMAPE also increase monotonically with T_V, as shown in

Table 2. GMAPE for the different T_V values considered.

TV (Days)	GMAE	GMAPE
21	6.00 × 10−4	5.4%
63	1.10 × 10−3	10.1%
126	1.60 × 10−3	15.4%
252	2.50 × 10−3	24.3%

.

4. Discussion

Natural selection in biology is backed by rigorous explanatory power and quantitatively verifiable predictions (e.g., recent quantitate predictions for COVID-19 dynamics can be found in ref. [26]). Conversely, its economic counterpart has not garnered much empirical support [7,27]. As previously observed, a significant limitation lies in estimating fitness from empirical market data. One possibility involves the concept of frequency-dependent selection [15,19,23], where the fitness of a firm depends on its interactions with all other firms. A common approach to implementing such frequency-dependent selection is the Replicator Dynamics Equation (RDE) [28]. But, to compute the N fitness {f_i}, the RDE requires an N × N “payoff matrix”, whose entry i-j corresponds to the payoff obtained by firm i when interacting with firm j. Estimating this payoff matrix is far from trivial. For a method to undertake this for a set of firms and a discussion of its drawbacks, we refer the reader to ref. [29].

Here, we propose a more straightforward method for estimating the fitness of companies using the time-series data of their market capitalizations. The resulting fluctuating selection from market caps (FSMC) method offers quantitative, testable predictions based on natural-selection-like concepts in relation to companies.

A main finding is that the FSMC method, when applied to a dataset of America’s top revenue companies considering daily market capitalizations from 2000 to 2021, performs well in forecasting their proportions of total market capitalization. This was a necessary first check of the method to subsequently address a significant challenge for evolutionary economic models, namely to generate and explain empirical observations as emergent properties stemming from the fluctuating forces driving markets [30]. Examples of the observed results that need to be explained include the distribution of firm growth rates and firm size distribution, both closely related to key aspects of market structure, like firm concentration. In fact, the dynamics of concentration derived from firm fitness will be examined using the FSMC method in a separate paper [31].

It is worth noticing that a limitation of the method is its difficulty to quantitatively reproduce catastrophic changes, such as those experienced by AIG and Fanny Mae during the Great Recession. However, the FSMC method is a useful tool to dissect the effects of the business cycle on the dynamics of companies and the market’s structure [31].