Game Theory of Tumor–Stroma Interactions in Multiple Myeloma: Effect of Nonlinear Benefits

Abstract

Cancer cells and stromal cells often exchange growth factors with paracrine effects that promote cell growth: a form of cooperation that can be studied by evolutionary game theory. Previous models have assumed that interactions between cells are pairwise or that the benefit of a growth factor is a linear function of its concentration. Diffusible factors, however, affect multiple cells and generally have nonlinear effects, and these differences are known to have important consequences for evolutionary dynamics. Here, we study tumor–stroma paracrine signaling using a model with multiplayer collective interactions in which growth factors have nonlinear effects. We use multiple myeloma as an example, modelling interactions between malignant plasma cells, osteoblasts, and osteoclasts. Nonlinear benefits can lead to results not observed in linear models, including internal mixed stable equilibria and cyclical dynamics. Models with linear effects, therefore, do not lead to a meaningful characterization of the dynamics of tumor–stroma interactions. To understand the dynamics and the effect of therapies it is necessary to estimate the shape of the benefit functions experimentally and parametrize models based on these functions.

1. Introduction

1.1. From Intra-Tumor Cooperation to Tumor–Stroma Interactions

Cancer cells secrete growth factors that promote proliferation by stimulating growth, resistance against apoptosis and against the immune system, or angiogenesis. Clones producing different essential growth factors can rely on each other for survival [1], and non-producer clones can free-ride on the growth factors produced by its cooperative neighbours [2]. The production of growth factors by cancer cells therefore is an example of intra-tumor cooperation and can be studied in the framework of evolutionary game theory. Concepts from game theory have been used to describe competition between cancer cells as pairwise games [3,4] with up to four strategies [5,6,7,8,9], and more recently as multiplayer public goods games to model the interactions between different clones within a tumor in well-mixed populations [10,11], in spatially structured populations that resemble monolayers in vitro [12,13,14], and to analyse the Warburg effect [15,16,17,18]; experiments with cancer cells have been used to test the theory [2].

Cancer cells also interact with non-cancerous stromal cells by exchanging diffusible factors [19,20,21,22] and these interactions with the tumor microenvironment are a fundamental part of cancer progression. While this is often referred to as “cooperation” between cancer and stroma, the secretion of growth factors by the stroma is often triggered by the cancer cells themselves, which recruit and activate stromal cells by producing diffusible cytokines; these cytokines stimulate the stroma to release growth factors that sustain tumor growth. Stromal cells, therefore, are manipulated by the cancer cells to produce growth factors that are beneficial to the tumor. A form of proper cooperation exists among the cancer cells themselves, as the cytokines they produce are a type of public good that can be exploited by non-producer cancer cells. Such a paracrine tumor–stroma signaling system, therefore, has non-trivial strategic interactions, which can also be studied in the framework of evolutionary game theory, but have received relatively less attention so far. Tumor–stroma interactions have been analysed in the context of prostate cancer [8], multiple myeloma [5,23], and MMP–TIMP (matrix metalloproteinases and tissue inhibitors of metalloproteinases) interactions [24].

1.2. From Two-Player Games to Collective Interactions with Nonlinear Effects

One problem with early models of tumor–stroma interactions, as with early models of intra-tumor cooperation, is that they assume pairwise interactions between cells. Cells, however, compete with more than one neighbor, and multiple cells contribute to the overall amount of growth factors available to a cell. A model with collective interactions [23] has been shown to lead to different results compared to a model [5] with pairwise interactions, with implications for disease progression and treatment (in particular, this new model suggests that reducing the number of malignant plasma cells below a critical threshold can lead to their extinction and thus to restore a healthy balance between osteoclast and osteoblast).

Even this approach, however, made an unrealistic assumption: that the effect of the growth factors exchanged by the cells is a linear function of its concentration. While linear effects are more amenable to analytical study, it is known that most growth factors have a nonlinear, often sigmoid, effect on cell proliferation, and that nonlinear effects can lead to very different dynamics [25,26]. We decided, therefore, to analyse tumor–stroma interactions using a multiplayer public goods game with nonlinear benefits, sacrificing analytical tractability for the sake of realism. Our goal is to understand whether nonlinear effects lead to qualitatively different results in the dynamics and equilibria in the study of tumor–stroma interactions, in particular in multiple myeloma (a type of cancer of plasma cells [27,28]).

1.3. Multiple Myeloma as a Modelling Case Study

We use multiple myeloma as a case study, continuing a previous extension to multiplayer linear games [23] of an earlier model with pairwise interactions [5]. In multiple myeloma, interleukin-6 (IL6), which promotes tumor proliferation, is produced not only by malignant plasma cells, but also by bone marrow stromal cells and osteoblasts; the stroma, however, is induced to produce IL6 by cytokines secreted by the multiple myeloma cells themselves [29,30], including FGF-beta [31], IL-1 [32], TNF-alpha [33], TGF-1 [34], and VEGF [35]. More in general, in multiple myeloma three types of cells contribute to bone resorption and bone formation by exchanging diffusible factors: malignant plasma cells (multiple myeloma, MM), osteoclasts (OC), and osteoblasts (OB). MM cells produce cytokines that activate OC and inhibit OB, altering the normal OB–OC equilibrium of healthy cells (and inducing, among other symptoms, bone fracture typical of multiple myeloma). OC and OB also secrete growth factors that affect each other and MM cells [36,37]. Our model is consistent with assumptions used previously in a model with linear benefits [23]—which was itself an extension with collective interactions of a pairwise game [5]—with the difference that here we use nonlinear benefit functions.

2. Results

2.1. Stability (Bistability) Depends on the Shape of the Benefit Functions

The most fundamental result of previous models is described in Figure 1: OB and OC cells are at a mixed equilibrium, balancing each other until MM cells are introduced in the population: if enough MM cells are introduced, the population evolves to a stable equilibrium on the MM–OC edge, which explains the potential for bone fractures observed in multiple myeloma. This result is obtained for the special case s_i,j→0 (linear benefits) in our model, and is consistent with previous models of linear benefits [23]. Observe, however (Figure 2), that if benefits are nonlinear this is not necessarily the case: the OB–OC equilibrium can persist irrespective of the number of MM cells introduced in the population or disappear entirely, depending on the shape of the benefit functions (in Figure 2 the position of the inflection point). While a model with linear benefits predicts a bistable system, nonlinear benefits can have both or only one of the two stable equilibria observed in linear models. Further examples are shown in Figure 3 and Figure 4, where additional mixed equilibria, not observed with linear benefits, arise with nonlinear benefits; in other cases, instead, the dynamics are similar (Figure 5). Unfortunately, given the huge number of combinations of parameters in the nonlinear model, any useful generalization is impossible. To understand and predict the dynamics of the system we need to know the shape of the benefit functions.

2.2. Nonlinear Benefits Can Lead to the Coexistence of Three Types and Cyclical Dynamics

Nonlinear benefits introduce a new type of equilibrium, never observed in linear models, in which all three types coexist (Figure 6). Coexistence of the three types can also be reached with cyclical oscillations that converge to a stable equilibrium (Figure 7) and even persist as a limit cycle (Figure 8): two types of dynamics that are never observed in linear models (and, more in general, in public goods games with constant group size).

2.3. Therapies That Target Growth Factors May Be More Effective Than Chemotherapy

Perturbations of the equilibria can reveal the effect of therapies against multiple myeloma and help understand the evolution of resistance to therapies. Figure 9 shows an example. At the OB–OC equilibrium, a mutant MM clone invades the population and leads to a MM–OC equilibrium: the typical fundamental result of current models (see Figure 1). A therapy that reduces the fraction of MM cells (such as chemotherapy that targets MM cells) immediately reduces the average fitness of the population; MM cells, however, eventually increase in frequency again and the population moves back to the previous MM–OC equilibrium. A therapy that increases the position of the inflection point (see Figure 2), for instance targeting the growth factors produced by the stroma and by the tumor (the inflection point increases when some of the growth factors are removed or made unavailable by the therapy because additional growth factors must be secreted—hence more producer cells are necessary—in order to achieve the same pre-therapy benefit level), instead, could lead to a spontaneous extinction of the MM types and a reduction of fitness; that is, a reduction in the speed of tumor growth. Understanding the effects of the two approaches quantitatively requires an estimation of the shape of the benefit functions.

3. Discussion

In summary, in addition to the pure equilibria and the two-type mixed equilibria observed in linear models [23], two additional results arise when benefits are nonlinear benefits: a mixed stable equilibrium with three types; and cyclical dynamics of the three types. This reconciles evolutionary game theory models with observations in real tumors (where the three types of cells can clearly coexist — something that previous linear models could not account for).

Our results, however, also show that it is impossible to generalize the results and predict the dynamics of the system without knowing exactly the shape of the benefit functions; small changes can lead to very different dynamics. In order to understand the dynamics and to be able to predict the effect of therapies, the shape of these benefit functions must be estimated from experimental data.

Even though, in the absence of measures of the shape of the benefit functions, we cannot yet provide an exact description of the dynamics and suggest how to calibrate therapies, our results suggest that a therapy will be more effective if, in addition, or even in alternative, to reducing the amount of MM cells, it increases the value of the inflection point of the benefit functions, which could be achieved, for example, using monoclonal antibodies against the growth factors produced by the tumor and the stroma (still an experimental approach [38,39]). Monoclonal antibodies increase the value of the inflection point because they make some of the secreted growth factors unavailable to the tumor; hence, further growth factors must be produced—hence more producer cells are necessary—in order to achieve the same pre-therapy benefit level. Reducing the amount of MM cells, instead, does not affect the shape of the benefit function. In other words, therapies that change the parameters of the game may be more effective, and lead to a stable outcome, than therapies that change the frequency of the players.

More in general, besides multiple myeloma, our results show that models with linear benefits (not to mention models with pairwise interactions) do not provide a reliable characterization of the dynamics of tumor–stroma interactions, and that unexpected types of dynamics, such as cyclical dynamics, can arise due to nonlinearities.

4. Materials and Methods

Let us call x_OC, x_OB, and x_MM the frequencies of three types of cells, respectively OC (osteoclasts), OB (osteoblasts), and MM (malignant plasma cells), in the population (x_OC + x_OB + x_MM = 1). N is the number of cells within the diffusion range of the growth factors (hence, equal to group size), which we assume is the same for all types and all growth factors. Fitness is calculated by considering the payoffs obtained in the randomly formed groups weighted by the probability that such groups occur. In a well-mixed population, the probability that a group contains n_OC, n_OB, and N − n_OC − n_OB individuals (excluding the focal cell itself) of type OC, OB, and MM, respectively, is given by

$$\left(\begin{array}{c}N-1\\ n_{OC},n_{OB}\end{array}\right)x_{OC}{}^{n_{OC}}·x_{OB}{}^{n_{OB}}·x_{MM}{}^{N-n_{OB}-n_{OC}-1}.$$

(1)

The payoffs for OC, OB, and MM are, respectively:

V_OC = b_OC,OC(n_OC + 1) + b_OB,OC(n_OB) + b_MM,OC(N − 1 − n_OC − n_OB) − c_OC

(2)

V_OB = b_OC,OB(n_OC) + b_OB,OB(n_OB + 1) + b_MM,OB(N − 1 − n_OC − n_OB) − c_OB

(3)

V_MM = b_OC,MM(n_OC) + b_OB,MM(n_OB) + b_MM,MM(N − n_OC − n_OB) − c_MM

(4)

where c_i is the cost of producing growth factors by type i and b_i,j is a function that indicates the effect on type j of the growth factors produced by type i. Note that each player’s strategy has also an effect on itself. The fitness of type i is given by

$$W_i={\displaystyle \sum }_{n_{OC}=1}^{N-1}{\displaystyle \sum }_{n_{OB}=0}^{N-1-n_{OC}}\left(\begin{array}{c}N-1\\ n_{OC},n_{OB}\end{array}\right)x_{OC}{}^{n_{OC}}·x_{OB}{}^{n_{OB}}·x_{MM}{}^{N-n_{OB}-n_{OC}-1}·V_i.$$

(5)

The frequencies change based on the replicator dynamics:

x_OC′ = x_OC(W_OC − W *)

(6)

x_OB′ = x_OB(W_OB − W *)

(7)

x_MM′ = x_MM(W_MM − W *)

(8)

where W * = x_OCW_OC + x_OBW_OB + x_MMW_MM. To introduce nonlinearities, we assume a benefit function defined by

$$b_{i,j}\left(n_i\right)=\frac{\left[l_{i,j}\left(n_i\right)-l_{i,j}\left(0\right)\right]}{\left[l_{i,j}\left(N\right)-l_{i,j}\left(0\right)\right]}$$

(9)

the normalized version of the logistic function

$$l_{i,j}\left(n_i\right)=\frac{B_{i,j}}{1+e^{s_{i,j}\left(h_{i,j}-n_i/N\right)}}$$

(10)

where n_i is the number of cells of type i; the parameters define the effect of the growth factors produced by type i on type j: B_i,j defines the maximum benefit; h_i,j defines the position of the inflection point: h_i,j→1 gives increasing returns and h_i,j→0 diminishing returns; s_i,j defines the steepness of the function at the inflection point (s_i,j→∞ models a threshold public goods game; s_i,j→0 models a linear benefit (Figure 10); and the normalization in (9) in prevents the logistic function (10) from becoming constant for s_i,j→0).