Using Hazard and Surrogate Functions for Understanding Memory and Forgetting

Abstract

The retention of human memory is a process that can be understood from a hazard-function perspective. Hazard is the conditional probability of a state change at time t given that the state change did not yet occur. After reviewing the underlying mathematical results of hazard functions in general, there is an analysis of the hazard properties associated with nine theories of memory that emerged from psychological science. Five theories predict strictly monotonically decreasing hazard whereas the other four theories predict a peaked-shaped hazard function that rises initially to a peak and then decreases for longer time periods. Thus, the behavior of hazard shortly after the initial encoding is the critical difference among the theories. Several theorems provide a basis to explore hazard for the initial time period after encoding in terms of a more practical surrogate function that is linked to the behavior of the hazard function. Evidence for a peak-shaped hazard function is provided and a case is made for one particular psychological theory of memory that posits that memory encoding produces two redundant representations that have different hazard properties. One memory representation has increasing hazard while the other representation has decreasing hazard.

1. Introduction

The hazard function first emerged from the context of actuarial studies that dealt with product failure or death [1]. Given a system that can undergo a state change, such as going from being alive to being deceased, the hazard $h\left(t\right)$ at time t is defined as the instantaneous current risk of the state change, conditioned by the fact that the transition has not yet occurred. Thus,

$$\begin{array}{c}\hfill h\left(t\right)=\frac{f\left(t\right)}{S\left(t\right)},\end{array}$$

(1)

where $f\left(t\right)$ is the probability density function of the distribution of lifetimes and where $S\left(t\right)$ is the proportion of all the cases at time t where the system remains in its initial state. $S\left(t\right)$ is thus called the survivor function, and it is the right tail of the lifetime distribution. The cumulative probability for the cases where the state transition has occurred is $F\left(t\right)={\int }_0^{t}f\left(x\right)dx$, which is the left tail of the lifetime distribution. Furthermore, since the density function is equal to $-\frac{dS}{dx}$, it follows from (1) that $h\left(x\right)=\frac{-dS}{Sdx}$. Hence,

$$\begin{array}{c}\hfill -{\int }_0^{t}h\left(x\right)dx={\int }_0^t\frac{dS}{S}.\end{array}$$

(2)

Also, because the initial value for the survivor function is $S\left(0\right)=1$, it follows from (2) that

$$\begin{array}{c}\hfill S\left(t\right)=e^{-{\int }_0^{t}h\left(x\right)dx}.\end{array}$$

(3)

From (1) we know $f\left(t\right)=h\left(t\right)S\left(t\right)$; thus the probability density can be re-expressed using Equation (3) as the following function of hazard.

$$\begin{array}{c}\hfill f\left(t\right)=h\left(t\right)e^{-{\int }_0^{t}h\left(x\right)dx}.\end{array}$$

(4)

Hazard can also be defined for a discrete random variable; however, in this paper hazard is used for only the continuous case.

For a comprehensive exploration of the mathematical and statistical properties of hazard functions see [2,3,4,5]. In mathematical psychology, a number of researchers have developed and applied hazard functions [6,7,8,9,10]. The topics of human memory retention and forgetting are the focus of attention in this paper, and the analysis of hazard functions provides a powerful framework for achieving important insights about these processes. However, first to aid the reader, it is useful to briefly describe the organization of the paper.

Because some readers are unfamiliar with the mathematical facts about hazard and hazard-surrogate functions, there is a review provided in Section 2 that discusses how these functions have the potential for uncovering essential properties about human memory. This section provides the proofs for five essential theorems and two corollaries. Although these results have been previously established, they are reworked here in a common notation because some of the results are relatively obscure and counterintuitive. Nine candidate models for a memory retention function, which were previously developed from the psychological literature, are discussed in Section 3 from a hazard function perspective, while hazard functions for the various models differ and thereby provide a means for discriminating the theories, the hazard function per se is statistically difficult to empirically measure directly. However, in Section 4, another hazard-surrogate function is discussed, along with two theorems. This function provides for a statistically sensitive method for critically testing the hazard function for human memory. This section also contains the findings from three experiments that use the surrogate function to uncover the key property of peak-shaped hazard. Finally in Section 5, there is a theoretical discussion about the memory structure that is consistent with the mathematical and empirical findings.

2. Fundamentals for the Hazard Analysis of Memory and Forgetting

2.1. Linking Forgetting to the Product Failure Context

To see why hazard functions are connected to memory research, let us consider the basic question of forgetting as studied in psychological experiments. Carefully controlled experiments that examine forgetting first need to train the research participants to learn some novel facts or associations. If at a later time the participants are correct when tested about that information, then we know that the memory, which was originally established at time $t_0$, has survived until time t. However, for the duration to be a valid quantity, it is important (1) that the to-be-remembered targets are novel and not part of the person’s pre-experimental knowledge and (2) that the post-encoding processes did not involve rehearsal of the target information. If there were a surreptitious rehearsal, say one minute after encoding, then a measured retention interval of say two minutes would be mistaken. Thus, to avoid post-encoding rehearsal, it is important that the research participants are kept busy doing other activities over the retention interval that is unrelated to the target information, although for very long retention intervals it is not possible to control all the post-encoding experiences of the participant. (Hazard models for manufactured products are different when the product is repaired or adjusted subsequent to the initial production. Such repairs prolong the survival of the product and is analogous to the post-encoding rehearsal of a memory target item.) Psychologists have invented a number of procedures that enable a study of memory retention as a function of time. Experimental tasks such as the serial-learning procedure [11], paired-associate training [12], the Brown-Peterson task [13,14], and the continuous testing method [15] are standard memory procedures that meet the above mentioned design requirements.

Given the experimental procedures developed by psychologists to study memory, it is clear that memory retention is a similar process to that of the survival of a manufactured product. In both cases there is something new created that is initially in a state of functionality, but later only a proportion $S\left(t\right)$ of the original items survived to time t. For both cases, models for the survival will be assumed to be a function of time; yet, it is very likely in both cases that it is not time per se that is responsible for the failures. A manufactured product usually fails because of the wear that occurs from usage, so without usage the product’s lifetime is discontinuously increased. Additionally, the survival of a manufactured product varies with the roughness of usage. Thus, when we model product reliability as a function of time, it is only an idealization for the events that occur in time. Given more time, there is more product usage and wear. By using time as a proxy for the amount of wear, the system is characterized without delineating exactly the sequence of events that occurred with product use. Similarly memory loss over the retention interval could either be due to other interfering events that occurred subsequent to studying the target or due to the amount of time elapsed. Some investigators [16,17] varied the amount of information per unit time during the retention interval to see the relative contribution of interfering events and time itself. Although there is a small unique role for time, most of the variability in memory loss is due to the number of intervening events that occurred. Nonetheless in many studies with very long retention intervals, we do not know how many events occurred, so time is used as an approximation for the amount of interfering events.

2.2. Initial Conditions and Rationality Constraints for Memory Reliability Modeling

To assure the initial condition that $S\left(0\right)=1$, the presentation of the memory targets during the learning phase of a memory experiment needs to be sufficiently long to assure that the items are adequately encoded. This condition requires experience with the stimuli class. In the real world, people are bombarded with information. For example, in a one-second time slice of a naturalistic real-world scene, there are more than 14,000 bits of information that are available for processing, but on average people can only acquire about 19 bits per second [18]. However, if only three letters (without consecutive letter repetitions), such as LDQ, is presented as a target item, then this unit has $\frac{log(26·{25}^2)}{log2}\approx 14$ bits of information, which is within the memory processing capacity limit, so it should be encoded completely within one second. In general researchers need to be careful to have knowledge about how to present the memory targets to assure near perfect initial encoding for the type of stimuli used [19,20].

There is also a complicating measurement issue for memory retention studies because there is not a perfect dependent variable that is a pure measure of the probability of target storage. The dependent variables obtained in behavioral studies involve a host of entangled underlying psychological processes such as storage, retrieval, and guessing [18]. For example, a correct response might be a lucky guess, but an incorrect response might be only a temporary retrieval problem of recovering a successfully stored memory. Although no process-pure measure is known, it is nonetheless possible to model how the underlying processes tap a set of separate dependent variables. With a cognitive psychometric model, a validated measurement of the desired target storage probability at time t can be extracted. There are a number of these model-based measurement methods developed in psychological science [21,22,23,24,25].

By following the above mentioned experimental and psychometric methods, it is possible for memory researchers to obtain data on the survival of memory as a function of time. However, a rational mathematical model must also satisfy a few constraints. First, any proposed theory must satisfy the initial condition that $S\left(0\right)=1$. Second, since the probability density function is equal to $-\frac{dS\left(t\right)}{dt}$, it follows that any proposed mathematical function for $S\left(t\right)$ must satisfy the constraint that $-\frac{dS\left(t\right)}{dt}\ge 0,\forall t\ge 0$.

2.3. Probability and Subprobability Hazard Models

It is important to distinguish between probability and subprobability models for hazard. For both model types, the support for the random variable is on a continuous interval, which is denoted generally as $[L,U]$. The bounds of L and U on the support interval can be either limits such as, respectively, $-\infty$ and $+\infty$ or they can finite real values. For a probability model of product survival as function of time (t), $L=0$ and the boundary conditions are: (1) $S\left(0\right)=1$ and $F\left(0\right)=0$, which corresponds to all units functioning at $t=0$, and (2) $S\left(U\right)=0$ and $F\left(U\right)=1$, which corresponds to all units failing by time $t=U$. Yet there are applications where the upper limit of the support is finite and some units are still functioning at the upper limit of the support. For human memory there are some memories that are still stored over the entire lifetime of the individual. For example, Bahrick reported cases of memories that were stored over a 50-year duration and where the participants reported not having rehearsed or used the information during the retention interval [26]. A subprobability model can capture cases where some units do not fail. Subprobability is the term that a number of mathematicians have used for a probabilistic measure where the integration of the density function over the support of the random variable is less than $1.0$ (i.e., $F\left(U\right)<1$) [27,28,29]. For a subprobability model of product survival, the boundary conditions are: (1) $S\left(0\right)=1$ and $F\left(0\right)=0$, and (2) $S\left(U\right)=1-\delta$ and $F\left(U\right)=\delta$ where $1>\delta >0$. The equation $S\left(t\right)=1-F\left(t\right)$ holds for both probability and subprobability models.

For any probability model it is possible to produce a subprobability counterpart model by multiplying the regular normalized density function $f_{*}\left(t\right)$ by a positive constant $\delta <1$, $f\left(t\right)=\delta f_{*}\left(t\right)$. It thus follows that $F\left(t\right)=\delta F_{*}\left(t\right)$ and $S\left(t\right)=1-\delta F_{*}\left(t\right)$. Although the multiplication by a positive $\delta$ results in linear changes in the density function as well as in the cumulative and survivor functions, it has important consequences for the hazard and the integrated hazard. To see this point, consider the simplest hazard probability model of an exponential distribution where $f_{*}\left(x\right)=k{e}^{-kx}$ for x on the support of the nonnegative reals and for $k>0$. The value for the cumulative distribution is $F_{*}\left(t\right)={\int }_0^{t}k{e}^{-kx}dx=1-e^{-kt}$. Thus, the hazard is $h_{*}\left(t\right)=\frac{k{e}^{-kt}}{e^{-kt}}=k$. This property of constant hazard is a well known and unique feature of the exponential distribution [30]. Although the hazard for the exponential probability model is a constant, the hazard for the counterpart exponential subprobability model is not a constant. For the exponential subprobability model, the hazard is

$$\begin{array}{ccc}\hfill h\left(t\right)& =& \frac{\delta k{e}^{-kt}}{1-\delta {\int }_0^{t}k{e}^{-kx}dx}.\hfill \\ & =& \frac{k}{\frac{1-\delta }{\delta }{e}^{kt}+1}.\hfill \end{array}$$

Thus, the hazard for the subprobability exponential model is monotonically decreasing, whereas the exponential model has constant hazard.

Subprobability models are not uncommon because stochastic processes are often right-tail censored for practical limitations, such as employing a time limit for a measured response. Right-tail censoring has also led some theorists to use the concept of reverse hazard, which is defined as $r\left(x\right)=\frac{f\left(x\right)}{F\left(x\right)}$ since reverse hazard is not a function of $\delta$ when $F\left(U\right)<1$ [31,32,33,34]. Consequently reverse hazard is especially useful when the quantity $\delta$ is unknown [35]. However, in this paper reverse hazard is not needed for learning some critical details about human memory retention.

2.4. Weibull Distribution and Hazard

The Weibull distribution is an important survival model because it can capture monotonically decreasing hazard, constant hazard, or monotonically increasing hazard depending on the value of a shape parameter. This model will also play an important role for describing the survivor function for memory.

The Weibull cumulative function is $F\left(t\right)=1-exp(-k{t}^c)$ on the support of the nonnegative reals and where k is a positive scale factor and c is a positive shape parameter [30]. The resulting hazard function is $h\left(t\right)=ck{t}^{c-1}$. Note that at the value of $c=1$, the Weibull distribution becomes the exponential distribution and has constant hazard. For $0<c<1$, the Weibull has monotonically decreasing hazard, and for $c>1$, the hazard is monotonically increasing.

As with the exponential distribution, there is also a Weibull subprobability model where the cumulative probability model is $F\left(t\right)=\delta [1-exp(-k{t}^c)]$ for $0<\delta <1$. The hazard for the Weibull subprobability model is

$$\begin{array}{ccc}\hfill h\left(t\right)& =& \frac{ck{t}^{c-1}}{\frac{1-\delta }{\delta }{e}^{kt}+1}.\hfill \end{array}$$

(5)

For the Weibull subprobability model, the hazard is monotonically decreasing for $0<c\le 1$, but for $c>1$, the hazard has a peak shape. That is, $h\left(0\right)=0$, but for some small t values the hazard is increasing until reaching a maximum at $t_p$. For $t>t_p$, the Weibull subprobability model has monotonically decreasing hazard. For example, the case where $\delta =0.5$, $c=2$, and $k=1$ results in a hazard peak at approximately $t=1.278$.

2.5. A Surrogate Function and Proofs of Monotonicity

For the analysis of the monotonicity and other properties of the hazard function, it is sometimes helpful to use other functions. For cases that have a known density function $f\left(x\right)$ for $x\in [L,U]$, it has been shown that there is a convenient surrogate function for understanding monotonicity properties [6].This function is

$$\begin{array}{ccc}\hfill g\left(x\right)& =& -\frac{f^{\prime }\left(x\right)}{f\left(x\right)}.\hfill \end{array}$$

(6)

The following two general theorems about hazard functions illustrate the importance of the $g\left(x\right)$ function. These theorems are variations of several other previously published proofs that deal with a class of functions where the hazard either has a monotone rate or is a constant [9].

Theorem 1.

For a continuous differentiable density function $f\left(x\right)$ on support $[L,U]$, the hazard is: monotonically increasing if and only if $h\left(x\right)>g\left(x\right)\forall x$, monotonically decreasing if and only if $h\left(x\right)<g\left(x\right)\forall x$, and a constant if and only if $h\left(x\right)=g\left(x\right)\forall x$.

Proof. 

It follows from the definition of hazard that

$$\begin{array}{ccc}{h}^{\prime }\left(x\right)\hfill & =\hfill & \frac{{\left[f\left(x\right)\right]}^2+f^{\prime }\left(x\right)S\left(x\right)}{{\left[S\left(x\right)\right]}^2},\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill & =\hfill & {\left[h\left(x\right)\right]}^2+\frac{f^{\prime }\left(x\right)f\left(x\right)}{S\left(x\right)f\left(x\right)},\hfill \end{array}$$

(8)

$$\begin{array}{ccc}\hfill & =\hfill & h\left(x\right)\left[h\right(x)-g(x\left)\right],\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\left(\frac{S\left(x\right)}{f\left(x\right)}\right)h^{\prime }\left(x\right)\hfill & =\hfill & h\left(x\right)-g\left(x\right).\hfill \end{array}$$

(10)

Since the term $\left(\frac{S\left(x\right)}{f\left(x\right)}\right)$ is non-negative for all $x\in [L,U]$, it follows that the sign of $h^{\prime }\left(x\right)$ is directly a function of the quantity $h\left(x\right)-g\left(x\right)$. Thus, if $h\left(x\right)>g\left(x\right)\forall x\Rightarrow h^{\prime }\left(x\right)>0$, and conversely if $h^{\prime }\left(x\right)>0\forall x\Rightarrow h\left(x\right)>g\left(x\right)$. Similarly if $h\left(x\right)<g\left(x\right)\forall x\Rightarrow h^{\prime }\left(x\right)<0$, and conversely if $h^{\prime }\left(x\right)<0\forall x\Rightarrow h\left(x\right)<g\left(x\right)$. Lastly if $h\left(x\right)=g\left(x\right)\forall x\Rightarrow h^{\prime }\left(x\right)=0$, and conversely if $h^{\prime }\left(x\right)=0\forall x\Rightarrow h\left(x\right)=g\left(x\right)$.    □

Theorem 1 is quite general because it only requires a continuous differentiable density function. It applies for both probability models and subprobability models. For example, using the theorem for the case of the exponential distribution $f\left(t\right)=k{e}^{-kt}$ for non-negative t, we find that both $h\left(t\right)$ and $g\left(t\right)$ are equal to k, so the hazard is a constant. However, for the subprobability version of the exponential when $f\left(t\right)=\delta k{e}^{-kt}$, we now find that $h\left(t\right)-g\left(t\right)=k[\frac{1}{\frac{1-\delta }{\delta }{e}^{kt}+1}-1]<0$, which results in correctly concluding that the hazard is monotonically decreasing. Theorem 1 is limited as a tool when there is not a closed form solution for the cumulative distribution that enables a closed-form formula for $h\left(x\right)$. However, the next theorem is useful even when the cumulative distribution function does not have a closed-form solution.

Theorem 2.

For a continuous differentiable density function $f\left(x\right)$ for $x\in [L,U]$ with $f\left(U\right)=0$, the hazard rate is: either monotonically increasing if and only if $g^{\prime }\left(x\right)>0\forall x$, monotonically decreasing if and only if $g^{\prime }\left(x\right)<0\forall x$, or a constant hazard rate if and only if $g^{\prime }\left(x\right)=0\forall x$.

Proof. 

It is convenient to find the expected value for $g\left(y\right)$ for $y\in [x,U]$. The conditional expected value is

$$\begin{array}{ccc}E\left(g\right(y\left)\right|y\ge x)\hfill & =\hfill & \frac{{\int }_x^{U}g\left(y\right)f\left(y\right)dy}{{\int }_x^{U}f\left(y\right)dy},\hfill \end{array}$$

(11)

$$\begin{array}{ccc}& =\hfill & \frac{-{\int }_x^U{f}^{\prime }\left(y\right)dy}{S\left(x\right)}=\frac{0+f\left(x\right)}{S\left(x\right)}=h\left(x\right).\hfill \end{array}$$

(12)

Note that $E\left(g\right(y\left)\right|y\ge x)=h(x)$ even for subprobability models because if a normalized density function were multiplied by $\delta$ where $0<\delta \le 1$, then the resulting $g\left(x\right)$ function is invariant to this change to the density function. Thus, it follows from (10) that

$$\begin{array}{ccc}\hfill \left(\frac{S\left(x\right)}{f\left(x\right)}\right)h^{\prime }\left(x\right)& =& E\left(g\right(y\left)\right|y\ge x)-g(x).\hfill \end{array}$$

(13)

The sign of $h^{\prime }\left(x\right)$ is thus directly related to the quantity $E\left(g\right(y\left)\right|y\ge x)-g(x)$, provided that $f\left(U\right)=0$. If $g^{\prime }\left(x\right)>0\forall x$, then $E\left(g\right(y\left)\right|y\ge x)$ and thus $h\left(x\right)$ is growing with x since the smallest values of $g\left(x\right)$ are being trimmed as x increases, which implies that $h^{\prime }\left(x\right)>0$. Conversely if $h^{\prime }\left(x\right)>0\forall x$ implies that $\frac{dE\left(g\right(y\left)\right|y\ge x)}{dx}>0$, so $g^{\prime }\left(x\right)>0$. Similarly if $g^{\prime }\left(x\right)<0\forall x$, then $h\left(x\right)=E\left(g\right(y\left)\right|y\ge x)<g(x)$, so $h^{\prime }\left(x\right)<0$. Again the converse is also true. Lastly if $g^{\prime }\left(x\right)=0\forall x\Rightarrow g\left(x\right)=k$, and because $h\left(x\right)=E\left(k\right)=k\Rightarrow h^{\prime }\left(x\right)=0$. Again the converse also holds.    □

Several useful corollaries follow from the above theorems.

Corollary 1.

The density function is monotonically decreasing for distributions that have monotonically decreasing hazard, and the lower limit for the support L must be finite where $f\left(x\right)$ has its maximum value.

Proof. 

In general if the hazard rate is monotonically decreasing $\forall x\in [L,U]$, then from Theorem 1 it follows that $g\left(x\right)>h\left(x\right)$. Thus, $-\frac{f^{\prime }\left(x\right)}{f\left(x\right)}>\frac{f\left(x\right)}{S\left(X\right)}$, so $-f^{\prime }\left(x\right)>\frac{{\left(f\left(x\right)\right)}^2}{S\left(x\right)}>0$. So, $f^{\prime }\left(x\right)<0\forall x$. Hence, the density function must be monotonically decreasing. Moreover, if the lower limit of support were $-\infty$, then the cumulative probability would grow without limit. Therefore L must be finite with $f\left(L\right)$ as the maximum density value.    □

Although a monotonically decreasing hazard function has monotonically decreasing probability density, the converse statement does not follow because there can be monotonically decreasing density functions that have monotonically increasing hazard. Consider the case where $f\left(x\right)=\frac{3}{2}-x$ on the support interval of $[0,1]$. For this density function $h\left(x\right)=(\frac{3}{2}-x){(1-\frac{3x}{2}+\frac{x^2}{2})}^{-1}$ and $g\left(x\right)={(\frac{3}{2}-x)}^{-1}$, so it follows that $\frac{h\left(x\right)}{g\left(h\right)}=\frac{{(\frac{3}{2}-x)}^2}{1-\frac{3}{2}x+\frac{x^2}{2}}>1$ for $x\in [0,1]$. Thus, from Theorem 1, $h\left(x\right)$ is monotonically increasing because $h\left(x\right)>g\left(x\right)$ for all x in the support.

The next corollary deals with the cases when hazard monotonicity does not hold.

Corollary 2.

For a continuous differentiable density function $f\left(x\right)$ for $x\in [L,U]$ that does not have either monotonically increasing hazard, monotonically decreasing hazard, or constant hazard, there is at least one point $x_0$ that is either a local hazard peak, valley or an inflection point. It is a hazard peak if $g\left(x_0\right)=h\left(x_0\right)$ with $g^{\prime }\left(x_0\right)g\left(x_0\right)>0$, it is a hazard valley if $g\left(x_0\right)=h\left(x_0\right)$ with $g^{\prime }\left(x_0\right)g\left(x_0\right)<0$, and it is an inflection point if $g\left(x_0\right)=h\left(x_0\right)$ with $g^{\prime }\left(x_0\right)g\left(x_0\right)=0$.

Proof. 

If $f\left(x\right)$ does not have a monotonic hazard rate, then there must be at least one $x_0$ that corresponds to $h^{\prime }\left(x_o\right)=0$. From Theorem 1, it is clear that a hazard peak, valley or an inflection point occurs if $g\left(x_0\right)=h\left(x_0\right)$. Additionally, from Equation (10), it follows that

$$\begin{array}{ccc}g\left(x\right)\hfill & =\hfill & h\left(x\right)-\frac{h^{\prime }\left(x\right)}{h\left(x\right)},\hfill \end{array}$$

(14)

$$\begin{array}{ccc}{g}^{\prime }\left(x\right)\hfill & =\hfill & h^{\prime }\left(x\right)-\frac{h^{″}\left(x\right)h\left(x\right)-{\left[h^{\prime }\left(x\right)\right]}^2}{{\left[h\left(x\right)\right]}^2}.\hfill \end{array}$$

(15)

From Theorem 1 and Equation (15), a hazard function has a peak, valley, or inflection point when $g\left(x_0\right)=h\left(x_0\right)$ and when $h^{\prime }\left(x_0\right)=0$, which corresponds to when $g^{\prime }\left(x_0\right)=-\frac{h^{″}\left(x_0\right)}{h\left(x_o\right)}=-\frac{h^{″}\left(x_0\right)}{g\left(x_0\right)}$. Thus, $h^{″}\left(x_0\right)=-g^{\prime }\left(x_0\right)g\left(x_0\right)$. So, at $x_0$ the hazard has either a peak if $g^{\prime }\left(x_0\right)g\left(x_0\right)>0$, a valley if $g^{\prime }\left(x_0\right)g\left(x_0\right)<0$, or an inflection point if $g^{\prime }\left(x_0\right)g\left(x_0\right)=0$.    □

As an illustration of the utility of the $g\left(x\right)$ surrogate function, consider the case of the Gaussian distribution, which has a density function of $f\left(x\right)=\frac{1}{\sigma \sqrt{2\pi }}exp[-\frac{1}{2}{\left(\frac{x-\mu }{\sigma }\right)}^2]$, but it lacks a closed form-function for $S\left(x\right)$ and $h\left(x\right)$. The support for the distribution is the real numbers, and clearly $f(U=\infty )=0$. For this distribution, $g\left(x\right)=\frac{x-\mu }{{\sigma }^2}$, so $g^{\prime }\left(x\right)=\frac{1}{{\sigma }^2}$, which is positive for all x. Thus, from Theorem 2 it is clear that the Gaussian distribution has monotonically increasing hazard.

2.6. Using Hazard to Create Novel Probability Models

Because the hazard function captures the change in the tendency for a state change, it is possible to design new probability models that have a particular desired hazard rate. As an illustration of this operation, suppose that the stochastic theorist wants a continuous density function that has a hazard function of $h\left(x\right)=\frac{1}{6}(x^3-6x^2+9x+6)$ over the support interval of $[0,\infty )$. This hazard function rises to a peak at $x=1$, decreases to a valley at $x=3$, and increases for $x>3$. From Equation (3) the survivor function for this desired distribution is

$$\begin{array}{ccc}\hfill S\left(x\right)& =& exp\left(-\frac{x^4}{24}+\frac{x^3}{3}-\frac{3x^2}{4}-x\right).\hfill \end{array}$$

From Equation (4) the corresponding density for the distribution is

$$\begin{array}{ccc}\hfill f\left(x\right)& =& \frac{1}{6}(x^3-6x^2+9x+6)exp\left(-\frac{x^4}{24}+\frac{x^3}{3}-\frac{3x^2}{4}-x\right).\hfill \end{array}$$

2.7. Hazard for Mixture Models

Up to this point, the reliability models assumed that all units are identically and independently distributed. For mixture models the assumption of identically distributed is relaxed. Mixture models can exhibit hazard properties that are not shared by any of the component distributions. Suppose for example that there is a mixture with two Gaussian components (i.e., $\theta$ proportion of the units have a Gaussian distribution with mean $\mu =0$ and standard deviation $\sigma =1$, whereas the rest of the units have a Gaussian distribution with mean $\mu =\Delta >0$ and standard deviation $\sigma =1$). As shown above, a Gaussian random variable has monotonically increasing hazard regardless of its mean and standard deviation. However, a mixture of say two components can exhibit a hazard rate that is either monotonically increasing or it can have a local peak and a local valley [9]. Which one of these two different hazard profiles occurs depends on the mixing rate (i.e., the value for $\theta$) and depends on the separation between the means ($\Delta$). For example, when $\theta =0.5$ and $\Delta <2.70$ the mixture has monotonically increasing hazard, but if $\theta =0.5$ and $\Delta \ge 2.70$, then the hazard has a profile that initially rises to a local peak, decreases to a local valley, and increases again [36]. However, when the mixing rates are $\theta =0.05$ and $1-\theta =0.95$, the critical separation between the two Gaussian components has to be $\Delta \ge 3.61$ before the phase change in hazard profile occurs. These general features about a hazard function for a mixture has been demonstrated for other distributions where the components of the mixture have monotonically increasing hazard [36]. Furthermore, this phase change has also been shown for sufficiently separate components that consists of more than two components. For example, a mixture of three components has been shown to exhibit a profile that increases to a local peak, decreases to a local valley, increases to a second local peak, decreases to a second local valley, and increases for larger values of x [9]. Consequently the finding of a hazard function that has multiple peaks and valleys is a potential signal that the distribution consists of a mixture, and it can suggest how many components there are in the mixture. However, as shown in the previous subsection about creating novel probability models, there can also be a homogeneous random process that has the same hazard profile.

Although a hazard rate for a mixture of components that individually have an increasing hazard rate is complicated, the mixture of component distributions that individually have non-increasing hazard has a monotonic rate [9]. A formal proof of this assertion is given below.

Theorem 3.

The hazard is monotonically decreasing for a mixture of components that individually have non-increasing hazard with $f_i\left(U\right)=0$.

Proof. 

First consider the case where there are two components in the mixture. Moreover, if one or both of the components has constant hazard, then it is an exponential distribution $f_i\left(x\right)=k_{i}exp[-k_i(x-L)]$, so $f_i(U=\infty )=0$. Since $f_i\left(U\right)=0$ it follows from Theorem 2 that $g_i^{\prime }\left(x\right)={\left(\frac{1}{f_i\left(x\right)}\right)}^2{H}_i\left(x\right)\le 0$ for $i=1,2$, where

$$\begin{array}{ccc}\hfill H_i\left(x\right)& =& {\left[f_i^{\prime }\left(x\right)\right]}^2-f_i\left(x\right)f_i^{″}\left(x\right)\le 0.\hfill \end{array}$$

(16)

Moreover, the equations for $f\left(x\right)$, $f^{\prime }\left(x\right)$, and $f^{″}\left(x\right)$ for the mixture are:

$$\begin{array}{ccc}\hfill f\left(x\right)& =& \theta f_1\left(x\right)+(1-\theta )f_2\left(x\right),\hfill \\ \hfill f^{\prime }\left(x\right)& =& \theta f_1^{\prime }\left(x\right)+(1-\theta )f_2^{\prime }\left(x\right),\hfill \\ \hfill f^{″}\left(x\right)& =& \theta f_1^{″}\left(x\right)+(1-\theta )f_2^{″}\left(x\right).\hfill \end{array}$$

It thus follows that $H\left(x\right)$ for the mixture is

$$\begin{array}{ccc}\hfill H\left(x\right)& =& {\theta }^2{H}_1\left(x\right)+{(1-\theta )}^2{H}_2\left(x\right)+\theta (1-\theta )f_1\left(x\right)f_2\left(x\right)G\left(x\right),\hfill \end{array}$$

(17)

$$\begin{array}{ccc}\hfill G\left(x\right)& :=& \frac{2f_1^{\prime }\left(x\right)f_2^{\prime }\left(x\right)}{f_1\left(x\right)f_2\left(x\right)}-\frac{f_1^{″}\left(x\right)}{f_1\left(x\right)}-\frac{f_2^{″}\left(x\right)}{f_2\left(x\right)}.\hfill \end{array}$$

(18)

Because $H_i\left(x\right)\le 0$, it follows that for $i=1,2$ that $f_i\left(x\right)f_i^{″}\left(x\right)\ge {\left[f_i^{\prime }\left(x\right)\right]}^2$. Hence, $\frac{f_i^{″}\left(x\right)}{f_i\left(x\right)}\ge {\left[\frac{f_i^{\prime }\left(x\right)}{f_i\left(x\right)}\right]}^2$. Thus, $\frac{f_i^{″}\left(x\right)}{f_i\left(x\right)}={\left[\frac{f_i^{\prime }\left(x\right)}{f_i\left(x\right)}\right]}^2+a_i$ where $a_i\ge 0$. Substituting this last equation into Equation (18) yields

$$\begin{array}{ccc}\hfill G\left(x\right)& =& \frac{2f_1^{\prime }\left(x\right)f_2^{\prime }\left(x\right)}{f_1\left(x\right)f_2\left(x\right)}-{\left[\frac{f_1^{\prime }\left(x\right)}{f_1\left(x\right)}\right]}^2-a_1-{\left[\frac{f_2^{\prime }\left(x\right)}{f_2\left(x\right)}\right]}^2-a_2,\hfill \end{array}$$

(19)

$$\begin{array}{ccc}\hfill & =& -{\left(\frac{f_1^{\prime }\left(x\right)}{f_1\left(x\right)}-\frac{f_2^{\prime }\left(x\right)}{f_2\left(x\right)}\right)}^2-a_1-a_2.\hfill \end{array}$$

(20)

Even if both components of the mixture has constant hazard (i.e., the distributions are exponential distributions of the from $f_i\left(x\right)=k_{i}exp[-k_i(x-L)]$), the first term in Equation (20) is negative, so $G\left(x\right)<0$ and thus $H\left(x\right)$ from Equation (17) is also negative. Hence, the theorem holds for any two-component mixtures where the components have non-increasing hazard because $g^{\prime }\left(x\right)<0$. The general proof is established by mathematical induction. That is, assume the theorem holds for a mixture of $k-1$ components. The case with k components is then considered as a mixture of two components where the first component is a mixture of $k-1$ components and the second component is the kth component. The same argument as above results in concluding that the k-component mixture has monotonically decreasing hazard.    □

Theorem 3 will play an important role for subsequent applications of hazard for understanding models for memory retention.

Finally, one possible interpretation for a subprobability model is as a mixture of two distributions. For $1-\delta$ proportion of the units, the survivor function is 1. Thus, the probability density for the lifetime failure distribution for this proportion is 0, so the value of $F\left(U\right)$ for these failure-resistant units is 0.

2.8. Disjunctive Hazard Systems

Virtually all units consist of essential constituent parts that must all work properly for the unit to be functional. A failure of any of the subsystems results in an overall system failure. Units of this type can be called a disjunctive hazard system or more informally a weakest-link system. For this type of product, it does matter what the average lifetime of the constituent subsystems is because the system fails when the weakest link fails. Statisticians and probability theorists recognized that there were three limiting or asymptotic probability distributions for problems of this type [37,38,39]. That is, the distribution of the minimum value for a system of n components has an asymptotic limit for large n that must be one of three distributions, which are denoted as $L^{\left(1\right)}$, $L^{\left(2\right)}$, and $L^{\left(3\right)}$. Hence, all probability distributions belong to one of three domains of attraction for the limiting case for the weakest link. For example, if the subsystems have a Gaussian distribution, then the weakest link has a $L^{\left(1\right)}$ limiting form; whereas if the subsystems have either an exponential or uniform distribution, then the weakest link has a $L^{\left(2\right)}$ limiting form [40]. However, for lifetime models where $t_i$ is the survival time for the ith component, only the $L^{\left(1\right)}$ and $L^{\left(2\right)}$ limiting distributions are suitable, and these distributions are, respectively, the Gumbel and Weibull distributions [41]. Thus, it is not accidental that the Gumbel and Weibull models are frequently used to capture the survivor function for manufactured products. It was shown previously in this paper that the Weibull distribution hazard for $t\ge 0$ is $h\left(t\right)=ck{t}^{c-1}$ for $c>0$, so this function can be either monotonically decreasing, constant hazard, or monotonically increasing hazard, depending on the value of the shape parameter c. A standard Gumbel distribution is on the support of the real line $(-\infty ,\infty )$. A truncated Gumbel distribution on the support of $t\ge 0$, with a scale factor of b, has the density of $f\left(t\right)=\frac{exp[-\frac{t}{b}-exp(-\frac{t}{b})]}{b[1-exp(-1\left)\right]}$. The resulting $g\left(t\right)$ function is $\frac{1}{b}[1-exp(-\frac{t}{b})]$. So, $g^{\prime }\left(t\right)=\frac{exp(-\frac{t}{b})}{b}>0\forall t\in [0,\infty )$, and it thus follows from Theorem 2 that the truncated Gumbel distribution has monotonically increasing hazard. Note that a subprobability version of the truncated Gumbel distribution has the same value for $g^{\prime }\left(t\right)>0\forall t\in [0,\infty )$, so it too has monotonically increasing hazard. These facts about the Gumbel distribution will be discussed later in the application of hazard functions to the question of memory retention.

If $S_i\left(x\right)$ is the survivor function for the ith component of a disjunctive hazard system, then a key property for disjunctive hazard systems follows when the $S_i\left(x\right)$ are independent.

Theorem 4.

The hazard for a disjunctive hazard system of n independent component is $h_{dis}={\sum }_{i=1}^n{h}_i\left(x\right)$, where $h_i\left(x\right)$ is the hazard for the ith component.

Proof. 

The survivor function for a system of n independent components is $S_{dis}={\prod }_{i=1}^n{S}_i\left(x\right)$. Thus, it follows that

$$\begin{array}{ccc}\hfill f_{dis}\left(x\right)& =& \sum _{i=1}^n-S_i^{\prime }\left(x\right)\prod _{j\ne i}{S}_j\left(x\right),\hfill \\ & =& \sum _{i=1}^n{f}_i\left(x\right)\prod _{j\ne i}{S}_j\left(x\right),\hfill \\ \hfill h_{dis}\left(x\right)& =& \frac{{\sum }_{i=1}^n{f}_i\left(x\right){\prod }_{j\ne i}{S}_j\left(x\right)}{{\prod }_{i=1}^n{S}_i\left(x\right)},\hfill \\ & =& \sum _{i=1}^n{h}_i\left(x\right).\hfill \end{array}$$

   □

If each unit of the system has the same unit hazard function (i.e., $h_i\left(x\right)=h\left(x\right)$), then the hazard for a disjunctive system where all n are required is $h_{dis}\left(x\right)=nh\left(x\right)$. Clearly complex systems with many essential constituent components have a short lifetime. For example, if we define the half-life duration ${\tau }_{0.5}$ as the time a system survivor function $S_{dis}\left({\tau }_{0.5}\right)=\frac{1}{2}$, the value of ${\tau }_{0.5}$ will be smaller as the number of essential components increases. For example, if $S_i\left(t\right)=e^{-kt}$ for each component, then $S_{dis}=e^{-nk{\tau }_{0.5}}=\frac{1}{2}$, so ${\tau }_{0.5}=\frac{log2}{kn}$.

2.9. Conjunctive Hazard Systems

A key idea underlying the engineering of a highly reliable system is having independent redundant systems. For example, an air-traffic control system has one or more backup systems. If the main system fails, then the backup system is able to maintain the functionality of air-traffic guidance. For systems of this type, the failure of the system requires the failure of all the independent subsystems. Let us call such arrangements a conjunctive hazard system. The cumulative distribution for a conjunction arrangement is $F_{con}\left(x\right)={\prod }_{i=1}^n{F}_i\left(x\right)$.

Theorem 5.

The hazard of a conjunctive system, which only fails when all the of the n independent component systems fail, is $h_{con}\left(x\right)=\frac{{\sum }_{i=1}^n{f}_i\left(x\right){\prod }_{j\ne i}{F}_j\left(x\right)}{1-{\prod }_{i=1}^n{F}_i\left(x\right)}$.

Proof. 

The probability density for the conjunction system is $f_{con}\left(x\right)={\sum }_{i=1}^n{f}_i\left(x\right){\prod }_{j\ne i}{F}_j\left(x\right)$. It thus follows that $h_{con}\left(x\right)=\frac{{\sum }_{i=1}^n{f}_i\left(x\right){\prod }_{j\ne i}{F}_j\left(x\right)}{1-{\prod }_{i=1}^n{F}_i\left(x\right)}$.    □

To see how a conjunctive hazard system improves the half-life duration, let us consider the case where each subsystem has a survivor function of $S_i\left(t\right)=e^{-kt}$. The resulting survivor function for the conjunctive system at the system half-life is $S_{con}\left({\tau }_{0.5}\right)=1-{(1-e^{-k{\tau }_{0.5}})}^n=\frac{1}{2}$. So, it follows that ${\tau }_{0.5}=\frac{1}{k}log[\frac{1}{1-{\left(\frac{1}{2}\right)}^{\frac{1}{n}}}]$. To see the benefit for the case of a single redundant backup (i.e., $n=2$), let us compute the quantity $\frac{{\tau }_{0.5}(n=2)}{{\tau }_{0.5}(n=1)}$. This ratio for the exponential individual survivor function is approximately $1.772$. Thus, the backup results in a $77.2$-percent increase in the system half-life duration.

It is also instructive to see how the hazard of a two-component conjunctive system compares to the hazard rate of the individual components. From Theorem 5, if both components are identical, then

$$\begin{array}{ccc}\hfill h_{con}& =& \frac{2f\left(x\right)F\left(x\right)}{1-{\left[F\left(x\right)\right]}^2},\hfill \\ & =& \frac{2f\left(x\right)F\left(x\right)}{[1-F(x\left)\right][1+F(x\left)\right]},\hfill \\ & =& \frac{2F\left(x\right)}{1+F\left(x\right)}h\left(x\right).\hfill \end{array}$$

Note for all values for the cumulative distribution where $0<F\left(x\right)<1$, $h_{con}\left(x\right)<h\left(x\right)$, but the magnitude of this inequality is dependent on the value for $F\left(x\right)$. When the overall risk is low, say when $F\left(x\right)=0.05$, then $h_{con}\left(x\right)$ is approximately $0.0952h\left(x\right)$. However, when the overall risk is high, say when $F\left(x\right)=0.8$, then $h_{con}\left(x\right)$ is approximately $0.8889h\left(x\right)$. Thus, a conjunctive system is particularly advantageous for reducing system failure when the overall risk is low.

3. Hazard Functions for the Candidate Models for Memory Retention

Since the original experimental study of human memory by Ebbinghaus [11] in 1885, there have been more than 200 published papers in the psychological literature that have studied memory retention as a function of the time after encoding. See [42,43] for reviews of this literature and for a discussion about some models for a memory retention function. Nine candidate models for the survivor function $S\left(t\right)$ are provided in

Table 1. Nine candidate models for the memory survivor function $S\left(t\right)$ are shown along with the corresponding hazard function. The parameters a, b, c and d are positive. The cumulative functions for the two respective traces of the two-trace hazard model are $F_1\left(t\right)=1-exp(-d{t}^2)$ and $F_2\left(t\right)=b(1-exp(-a{t}^c)$.

Model Name	Survivor Function	Hazard Function
modified hyperbolic	$1-b+\frac{b}{at+1}$	$\frac{ab}{{(at+1)}^2[1-b+\frac{b}{at+1}]}$
modified exponential	$1-b+b{e}^{-at}$	$\frac{a}{\frac{1-b}{b}{e}^{at}+1}$
modified logarithm	$1-b\frac{log(t+1)}{log(U+1)}$	$\frac{b}{(t+1)log\frac{(U+1)}{{(t+1)}^b}}$
modified power	$1-b+\frac{b}{{(t+1)}^c}$	$\frac{cb}{(1-b){(t+1)}^{c+1}+b(t+1)}$
modified single-trace fragility	$1-d+\frac{d{e}^{-at}}{{(1+bt)}^c}$	$\frac{da+\frac{cbd}{1+bt}}{d+e^{at}(1-d){(1+bt)}^c}$
modified Anderson-Schooler	$1-d+\frac{db}{b+t^c}$	$\frac{dbc}{t^{1-c}[(1-d){(b+t^c)}^2+db(b+t^c)]}$
Ebbinghaus	$\frac{b}{b+{[log(t+1)]}^c}$	$\frac{c{[log(t+1)]}^{c-1}}{(t+1)(b+{[log(t+1)]}^c)}$
trace susceptibility theory	$1-b+b{e}^{-a{t}^2}$	$\frac{2abt{e}^{-a{t}^2}}{1-b+b{e}^{-a{t}^2}}$
two-trace hazard theory	$1-F_1\left(t\right)F_2\left(t\right)$	$\frac{2dt{e}^{-d{t}^2}{F}_2\left(t\right)+bac{t}^c{e}^{-a{t}^c}{F}_1\left(t\right)}{1-F_1\left(t\right)F_2\left(t\right)}$

along with the corresponding hazard function. Each of these models meet the aforementioned constraints of $S\left(0\right)=1$ and $S\left(U\right)=1-\delta$ for $0\le \delta \le 1$ because we want to study the survival of memories that were initially encoded correctly at time $t=0$ and because we want to incorporate the feature that some proportion of the memories might survive over the entire lifetime of the individual. Consequently some of the models listed in Table 1 are modified from the form that was originally proposed so as to satisfy the above two boundary-conditions. In this section each of the candidate models is discussed briefly.

3.1. Modified-Hyperbolic Model

Several researchers in animal learning proposed the retention function of $S\left(t\right)=\frac{1}{at+c}$ [44,45]. Note that for $c>1$ this function does not satisfy the constraint that $S\left(0\right)=1$. This feature is likely to be intentional because animals rarely perform the task perfectly initially. These nonverbal organisms need to be trained to do the task. Yet after extensive training the animals will often not be 100 percent consistent in executing the task properly. To modify the original equation to meet the $S\left(0\right)=1$ constraint, the value of c can be set to 1. Chechile [43] further modified the hyperbolic model to include for the possibility of long-term memory retention for a subset of the memories. Hence, in the modified model the survivor function is $1-b+\frac{b}{at+1}$. If $b=1$, then there are no lifetime memories, but for $0<b<1$, there are $1-b$ long-term memories.

The hazard function for the modified-hyperbolic model has a constant numerator along with a denominator that increases with time. Consequently, the hazard is monotonically decreasing with time. Because the hazard is monotonically decreasing, it also follows from Corollary 1 that the density $f\left(t\right)$ is also monotonically decreasing. Moreover ${lim}_{t\to U}f\left(t\right)\to 0$.

The fact that memory hazard for long times is decreasing is unlike the typical hazard for manufactured products. This property about memory has been articulated by Jost as far back as 1897 [46]. Jost’s law is the claim that if there are two memories of equal strength, then it is more likely the more recent memory will be lost.

3.2. Modified Exponential Model and the Multiple-Store Model

The modified exponential model was discussed previously in Section 2.3 as an example of a subprobability model. Recall that the exponential model has constant hazard, so it is inconsistent with Jost’s law. However, the subprobability version of the exponential model had monotonically decreasing hazard and thus is consistent with Jost’s law. However, there is another more important reason for justifying the modified exponential model as a viable candidate for the retention function because it represents a form of a highly regarded multiple-store model of memory [47,48].

For the multiple-store memory framework, the initial encoding results in either (1) the information being transferred from a temporary storage buffer to a permanent store, which is postulated to last over the lifespan of the person, or (2) the information remains in a short-term memory. An item in the short-term store can survive for awhile until some new information knocks it out. This theory is a mixture model where $1-b$ proportion of the memories are immune to memory loss, and b proportion of the studied items are 100-percent susceptible to loss eventually [47]. Thus, the multiple-store model does not have two representations that are both susceptible to loss of information [47]. Memory failures for information in the long-term store is assumed to be a temporary retrieval problem; the target is still assumed to be available for recall at a later time. Thus, storage loss occurs when information is not transferred to the long-term store, and it is knocked out of the short-term store by other incoming information. A Markov-chain model can be used to characterize the storage loss from the short-term store. Suppose b proportion of the targets are exclusively in the short-term store, so $1-b$ proportion of the targets are in the long-term store. Furthermore, suppose each post-encoding competing event has a probability of $1-q$ for displacing the target in the short-term store. The resulting cumulative discrete probability for memory loss as a function of n post-encoding events is

$$\begin{array}{ccc}\hfill F\left(n\right)& =& b(1-q)[1+q+q^2+\cdots +q^{n-1}],\hfill \\ & =& b(1-q)\frac{(1-q^n)}{1-q}=b(1-q^n).\hfill \end{array}$$

Since in general we do not know the value of n, it is reasonable to assume that $n=mt$ where $m>0$. This assumption enables a re-expression of the cumulative probability as a function of time. It is also convenient to express the term $q^{mt}$ as $e^{-at}$, where $a=mlog\frac{1}{q}>0$. Thus, the survivor function is $S\left(t\right)=1-b+b{e}^{-at}$. As shown in Section 2.3, the resulting hazard function is monotonically decreasing. It also follows from Corollary 1 that the density function is monotonically decreasing with ${lim}_{t\to U}f\left(t\right)\to 0$.

3.3. Modified Logarithm Model

In a classic book on experimental psychology [49], the memory retention function was hypothesized to be $b-mlog\left(t\right)$, but this proposal does not satisfy two memory boundary conditions. The modified logarithm version shown in Table 1 corrects the original proposal so as to enable $S\left(0\right)=1$ and to allow for some long-term memory. The modified survivor function is $1-\frac{blog(t+1)}{log(U+1)}$ for $t\in [0,U]$. The corresponding density function is $f\left(t\right)=\frac{b}{(t+1)log(U+1)}$ and ${lim}_{t\to U}f\left(t\right)={lim}_{t\to U}\frac{b}{(t+1)log(U+1)}\to 0$. Furthermore the corresponding $g\left(t\right):=\frac{-f^{\prime }\left(t\right)}{f\left(t\right)}$ function is simply $\frac{1}{(t+1)}$. Because $g^{\prime }\left(t\right)=\frac{-1}{{(t+1)}^2}<0\forall t\in [0,U]$, it follows from Theorem 2 that the hazard is monotonically decreasing.

3.4. Modified Power Model

Several investigators (e.g., [50,51]) argued for a power function survivor model of the form $\frac{b}{t^c}$. Similar to the other modified models, the original function is modified to satisfy the two memory boundary conditions. The resulting hazard function shown in Table 1 is monotonically decreasing because the numerator is a constant and the denominator increases with t. It also follows from Corollary 1 that the density function is monotonically decreasing with ${lim}_{t\to U}f\left(t\right)={lim}_{t\to U}\frac{bc}{{(t+1)}^{c+1}}\to 0$.

3.5. Modified Single-Trace Fragility Theory

Wickelgren [52] developed a single-trace model where the strength decreased as a function of time. If the maximum strength at $t=0$ is rescaled to be 1 rather than a parameter $\rho$, then the single-trace fragility theory would predict a survivor function of $\frac{exp(-at)}{{(1+bt)}^c}$. For this form of the single-trace fragility model, the hazard function would be $a+\frac{cb}{1+bt}$, which is monotonically decreasing. However, this form of the survivor function does not allow for any lifetime memories, but with the modification shown in Table 1, $1-d$ proportion of the traces survive indefinitely. Nonetheless the hazard function shown in the table is still monotonically decreasing in time because the numerator is decreasing with time while the denominator is increasing with increasing time. Because of Corollary 1, it is known that the density function is monotonically decreasing. Moreover ${lim}_{t\to U}f\left(t\right)\to 0$.

3.6. Modified Anderson-Schooler Model

Anderson and Schooler [53] proposed another version of a power model for a memory retention function. These authors suggest the function $S\left(t\right)=\frac{b}{b+t^c}$, which has the hazard function of $h\left(t\right)=\frac{c}{t+b{t}^{1-c}}$. The hazard is monotonically decreasing for $0<c\le 1$ because the denominator is increasing with time and the numerator is a constant. For $c>1$, $h\left(0\right)=0$, but for larger values of t, the hazard is decreasing because the denominator is approximately t. Consequently, for $c>1$ this function has a peak-shaped hazard profile. However, the Anderson-Schooler version of the retention function does not enable the possibility of some lifetime memories. Hence, the modified survivor function shown in Table 1 provides for long-term memories. Despite this modification to the Anderson-Schooler function, the hazard has the same properties as did the original model. That is, the hazard is monotonically decreasing if $0<c\le 1$, but it has a peak shape for $c>1$.

3.7. Ebbinghaus Model

This model is the original memory retention function from the first experimental psychologist to examine memory [11], although the logarithm term has been adjusted to $log(t+1)$ rather than $logt$ to avoid the problem of the term being undefined at $t=0$. The resulting hazard function shown in Table 1 is only defined for $c>1$ because if c were less than 1, then the hazard would diverge at $t=0$. However, with $c>1$, the hazard function is zero at $t=0$. Moreover, for large t, the hazard is monotonically decreasing. Thus, the hazard function has a peak shape for $c>1$.

3.8. Trace Susceptibility Theory

Unlike the previously discussed models, the trace susceptibility theory was designed with hazard in mind from the outset [17]. This theory is based on the premise that the memory consists of a network of features that becomes insufficiently stored when the weakest link is destroyed. Given the fact that the Weibull distribution is one of the two asymptotic limiting forms for a disjunction collection of features, the trace susceptibility theory is based on an assumed Weibull model, but to enable lifetime memories, it is a Weibull subprobability model. The shape parameter of the Weibull subprobability model is stipulated to be 2. Recall that a Weibull probability model with a shape parameter greater than 1 has increasing hazard, but a Weibull subprobability model with a shape parameter greater than 1 has peak-shaped hazard as discussed in Section 2.4.

3.9. Two-Trace Hazard Model

The previously discussed hazard models for memory were based on a single memory encoding or trace. In contrast to these single-representation models, the two-trace hazard model is a conjunctive hazard system [43]. Thus, a key feature of this model is the redundant encoding of the information. Survival of either of two traces is sufficient for the memory of an event to be retained. Trace 1 is hypothesized to be a Weibull probability model with a shape parameter of 2. The survivor function for Trace 1 is $S_1\left(t\right)=e^{-d{t}^2}$ and the hazard function for this trace is $h_1\left(t\right)=2dt$, which is clearly an increasing hazard function. Thus, Trace 1 is not likely to survive over a long time period. Trace 2 is based on a Weibull subprobability model with a shape parameter $0<c\le 1$. The survivor function for Trace 2 is $S_2\left(t\right)=1-b+b{e}^{-a{t}^c}$. The corresponding hazard function for Trace 2 is

$$\begin{array}{ccc}\hfill h_2\left(t\right)& =& \frac{ac}{t^{1-c}\left(e^{at}\frac{1-b}{b}+1\right)}.\hfill \end{array}$$

The hazard for Trace 2 is monotonically decreasing because the numerator is a constant and the denominator is an increasing function of time. Consequently only Trace 2 is consistent with Jost’s law. So what is the hazard function for the system of two traces? Given $F_1\left(t\right)=1-S_1\left(t\right)$ and $F_2\left(t\right)=1-S_2\left(t\right)$, the resulting hazard function is

$$\begin{array}{ccc}\hfill h\left(t\right)& =& \frac{2dt{e}^{-d{t}^2}{F}_2\left(t\right)+bac{t}^c{e}^{-a{t}^c}{F}_1\left(t\right)}{1-F_1\left(t\right)F_2\left(t\right)},\hfill \end{array}$$

Because $F_1\left(0\right)=0$ and $F_2\left(0\right)=0$, it follows that $h\left(0\right)=0$. However, for larger values for t, the conjunctive hazard converges to $h_2\left(t\right)$, which is a monotonically decreasing function of t. Thus, the hazard profile for the system of two traces is peak shaped (i.e., rising from zero initially before eventually decreasing for increasing time).

3.10. Scale Invariance and the Number of Fitting Parameters

Although all the survivor functions listed in Table 1 are a function of t, not all of the functions are scale invariant. The modified exponential model is an example of a model that is time-scale invariant. Suppose time is measured in units of seconds and with the parameter $a=0.3$, the same survivor function can also represent the case where time is measured in units of minutes and where $a=18$ because $a=\frac{0.3}{sec}\times \frac{60sec}{min}=\frac{18}{min}$. The value of the a parameter is scale sensitive, but the survivor function is free of measurement units. As in the physical sciences, it is important for the parameters to have measurement-scale dependence rather than the physical law having scale dependence. For example, in the classical formula for gravitation, $F=G\frac{m_1{m}_2}{r^2}$, the force F, the masses $m_1$ and $m_2$, and the distance r can be expressed in a variety of unit systems without changing the law. The only thing that does change is the value and the units of the G parameter. Five models in Table 1 are time-scale invariant in a similar fashion, but the other four models in the table are dependent on the units of time measurement employed. Consequently, these four models for memory hazard ought to be modified to be time-scale invariant.

The four models that require recasting for time-scale invariance are: the modified logarithm, the modified power, the modified Anderson-Schooler, and the Ebbinghaus function. The adjusted survivor functions for these models are shown in

Table 2. The nine candidate models for the memory survivor function $S\left(t\right)$ that are time-scale invariant. The parameters a, b, c and d are positive. The cumulative functions for the two respective traces of the two-trace hazard model are $F_1\left(t\right)=1-exp(-d{t}^2)$ and $F_2\left(t\right)=b(1-exp(-a{t}^c)$.

Model Name	Survivor Function	# Parameters
modified hyperbolic	$1-b+\frac{b}{at+1}$	2
modified exponential	$1-b+b{e}^{-at}$	2
modified logarithm	$1-b\frac{log(at+1)}{log(aU+1)}$	3
modified power	$1-b+\frac{b}{{(at+1)}^c}$	3
modified single-trace fragility	$1-d+\frac{d{e}^{-at}}{{(1+bt)}^c}$	4
modified Anderson-Schooler	$1-d+\frac{db}{b+{\left(at\right)}^c}$	4
Ebbinghaus	$\frac{b}{b+{[log(at+1)]}^c}$	3
trace susceptibility theory	$1-b+b{e}^{-a{t}^2}$	2
two-trace hazard theory	$1-F_1\left(t\right)F_2\left(t\right)$	4

along with the other functions.

The adjustment to the four models to meet the criterion of being time-scale invariant alters the specific hazard function, but not so much to change either their monotonicity or peakedness. The hazard for the time-scale invariant form of the modified logarithm model is

$$\begin{array}{ccc}\hfill h\left(t\right)& =& \frac{ab}{(at+1)log\frac{aU+1}{{(at+1)}^b}}.\hfill \end{array}$$

The hazard for the time-scale invariant version of the modified power model is

$$\begin{array}{ccc}\hfill h\left(t\right)& =& \frac{abc}{(1-b){(at+1)}^{c+1}+b(at+1)}.\hfill \end{array}$$

The hazard for the time-scale invariant form of the modified Anderson-Schooler model is

$$\begin{array}{ccc}\hfill h\left(t\right)& =& \frac{dbc{a}^c}{t^{1-c}\left((1-d){[b+{\left(at\right)}^c]}^2+db[b+{\left(at\right)}^c]\right)}.\hfill \end{array}$$

Finally the time-scale invariant version of the Ebbinghaus function is

$$\begin{array}{ccc}\hfill h\left(t\right)& =& \frac{ac{[log(at+1)]}^{c-1}}{(at+1)(b+{[log(at+1)]}^c)}.\hfill \end{array}$$

4. Some Empirical Evidence about Memory Hazard

The nine proposals for the memory hazard were informed by results from experiments of different types that span different time frames [42]. Nonetheless, there is one agreed upon pattern – after some initial duration, the hazard is considered to be decreasing. The first five functions shown in Table 1 hypothesize a monotonically decreasing hazard function for all time, but the last four functions predict a peak-shaped hazard function. Consequently, a critical question to assess is: does hazard immediately after encoding increase or does hazard decrease monotonically? To answer this question, experiments need to focus on the time period shortly after memory encoding because that is the temporal region where the models differ.

Yet it is technically challenging to obtain an empirical hazard value. If an experiment measures the memory after five different times, say $t_1,\cdots ,t_5$, then it is possible to find the proportion of times where the target information is correct for each time (i.e, values for $S_i\left(t_i\right)$ for $i=1,\cdots ,5$ can be measured). However, to obtain a valid $S\left(t\right)$ measurement, corrections are needed to take into account guessing processes and to account for the possibility that the performance might be also a function of retrieval processes. Thus, to obtain credible values for $S\left(t_i\right)$ requires a validated psychometric model so as to extract a measure of the probability of target storage [18]. However, even with these correction procedures, an empirical measurement of hazard has further challenges to obtain an estimate of $f\left(t\right)$, which is required to compute hazard. To estimate $f\left(t\right)$ a second $S(t_i+\Delta t)$ measurement is required. Empirical hazard estimates have potentially high statistical error because there is likely a large error in estimating $S\left(t_i\right)-S(t_i+\Delta t_i)$ that is compounded by dividing by $S\left(t_i\right)\times \Delta t$. Consequently, valid and statistically reliable empirical hazard estimates are very difficult to obtain in practice. However, it is possible to use another surrogate function to glean some useful information about the hazard function without having to actually estimate the hazard function directly.

4.1. Using a Surrogate Function for Assessing the Hazard Function

The function $u\left(t\right):=\frac{F\left(t\right)}{t}$ can be used to ascertain if the early part of the hazard function is increasing or decreasing [43,54]. Experimentally $u\left(t\right)$ can be readily estimated because t is known as is $F\left(t\right)=1-S\left(t\right)$. The rationale for how $u\left(t\right)$ can be used as a surrogate function for ascertaining hazard properties is established with the following two theorems.

Theorem 6.

If a continuous probability or subprobability on support $t\ge 0$ has monotonically decreasing hazard, then $u\left(t\right)=\frac{F\left(t\right)}{t}$ is monotonically decreasing with a maximum value of $u\left(0\right)=f\left(0\right)$.

Proof. 

Given a monotonically decreasing hazard function on the support $t\ge 0$, it is known from Corollary 1 that the density $f\left(t\right)$ is monotonically decreasing for all t with the maximum at $t=0$. From L’Hospital’s rule it follows that ${lim}_{t\to 0}u\left(t\right)={lim}_{t\to 0}\frac{F\left(t\right)}{t}=f\left(0\right)$. Furthermore,

$$\begin{array}{ccc}{u}^{\prime }\left(t\right)\hfill & =\hfill & \frac{F^{\prime }\left(t\right)t-F\left(t\right)}{t^2}\hfill \end{array}$$

(21)

$$\begin{array}{ccc}\hfill & =\hfill & \frac{1}{t}[f\left(t\right)-u\left(t\right)],\hfill \end{array}$$

(22)

For any $t>0$, $u\left(t\right)=\frac{{\int }_0^{t}f\left(r\right)dr}{t}$. However, from the mean-value theorem of integral calculus, the integral ${\int }_0^{t}f\left(r\right)dr=tf\left(t_m\right)$, where $0<t_m<t$, so $u\left(t\right)=\frac{tf\left(t_m\right)}{t}=f\left(t_m\right)$. Thus, from (22), it follows that the sign of $u^{\prime }\left(t\right)$ is directly a function of the term $f\left(t\right)-f\left(t_m\right)$. Because $f\left(t\right)$ is monotonically decreasing and because $t_m<t$, it follows that $u^{\prime }\left(t\right)<0$. Hence for monotonically decreasing hazard functions on support of $t\ge 0$, $u\left(t\right)$ must be monotonically decreasing for all t with a maximum at $u\left(0\right)$.    □

Theorem 7.

If the function $u\left(t\right)=\frac{F\left(t\right)}{t}$ is monotonically increasing for $t\in [0,t_p]$, then the hazard $h\left(t\right)$ is increasing in that interval.

Proof. 

For all $t\in [0,t_p]$, $u^{\prime }\left(t\right)=\frac{1}{t}[f\left(t\right)-u\left(t\right)]$ with $u\left(t\right)=\frac{tf\left(t_m\right)}{t}=f\left(t_m\right)$ where $0<t_m<t_p$. Because $u\left(t\right)$ is increasing over the interval $[0,t_p]$, it follows that $f\left(t\right)-f\left(t_m\right)>0$ over this interval. Thus, $f^{\prime }\left(t\right)>0$ over this interval, and it follows that $g\left(t\right)=\frac{-f^{\prime }\left(t\right)}{f\left(t\right)}<0$. However, from Equation (10), it follows that $h\left(t\right)$ is increasing because $h\left(t\right)>g\left(t\right)$ in this interval.    □

Theorems 6 and 7 indicate that there is a way to critically falsify any theory for memory hazard that predicts strictly monotonically decreasing hazard. Namely if evidence can be found that $u\left(t\right)$ is initially increasing, then that finding would be falsifying evidence for theories that predict monotonically decreasing hazard. Yet to find a statistically reliable difference in $u\left(t\right)$ estimates requires a reasonably large sample size, which leads to the practical reality that experimentation on $u\left(t\right)$ requires pooling data over different research participants and test trials. However, from Theorem 3, it is known that a mixture of non-increasing hazard functions results in a monotonically decreasing hazard. Consequently if all participants in a group have monotonically decreasing hazard, then the hazard function for the pooled data is also monotonically decreasing. Thus, an experiment that has a large sample size for each retention interval and has research protocols that enable model-based correction estimates for the probability for memory storage has a chance to see if $u\left(t\right)$ is increasing in the early part of the retention interval. Of course if the experiment does not examine time values in the critical period where a hazard function peak exits, then the study might miss detecting the hazard peak.

Experiment 3 from [25] was designed in a fashion to enable estimates of $u\left(t\right)$ for various short retention intervals, although the measurement of $u\left(t\right)$ was not the purpose of the study.(The study addressed a different set of questions associated with implicit and explicit memory. The experiment was designed to see if implicit memory had a different retention function than explicit memory. Implicit memory is exemplified when a participant cannot recall or confidently recognize an item but nonetheless can correctly select the item from a lineup list.) The study examined the memory of novel nonsense letters taken from a stimulus set that was scaled to be of low meaningfulness [55]. Following the presentation of a target letter triad, there was a series of digits that participants had to verbally repeat as soon as they heard the digits. The target letters and the digits were presented over headphones. The testing involved a decision about four visually displayed letter triads. For half of the test probes none of the stimuli matched the target item for that trial, and the other half had one of the stimuli match the target. The participant had to respond yes or no if any of the triads on the four-item list of triads was the target for that trial. The participants also had to indicate if they were of high or low confidence in their decision.(Given that the original paper was focused on measuring explicit and implicit memory, there was also a series of forced-choice tests after the yes/no decision; however, those forced-choice data are not pertinent for the $u\left(t\right)$ analysis.) For each retention interval there were 696 tests where the target is on the list and another 696 tests where the target was not on the list.

The analysis in the current paper is based on model 7B from [23] for estimating the storage probability for each retention interval. The data needed for using model 7B is based on a yes-no recognition memory test along with high-versus-low confidence judgments.

Table 3. The frequencies labels are shown for the various response categories for the experimental task that is associated with model 7B from [23]. Participants responded either yes or no to the test probe and rated their decision either as sure or unsure. The test probes were either old or new corresponding to the probe, respectively, being a memory target or a novel item. For each outcome in the Table there is also a population proportion ${\varphi }_i$, $i=1,\cdots ,8$.

Test Type	No Sure	No Unsure	Yes Unsure	Yes Sure
old	$n_1$	$n_2$	$n_3$	$n_4$
new	$n_5$	$n_6$	$n_7$	$n_8$

defines the response cells frequencies $n_i$ associated with the experimental task. The frequencies for old recognition tests $n_1,\cdots ,n_4$ form a four-cell multinomial that has in the population the proportions ${\varphi }_1,\cdots ,{\varphi }_4$ where ${\sum }_1^4{\varphi }_i=1$. Similarly for the new recognition test there are population proportions ${\varphi }_5,\cdots ,{\varphi }_8$ where ${\sum }_5^8{\varphi }_i=1$. The proportions for the cells of the two multinomials are a function of memory and task factors. Psychological processes such as sufficient storage, guessing when target storage is insufficient, and rating processes influence the proportions in the multinomial outcome cells. Model 7B is a psychometric model that describes how those psychological processes map onto the multinomial outcomes. The parameters of the 7B model are denoted by subscripted $\theta$ symbols for processes such as for target storage (${\theta }_S$), guessing yes on old recognition (${\theta }_g$), guessing no on new recognition (${\theta }_{g^{\prime }}$). There are also parameters for using the sure versus unsure rating when guessing. The model structure consists of two probability trees for the old and new recognition testing procedures. Psychometric models of this type have come to be called multinomial processing tree (MPT) models.

The latent $\theta$ parameters can be statistically estimated by means of a procedure called population parameter mapping (PPM) [56,57,58]. The parameters of model 7B can also be estimated with either a frequentist maximum likelihood method or with a conventional Bayesian method, but the PPM procedure provides additional information about model coherence and is computationally faster [23]. PPM is a specialized Bayesian Monte Carlo sampling procedure developed to estimate the latent parameters of a scientific model for multinomial data. This procedure was invented as a Bayesian Monte Carlo sampling method that avoided the lower efficiency of Markov chain Monte Carlo (MCMC) algorithms. Unlike MCMC procedures, the PPM procedure is not an approximate method, and it does not require a burn-in period [58]. Moreover, the sampled vectors are independent, so the issue of correlated samples, which can occur with MCMC algorithms, is not a problem. With multinomial data there is an exact Monte Carlo sampling method for sampling vectors of points $\left({\varphi }_i\right)$ from the posterior distribution for sets of multinomial data [56,57,58]. Each vector from $\varphi$ spaced is mapped to a corresponding $\theta$-space vector for the scientific model parameters. The collection of coherently mapped points from $\varphi$ space to $\theta$ space are used for point and interval estimation of the latent model parameters. Chechile [23] proved that the 7B model is identifiable and demonstrated that the PPM estimation method out performed (in terms of accuracy) the frequentist maximum likelihood estimation, especially for sample sizes less than 1000. For sample sizes greater than or equal to 1000, the maximum likelihood estimate and the PPM estimate are virtually the same. See the Appendix A for additional details about the PPM method for estimating the storage parameter for model 7B as well as for seeing the software for implementing the estimation.

The frequencies for the various temporal delays for experiment 3 from [25] are provided in

Table 4. The table provides the response frequencies from experiment 3 in [25] for each temporal delay.

Time	$({\mathbf{n}}_1,\cdots ,{\mathbf{n}}_4)$	$({\mathbf{n}}_5,\cdots ,{\mathbf{n}}_8)$
1.33	(13, 4, 9, 670)	(658, 28, 4, 6)
2.33	(33, 21, 39, 603)	(610, 69, 6, 11)
5	(81, 79, 165, 371)	(378, 223, 76, 19)
13	(42, 194, 275, 185)	(163, 341, 177, 15)
31	(36, 294, 265, 101)	(123, 369, 188, 16)

. The time values shown in the table are one second longer than the digit-repeating interval in order to give the participants a chance to read the four-item visual list to which they had to respond either yes or no. The participants were instructed to respond yes if any of the four items on the test probe was the target stimulus for that trial. Perfect performance on this task would correspond to 696 trials of correct performance with high confidence on the target-present tests and 696 trials of correct performance with high confidence on the target-absent tests. These values were used to create an initial set of ${\theta }_S\left(0\right)$ values as a baseline for finding $u\left(t\right)$.

The key information for the $u\left(t\right)$ analysis for the data from experiment 3 in [25] is provided in

Table 5. Estimates for $u\left(t\right)$ for data from experiment 3 in [25].

Time in s.	Storage Estimate	u(t)	95 % u(t) Interval
1.33	0.938	0.041	[0.002, 0.088]
2.33	0.764	0.098	[0.061, 0.178]
5	0.470	0.104	[0.091, 0.116]
13	0.230	0.059	[0.054, 0.062]
31	0.111	0.028	[0.027, 0.030]

. For each t value the posterior median for ${\theta }_S\left(t\right)$ is provided. The corresponding $u\left(t\right)$ is the median of the set of mapped values for $u\left(t\right)=\frac{1}{t}[{\theta }_S\left(0\right)-{\theta }_S\left(t\right)]$. The 95-percent interval for each condition is based on the quantiles for this set of $u\left(t\right)$ values. See the Appendix A for the details about the storage estimation software.

There is a clear maximum for $u\left(t\right)$ for $t>1.33$. There is a posterior probability of $0.9788$ that $u(t=2.33)>u(t=1.33)$, and there is a probability of $0.9891$ that $u(t=5)>u(t=1.33)$. For times greater than $t=5$, there is a decline in $u\left(t\right)$ because (1) the probability that $u(t=5)>u(t=13)$ is $0.9975$ and (2) because the probability that $u(t=13)>u(t=31)$ is $0.9994$.

Thus, via Theorem 6 the data from this experiment are inconsistent with any theory that predicts strictly monotonically decreasing hazard. Additionally, via Theorem 7, it is clear that the hazard is increasing over a brief initial interval of approximately five seconds after the initial encoding. Thus, this experiment has falsifying evidence against the first five models listed in Table 2. However, more strongly, the experiment is inconsistent with any model (either an existing theory or a yet to be generated theory) that predicts strictly monotonically decreasing hazard. Yet, given the import of this strong conclusion, it is prudent to explore if this general conclusion can be replicated and supported by other experiments.

Experiment 2 from [17] is a study that provides an assessment of $u\left(t\right)$ over a time frame from 1.33 to 76 s. In this experiment the targets are again letter triads of low meaningfulness. Additionally, the retention interval is filled with digits that had to be repeated (i.e., letter shadowing). The target stimulus, the test probe, and the interpolated digits are all auditory, but the gender for the articulation of the target item and test probe is different from the gender used for the interpolated digits. The probe stimulus for each recognition test was either the original target or a novel triad of nonsense letters. The data for each condition is provided in

Table 6. The table provides the response frequencies from experiment 2 in [17] for each temporal delay.

Time	$({\mathbf{n}}_1,\cdots ,{\mathbf{n}}_4)$	$({\mathbf{n}}_5,\cdots ,{\mathbf{n}}_8)$
1.33	(1, 0, 1, 898)	(880, 6, 9, 5)
2.33	(8, 3, 13, 876)	(858, 26, 15, 1)
5	(49, 80, 142, 629)	(646, 179, 53, 22)
13	(74, 226, 283, 317)	(434, 329, 99, 38)
32.67	(63, 281, 314, 242)	(296, 412, 143, 49)
76	(61, 322, 309, 208)	(244, 462, 166, 28)

.

Similar to the analysis in Table 5, the key results for experiment 2 from [17] are provided. The time values listed in the table are one second longer than the time for digit shadowing in order for the participant to hear the probe stimulus. Perfect performance in this experiment would consist of 900 trials with high confidence correct old recognition, and 900 trials with high confidence correct rejection of the foil probes. Those values were used for generating the vector of baseline ${\theta }_S\left(0\right)$ values need for the $u\left(t\right)$ calculations. For each temporal delay the values for posterior median for ${\theta }_S\left(t\right)$, $u\left(t\right)$ and the 95-percent interval for $u\left(t\right)$ are provided. There is a clear maximum for $u\left(t\right)$ at $t=5$. The posterior probability that $u(t=5)>u(t=2.33)$ is greater than 0.9999, and the probability that $u(t=5)>u(t=13)$ is 0.9971. Furthermore there is a probability of 0.9994 that $u(t=13)>u(t=32.67)$, and a probability of 0.9997 that $u(t=32.67)>u(t=76)$. Thus, this experiment provides further support for the conclusion that memory hazard initially increases before the hazard declines with time.

Experiment 1 from [17] is a third study for which $u\left(t\right)$ can be estimated for several times in the critical time period after memory encoding. The experimental protocols for this experiment are the same as for the study upon which

Table 7. Estimates for $u\left(t\right)$ for data from experiment 2 in [17].

Time in s.	Storage Estimate	u(t)	95 % u(t) Interval
1.33	0.994	0.001	[0.000, 0.007]
2.33	0.969	0.011	[0.000, 0.017]
5	0.630	0.073	[0.062, 0.084]
13	0.230	0.059	[0.054, 0.064]
32.67	0.148	0.026	[0.024, 0.028]
76	0.172	0.011	[0.010, 0.011]

is based, except for the sample size. In this study, there were in each condition 240 old recognition tests and 240 new recognition tests.(There is another set of four retention intervals that were examined with a slower rate for the interpolated digit-shadowing task. These four conditions are omitted because that rate of presenting digits in the retention interval was too slow to prevent participants from rehearsing the target items [17].) For this study there is not a five-second condition, but instead there are two intervals bracketing the five-second retention interval (i.e., intervals of $t=3.67$ and $t=6.33$). The frequency data are in

Table 8. The table provides the response frequencies from experiment 1 in [17] for each temporal delay.

Time	$({\mathbf{n}}_1,\cdots ,{\mathbf{n}}_4)$	$({\mathbf{n}}_5,\cdots ,{\mathbf{n}}_8)$
2.33	(5, 5, 6, 224)	(224, 13, 1, 0)
3.67	(10, 29, 48, 153)	(187, 52, 1, 0)
6.33	(26, 55, 95, 64)	(114, 103, 20, 3)
11.67	(9, 83, 106, 42)	(45, 133, 59, 3)

and the $u\left(t\right)$ results are in

Table 9. Estimates for $u\left(t\right)$ for data from experiment 1 in [17].

Time in s.	Storage Estimate	u(t)	95 % u(t) Interval
2.33	0.908	0.031	[0.000, 0.071]
3.67	0.551	0.116	[0.075, 0.233]
6.33	0.194	0.124	[0.102, 0.144]
11.67	0.147	0.071	[0.060, 0.077]

. Interestingly $u\left(t\right)$ is not statistically different between the two retention intervals that bracket five seconds (i.e., the $t=3.67$ and $t=6.33$ conditions). Both of these conditions did have a reliably larger $u\left(t\right)$ than the $u\left(t\right)$ for the $t=2.33$ condition (with probability greater than $0.985$ for both cases), and both of these conditions had a $u\left(t\right)$ larger than $u\left(t\right)$ for the $t=11.67$ condition (with a probability greater than $0.986$ for both cases).

The above three experiments indicate that the human memory hazard function is not strictly monotonically decreasing as predicted by the first five models listed in Table 2. These studies when analyzed in light of Theorems 6 and 7 establish a framework for using an experimentally practical surrogate function $u\left(t\right)=\frac{F\left(t\right)}{t}$ to ascertain that the human memory system has a peaked-shaped hazard function. This approach has the advantage of learning about hazard empirically without actually estimating hazard per se. The method thus avoids the technical and statistical difficulties of estimating a density-based metric. Yet the research still has the limitation of only examining human participants. It might be possible that some animal species might exhibit a different memory hazard function. However, it is difficult to measure the $u\left(t\right)$ surrogate function in a careful fashion with animals because they need to be trained to do a complex recognition model task. Human participants are capable of following complex instructions that allow for model-based corrections to obtain a measure of memory storage after various retention intervals. Nonetheless it is important to understand the properties of the human memory system even if humans differ from other animals. In the next subsection a case is made for a particular function that has peak-shaped hazard for human memory.

4.2. Evidence for the Two-Trace Hazard Model

Given the results from the previous subsection, attention is focused on the four models that predict a peak-shaped hazard function (i.e., the modified Anderson-Schooler model, the Ebbinghaus function, the trace susceptibility theory, and the two-trace hazard model). For each theory there is a survivor function that predicts target storage as a function of time, and each theory has fitting parameters that can be adjusted for each individual. Although the pooled data were used to assess the $u\left(t\right)$ function, the averaging of nonlinear functions with different fitting parameters can be misleading as shown by Estes [59]. Thus, for testing the fit quality of the various models, it is important to do the testing separately for each person. Moreover, to test the fit quality on an individual basis requires a sizable sample size for each retention interval to reduce the uncertainty in the estimated storage probability. It is also important to assess the storage probability of memory targets over both the short-term and the long-term temporal regions.

In the short-term time frame, experiment 2 from [17] is particularly important because each of 30 participants was evaluated over six retention intervals with 90 replication tests per interval; thus the data in this study enables a reasonably precise estimate of target storage for each condition and for each participant. For this analysis the two-trace hazard model, with its four parameters, was omitted because it could always be as effective as the trace susceptibility theory. Both the two-trace hazard model and the trace susceptibility theory employ a Weibull model. Consequently, if the b parameter of both models are equivalent and if the d parameter of the two-trace hazard model is equated to the a parameter of the trace susceptibility theory, then there are always values for the extra a and c parameters of the two-trace hazard model that effectively match the prediction equation of the trace susceptibility theory.

The measure of fit quality for each model was the correlation between the estimated storage probability ${\theta }_S$ and the predicted value from the theory. The $r^2$ means for the trace susceptibility theory, the Ebbinghaus function, and the modified Anderson-Schooler model are, respectively, $0.938$, $0.905$, and $0.846$ [43]. Importantly, the modified Anderson-Schooler model was not the best fit for any of the 30 participants. The trace susceptibility theory was the best fit for 22 participants and the Ebbinghaus function was the best fit for the other 8 participants. Given the fact that the Ebbinghaus function had a better fit than the trace susceptibility theory for 8 participants, these people were also examined in terms of the two-trace hazard model. The two-trace hazard model had a better fit for each of those 8 participants than did the Ebbinghaus function. Thus, in the short-term time frame, the trace susceptibility and the two-trace hazard model have a better fit to the storage probability than either the modified Anderson-Schooler or the Ebbinghaus model.

There have not been very many studies that report data for individuals in the long-term temporal domain. Nearly all the data sets are based on averaging data over participants. Yet Estes [59] showed that learning and retention curves based on grouped data are problematic. Consequently, Chechile [43] was only able to discuss two studies that had individual-participant data that enabled estimates of the storage probability over a long-term temporal region [11,60]. Chechile [43], who developed both the trace susceptibility theory and the two-trace hazard model, reported that the trace susceptibility theory failed to fit this long-term data, but the two-trace hazard model did fit each participant well. However, the two studies reported by Chechile only had results from three people, who were either the author or a relative of the author [11,60]. However, another study by Sloboda [61] examined nine participants over ten retention intervals in a fashion that enables the estimation of the model 7B storage probability value for each person for each interval. The ten intervals examined were: 30 s, 5 min, 15 min, 30 min, 1 h, 3 h, 9 h, 24 h, 72 h, and 144 h. The target stimuli in the Sloboda experiment were meaningful words selected from a word frequency norm [62]. There were 120 tests for each of the ten retention intervals (i.e., 60 target-present trials and 60 target-absent trials). Sloboda assumed that trace 1 of the two-trace hazard model was no longer available for retention intervals of 5 min or longer. If we denote the storage probabilities for traces 1 and 2 as, respectively, ${\theta }_{S_1}$ and ${\theta }_{S_2}$, then ${\theta }_{S_1}=e^{-d{t}^2}$ and ${\theta }_{S_2}=1-b(1-e^{-a{t}^c})$. The mean value for d was 12.96 min⁻² for the previously reported 30 participants in the short-term time domain; consequently by 5 min the value for ${\theta }_{S_1}$ would be less than ${10}^{-140}$. Thus, Sloboda only fitted ${\theta }_{S_2}=1-b(1-e^{-a{t}^c})$ to the long-term storage probability values. The mean and standard deviation for the fits of the model for the a, b, and c parameters, respectively, are: 1.9(1.1) min⁻², $0.991\left(0.009\right)$, and $0.238\left(0.043\right)$. The resulting fit is excellent for each of the nine participants; the mean correlation between the model predicted values and storage probabilities is $0.947$.

Thus, the currently available data are sufficient to indicate that the modified Anderson-Schooler model, the Ebbinghaus model, and the trace-susceptibility theory have problems fitting all the participants; whereas the two-trace hazard model has an excellent fit to each participant. Consequently, until a better theoretical function is found, the two-trace hazard model is the current best option for a model of memory retention as a function of time. In this model, the storage of the target is ${\theta }_S=1-(1-{\theta }_{S_1})(1-{\theta }_{S_2})$ with ${\theta }_{S_1}=e^{-d{t}^2}$ and ${\theta }_{S_2}=1-b(1-e^{-a{t}^c})$.

5. Discussion

The basis of any human memory is the development of an internal representation of experienced events. However, after the initial encoding there are dynamic changes to the internal representation that alter the likelihood for remembering the original event. The hazard function is a natural mathematical tool for studying the dynamic changes of a stochastic system. Thus, much can be learned about the properties of the mental representation and the process of forgetting by an examination of the memory hazard function. This system-level approach for studying the memory retention process avoids neuroscience measurements of the memory process, but nonetheless the system-level approach results in powerful constraints on memory theory and the structure of the memory representation.

In this paper, the hazard properties of nine models for memory retention were studied; each of these theories emerged from previous research in psychological science. For all of these theories the hazard decreases for long delays, but four of the models predicted hazard is initially increasing in the early stages of memory retention before a peak is reached. By examining a surrogate function, which is easier to experimentally measure than the hazard function, a case is made that hazard has a peak shape. Furthermore, goodness-of-fit analyses of selective studies found that there was evidence against three of the four theories that correctly predicted peak-shape hazard. The sole surviving model is the two-trace hazard model. A central assumption for this theory is that the initial encoding establishes two independent representations or traces. Trace 1 is assumed to have storage probability at time t that is ${\theta }_{S_1}=e^{-d{t}^2}$, which is a Weibull distribution with a shape parameter of 2. The hazard for trace 1 is an increasing function of time, so this trace is not likely to survive very long. Trace 2 is assumed to have a storage probability at time t that is ${\theta }_{S_2}=1-b(1-e^{-a{t}^c})$. Trace 2 is a Weibull subprobability distribution with a shape parameter of $0<c\le 1$. Trace 2 has a monotonically decreasing hazard function. It is possible that some proportion of the trace 2 representations survive over the lifetime of the individual (i.e., $1-b$ proportion of the events can survive where $0\le b\le 1$). The redundant memory traces, where the availability of either trace is sufficient for target memory retention, results in a system that has peak-shaped hazard. It is also a system that has an increased memory span for retaining a set of m concurrent items without memory loss. Given this theory there are a number of interesting questions to discuss.

To begin, it is reasonable to ask: why are there two independent encodings, and why do these traces have survivor functions that are related to a Weibull distribution? The answer to the second of these questions has to do with special properties of the Weibull distribution as a survivor model. Products, which are complex systems with many components that must all function properly, will fail whenever one component of the system fails. It is reasonable to consider a memory encoding as a complex combination of sensory features activated by the target stimuli along with an elaborated set of participant-generated cognitive processes that interpret the current stimulus and link it to temporal and spatial markers as well as link it to previous associations [18]. The encoding is not simply a mapping of an external stimulus to the activation of sensory features, but rather it involves a wide range of participant-generated elaborations that situates the current event into a larger context. This system of activated features and associations is bound together. Although some features of the encoding are trivial aspects of the event, which can be lost without losing the memory of the event, there is a set of essential components that must be retained to have a complete memory of the event. With the loss of any one of the critical components of this bound network results in a change from the initial state of complete memory storage. For a complex system such as a memory representation, the failure of the system is not captured by the mean of the distribution of the component lifetimes; rather it is the distribution for the weakest component. Since time is a nonnegative value, there are two mathematical distributions for the asymptotic limiting form for a minimum lifetime (i.e., the Gumbel and Weibull distributions [41]). However, the Gumbel distribution and a Gumbel subprobability distribution have monotonically increasing hazard for all t, so it is not a suitable model for memory failure because all the theories of memory hazard predict that the hazard is decreasing for long temporal delays. However, the Weibull distribution can have monotonically decreasing hazard, constant hazard, or monotonically increasing hazard, depending on the value of a shape parameter. Thus, the model for trace 1 is a Weibull probability model with a shape parameter set equal to 2, so this trace has increasing hazard. The model for trace 2 is a Weibull subprobability model that has a shape parameter of $0<c<1$, so it has monotonically decreasing hazard. Note that a single Weibull distribution does not have a peak-shaped hazard function, but a conjunctive system of two independent Weibull-related traces with different hazard monotonicity has a peaked-shape hazard function. Hence, one reason for having encoding redundancy is it enables a combination of Weibull systems to have a peak-shaped hazard function.

There is a neuropsychological model of memory encoding that posits multiple representations with different temporal scales (i.e., the Teyler-DiScenna index theory [63]). This neural model describes memory encoding as an interaction between two different brain regions– the hippocampus (a deeper brain structure in both hemispheres) and the neocortex. Previous research has linked the hippocampus with learning (e.g., [64]); whereas other researchers have shown an important role of the neocortex in learning (e.g., [65]). In the indexing theory, the sensory detection of the stimulus activates neocortical cells, but these activated neocortical cells map to the two hippocampi, which in turn can reactivate at a later time the neocortical cells associated with the stimulus. It is outside the scope of this paper to elaborate in detail the indexing theory, but the theory is mentioned here to support the plausibility of two representations of the target event being established.

The two-trace hazard model also has some similarity to an information-processing model of learning and memory – namely the multiple-store model that was discussed in Section 3.2 [47,48]. As the name implies, the multiple-store or the working-memory model has two stores, but the long-term store is hypothesized as a permanent storage system, so not all targets are stored in this system. The essential differences between the two-trace hazard model and the multiple-store model are: (1) all targets in the two-trace hazard model, which have been encoded so that $S\left(0\right)=1$, have two representations, (2) each representation in the two-trace model is susceptible to memory loss rather than only the items in the short-term store, and (3) the overall hazard for the two-trace model has a peak shape, whereas the multiple-store model has monotonically decreasing hazard.

It is also reasonable to ask: why is it that memory hazard is decreasing after the initial short-term increase? Manufactured products tend to wear in time after an initial high-risk, so the hazard for these products is typically a U-shaped function rather than a peak-shaped function. Memory retention has clearly different hazard properties from manufactured products. Two hypotheses have been proposed to explain this difference. The first hypothesis proposes that the memories that survive early threats from competing new information become hardened or immune to the effects of interference. This hypothesis is typically called memory consolidation theory [66,67,68]. Consolidation theory assumes that new encodings are fragile like a fresh painting. However, with the passage of time, the encoding is hardened into a permanent representation like the drying of wet paint. While consolidation theory can account for the reduced tendency for forgetting for older memories, this theory has been critiqued by several investigators [18,69,70]. Among the reasons for doubting the consolidation hypothesis is the forgetting that has been observed when the memory targets are given more training [69]. This effect has caused consolidation theorists to argue that additional learning softens a previously hardened trace [68]. However, critics find this position theoretically problematic. Another reason for doubting the consolidation hypothesis is its failure to account for the phenomenon of storage proactive interference. Proactive interference is the higher rate for forgetting of the recent memories that share some similarity to prior events [17,71]. The hardening of a memory trace with time alone is insufficient to account for proactive interference.

The second hypothesis to account for the reduced risk of older memories stresses the fact that time itself is part of the encoding of information [18]. For example, if the target is the nonsense triad such as BXT, then this stimulus with its ordinal letter positions are linked to a larger spatial and temporal context. Subsequent events are also encoded in terms of their stimulus features along with its temporal/spatial cues. New events compete with the prior events for storage, and the vigor of the interference is related to the degree of similarity between the events. Time is a feature of this assumed similarity-based competition, so events after a much greater time are more likely to compete with other events in that time frame and not with the older memories. Hence the hazard for an older item is reduced from that of the more recent item. Thus, this alternative account for the reduced risk with time is based on the importance of time as an important aspect of memory encoding. Yet the study of time as a separate aspect of memory encoding is still at an early stage of development, so further research is needed to delineate its role in memory organization and recollection [72].

Finally it is important to acknowledge the limitations to the conclusions of this paper. Memory is a basic property of living systems. It is important for all organisms to model the environment to increase the likelihood for survival. However, the hazard properties of memory studied in this paper has been limited to a single species – humans. It may well be the case that other species, which have a divergent evolutionary history, have different memory mechanisms. Consequently the two-trace hazard model might not be valid for other species. Given the importance of having a good model of the environment, it is reasonable to expect that evolutionary mutations that improve memory are likely to emerge. For example, having two representations of events increases the chance that the information is preserved longer to enable more complex cognitive operations. Yet not all evolutionary paths might have this information-processing advantage. The study of animal cognitive processes is methodologically challenging, but a memory hazard function is quite general as are the theorems highlighted in this paper.