Application of an Intensive Longitudinal Functional Model with Multiple Time Scales in Objectively Measured Children’s Physical Activity

Abstract

This study proposes an intensive longitudinal functional model with multiple time-varying scales and subject-specific random intercepts through mixed model equivalence that includes multiple functional predictors, one or more scalar covariates, and one or more scalar covariates. An estimation framework is proposed for estimating a time-varying coefficient function that is modeled as a linear combination of time-invariant functions with time-varying coefficients. The model takes advantage of the information structure of the penalty, while the estimation procedure utilizes the equivalence between penalized least squares estimation and linear mixed models. A number of simulations are conducted in order to empirically evaluate the process. In the simulation, it was observed that mean square errors for functional coefficients decreased with increasing sample size and level of association. Additionally, sample size had a greater impact on a smaller level of association, and level of association also had a greater impact on a smaller sample size. These results provide empirical evidence that ILFMM estimates of functional coefficients are close to the true functional estimate (basically unchanged). In addition, the results indicated that the AIC could be used to guide the choice of ridge weights. Moreover, when ridge weight ratios were sufficiently large, there was minimal impact on estimation performance. Studying two time scales is important in a wide range of fields, including physics, chemistry, biology, engineering, economics, and more. It allows researchers to gain a better understanding of complex systems and processes that operate over different time frames. Consequently, studying physical activities with two time scales is critical for advancing our understanding of human performance and health and for developing effective strategies to optimize physical activity and exercise programs. Therefore, the proposed model was applied to analyze the physical activity data from the Active Schools Institute of the University of Northern Colorado to determine what kind of time-structure patterns of activities could adequately describe the relationship between daily total magnitude and kids’ daily and weekly physical activity.

1. Introduction

Functional Data Analysis (FDA) deals with the analysis and theory of data that are in the form of functions, images and shapes, or more general objects [1,2]. The atom of functional data is a function, where for each subject in a random sample one or several functions are recorded. While the term “functional data analysis” was coined by [1,3], the history of this area is much older and dates back to Greven et al. (2020) [4] and Rao et al. (1950) [5]. There are several ways that the data can called functional. For instance, in archaeological studies the form of a three-dimensional image of each bone can be presented as functional data [6].

FDA has many applications in areas such as biostatistics, environmental monitoring, finance, and image analysis. For example, in biostatistics, FDA can be used to study the patterns of gene expression over time or to model the progression of a disease. In finance, FDA can be used to analyze the dynamics of financial markets over time, and to develop models for predicting future stock prices or other financial variables [7].

The functional data comes in many forms; however, their definition is a based on functions which are often smooth curves [8]. The smoothness is a characteristic that filter the noises inside the raw data as efficiently as possible [9]. They are several approaches for representing functional data in smoothing form: basis expansion, least-squares, and roughness penalty. One of basis expansions is based on B-splines. In this approach, both functional coefficient and functional predictor expand in the form of B-splines and thereafter the regularized estimate of functional coefficient will be applied [10]. Müller (2005) [11] expressed that the regression coefficient can be expressed in orthonormal basis determined by the eigenfunctions of the covariance of functional regressor. Goldmsmith [12] in 2011, combined both approaches introduced by Ramsay [10] and Müller [11]. This method has been considered in this study for smoothness and it is called roughness penalty because it penalizes fits that are too rough [13].

In many cases, data are observed longitudinally and collected over time. Longitudinal data is epansion of a time series, which is a sequence of data points that are ordered based on time, such that each data point corresponds to a specific moment in time [14]. Longitudinal data analysis can provide more accurate estimates of the effects of interventions or treatments over time [15]. The mixed-effects regression models have been used to study the longitudinal data [16]. These models allow us to make subject-specific conclusions and present statements about individuals changes, trends, or effects over time.

Sometimes in practice, the data has both characteristics of functional and longitudinal. One of the case can be when the outcome is not functional; however, covariates are collected functionally and longitudinally over one-time scale. People often use two approaches. One is based on function models where functional covariates have been smoothed according to roughness penalty and it is called Longitudinal Penalized Functional Regression (LPFR) [17]. The other one is built on according to principal component analysis for longitudinal functional data [18]. One of the main disadvantage of both approaches is the functional covariates are remained constant over time. But sometime in practice, we might have situation where functional covariates and scalar outcome have been changed over time which is considered in this study. Longitudinal functional models have not been studied to incorporate longitudinal functional for multiple time scales; however; Kundu et al. (2016) [19] published their work about the same data situation for one time-scale. Therefore, this study is an extension of Kund’s work to incorporate multilevel longitudinal functional covariates.

Studying two time scales is important in a wide range of fields, including physics, chemistry, biology, engineering, economics, and more. It allows researchers to gain a better understanding of complex systems and processes that operate over different time frames [20].

One of the key benefits of studying two time scales is that it can help researchers identify the underlying mechanisms that contribute to the behavior of a system. For example, in physics, studying the behavior of particles over different time scales can help researchers understand how those particles interact with each other and how their interactions lead to the formation of larger structures.

In chemistry, studying chemical reactions over different time scales can help researchers identify the intermediates and transition states that are involved in the reaction, and help them design more efficient and effective reactions [21].

Studying two time scales can also help researchers design and optimize systems for specific applications. For example, in engineering, studying the behavior of materials over different time scales can help researchers design materials with specific mechanical, electrical, or chemical properties.

Overall, studying two time scales is critical for advancing our understanding of complex systems and processes, and for developing new technologies and applications that can improve our lives.

Studying physical activities with two time scales is important because it allows us to understand how different processes operate at different timescales and how they interact with each other. Physical activities involve a wide range of processes that occur at different timescales, from short-term processes such as muscle activation and joint movements, to longer-term processes such as training adaptations and recovery [22].

By studying physical activities with two time scales, researchers can better understand the underlying mechanisms that contribute to performance and health outcomes [23]. For example, studying the short-term and long-term effects of exercise on muscle function and metabolism can help us understand how exercise training improves overall health and athletic performance [24].

Furthermore, studying physical activities with two time scales can help inform the development of effective training programs and interventions. By understanding how different processes interact over time, we can design training programs that are optimized for different goals, such as improving strength or endurance, and minimizing the risk of injury or overtraining [25].

Overall, studying physical activities with two time scales is critical for advancing our understanding of human performance and health and for developing effective strategies to optimize physical activity and exercise programs [26].

The proposed model was appropriate to apply on the data collected by the Active Schools Institute of the University of Northern Colorado. They were interested in the associations between daily and weekly activity profile, as measured by accelerometers, and academic and behavioral outcomes. To address these interests, students from one primary school in a suburban area in the western United States wore accelerometers during the school day for 5 consecutive days over 4 different weeks of the year. Corresponding academic and behavioral data were also obtained.

The data collected from the Active Schools Institute assumed daily student activity crossed within weeks. Due to nature of this data, establishing a multilevel mixed-effects regression model, including demographics or teacher effects, and accelerometer wear-time was essential. Hence, the proposed model was suited to apply on the physical activity data set.

As a consequence, this article addresses the following questions:

According to Section 2, the following question has been addressed: How can we represent a longitudinal functional regression model with time-varying regression functions for multiple time scales when time components are crossed?

Under Section 2.3, the following question is addressed: How can the mean parameters of the propsed model be estimated?

A response to this question can be found in Section 5: How can the model and estimation be implemented in software?

A detailed answer to this question can be found in Section 3: How do the proposed model compare to similar existing models with single time scales in terms of model MSE, across sample sizes and levels of association?

2. Intensive Longitudinal Functional Model with Multiple Time Scales

The intensive longitudinal method represents sequences of repeated measurements frequently recorded to characterize a separate change process for each subject [27].

Intensive longitudinal data refers to data that is collected repeatedly and frequently over a relatively short period of time, often at a high frequency (e.g., several times a day). This type of data is particularly useful for studying processes that unfold over time, such as mood fluctuations, stress reactivity, or medication effects [27,28].

Intensive longitudinal data can be collected using a variety of methods, including ecological momentary assessment (EMA), experience sampling methods (ESM), ambulatory monitoring, and diary or journal methods [29]. These methods typically involve asking participants to report on their experiences, thoughts, and behaviors at multiple points throughout the day, either through self-reports on electronic devices (such as smartphones or smartwatches), or by filling out paper or online forms [30].

The analysis of intensive longitudinal data requires specialized statistical methods that can capture the dynamic nature of the data, such as time-series analysis, multilevel modeling, and dynamic systems modeling. These methods allow researchers to examine how variables change over time, how they are related to each other, and how they are influenced by external factors [31].

Intensive longitudinal data has been used in a variety of fields, including psychology, medicine, and public health, to study phenomena such as mood disorders, substance use, physical activity, and sleep patterns.

This study considers a longitudinal model where the time scales of functional predictors cross. Additionally, the proposed model is suitable when data is collected frequently over a short period of time, often at a high frequency (e.g., several times a day). As a consequence, the proposed model may be referred to as an Intensive Longitudinal Functional Model with Multiple Time Scales (ILFMM) [32].

A specific estimation process is also included in the ILFMM. Estimating parameters is performed using a generalized ridge estimate. As a result of this estimation process, a linear mixed model representation can be derived. By using a mixed model approach, tuning parameters can be automatically selected. It is also possible to estimate parameters by fitting a linear mixed model based on this linear mixed model representation. It is worthwhile to note that the model presented has some advantages. There is no restriction on the time course of regression functions. In addition, it is possible to incorporate the structure of the regression function directly into the estimation process [19]. The approach described by Kundu et al. (2016)[19] was extended here to incorporate longitudinal functional predictors spanning multiple time scales.

2.1. Statistical Model of ILFMM

To represent a longitudinal functional regression model with time-varying regression functions for multiple time scales when time components are crossed, assume $\mathbf{X}$ presents univariate predictors, ${\mathbf{W}}_{i{t}_w}$ denotes a predictor function for the ith subject at timepoint $t_w$, and ${\mathbf{D}}_{i{t}_d}$ denotes a predictor function for the ith subject at timepoint $t_d$, where $i=\{1,2,\cdots ,N\}$, $t=\{t_1,t_2,\cdots ,t_{n_i}\}$, and longitudinal-time points t can be decomposed in terms of $t_w$ and $t_d$, where each $t_k$ ($k=\{1,2,\dots ,n_i\}$) corresponds to one value each from $t_{w_k}$ and $t_{d_k}$. It is assumed that each observed predictor is sampled at the same p locations, $s_1,\cdots ,s_p\in \mathsf{\Omega }$, and there are an equal number of observations per subject (i.e., $n_i$ are equal). Let ${\mathbf{W}}_{i{t}_w}:={\left[{\mathbf{W}}_{i{t}_w}\left(s_1\right),\cdots ,{\mathbf{W}}_{i{t}_w}\left(s_p\right)\right]}^{\top }$ and ${\mathbf{D}}_{i{t}_d}:={\left[{\mathbf{D}}_{i{t}_d}\left(s_1\right),\cdots ,{\mathbf{D}}_{i{t}_d}\left(s_p\right)\right]}^{\top }$ be the $p\times 1$ vector of values sampled from the realized functions ${\mathbf{W}}_{i{t}_w}$ and ${\mathbf{D}}_{i{t}_d}$, respectively. Then, the observed data have a form of

$$\{y_{it};{\mathbf{X}}_{it};{\mathbf{W}}_{i{t}_w};{\mathbf{D}}_{i{t}_d}\},$$

where $y_{it}$ is a scalar outcome, ${\mathbf{X}}_{it}$ is a $K\times 1$ column vector of measurements on K scalar predictors, ${\mathbf{W}}_{i{t}_w}$ is the sampled predictor from the ith subject corresponding to time $t_w$, and ${\mathbf{D}}_{i{t}_d}$ is the sampled predictor from the ith subject corresponding to time $t_d$. The longitudinal functional linear model with the scalar outcome and functional predictor along with multiple time scales could be written as

$$y_{it}={\mathbf{X}}_{it}^{\top }\mathit{\beta }+{\int }_{\mathsf{\Omega }}{\mathit{W}}_{i{t}_w}\left(s\right)\mathit{\gamma }(t_w,s)ds+{\int }_{\mathsf{\Omega }}{\mathit{D}}_{i{t}_d}\left(s\right)\mathit{\eta }(t_d,s)ds+{\mathbf{z}}_{it}^{\top }{\mathbf{b}}_i+{\epsilon }_{i{t}_w{t}_d},$$

(1)

where $\mathit{\gamma }\equiv \mathit{\gamma }(t_w,s)$ are functional coefficients at time $t_w$, $\mathit{\eta }\equiv \mathit{\eta }(t_d,s)$ are functional coefficients at time $t_d$, ${\mathbf{b}}_i$ is the vector of r random effects pertaining to subject i, and ${\epsilon }_{i{t}_w{t}_d}$ denote the subject-specific random effect and random error term, respectively.

Similar to a linear mixed model with time-related slope for longitudinal data, it is assumed that $\mathit{\gamma }$ and $\mathit{\eta }$ can be decomposed into several time-invariant component functions ${\mathit{\gamma }}_0,\cdots ,{\mathit{\gamma }}_W$ and ${\mathit{\gamma }}_0,\cdots ,{\mathit{\gamma }}_D$ as follows:

$$\mathit{\gamma }(t_w,s)={\mathit{\gamma }}_0\left(s\right)+f_1\left(t_1\right){\mathit{\gamma }}_1\left(s\right)+\dots +f_W\left(t_W\right){\mathit{\gamma }}_W\left(s\right),$$

(2)

$$\mathit{\eta }(t_d,s)={\mathit{\eta }}_0\left(s\right)+g_1\left(t_1\right){\mathit{\eta }}_1\left(s\right)+\dots +g_D\left(t_D\right){\mathit{\eta }}_D\left(s\right),$$

(3)

where $f_1,\dots ,f_W$ and $g_1,\dots ,g_D$ are functions of $t_w$ and $t_d$, respectively. Additionally, $\mathit{\gamma }$ and $\mathit{\eta }$ are functions of s. In general, $\mathit{\gamma }(t_w,s)$ and $\mathit{\eta }(t_d,s)$ have three components: $t_w$, $t_d$, and s. The time component $t_w$ enters into $\mathit{\gamma }$ through $f_1\left(t_1\right),\dots ,f_W\left(t_W\right)$ and the functional component enters into $\mathit{\gamma }(t_w,s)$ through ${\mathit{\gamma }}_0\left(s\right),\cdots ,{\mathit{\gamma }}_W\left(s\right)$. Additionally, the time component $t_d$ enters into $\mathit{\eta }(t_d,s)$ through $g_1\left(t_1\right),\dots ,g_D\left(t_D\right)$ and the functional component enters into $\mathit{\eta }(t_d,s)$ through ${\mathit{\eta }}_0\left(s\right),\cdots ,{\mathit{\eta }}_D\left(s\right)$. The model is flexible with any function of $t_w$ and $t_d$ with $f\left(0\right)=0$ (e.g., $f\left(t_w\right)=t_w$ or $t_w{e}_w^t$ and $f\left(t_d\right)=t_d$ or $t_d{e}_d^t$). Additionally, ${\epsilon }_{i{t}_w{t}_d}\sim N\left(0,{\sigma }_{\epsilon }^2\right)$ and ${\mathbf{b}}_i$ is distributed as $N\left(0,{\Sigma }_{{\mathbf{b}}_i}\right)$. It is assumed that ${\epsilon }_{i{t}_w{t}_d}\mathrm{and}{\mathbf{b}}_i$ are independent, ${\epsilon }_{i{t}_w{t}_d}\mathrm{and}{\epsilon }_{i^{\prime }{t_w}^{\prime }{t_d}^{\prime }}$ are independent whenever $i\ne i^{\prime }\mathrm{or}{t}_w\ne {t_w}^{\prime }\mathrm{or}{t}_d\ne {t_d}^{\prime }$ or all, and ${\mathbf{b}}_i$ and ${\mathbf{b}}_{i^{\prime }}$ are independent if $i\ne i^{\prime }$.

In Model (1) ${\mathit{X}}_{it}^{\top }\mathbf{\beta }$ is the standard fixed effect. In addition, ${\mathit{Z}}_{it}{\mathit{b}}_i$ is the standard multilevel random effect and ${\int }_{\mathsf{\Omega }}{\mathit{W}}_{i{t}_w}\left(s\right)\mathit{\gamma }(t_w,s)ds$ is the subject and time specific functional effect corresponding to level w. Here ${\int }_{\mathsf{\Omega }}{\mathit{D}}_{i{t}_d}\left(s\right)\mathit{\eta }(t_d,s)ds$ is the subject and time specific functional effect corresponding to level d. And both the variances and covariances across time are assumed to be constant as $Var\left(y_{i{t}_w{t}_d}\right)={\sigma }_b^2+{\sigma }_{ϵ}^2$ and $Cov\left(y_{i{t}_w{t}_d},y_{i{t}_w^{\prime }{t}_d^{\prime }}\right)={\sigma }_b^2$, where $t_w\ne t_w^{\prime }$ and $t_d\ne t_d^{\prime }$. This implies a compound symmetry assumption for the variances and covariances.

Model (1) can be written based on Equations (2) and (3) as

$$\begin{array}{ccc}\hfill y_{it}& =& {\mathbf{X}}_{it}^{\top }\mathit{\beta }+{\int }_{\mathsf{\Omega }}{\mathit{W}}_{i{t}_w}\left(s\right)({\mathit{\gamma }}_0\left(s\right)+f_1\left(t_1\right){\mathit{\gamma }}_1\left(s\right)+\dots +f_W\left(t_W\right){\mathit{\gamma }}_W\left(s\right))ds\hfill \\ & +& {\int }_{\mathsf{\Omega }}{\mathit{D}}_{i{t}_d}\left(s\right)({\mathit{\eta }}_0\left(s\right)+g_1\left(t_1\right){\mathit{\eta }}_1\left(s\right)+\dots +g_D\left(t_D\right){\mathit{\eta }}_D\left(s\right))ds\hfill \\ & +& {\mathbf{z}}_{it}^{\top }{\mathbf{b}}_i+{\epsilon }_{i{t}_w{t}_d}.\hfill \end{array}$$

(4)

A primary goal in developing Model (4) is the estimation of $\mathit{\beta }$, ${\mathit{\gamma }}_w={\left[{\mathit{\gamma }}_w\left(s_1\right),\dots ,{\mathit{\gamma }}_w\left(s_p\right)\right]}^{\top }$, $0\le w\le W$, and ${\mathit{\eta }}_d={\left[{\mathit{\eta }}_d\left(s_1\right),\dots ,{\mathit{\eta }}_d\left(s_p\right)\right]}^{\top },0\le d\le D$.

2.2. Matrix Form of ILFMM

In the following an explanation will be given demonstrating how the proposed model can be written in matrix form. For this purpose, the observed values and the design matrix of fixed effect of Model (1) can be considered as:

$$\begin{array}{ccc}\hfill \mathbf{y}& =& \left(\begin{array}{c}{y}_{1t_1}\\ y_{1t_2}\\ ⋮\\ y_{1t_{n_1}}\\ y_{2t_1}\\ y_{2t_2}\\ ⋮\\ y_{2t_{n_2}}\\ ⋮\\ y_{N{t}_1}\\ y_{N{t}_2}\\ ⋮\\ y_{N{t}_{n_N}}\end{array}\right)\in {\mathbb{R}}^{{\sum }_{i=1}^N{n}_i\times 1},\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill \mathbf{X}& =& \left(\begin{array}{cccc}{X}_{1t_1}& X_{1t_2}& \cdots & X_{1t_{n_1}}\\ X_{2t_1}& X_{2t_2}& \cdots & X_{2t_{n_2}}\\ ⋮& ⋮& \dots & ⋮\\ X_{N{t}_1}& X_{N{t}_2}& \cdots & X_{N{t}_{n_N}}\end{array}\right)\in {\mathbb{M}}_{{\sum }_{i=1}^N{n}_i\times k}\left(\mathbb{R}\right).\hfill \end{array}$$

(6)

It is important to point out that it is assumed there are an equal number of observations per subject (i.e., $n_i$ are equal). In addition, the matrices of functional coefficients, $\mathbf{W}$ and $\mathbf{D}$, have the form

$$\begin{array}{c}\hfill \mathbf{W}=\left(\begin{array}{c}{\mathbf{W}}_1\\ ⋮\\ {\mathbf{W}}_N\end{array}\right)\in {\mathbb{M}}_{{\sum }_{i=1}^N{n}_i\times (W+1)p}\left(\mathbb{R}\right)\mathrm{and}\mathbf{D}=\left(\begin{array}{c}{\mathbf{D}}_1\\ ⋮\\ {\mathbf{D}}_N\end{array}\right)\in {\mathbb{M}}_{{\sum }_{i=1}^N{n}_i\times (D+1)p}\left(\mathbb{R}\right),\end{array}$$

(7)

where

$$\begin{array}{ccc}\hfill {\mathbf{W}}_i& =& \left(\begin{array}{cccc}{\mathbf{W}}_{i{t}_1}^{\top }& f_1\left(t_1\right){\mathbf{W}}_{i{t}_1}^{\top }& \cdots & f_W\left(t_1\right){\mathbf{W}}_{i{t}_1}^{\top }\\ ⋮& ⋮& \ddots & ⋮\\ {\mathbf{W}}_{i{t}_{n_i}}^{\top }& f_1\left(t_{n_i}\right){\mathbf{W}}_{i{t}_{n_i}}^{\top }& \cdots & f_W\left(t_{n_i}\right){\mathbf{W}}_{i{t}_i}^{\top }\end{array}\right)\hfill \end{array}$$

(8)

and

$$\begin{array}{ccc}\hfill {\mathbf{D}}_i& =& \left(\begin{array}{cccc}{\mathbf{D}}_{i{t}_1}^{\top }& g_1\left(t_1\right){\mathbf{D}}_{i{t}_1}^{\top }& \cdots & g_D\left(t_1\right){\mathbf{D}}_{i{t}_1}^{\top }\\ ⋮& ⋮& \ddots & ⋮\\ {\mathbf{D}}_{i{t}_{n_i}}^{\top }& g_1\left(t_{n_i}\right){\mathbf{D}}_{i{t}_{n_i}}^{\top }& \cdots & g_D\left(t_{n_i}\right){\mathbf{D}}_{i{t}_i}^{\top }\end{array}\right).\hfill \end{array}$$

(9)

So Model (4) would be

$$\mathbf{Y}=\mathbf{X}\mathit{\beta }+\mathbf{W}\mathit{\gamma }+\mathbf{D}\mathit{\eta }+\mathbf{Zb}+\epsilon ,$$

(10)

where $\mathit{\beta }$ is the associated coefficient vector, $\mathit{\gamma }={\left[{\mathit{\gamma }}_0^{\top },{\mathit{\gamma }}_1^{\top },\cdots ,{\mathit{\gamma }}_W^{\top }\right]}^{\top }\in {\mathbb{R}}^{(W+1)p\times 1}$ and $\mathit{\eta }={\left[{\mathit{\eta }}_0^{\top },{\mathit{\eta }}_1^{\top },\cdots ,{\mathit{\eta }}_D^{\top }\right]}^{\top }\in {\mathbb{R}}^{(D+1)p\times 1}$ are coefficient vectors of function predictors, $\mathbf{b}\in {\mathbb{R}}^{rN\times 1}$ is a vector of random effects and $\mathbf{Z}\in {\mathbb{M}}_{{\sum }_{i=1}^N{n}_i\times rN}\left(\mathbb{R}\right)$ is the corresponding design matrix.

In this framework, the estimation approach is an extension of generalized ridge regression that takes the form of a penalty to the longitudinal setting in a manner that allows the estimated regression functions $\mathit{\gamma }(t_w,s)$ and $\mathit{\eta }(t_d,s)$ to vary with time.

2.3. Estimation Based on the Mixed Model Representation

The goal of this section is to estimate parameters based on the matrix form of Model (4). This model was described in Section 2.2 and it can be written as:

$$Y=\mathit{X}\mathit{\beta }+\mathit{W}\mathit{\gamma }+\mathit{D}\mathit{\eta }+\mathit{Z}b+\mathit{ϵ},$$

(11)

where $\mathbf{X}$, $\mathbf{W}$, $\mathbf{D}$, and $\mathbf{Z}$ are known matrices, $\mathit{\beta }$ is an unobservable vector of fixed effects, $\mathit{\gamma }$ and $\mathit{\eta }$ are unobservable functional vectors of fixed effects of two time scales, and $\mathit{b}$ is an unobservable vector of random effects with $E\left(\mathit{b}\right)=\mathbf{0},Cov\left(\mathit{b}\right)=\mathit{B}$, and $Cov(\mathit{b},\mathit{ϵ})=\mathit{0}$. Let $Cov\left(\mathit{ϵ}\right)=\mathit{R}$ and $E\left(\mathit{ϵ}\right)=\mathbf{0}$, then $\mathbf{Z}\mathbf{b}+\mathit{\epsilon }$ is distributed as $N(\mathbf{0},\mathbf{V})$ because $E(\mathbf{Z}\mathbf{b}+\mathit{\epsilon })=E\left(\mathit{\epsilon }\right)=\mathbf{0}$ and $Cov(\mathbf{Z}\mathbf{b}+\mathit{\epsilon })=\mathbf{Z}\mathbf{B}{\mathbf{Z}}^{\prime }+\mathbf{R}\equiv \mathbf{V}.$ In addition, it is assumed that $\mathbf{B}$ and $\mathbf{R}$ are known.

The well-known mixed model equation, developed by Henderson (1950) [33], can be applied at this time. Henderson’s mixed model equation is similar in spirit to normal equations, but the mixed model equation simultaneously provides Best Linear Unbiasted Estimators (BLUEs) and Best Linear Unbiased Predictors (BLUPs) by estimating the variance component $\mathbf{V}$.

Theorem 1.

If $\left(\begin{array}{c}{\widehat{\mathit{\alpha }}}^{\prime }{\widehat{\mathbf{b}}}^{\prime }\end{array}\right)$ is a solution to the mixed model

$$\mathbf{Y}=\mathbf{C}\mathit{\alpha }+\mathit{Z}\mathbf{b}+\mathit{ϵ},$$

(12)

where $\mathrm{C}=\left[\mathbf{X}\mathbf{W}\mathbf{D}\right]$, $\mathit{\alpha }=({\mathit{\beta }}^{{}^{\prime }}{\mathit{\gamma }}^{{}^{\prime }}{\mathit{\eta }}^{{}^{\prime }})$, $Cov\left(\mathbf{b}\right)=\mathbf{B}$, and $Cov\left(\mathit{ϵ}\right)=\mathbf{R}$, then $\mathbf{C}\widehat{\mathit{\alpha }}$ is a BLUE of $\mathbf{C}\mathit{\alpha }$ and simultaneously $\widehat{\mathbf{b}}$ is a BLUP of $\mathbf{b}$ if $\mathbf{B}$ and $\mathbf{R}$ are known and also by estimating the variance component $\mathbf{V}$.

Theorem 1 demonstrates how the mixed Model (11) can provide BLUEs and BLUPs simultaneously. Appendix A.1 includes the proof of the Theorem 1.

2.4. Roughness Penalties Estimate

In this section a roughness penalties estimate based on the matrix form of Model (4) is discussed, which was described in Section 2.2. As mentioned, the mixed model can be written as:

$$Y=\mathbf{X}\mathit{\beta }+\mathbf{W}\mathit{\gamma }+\mathbf{D}\mathit{\eta }+\mathit{Z}b+\mathit{ϵ},$$

(13)

where $\mathbf{X}$, $\mathbf{W}$, $\mathbf{D}$, and $\mathbf{Z}$ are known matrices, $\mathit{\beta }$ is an unobservable vector of fixed effects, $\mathit{\gamma }$ and $\mathit{\eta }$ are unobservable functional vectors of fixed effects for two time scales, and $\mathit{b}$ is an unobservable vector of random effects with $E\left(\mathit{b}\right)=\mathbf{0},Cov\left(\mathit{b}\right)=\mathit{B}$, and $Cov(\mathit{b},\mathit{ϵ})=\mathbf{0}$. Let $Cov\left(\mathit{ϵ}\right)=\mathit{R}$ and $E\left(\mathit{ϵ}\right)=\mathbf{0}$, then $\mathbf{Z}\mathbf{b}+\mathit{\epsilon }$ is distributed as $N(\mathbf{0},\mathbf{V})$ (see Section 2.3).

It is also assumed that $\mathbf{V}$ is known.

Theorem 2.

For each $w=0,\cdots ,W$ and $d=0,\cdots ,D$, let ${\mathcal{L}}_d$ and ${\mathcal{L}}_w$ be penalty operators for ${\mathit{\eta }}_w$ and ${\mathit{\gamma }}_d$ with ${\tau }_w^2$ and ${\lambda }_d^2$ as the associated tuning parameters. A generalized ridge estimate associated with Model (13) for minimizing β, γ, and η

$$\begin{array}{ccc}\hfill {\parallel \mathbf{y}-\mathbf{X}\mathit{\beta }-\mathbf{W}\mathit{\gamma }-\mathbf{D}\mathit{\eta }\parallel }_{{\mathbf{v}}^{-1}}^2& +& {\tau }_0^2{∥{\mathit{\eta }}_{\mathbf{0}}∥}_{{\mathcal{L}}_0^{\top }{\mathcal{L}}_0}^2+\cdots +{\tau }_W^2{∥{\mathit{\eta }}_W∥}_{{\mathcal{L}}_W^{\top }{\mathcal{L}}_W}^2\hfill \\ & +& {\lambda }_0^2{∥{\mathit{\gamma }}_{\mathbf{0}}∥}_{{\mathcal{L}}_0^{\top }{\mathcal{L}}_0}^2+\cdots +{\lambda }_D^2{∥{\mathit{\gamma }}_D∥}_{{\mathcal{L}}_D^{\top }{\mathcal{L}}_D}^2,\hfill \end{array}$$

(14)

is obtained as

$$\left(\begin{array}{c}\widehat{\mathit{\beta }}\\ \widehat{\mathit{\eta }}\\ \widehat{\mathit{\gamma }}\end{array}\right)={\left({\mathbf{C}}^{\top }{\mathbf{V}}^{-1}\mathbf{C}-\mathbf{M}\right)}^{-1}{\mathbf{C}}^{\top }{\mathbf{V}}^{-1}\mathbf{y},$$

(15)

where

$$\mathrm{C}=\left[\mathbf{X}\mathbf{W}\mathbf{D}\right],\mathbf{M}=\mathit{blockdiag}\left\{0,{{\mathcal{L}}_{\Gamma }}^{\top }{\mathcal{L}}_{\Gamma },{{\mathcal{L}}_{\mathrm{H}}}^{\top }{\mathcal{L}}_{\mathrm{H}}\right\},$$

$${\mathcal{L}}_{\Gamma }=\mathit{blockdiag}\left\{{\tau }_0{\mathcal{L}}_0,\cdots ,{\tau }_W{\mathcal{L}}_W\right\},$$

and

$${\mathcal{L}}_{\mathrm{H}}=\mathit{blockdiag}\left\{{\gamma }_0{\mathcal{L}}_0,\cdots ,{\gamma }_D{\mathcal{L}}_D\right\}.$$

In Theorem 2, an expression for the generalized ridge estimate of Model (13) will be derived. The results of both theorems illustrated that the ridge estimate is BLUP from an equivalent mixed model; hence, the estimation procedure takes advantage of the equivalence between penalized least-squares estimation and linear mixed model representation. An Appendix A.2 contains the proof of the Theorem 2.

2.5. Selection of Time-Structure in $\gamma (t_w,s)$ and $\eta (t_d,s)$

In the proposed approach there is freedom to choose a time-structure for both $\mathit{\gamma }(t_w,s)$ and $\mathit{\eta }(t_d,s)$. The matter of selecting appropriate time-structure in $\mathit{\gamma }(t_w,s)$ and $\mathit{\eta }(t_d,s)$ are analogous, in essence, to that of selecting time structure in a linear mixed-effects model.

For example, selection of time-structure ${\mathit{\eta }}_1+t_d{\mathit{\eta }}_2$ or ${\mathit{\eta }}_1+t_d{\mathit{\eta }}_1+t_d^2{\mathit{\eta }}_2$ or ${\mathit{\eta }}_1+t_d{\mathit{\eta }}_1+t_d^2{\mathit{\eta }}_2+t_d^3{\mathit{\eta }}_3$ is similar to selection of variables in mutiple linear regression models based on linear mixed-effects model.

When comparing different time structures, it is common to apply Akaike Information Criterion (AIC). The formula $-2l+2p$, where l refers to a Restricted, or a Residual Maximum Likelihood (REML) estimation of mixed models and p refers to the number of parameters of the model, for information criterion (AIC) was used to determine time-structure in $\mathit{\gamma }(t_w,s)$ and $\mathit{\eta }(t_d,s)$.

2.6. Selection of a, b, c, and d for Decomposition Penalties

The set of values $\{1,2,3\}$ for each of a, b, c, and d are considered in this study. Those are treated as weights of a trade-off between preferred and non-preferred subspaces with $ab=$ constant and $cd=$ constant.

Additionally, REML is used to estimate tuning parameters when b and d are considered as fixed values and a grid search determines a and c values. These values are used to jointly select the tuning parameters.

Then, the selected values of a and c can be applied to the preferred subspace. The formula for information criterion (AIC) $-2l+2p$ creates values for a and c, where l refers to a Restricted, or a Residual Maximum Likelihood (REML) estimation of mixed models and p refers to the number of parameters of the model.

3. Simulation Steps

In this section, the process of simulating the proposed model is explored. Without loss of generality, it was assumed that $b=1$ and $d=1$; however, a and c could be varied on the set values of $\{1,2,3\}$. The largest values of a and c indicate there was much prior information in the estimation process. The following was used to generate the response data for 100 subjects by simulating functional covariates $\mathit{W}$ in 4 time points $(t_w=1,2,3,4)$ and functional covariates $\mathit{D}$ in 5 time points $(t_d=1,2,3,4,5)$:

$$\begin{array}{ccc}\hfill y_{it}& =& {\beta }_0+{\beta }_{1}t+{\int }_0^1{\mathit{W}}_{i{t}_w}\left(s\right)\mathit{\gamma }(s,t_w)ds\hfill \\ & +& {\int }_0^1{\mathit{D}}_{i{t}_d}\left(s\right)\mathit{\eta }(s,t_d)ds+b_i+{\epsilon }_{i{t}_w{t}_d},i=1,\cdots ,100,\hfill \end{array}$$

(16)

where $\mathit{\gamma }(s,t_w)={\mathit{\gamma }}_1\left(s\right)+t_w{\mathit{\gamma }}_2\left(s\right)$, $\mathit{\eta }(s,t_d)={\mathit{\eta }}_1\left(s\right)+t_d{\mathit{\eta }}_2\left(s\right)$. Functional covariates $\mathbf{W}$ and $\mathbf{D}$ have the following form: $\mathbf{W}=\left(\begin{array}{cc}{\mathbf{W}}_1& {\mathbf{W}}_2\end{array}\right)$ and $\mathbf{D}=\left(\begin{array}{cc}{\mathbf{D}}_1& {\mathbf{D}}_2\end{array}\right)$. Consequently, a rewritten of Model (16) would be

$$\begin{array}{ccc}\hfill y_{it}& =& {\beta }_0+{\beta }_{1}t+{\int }_0^1{\mathit{W}}_{{\mathbf{1}}_{\mathit{i}{\mathit{t}}_{\mathit{w}}}}\left(s\right){\mathit{\gamma }}_{\mathbf{1}}\left(s\right)ds+{\int }_0^1{t}_w{\mathit{W}}_{{\mathbf{2}}_{\mathit{i}{\mathit{t}}_{\mathit{w}}}}\left(s\right){\mathit{\gamma }}_{\mathbf{2}}\left(s\right)ds\hfill \\ & +& {\int }_0^1{\mathit{D}}_{{\mathbf{1}}_{\mathit{i}{\mathit{t}}_{\mathit{d}}}}\left(s\right){\mathit{\gamma }}_{\mathbf{1}}\left(s\right)ds+{\int }_0^1{t}_d{\mathit{D}}_{{\mathbf{2}}_{\mathit{i}{\mathit{t}}_{\mathit{d}}}}\left(s\right){\mathit{\gamma }}_{\mathbf{2}}\left(s\right)ds\hfill \\ & +& b_i+{\epsilon }_{i{t}_w{t}_d},i=1,\cdots ,100.\hfill \end{array}$$

(17)

Functional covariates $\mathit{W}$ were generated by considering three sets of sampling points: wide, moderate, and narrow corresponding to one, five, and two bumps, respectively [19].

As an example, the first set of 100 columns in $\mathbf{W}$ was represented as follows:

a.

We have recorded all of the observed values for the first subject at the first time point in the first $100\times 1$ row of the $\mathbf{W}$ matrix.

b.

As a result, all observation values for $\mathbf{W}$ at all time points and all subjects in each following row have been arranged. This is so that each row represents an observation value for $\mathbf{W}$ from a specific time point and subject. In accordance with $\mathbf{y}$, the rows were ordered by time and subject.

Having constructed the first set of 100 columns, we next proceeded to construct the second set of 100 columns by multiplying the first 100 columns by $t_w$.

Since it was impossible to generate the curves with functional form only by considering discrete observations at different points of the domain interval, it was necessary to consider some set of sampling points on the interval $[0,1]$ [19]. To simulate the true functional forms of the curves, prior knowledge about the functional variable was applied. Due to this, the observed value of $\mathit{W}$ was calculated considering the degrees of curvature at the wide, moderate, and narrow bumps [19]. White noise was added to the predictor functions to account for the instrumental measurement noise [19].

Functional covariates $\mathit{D}$ were generated by considering three sets of sampling points: wide, moderate, and narrow corresponding to one, five, and two bumps respectively as $H_{\mathrm{wide}}=\left\{0.40\right\},H_{\mathrm{moderate}}=\{0.10,0.25,0.65,0.75,0.80\},$ and $H_{\mathrm{narrow}}=\{0.50,0.85\}.$ The first set of 100 columns in $\mathbf{D}$ were represented similar to process of presenting $\mathbf{W}$ as discussed ealier. Consequently, the observed value of $\mathit{D}$ was generated in the same way as $\mathit{W}$.

The functional coefficients ${\mathit{\gamma }}_1$, ${\mathit{\gamma }}_2$ were generated according to [19] considering different degrees of curvature at the wide, moderate, and narrow bumps. But ${\mathit{\eta }}_1$, and ${\mathit{\eta }}_2$ were generated as follows with bumps centered at $H_{{\mathit{\eta }}_1}=\{0.10,0.45,0.75\}$, and $H_{{\mathit{\eta }}_2}=\{0.25,0.65\}$,

$$\begin{array}{ccc}\hfill {\eta }_1\left(s\right)& =& \sum _{h\in H_{{\eta }_1}}{a}_w\left(h\right)exp\left[-c_w\left(h\right)\ast {\left(\frac{s-h}{100}\right)}^2\right],\hfill \\ \hfill {\eta }_2\left(s\right)& =& \sum _{h\in H_{{\eta }_n}}{a}_n\left(h\right)exp\left[-c_n\left(h\right)\ast {\left(\frac{s-h}{100}\right)}^2\right],\forall s\in [0,1],\hfill \end{array}$$

(18)

where $a_w\left(h\right)$, $c_w\left(h\right)$, $a_n\left(h\right)$, and $c_n\left(h\right)$ correspond to amplitude and degree of curvature respectively. In fact, “bump” is the same as the “turning” of the polynomial graph. While the proper term is local maximum, the term “bump” referred to the “turning” of functional coefficients. The amplitude is the height from the centerline to the peak (dip) of the polynomial, and it measures the height from highest to lowest points divided by two. The amplitude values as $0.1,0.4,0.5,$ and $0.6$ were considered to simulate functional coefficients [19]. Another term that played an important role in simulating was degree of curvature. It was referred to the degree of a polynomial [34]. The amplitude and curvature used to generate the functional coefficients ${\mathit{\eta }}_1$, and ${\mathit{\eta }}_2$ are specified in

Table 1. The amplitude and curvature parameters for ${\mathit{\eta }}_{\mathbf{1}}$ and ${\mathit{\eta }}_{\mathbf{2}}$.

h	Amplitude ${\mathit{a}}_{\mathit{w}}\left(\mathit{h}\right)$	Curvature ${\mathit{c}}_{\mathit{w}}\left(\mathit{h}\right)$	Amplitude ${\mathit{a}}_{\mathit{m}}\left(\mathit{h}\right)$	Curvature ${\mathit{c}}_{\mathit{m}}\left(\mathit{h}\right)$
5	0.15
15			0.15	500
30	−0.10	250
50
70	0.10	250
80			−0.15	1000
90
95

The second and third columns are parameters for ${\mathit{\eta }}_{\mathbf{1}}$, and the last two columns are for ${\mathit{\eta }}_{\mathbf{2}}$.

.

The model was applied to analyze the physical activities of kids with logitudonal functional data with two-time scale. The results suggested that ${\beta }_0=0.06$ and ${\beta }_1=0.4$ were reasonable choices for fixed parameters in the simulation study. Having made these choices the fixed effect, the random effect, and errors were generated and are shown in

Table 2. Choices for fixed effects, random effect, and errors in the simulation study.

Fixed Effect	${\beta }_0$	0.06
Fixed Effect	${\beta }_1$	0.04
Random Effect	$b_i$	$\sim \mathcal{N}(0,0.{05}^2)$
Errors	${ϵ}_{i{t}_w{t}_d}$	$\sim \mathcal{N}(0,0.{02}^2)$

.

Throughout this simulation the following decomposition penalties, an extension of Kundu et al. (2016) [19] for multiple time scales, were considered: $b=1$ and $d=1$ while a and c were varied over $1,2,$ and 3, also structured penalties were ${\mathcal{L}}_{{\mathbf{Q}}_w}={10}^a\left(\mathit{I}-P_{{\mathbf{Q}}_w}\right)+b{P}_{{\mathbf{Q}}_w}$ and ${\mathcal{L}}_{{\mathbf{Q}}_d}={10}^c\left(\mathit{I}-P_{{\mathbf{Q}}_d}\right)+d{P}_{{\mathbf{Q}}_d}.$ R 3.5.2 GUI 1.70 El Capitan build (7612) and RStudio Version 1.1.383 were used to analyze the simulated data. The method discussed in this section was implemented in the R package nlme via the lme function. The R code has been uploaded to GitHub and it is viewable at https://github.com/Mostafa2020-bit/PackageILFMM since November 2020.

About 100 data were generated for two different sample sizes(100 and 200) along with two different $R^2$ ($0.6$ and $0.9$).

The primary interests were in the estimation of the functional coefficients and the measurement of the squared distance between the fitted and the true value of $\mathbf{y}$. Estimation error was summarized in terms of the Mean Squared Errors (MSE) of the functional coefficients. Also, the MSE of $\tilde{\mathbf{y}}$ was obtained.

With respect to the MSE of both the functional covariates and the response, we investigated the effect of increasing the sample size and $R^2$ on these values of $MSE$. If these were increased, would $MSE\left({\mathit{\gamma }}_1\right)$, $MSE\left({\mathit{\gamma }}_2\right)$, $MSE\left({\mathit{\eta }}_1\right)$, $MSE\left({\mathit{\eta }}_2\right)$, and $MSE\left(\tilde{\mathbf{y}}\right)$ decrease or not? In this scenario, the following criteria were computed: $MSE\left({\mathit{\gamma }}_1\right)={\parallel {\mathit{\gamma }}_{\mathbf{1}}-\widetilde{{\mathit{\gamma }}_{\mathbf{1}}}\parallel }^2={\sum }_{s=1}^p{\int }_{\mathsf{\Omega }}{\left({{\gamma }_1}_s-{\widetilde{{\gamma }_1}}_s\right)}^{2}ds,$ where $\widetilde{{\mathit{\gamma }}_{\mathbf{1}}}$ is the estimate of ${\mathit{\gamma }}_{\mathbf{1}}$ and each observed predictor ${\gamma }_1$ is sampled at the same p locations, $s_1,\cdots ,s_p\in \mathsf{\Omega }$. Similarly, $MSE\left({\mathit{\gamma }}_2\right)$, $MSE\left({\mathit{\eta }}_1\right)$, $MSE\left({\mathit{\eta }}_2\right)$ were computed; however, the calculation of $MSE\left(\tilde{\mathbf{y}}\right)$ was different because of its discrete characteristics and it was computed as $MSE\left(\tilde{\mathit{y}}\right)=\frac{\parallel \mathit{y}-\tilde{\mathit{y}}{\parallel }^2}{N}=\frac{{\sum }_{i=1}^N{\sum }_{j=1}^{n_i}{(y_{i{t}_j}-{\tilde{y}}_{i{t}_j})}^2}{N},$ where $\tilde{\mathit{y}}$ denotes the model fitted values of the true $\mathit{y}$.

The $MSE\left({\mathit{\gamma }}_1\right)$, $MSE\left({\mathit{\gamma }}_2\right)$, $MSE\left({\mathit{\eta }}_1\right)$, and $MSE\left({\mathit{\eta }}_2\right)$ could provide empirical evidence on whether the estimates are close to the corresponding true function or not.

To determine the effect of the decomposed penalty weights a, b, c, and d without loss of generality, b and d were considered a fixed value of 1. Then a and c were increased up to 3 on an exponential scale (i.e., $\{1,2,3\}$). The reason for this was to determine which combination of a, b, c, and d values would improve the estimation of ${\mathit{\gamma }}_1$, ${\mathit{\gamma }}_2$, ${\mathit{\eta }}_1$, and ${\mathit{\eta }}_2$. In this manner, an evaluation of estimation performance would remain almost unchanged.

Decomposition penalty values were selected according to the $-2l+2p$ formula for AIC. It was expected that minimized AIC for selected decomposition penalty values leads to a minimized MSE for different sample sizes and $R^2$s. AIC values were computed for all combinations of a and c values. Consequently, the simulation parameters were a and c values, the four sample sizes, and the four $R^2$s.

4. Simulation Conclusions

The simulation study illustrated the potential advantage of an Intensive Longitudinal Functional Model with Multiple time scales (ILFMM) estimate in exploiting an informed, structured penalty. The simulation suggested that as the sample size and $R^2$ increased, the MSE for both functional coefficients and the fitted values decreased, Figure 1 and

Table 3. Averages $MSE\left({\mathit{\gamma }}_1\right)$, $MSE\left({\mathit{\gamma }}_2\right)$, $MSE\left({\mathit{\eta }}_1\right)$, $MSE\left({\mathit{\eta }}_2\right)$, and $MSE\left(\tilde{\mathit{y}}\right)$ for four cases of simulation.

Case	N	${\mathit{R}}^2$	$\mathit{MSE}\left({\mathit{\gamma }}_1\right)$	$\mathit{MSE}\left({\mathit{\gamma }}_2\right)$	$\mathit{MSE}\left({\mathit{\eta }}_1\right)$	$\mathit{MSE}\left({\mathit{\eta }}_2\right)$	$\mathit{MSE}\left(\tilde{\mathit{y}}\right)$
Case 1	100	0.6	$1.000522\times {10}^{-24}$	$1.991303\times {10}^{-25}$	0.5035614	0.3827327	0.01625859
Case 2	100	0.9	$2.912084\times {10}^{-25}$	$1.343186\times {10}^{-24}$	0.5027712	0.3765218	0.01378987
Case 3	200	0.6	$8.249432\times {10}^{-27}$	$8.167412\times {10}^{-27}$	0.5286615	0.3614322	0.01240092
Case 4	200	0.9	$3.903604\times {10}^{-27}$	$6.183226\times {10}^{-27}$	0.5178613	0.3525342	0.01182925

.

Furthermore, sample size had a larger impact for smaller $R^2$, and also $R^2$ had a greater impact for smaller sample size, Figure 2.

These results provided empirical evidence that the ILFMM estimates of functional coefficients were close to the true functional estimate (basically unchanged). In other words, the estimation of the functional coefficients rose to the level of the estimation of true functions. These results suggested that AIC could guide the choice of ridge weights, Figure 3 and Figure 4.

The results for AIC are displayed graphically in Figure 3. It has been demonstrated, when c = 3, AIC is minimized among all cases. Therefore, the choice of decomposition penalties is a = 3, b = 1, c = 3, and d = 1. Also with sufficiently large values there was minimal impact on the estimation performance.

In addition, the model with the selected ridge penalties had the lowest MSE values of $\tilde{\mathit{y}}$ as compared to the model with the defined selection of ridge penalties, Figure 4 and Table 3. These results implied that the model with the selected ridge penalties would perform better than the model with the basic selection of ridge penalties.

Furthermore, for smaller sample sizes and $R^2$s, the ILFMM estimate may oversmooth the estimated regression function. However, by increasing the sample size to 200 or $R^2$ to $0.9$, it was observed that the average ILFMM estimate of the functional coefficients approached the true functions.

All in all, the model with ridge penalties $a=3,b=1,c=3,$ and $d=1$ appears to perform better than the remaining models with other selections of the ridge penalties.

In the absence of a model with two time scales for comparison, it was decided to evaluate the effect of sample size and $R^2$ on MSE of functional coefficients, the fitted and AIC values, which would indicate whether the patterns are consistent with those seen in previous models.

Kundu’s approach was based on a single time scale. The results of the proposed approach was lined up with consideration to a single time scale comparing AIC values [19] (i.e., single versus multiple time scales).

Also, Kudu concluded the selected ridge penalties had the lowest MSE values as when compared to the model with the basic selection. This demonstrated that the selected ridge penalties were reasonable. In addition, when the sample size and $R^2$ were increased, MSE values were decreased. Furthermore, sample size had a larger impact for smaller $R^2$, and also $R^2$ had a greater impact for smaller sample size [19].

5. Physical Activity Study Application

The proposed method could be implemented in a ILFMM() (i.e., Intensive Longitudinal Functional Model with Multiple time scales) package in R. The author has written the ILFMM() package for a longitudinal functional model with multiple time scales along with scalar outcome, multiple functional predictors, one or more scalar covariates, and subject-specific random intercepts through mixed model equivalence. This package also works for a longitudinal functional model with a single time scale. The ILFMM() package has been uploaded to GitHub and it is viewable at https://github.com/Mostafa2020-bit/PackageILFMM sicne Novebmer 2020.

It is more common in the Physical Activity (PA) field to calculate the amount of PA a child gets in a single day, rather than a week. So this might be why it has been difficult to find information on weekly amounts of PA. Consequently, it is standard in PA projects to get week data by taking the average of all weekdays [35,36].

Up until about five years ago most physical activity researchers only measured and reported five days of activity at a time. Most reviews of handling accelerometer data focus on the epochs, the device or the type of outcome reported (Moderate-to-Vigorous Physical Activity (MVPA) or light PA) [37].

In PA field, the daily accelerometer-assessed time in MVPA over a week is usually calculated as the mean of data over 7 days. For participants with less than 7 valid days of data, the following formula is common to use for standardizing measurements to one week for all participants [37]:

$\left[\right(5\times$ mean daily weekday MVPA time $+2\times$ mean daily weekend MVPA time $\left)\right]/7.$

The dataset was stored in a secure password-protected Super R machine server utilized by IT at University of Northern Colorado for the Active Schools Institute. The author had permission from the Active Schools Institute to access the dataset. The data used included the five-second records from all students in school 1 for all weeks 1, 2, 3, and 4 over five days of school time. The standard approach was used to get week data.

For all weeks the “Axis-1” measurements ranged approximately from 0 to $2647.0$, with a mean of $20.75$, where “Axis-1” referred to the physical activities over x-axis coordinate.

For week 1, many observations were irrelevant (i.e., zeros recorded late in the evening), so observations were truncated to consider only the time between 8:00 AM and 3:30 PM. For example, by considering the first day of data collection the number of observations were $1,059,840$. Further limitation on the number of observations came from the restriction of the hours from 8:00 AM to 3:30 PM on the first day. This resulted in a final data set of $496,800$ observations. This is around 5900 observations per student. This number is close but a little high for 5-second observations. It would be anticipated that there would be 5400 observations per student (12 per minute, 60 min. per hour, $7.5$ h). However, 8 students show twice as many observations, suggesting that they have two days labeled as 9 November 2016. For all other weeks there was no such a problem.

For all weeks, the Stata default for time-stamps was New Year’s 1960. All official time stamps showed dates around this time, but for analysis, dates were inherited from the “Date” variable that the research team included.

Time stamps were translated in an unexpected way. For Day 1, all activity appeared to begin at 1:00 AM for each student. We assumed this should have been 8:00 AM and translated the time-stamps accordingly. For all other days, recording appeared to begin at 5:00 PM, but all activity appeared to begin at midnight. We again assumed these times should have been 8:00 AM and translated time-stamps accordingly. We then removed all observations outside the hours of 8:00 AM through 3:30 PM. Plots of the five-second data are usually “smoothed” to show the true patterns in the data. Smoothing is a method of taking the data in “windows of time” to estimate means or regression trends. The size of the window has a strong impact on the graphic produced but is subjective, and it is usually determined by a span value.

In designing the matrix for weekly activities, 162,000 data points per subject were considered for each week. The researcher chose 162,000 data points because it was large enough to capture physical activity patterns from 8:00 AM to 3:30 PM. This implies 684,000 observations per subject for four weeks (i.e., 162,000 × 4 = 684,000). And since 80 subjects were considered, the total number of observations were 51,840,000. These design matrices are graphically shown for 11 subjects in Figure 5 and Figure 6. It provides us with information on the overall patterns of physical activity on a daily and weekly basis.

For daily physical activity analysis, we chose 162,000 because it was large enough to capture physical activity patterns every five seconds from 8:00 AM to 3:30 PM. This implies 3,240,000 observations per subject for five days across four weeks (i.e., 162,000 × 5 × 4 = 3,240,000). And since 80 subjects were considered the total number of observations were 259,200,000. These design matrices are graphically shown in Figure 7 and Figure 8.

The proposed model with two-time scales along with $a=3$, $b=1$, $c=3$, and $d=1$ as ridge penalties was applied to real data as

$$y_{i{t}_w}={\beta }_0+{\beta }_{p}Teache{r}_p+{\int }_0^1{\mathit{W}}_{i{t}_w}\left(s\right)\mathit{\gamma }(s,t_w)ds+{\int }_0^1{\mathit{D}}_{i{t}_d}\left(s\right)\mathit{\eta }(s,t_d)ds+b_i+{\epsilon }_{i{t}_w{t}_d},$$

(19)

where $\mathit{\gamma }(s,t_w)={\mathit{\gamma }}_1\left(s\right)+t_w{\mathit{\gamma }}_2\left(s\right)$, $\mathit{\eta }(t_d,s)={\mathit{\eta }}_1\left(s\right)+t_d{\mathit{\eta }}_2\left(s\right)+t_d^2{\mathit{\eta }}_2\left(s\right)+t_d^3{\mathit{\eta }}_4\left(s\right)$, $i=1,\cdots ,100$, and $p=1,2,3,4,5,6$. The expanded form of Model (19) would be

$$\begin{array}{ccc}\hfill y_{i{t}_w{t}_d}& =& {\beta }_0+{\beta }_{2}Teache{r}_2+{\beta }_{3}Teache{r}_3+{\beta }_{4}Teache{r}_4\hfill \\ & +& {\beta }_{5}Teache{r}_5+{\beta }_{6}Teache{r}_6\hfill \\ & +& {\int }_0^1{\mathit{W}}_{{\mathbf{1}}_{\mathit{i}{\mathit{t}}_{\mathit{w}}}}\left(s\right){\mathit{\gamma }}_{\mathbf{1}}\left(s\right)ds+{\int }_0^1{t}_w{\mathit{W}}_{{\mathbf{2}}_{\mathit{i}{\mathit{t}}_{\mathit{w}}}}\left(s\right){\mathit{\gamma }}_{\mathbf{2}}\left(s\right)ds\hfill \\ & +& {\int }_0^1{\mathit{D}}_{{\mathbf{1}}_{\mathit{i}{\mathit{t}}_{\mathit{d}}}}\left(s\right){\mathit{\eta }}_{\mathbf{1}}\left(s\right)ds+{\int }_0^1{t}_d{\mathit{D}}_{{\mathbf{2}}_{\mathit{i}{\mathit{t}}_{\mathit{d}}}}\left(s\right){\mathit{\eta }}_{\mathbf{2}}\left(s\right)ds+{\int }_0^1{t}_d^2{\mathit{D}}_{{\mathbf{3}}_{\mathit{i}{\mathit{t}}_{\mathit{d}}}}\left(s\right){\mathit{\eta }}_{\mathbf{3}}\left(s\right)ds\hfill \\ & +& {\int }_0^1{t}_d^3{\mathit{D}}_{{\mathbf{4}}_{\mathit{i}{\mathit{t}}_{\mathit{d}}}}\left(s\right){\mathit{\eta }}_{\mathbf{4}}\left(s\right)\hfill \\ & +& b_i+{\epsilon }_{i{t}_w}.\hfill \end{array}$$

(20)

The ILFMM estimates were obtained as BLUP assuming ${\epsilon }_{i{t}_w{t}_d}\sim N\left(0,{\sigma }_{\epsilon }^2\right)$ and the subject-specific random intercepts $b_i\sim N\left(0,{\sigma }_b^2\right)$. The model was fitted into ILFMM() R package and the estimates of ${\sigma }_{\epsilon }^2$ and ${\sigma }_b^2$ were $0.000320$ and $0.7281$, respectively.

We compared models based on AIC values with a different selection of weight penalties and time structures (linear, quadratic, and cubic). The results are summarized in

Table 4. Comparison of AIC for selection of ridge weights and time structures.

Model	Time Structure for Days	Time Structure for Weeks	a	c	AIC
Model 1	${\mathit{\eta }}_1\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)$	1	1	2,096,978
Model 2	${\mathit{\eta }}_1\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)+t{\mathit{\gamma }}_2\left(s\right)$	1	1	1,978,254
Model 3	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)$	1	1	1,947,720
Model 4	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)+t{\mathit{\gamma }}_2\left(s\right)$	1	1	1,947,720
Model 5	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)+t^2{\mathit{\eta }}_3\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)$	1	1	1,947,720
Model 6	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)+t^2{\mathit{\eta }}_3\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)+t{\mathit{\gamma }}_2\left(s\right)$	1	1	1,947,720
Model 7	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)+t^2{\mathit{\eta }}_3\left(s\right)+t^3{\mathit{\eta }}_4\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)$	1	1	1,947,720
Model 8	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)+t^2{\mathit{\eta }}_3\left(s\right)+t^3{\mathit{\eta }}_4\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)+t{\mathit{\gamma }}_2\left(s\right)$	1	1	1,947,720
Model 9	${\mathit{\eta }}_1\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)$	3	3	1,789,975
Model 10	${\mathit{\eta }}_1\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)+t{\mathit{\gamma }}_2\left(s\right)$	3	3	16,898,224
Model 11	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)$	3	3	1,574,037
Model 12	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)+t{\mathit{\gamma }}_2\left(s\right)$	3	3	1,442,020
Model 13	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)+t^2{\mathit{\eta }}_3\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)$	3	3	1,347,340
Model 14	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)+t^2{\mathit{\eta }}_3\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)+t{\mathit{\gamma }}_2\left(s\right)$	3	3	1,304,740
Model 15	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)+t^2{\mathit{\eta }}_3\left(s\right)+t^3{\mathit{\eta }}_4\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)$	3	3	1,300,120
Model 16	${\mathit{\eta }}_1\left(s\right)+t{\mathit{\eta }}_2\left(s\right)+t^2{\mathit{\eta }}_3\left(s\right)+t^3{\mathit{\eta }}_4\left(s\right)$	${\mathit{\gamma }}_1\left(s\right)+t{\mathit{\gamma }}_2\left(s\right)$	3	3	1,235,622

.

Based on the AIC, Model 16, with a cubic time structure for days and linear time structure for weeks and weight penalties $a=3$, $b=1$, $c=3$, and $d=1$ appears to be better than the other models.

6. Discussion

The proposed intensive longitudinal functional model with multiple time-varying scales with scalar outcome, multiple functional predictors, one or more scalar covariates, and subject-specific random intercepts through mixed model equivalence has been defined. There were three primary advantages of this framework. First, this model estimated a time-dependent regression function. Second, it was also able to incorporate structural information into the estimation process. And third, it was easily implemented through linear mixed model equivalence.

One of the significant limitations of this study was the hardware needs for the organization and analysis of the data. Storage space to house the data, networking bandwidth to transfer it to and from analytics systems, and computing resources to perform those analytics were very time-consuming.

The physical activity data study results indicate that the proposed method is an appropriate one for the tested application. By using the proposed model, we could investigate what kind of time-structure for activity patterns would adequately describe the relationship between the subjects’ daily total magnitude and their weekly physical activities. Likewise, it described this relationship for daily physical activities.

So, the interest of the Active Schools Institute has been addressed through the proposed model. The proposed model concluded that the time structure of weekly activity was linear, but for daily activity was cubic. This conclusion was reasonable because the general patterns of movement throughout the day had more fluctuations than weekly movement. The weekly movements showed more smoothed patterns. Also, several dips in data points around the sedentary category for daily movement were observed. However, for weekly movement, it appeared this sedentary behavior was changed (i.e., it is needed to consider two separate definitions for sedentary behavior for weekly and daily activities). This analysis also gives us a slightly better sense of how movement intensity changes over time for both time scales.

The following are recommendations for future studies. One area of further investigation is the effect of different time structures. In this study, a linear form for simulation was considered; however, other forms of $f\left(t\right)$, such as $e^{\left(t\right)}-1$ or $log(t+1)$ could be explored. A relative improvement in AIC due to the use of these complex structures would be anticipated. For instance, it would be expected to have a minimal AIC associated with $\mathit{\gamma }(t,s)={\mathit{\gamma }}_0\left(s\right)+[e^{\left(t\right)}-1]{\mathit{\gamma }}_1\left(s\right)$ in comparison to AIC for $\mathit{\gamma }(t,s)={\mathit{\gamma }}_0\left(s\right)+t{\mathit{\gamma }}_1\left(s\right)$.

In the proposed model, the ridge weights were set as $b=d=1$ while a and c varied on an exponential scale. Larger values of a and c indicated greater emphasis of prior information on the estimation process. Future study could investigate other scales of ridge weights to determine more accurately and quickly the estimation process.

A possible extension of this work could be to incorporate multiple functional predictors. For example, in PA projects, the physical activity and patterns might be studied over an entire year. This implies there would be three-time scales: days, weeks, and years. Consequently, an extension of the proposed approach would be needed.

Additionally, an interesting and powerful use of the framework underlying the proposed model is when the response variable is binary. In this case when the response variable is binary, different problems arise. Multicollinearity and high dimensionality prejudice the estimation of the model and the interpretation of its parameters. This prejudice can be overcome by using an intensive longitudinal multilevel functional logistic regression model with principal component analysis. This study could be extended to any differentiable Hilbert space for analyzing of image, spatial, and spatial-temporal data.