Cognition Test Battery Survey: Development of a Single Alertness and Mood Score for Short- and Long-Duration Spaceflight

Abstract

Astronaut behavioral and mental health are key factors for space mission success. NASA’s Cognition test battery is often preceded by a brief 18-item Alertness and Mood Survey (AMS) adding subjective assessments to Cognition’s 10 objective neuropsychological tests. Therefore, the main objective of this study was to generate a single AMS summary score for short (<6 months) and long (>6 months) space missions based on the relevance of individual items. With the paired comparison (PC) method, 19 subject matter experts (SMEs) were asked to rate the relevance of 14 AMS items for astronaut behavioral health. Boredom (22.1% of comparisons), monotony (23.1%), and sleepiness (35.6%) were considered the least relevant, and health (74.3%), depression (76.5%), and crew conflicts (77.9%) were considered the most relevant by SMEs. Six of the fourteen items differed statistically significantly between PCs for short- and long-duration missions: sleepiness, tiredness, energy level, and mental status were considered more relevant for short-duration missions, while monotony and loneliness were considered more relevant for long-duration missions (all adjusted p < 0.05). We also demonstrated systematic changes in AMS summary scores during a 60-day 6° head-down tilt bed rest (HDBR) study, with increased alertness and mood disturbance during and after the HDBR period relative to pre-HDBR levels. This analysis identified the AMS domains considered most relevant for space mission success by SMEs, and highlighted differences between long- and short-duration missions. The resulting AMS summary scores were based on item relevance and will be useful for monitoring astronaut behavioral health on short- and long-duration space missions.

1. Introduction

1.1. Importance of Astronaut Behavioral Health for Mission Success

Astronauts are exposed to several environmental stressors that include, among others, microgravity, radiation, non-24-hour light–dark cycles, and elevated levels of carbon dioxide [1,2,3]. They are also exposed to psychological stress related to living in an isolated, confined and extreme (ICE) environment for prolonged periods of time. Reduced sensory stimulation and sensory monotony have been shown to contribute to sleep disruptions, impaired cognitive performance, negative affect, and interpersonal tension and conflict [4]. While any of these factors can be considered a significant stressor independently, it can be expected that they interact and synergistically aggravate an astronaut’s stress response [5]. Several studies have shown structural and functional brain changes in astronauts returning from six-month ISS missions, and some of those do not fully revert to pre-flight measurements even months after return to Earth [6]. Thus, it is not surprising that adverse cognitive or behavioral conditions and psychiatric disorders pose one of the greatest unmitigated risks of space exploration [7].

So far, only four astronauts have spent more than one year consecutively in space [8]. Thus, we have limited knowledge about how humans will react to the spaceflight environment on prolonged exploration missions such as a >1000-day future mission to Mars on which astronauts will have to endure unprecedented levels of radiation exposure outside Earth’s protective magnetic shield, confinement, and isolation aggravated by communication delays and threat to life.

1.2. The Role of Objective Cognitive Tests and Subjective Assessments in Spaceflight

Continuous high levels of astronaut cognitive performance are of high relevance as even small mistakes can have catastrophic consequences and cause mission failure. Astronauts are highly trained and selected, but they do not differ from the general population in their relative inability to self-assess performance, especially in chronic exposure situations [9]. This is why repeated objective cognitive performance testing is so important for mission success [6]. NASA’s Cognition test battery was developed for high-performing astronauts [10]. It consists of 10 brief cognitive tests that cover a range of cognitive domains relevant for spaceflight, yet it typically takes less than 20 min to administer. The fact that Cognition has 15 unique forms is a strength of Cognition that allows for the repeated testing and monitoring of astronauts on long-duration space missions. Practice and stimulus difficulty effects have been established for Cognition [11].

There is a long history of the administration of surveys in spaceflight and space-analog environments with the goal to assess various mental and emotional states. Commonly used instruments include the Profile of Moods States (POMS) [12] or the Beck Depression Inventory (BDI) [13,14]. Most of these scales contain multiple items to capture a single domain (e.g., confusion–bewilderment on the POMS), with sub-scores for each domain and a total score across domains. The sheer number of questionnaires and the time required to fill them out has been a problem in spaceflight and especially in space-analog environments, leading to response fatigue and complaints by study participants.

In many settings, Cognition administration is preceded by a brief 18-item Alertness and Mood Survey (AMS) that can be used to track astronaut self-assessed mental state before, during and after space missions. In order to reduce administration time and avoid response fatigue, Cognition AMS uses visual analog scales where each question covers a single domain of interest. So far, analyses have concentrated on individual AMS items as a combined AMS score does not exist. The latter would be a useful summary metric for astronaut behavioral health that could be tracked across space missions of different durations.

1.3. Study Objectives

The overarching objective of this study was to develop a combined AMS score using the paired comparison (PC) method. Specifically, we aimed to (1) identify the relevance of 14 self-reported ratings relative to alertness and mood for short- as well as long-duration space missions based on subject matter expert (SME) ratings, (2) combine the answers to individual AMS items into a single AMS score reflecting the relative importance of the individual items, and (3) describe the time course of the two newly derived AMS scores during a two-month bed rest study that served as a ground-based spaceflight analog mimicking several stressors commonly encountered in spaceflight.

2. Materials and Methods

2.1. Cognition Alertness and Mood Survey

The AMS is administered prior to the 10 tests of the Cognition test battery. Items #1 and #2 ask about the last time the astronaut went to bed and woke up. Items #3–#15 consist of the following 11-point Likert scales (0–10; anchors are provided in parenthesis after each question; the middle response category is labeled “neutral”): (#3) What was the quality of your sleep? (good–poor); (#4) What was today’s workload? (very high–very low); How are you feeling right now? (#5; not sleepy at all–very sleepy); (#6; happy–unhappy); (#7; sick–healthy); (#8; energetic–physically exhausted); (#9; mentally sharp–mentally fatigued); (#10; not stressed at all–very stressed); (#11; tired–fresh, ready to go); (#12; very depressed–not depressed at all); (#13; very bored–not bored at all); (#14; not lonely at all–very lonely); and (#15) What is your current everyday life like? (very monotonous–not monotonous at all). Item #16 asks about stimulant or depressant intake in the last 6 h. Item #17 asks about crew conflicts that may have occurred in the past week, and if so, item #18 asks how many of them were resolved (none/some/all). We only used items #3–#15 and item #17 for the AMS score (see below). We excluded self-reported sleep time and stimulant/depressant use as they are only indirectly related to self-assessed alertness and mood.

2.2. AMS Score Generation with the Paired Comparison Method

The easiest way to generate a single AMS score would be a simple average across all items. However, this approach does not reflect that some items may be more relevant for astronaut behavioral health and mission success than others. We therefore recruited 19 SMEs (13 male, mean ± SD age 39.3 ± 14.9 years, range 21–74 years) to assess the relative spaceflight importance of each of the AMS items with the goal to derive the AMS score as a weighted average of items using SME-determined item relevance as the weight. SMEs were chosen based on their experience and involvement in astronaut behavioral health research.

As it is hard to rank items when the number of items is large, we used the PC method [15] where each item is compared to each other item (i.e., a full PC). For the 14 investigated AMS items, this translates to 13 + 12 + 11 + … + 1 = 91 pairwise comparisons. The relevance of each item is then determined by the proportion of comparisons in which an item was scored as more relevant than its comparator. A proportion of 0 would mean the item was never considered more relevant while a proportion of 0.5 would mean the item was considered more relevant in 50% of the pairwise comparisons. This proportion was then used as the weight for generating the AMS score. As the importance of the individual AMS items may differ depending on mission duration, we asked SMEs to perform the PC twice: once for short duration missions (defined as 6 months or shorter) and once for long-duration missions (defined as > 6 months).

The PC was performed with a software developed in Visual Basics for Applications (Microsoft Excel). SMEs were first provided with background information and instructions on how to operate the software (see Appendix A). For each of the 91 comparisons they were asked the following: “Out of the two items below, please choose the one that you believe is more relevant for astronaut behavioral health during LONG duration spaceflight (>6 months)” or “Out of the two items below, please choose the one that you believe is more relevant for astronaut behavioral health during SHORT duration spaceflight (<6 months)”. SMEs were provided with a brief description and the anchors of each of the two items to be compared (see

Table 1. Item description and item anchors for the paired comparison.

Item #	Item Description	Item Anchors
1	Sleep Quality	Good–Poor
2	Workload	Very low–Very high
3	Sleepiness	Not sleepy at all–Very sleepy
4	Happiness	Happy–Unhappy
5	Health	Healthy–Sick
6	Energy Level	Energetic–Physically exhausted
7	Mental Status	Mentally sharp–Mentally fatigued
8	Stress Level	Not stressed at all–Very stressed
9	Tiredness	Fresh, ready to go–Tired
10	Depression	Not depressed at all–Very depressed
11	Boredom	Not bored at all–Very bored
12	Loneliness	Not lonely at all–Very lonely
13	Monotony	Not monotonous at all–Very monotonous
14	Crew Conflicts	No (0)–Yes (10)

). They were asked to press the number key 1 if they believed the left item was more relevant and the number key 3 if they believed the left item was more relevant. The date and time of the decision were recorded with 1 s resolution. The comparisons were randomized so that each SME received a unique order of paired comparisons. Additionally, the randomization process assured that each item was presented as the left item in 40–60% of comparisons. Eight SMEs performed the PC for short-duration missions first, the other eleven subjects performed it second.

Statistical Analysis of Paired Comparison Data

A coefficient of consistency (CoC) was calculated to measure the consistency of each SME’s ratings [15]. This measure is based on the number of circular mistakes (e.g., A > B, B > C, and C > A). The CoC value ranges from 0 to 1, with 1 indicating the complete absence of circular mistakes. The inclusion of individual SMEs was based on an a priori-defined cut-off of 0.35. Exact 95% confidence intervals were calculated for the PC proportions. We also investigated whether PC proportions and decision times differed between short- and long-duration PCs with a paired t-test.

2.3. Bed Rest Study

We used a 6° head-down tilt bed rest (HDBR) study performed at the German Aerospace Center (DLR) in Cologne, Germany to investigate how the AMS score for short-duration (AMS-SD) and long-duration (AMS-LD) missions was affected by HDBR (some of the descriptions below are replicated from Basner et al. [16]). HDBR is considered a space-analog environment as it mimics the reduced mobility and head-ward fluid shifts encountered by astronauts in microgravity. The study was titled Artificial Gravity Bed Rest—European Space Agency (AGBRESA). Participants were randomly assigned to one of three groups of eight participants each, all of them undergoing 60 days of strict 6⁰ HDBR: (1) Control group: no artificial gravity intervention; (2) continuous artificial gravity (cAG) group: one continuous 30 min bout of centrifugation daily; and (3) intermittent artificial gravity (iAG) group: six 5 min bouts of centrifugation with 3 min rest between bouts daily. Participants were pseudo-randomly assigned to groups. The age of the 24 participants averaged 33.3 ±9.2 years (range 23–54 years) and 14 (66.7%) were male. Study design and protocol details can be found in Basner at al. [16].

Participants performed the full Cognition battery including AMS twice for task familiarization 13 and 11 days before the start for the HDBR period. Cognition was performed three more times on days 9, 7 and 6 before HDBR (baseline bouts). Furthermore, Cognition was then administered on days 1, 3, 5, 14, 28, 42, and 57 after the initiation of the HDBR period. Finally, participants performed Cognition on days 1, 5 and 12 during the recovery period following HDBR. Cognition laptops were mounted vertically on an adjustable swivel arm and positioned at chest height in front of the participants for testing in the head-down tilt position.

Statistical Analysis of Bed Rest Data

Consistent with analyses reported in Basner et al. [16], linear mixed-effect models with random intercepts were used to account for the fact that test data were clustered within participants (SAS version 9.4, SAS Institute, Carey, NC, USA). Survey data were treated as continuous for analysis purposes [17]. Item #17 (crew conflicts) was not used for calculating AMS-SD and AMS-LD, as participants had private rooms throughout the HDBR period. As explained above, AMS-SD and AMS-LD reflect weighted averages of the survey responses with PC proportions used as weights. Higher AMS scores reflect greater alertness and mood disturbance (i.e., are worse and less desirable). AMS scores range from 0 (minimal alertness and mood disturbance) to 10 (maximal alertness and mood disturbance). All models were adjusted for sex and age (continuous variable). p-values were adjusted for multiple testing according to the false discovery rate method [18] for the 15 subjective outcomes (13 individual items plus AMS-SD and AMS-LD). We provide unadjusted confidence intervals as well as the alpha level that survived adjustment (i.e., p < 0.05, p < 0.01, p < 0.001, p < 0.0001).

Marginal means were estimated for the AG groups and controls during the HDT and recovery phase using observed marginal means for age and sex, and it was investigated whether values reported during the HDBR and recovery phase differed significantly from baseline. Furthermore, the iAG group and the cAG group were contrasted with the control group separately for the HDBR and the recovery phase. Finally, we investigated whether there was a linear change in assessments with time in HDBR, and whether the slope differed significantly between groups (i.e., group*time interaction). The last model was the only model that allowed for random intercepts and random slopes (unstructured covariance).

3. Results

3.1. Paired Comparison Results

The average ± SD CoCs for AMS-SD and AMS-LD were 0.778 ± 0.083 (range: 0.589–0.946) and 0.753 ± 0.090 (range: 0.589–0.973), respectively. Thus, all SMEs met the a priori-defined cut-off of 0.35 and contributed to the PC analysis. The main findings of the PC are shown in Figure 1 (numeric results for calculating AMS-SD and AMS-LD can be found in

Table A1. Proportion of times an item was considered more relevant compared to all other items for short-duration (SDM; up to 6 months) and long-duration (LDM; >6 months) space missions.

	Mission Duration	Rating	CI Low	CI High
Boredom	SDM	18.6%	7.1%	40.7%
	LDM	25.5%	11.3%	48.0%
	Average	22.1%	9.1%	44.4%
Monotony	SDM	17.8%	6.6%	39.8%
	LDM	28.3%	13.1%	50.8%
	Average	23.1%	9.7%	45.5%
Sleepiness	SDM	41.7%	22.8%	63.4%
	LDM	29.6%	14.0%	52.0%
	Average	35.6%	18.3%	57.8%
Workload	SDM	36.4%	18.9%	58.6%
	LDM	35.2%	18.0%	57.5%
	Average	35.8%	18.4%	58.0%
Tiredness	SDM	47.4%	27.3%	68.3%
	LDM	34.8%	17.7%	57.1%
	Average	41.1%	22.4%	62.8%
Happiness	SDM	40.1%	21.6%	61.9%
	LDM	49.4%	29.0%	70.0%
	Average	44.7%	25.2%	66.0%
Loneliness	SDM	34.0%	17.1%	56.3%
	LDM	56.7%	35.2%	75.9%
	Average	45.3%	25.7%	66.6%
Energy Level	SDM	53.0%	32.1%	73.0%
	LDM	38.1%	20.1%	60.1%
	Average	45.5%	25.9%	66.7%
Sleep Quality	SDM	55.1%	33.8%	74.6%
	LDM	51.4%	30.7%	71.7%
	Average	53.2%	32.2%	73.2%
Mental Status	SDM	63.2%	41.0%	80.9%
	LDM	49.4%	29.0%	70.0%
	Average	56.3%	34.8%	75.6%
Stress Level	SDM	66.8%	44.5%	83.5%
	LDM	70.0%	47.6%	85.8%
	Average	68.4%	46.0%	84.6%
Health	SDM	75.7%	53.3%	89.5%
	LDM	72.9%	50.4%	87.7%
	Average	74.3%	51.8%	88.6%
Depression	SDM	72.5%	50.0%	87.4%
	LDM	80.6%	58.4%	92.4%
	Average	76.5%	54.1%	90.0%
Crew Conflicts	SDM	77.7%	55.4%	90.8%
	LDM	78.1%	55.8%	91.0%
	Average	77.9%	55.6%	90.9%

CI: confidence interval; average reflects average of SDM and LDM ratings.

).

Boredom (22.1%), monotony (23.1%), and sleepiness (35.6%) were chosen least often by SMEs over other items, while health (74.3%), depression (76.5%), and crew conflicts (77.9%) were chosen most often (percentages reflect averages across scorings for short- and long-duration missions). Six out of the fourteen items differed statistically significantly between PCs for short- and long-duration missions: sleepiness, tiredness, energy level and mental status were considered more relevant by SMEs for short-duration missions, while monotony and loneliness were considered more relevant for long-duration missions (Table A1).

3.2. Bed Rest Study Results

Estimated means for AMS-SD and AMS-LD across the three phases of the HDBR study are shown in Figure 2 for the three groups (control, cAG and iAG). A small but statistically non-significant (all p > 0.18) increase in AMS-SD and AMS-LD was observed with time in HDBR. This linear trend did not differ between the three study groups (p = 0.67 for AMS-SD and p = 0.62 for AMS-LD for interaction, respectively).

Figure 3 shows how the 13 AMS items as well as AMS-SD and AMS-LD differed during the HDBR period relative to the baseline for the three study groups (left panel) and whether the cAG and iAG groups differed from the control group during HDBR (right panel). Statistically significant increases in at least one of the study groups were observed for feeling unhappy, sick, mentally fatigued, depressed, bored, lonely, and experiencing monotony during HDBR relative to the baseline, while workload was assessed to be statistically significantly lower in the control and cAG groups during HDBR. AMS-SD and AMS-LD were statistically significantly higher during HDBR relative to the baseline in all the study groups except for AMS-SD in the control group. Pooled across the three study groups, AMS-SD was 0.59 units higher (p = 0.0001) and AMS-LD was 0.66 units higher (p < 0.0001) during HDBR compared to the baseline, respectively. Survey responses and AMS scores in the cAG and iAG groups did not differ from the control except for workload, which was assessed to be statistically significantly higher in the intervention groups (both adjusted p < 0.05).

Figure 4 shows how the 13 AMS items as well as AMS-SD and AMS-LD differed during the recovery period relative to the baseline for the three study groups (left panel) and whether the cAG and iAG groups differed from the control group during recovery (right panel). Statistically significant increases in at least one of the study groups were observed for all outcomes except for sleep quality and boredom during recovery relative to the baseline. AMS-SD and AMS-LD were statistically significantly higher during recovery relative to the baseline, being more pronounced in the control and cAG groups compared to the iAG group. Pooled across the three study groups, AMS-SD was 1.29 units higher (p < 0.0001) and AMS-LD was 1.25 units higher (p < 0.0001) during recovery compared to the baseline, respectively. Survey responses and AMS scores in cAG and iAG groups did not differ from the control (all adjusted p > 0.05).

4. Discussion

In this study, we developed a single summary score for 14 items of the Alertness and Mood survey that precedes NASA’s Cognition test battery. The summary score is not a simple but a weighted average across items. The weights were informed by a PC analysis performed by 19 SMEs. A PC can be used to rank a large number of items relative to their importance or relevance. It breaks down the ranking problem into n × (n − 1)/2 pairwise comparisons, where the rater has to decide which of the two items is more relevant for the question at hand. The proportion of comparisons where an item was considered more relevant than the other items ranges from 0 to 1 and reflects its relevance or rank relative to the other items. This is an advantage of the PC as its results can be considered ratio-scaled as opposed to only interval-scaled results that a classical ranking would produce. The PC proportions were used as weights for generating the AMS summary score.

As the importance of each item may depend on space mission duration, SMEs were asked to perform the PC twice, for short- (up to 6 months) and long-duration (>6 months) space missions. Averaging across short- and long-duration PCs, AMS items considered the least relevant (chosen <40% of comparisons) were boredom, monotony, sleepiness and workload. Tiredness, happiness, loneliness, energy level, sleep quality, mental status, and stress were considered somewhat relevant (chosen in 40–70% of comparisons), while health, depression and crew conflicts were considered most relevant (chosen >70% of comparisons). These results highlight the importance of crew cohesion [19,20], on the one hand, and mental health, on the other hand, for mission success. They corroborate the aforementioned fact that adverse cognitive or behavioral conditions and psychiatric disorders are considered one of the greatest unmitigated risks of space exploration [7].

The SMEs considered sleepiness, tiredness, energy level and mental status significantly more relevant for short-duration compared to long-duration missions, while they considered monotony and loneliness significantly more relevant for long-duration compared to short-duration missions. This makes sense in light of the fact that the workload on shuttle and ISS missions has typically been high and considered one of the reasons for insufficient sleep [21,22]. In contrast, one of the main challenges of exploration-class missions to Mars and beyond will be to provide the crew with meaningful work while en route to their destination [23] and to cope with prolonged periods of isolation and confinement with the aggravating factor of communication delays [14]. Given the importance of sleep for cognitive performance, mood and health [24,25], we were surprised to see the low relevance scores for sleepiness, especially for long-duration missions. The SMEs may have been unaware of the fact that several of the states identified as highly relevant can result from acutely or chronically insufficient sleep (e.g., stress, aggressive behavior, depression), and that sleep loss and circadian misalignment have been shown to be prevalent in spaceflight [21,22] and to impair cognitive performance [5,21].

We used a 60-day HDBR study with an intermittent (iAG) and continuous (cAG) artificial gravity countermeasure [16] to investigate how individual AMS items as well as AMS-SD and AMS-LD changed during and after HDBR relative to pre-HDBR levels. We found significantly elevated AMS scores (indicating greater alertness and mood disturbance) during and after HDBR, with more pronounced elevations during the post-HDBR recovery phase. These elevations were similar across the control and intervention groups during HDBR, and somewhat less pronounced for the iAG group during post-HDBR recovery. The differences between AMS-SD and AMS-LD were negligible, though, which can likely be explained by the fact that those items most strongly affected by HDBR (e.g., health and stress) were considered equally important for short- and long-duration missions by SMEs.

The strengths of this study include the use of the PC method for deriving weights for generating AMS summary scores and that SMEs performed both PCs for short- and long-duration missions on a computer platform that guaranteed a randomized but balanced presentation of AMS items. However, this study also has limitations. While the PC analysis was based on 3460 comparisons, it is unclear how representative the pool of 19 SMEs, despite including both sexes and a wide age range, was for all the possible SMEs. Additionally, we only tested AMS scores in a single ground-based study in astronaut surrogate subjects. Thus, it is unclear how the findings translate to astronauts and the spaceflight environment, which should be investigated in future studies.

In conclusion, in deriving a weighted summary score for Cognition’s AMS survey with the PC method, this study generated a ranking of the individual AMS items in terms of their relevance for astronaut behavioral health, and it also shed some light on differences in relevance for short- and long-duration missions. The AMS summary scores will be a useful tool for monitoring astronaut behavioral health on short- and long-duration space missions with a very brief survey that takes less than two minutes to complete.