Next Article in Journal
Pharmacokinetic Outcomes of the Interactions of Antiretroviral Agents with Food and Supplements: A Systematic Review and Meta-Analysis
Previous Article in Journal
The Role of Macronutrients, Micronutrients and Flavonoid Polyphenols in the Prevention and Treatment of Osteoporosis
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Nutrients
Volume 14
Issue 3
10.3390/nu14030519
nutrients-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Relative Validity and Reproducibility of a Semi-Quantitative Food Frequency Questionnaire for Determining Nutrient Intake in Older Adults in New Zealand: The REACH Study

by
Angela D. Yu
1,
Karen D. Mumme
1,
Cathryn A. Conlon
1,
Pamela R. von Hurst
1,
Nicola Gillies
2,
Anne-Louise Heath
3,
Jane Coad
4 and
Kathryn L. Beck
1,*
1
College of Health, Massey University, Auckland 0632, New Zealand
2
The Liggins Institute, University of Auckland, Auckland 1023, New Zealand
3
Department of Human Nutrition, University of Otago, Dunedin 9016, New Zealand
4
School of Food & Advanced Technology, College of Sciences, Massey University, Palmerston North 4474, New Zealand
*
Author to whom correspondence should be addressed.
Nutrients 2022, 14(3), 519; https://doi.org/10.3390/nu14030519
Submission received: 28 November 2021 / Revised: 17 January 2022 / Accepted: 23 January 2022 / Published: 25 January 2022
(This article belongs to the Section Nutrition Methodology & Assessment)
Browse Figure
Versions Notes

Abstract

:
Dietary intake is an important predictor of health and disease outcomes. This cross-sectional study evaluated the relative validity and reproducibility of a semi-quantitative food frequency questionnaire (FFQ) for assessing energy and nutrient intake in older adults. Dietary data were collected 2018–2019 in Auckland, New Zealand from a convenience sample of community-dwelling adults (65–74 years, n = 294, 37% male) using a 109-item self-administered FFQ at baseline (FFQ1) and four weeks later to assess reproducibility. FFQ1 was compared to a four-day food record to determine relative validity. Agreement between dietary assessment tools was assessed for both raw and energy-adjusted nutrient intakes using paired t-tests, correlation coefficients, weighted kappa statistic, Bland–Altman plots, and linear regression analysis. Energy adjustments moderately improved the relative validity and reproducibility for most nutrients. For energy and energy-adjusted nutrient intakes, the mean correlation coefficients were 0.38 (validity) and 0.65 (reproducibility); the mean weighted kappa statistics were 0.27 (validity) and 0.51 (reproducibility). A significant slope of bias was present in 54% (validity) and 25% (reproducibility) of Bland–Altman plots. The Researching Eating, Activity, and Cognitive Health (REACH) FFQ has acceptable relative validity and good reproducibility for ranking nutrient intakes in older New Zealand adults, but is less suitable for assessing absolute nutrient intakes.

1. Introduction

New Zealand’s population is ageing. The proportion of adults aged 65 years and over in New Zealand is expected to be more than 23% by 2043 (from 15%, 2016) [1,2]. As life expectancy increases, people are living longer but not necessarily in good health. In New Zealand, older adults carry a heavier burden of chronic diseases such as coronary heart disease, bowel cancer, and musculoskeletal disorders, with vascular disease being a primary cause of health loss [3]. Dietary intake is a key factor contributing to the onset of chronic diseases [3,4].
Energy needs decrease in later years, but the required intake of nutrients remains relatively unchanged, thus suboptimal intake of both macro and micronutrients is a concern [4,5,6]. For example, low calcium intake is commonly observed in older adults and insufficient dietary intake of calcium may contribute to bone loss and a higher risk of fractures [7,8]. Additionally, in older adults, inadequate protein intake may result in protein deficiency, causing changes in body composition, to the point that there is an increased risk of sarcopenia and muscle wasting [9,10].
As dietary intake is closely associated with the health and wellbeing of a population, it is important that dietary intake is accurately assessed [11,12]. However, assessing typical dietary intake in older adults is challenging, due to declining cognition and changes in eating habits associated with the ageing process. Dietary intakes can also be affected, for instance by poor oral health, taste changes, poor muscle strength, and declining metabolic rates, although diets and habitual behaviours are possibly more established in older adults compared to younger populations [13,14,15,16].
Traditionally, a weighed or estimated food record has been the preferred method to measure dietary intake. However, this dietary assessment method is impractical in epidemiological research involving large populations [17,18]. In comparison, a food frequency questionnaire (FFQ) is inexpensive, readily computerised, and relatively easy for participants to complete, so is useful in larger sample sizes [19]. In an FFQ, participants are presented with a list of food items and answer how frequently each food item is consumed [20]. Although an FFQ, compared with a food record, is less useful for measuring absolute dietary intakes, a validated FFQ can reflect the typical diet and relative nutrient intakes of a population and allow researchers to identify regional and local dietary risks [20,21].
FFQs are specific to the population they are developed for and should be assessed for their relative validity and reproducibility prior to use [20]. While a range of FFQs have been validated in New Zealand adults [22,23,24,25,26,27,28], a recent, valid, and reproducible FFQ to assess multiple nutrient intakes in older New Zealand adults is not available. To our best knowledge, the most recent validated FFQ study in older New Zealand adults was conducted over 25 years ago in 53 adults (54–86 years) living in Dunedin, New Zealand [26]. Food trends change over time, so FFQs need to be updated. The aim of this study was to evaluate the relative validity and reproducibility of a semi-quantitative FFQ adapted for use in older adults and its ability to determine nutrient intake in older adults living in New Zealand.

2. Materials and Methods

This semi-quantitative FFQ validation and reproducibility study was undertaken as part of the REACH (Researching Eating, Activity, and Cognitive Health) study at Massey University, Auckland, New Zealand. The REACH study aimed to identify the dietary patterns of older adults and their associations with cognitive function and metabolic syndrome [29]. The 109-item self-administered semi-quantitative REACH FFQ (FFQ1) was designed to assess nutrient intake, food group intake, and dietary patterns in older adults [30]. FFQ1′s ability to measure nutrient intake was assessed for validity against a four-day food record (4-DFR), and the reproducibility was assessed against a repeat administration of FFQ1 four weeks later (FFQ2). Ethical approval for the REACH study was obtained from the Massey University Human Ethics Committee Southern A, Application 17/69. Written informed consent was obtained from all participants.

2.1. Participants and Recruitment

Participants were males and females aged 65 to 74 years, living independently (i.e., not requiring assistance with daily activities or full-time care) and proficient in English. Participants were excluded if there was a previous diagnosis of dementia; they were taking medication which may affect cognitive function; or if they had or previously had any health condition that may influence cognitive function. Participants were recruited through social media, posters in public areas, radio stations, retirement villages, and by word of mouth. Participants were screened based on the eligibility criteria through telephone or email. Only one person per household was accepted. The sample size was determined by the number of REACH participants who completed three dietary assessments (FFQ1, FFQ2, and the 4-DFR). Recruitment and data collection occurred between March 2018 and May 2019. Figure 1 shows the flow of participants through the current study.

2.2. Development of the Semi-Quantitative FFQ

The semi-quantitative FFQ was derived from a validated New Zealand FFQ designed to measure iron-related dietary patterns in young women [31]. Changes were made to include serving sizes and food items not included in the original validation study as they were not typically related to iron nutrition, e.g., confectionary. Serving sizes were based on commonly eaten amounts guided by FOODfiles, the New Zealand food composition database [32]. Food items were combined to shorten the original FFQ from 144 to 109 food items, e.g., two seed food items became one. The FFQ was further cross-checked against the New Zealand Women’s Food Frequency Questionnaire to ensure all relevant food groups were covered [23]. The final semi-quantitative FFQ asked how often a serving of a food item was eaten in the last month. There were ten frequency response options ranging from “I never eat this food” to “six or more times per day”. A final open question invited participants to add any recently consumed foods not captured by the 109 food item questions. The frequency and portion size of these extra items were incorporated into the FFQ by the researcher at the time of analysis. Three example questions were provided at the beginning of the FFQ to demonstrate how the FFQ should be completed. The FFQ was pilot tested on ten individuals in the study age range for understanding and readability, and no changes were made to the FFQ as a result of pilot testing. The FFQ was administered online using SurveyMonkey [33]. The 109 food items and their serving size are provided in Table S1.

2.3. Data Collection

During an appointment at the Human Nutrition Research Unit, Massey University, Auckland, New Zealand, demographic information (age, sex, ethnicity, education levels) were collected using a self-administered questionnaire. Anthropometric measurements were collected by trained researchers using a standardised protocol [34]. Body mass index (kg/m2) was calculated from height (using a stadiometer (SECA)) and weight (using Tanita electronic scales).
FFQ1, the first self-administered online FFQ, was conducted during this onsite appointment, with a researcher available during completion of FFQ1 to answer any questions. At this same appointment, participants were asked to subsequently complete a 4-DFR at home. Participants were advised to record their usual diet over four consecutive days which included at least one weekend day. Participants watched an instructional video describing how to complete the 4-DFR including instructions on describing foods (e.g., type, brand, cooking methods) and estimating food quantities. There was an opportunity to ask questions after the video and an email address was provided if the participants had further questions. To assist with estimating food quantities, supplementary pictures of food portions on a standard size plate were provided [35]. Recipes used in home-cooked items were requested to be mailed back with the completed 4-DFR via a pre-paid envelope. Where missing or incorrect information was suspected in the 4-DFR, participants were contacted by telephone or email for further clarification. Four weeks after their first appointment participants were contacted by email with a request to complete the second FFQ (identical to the first) using a link to the online survey platform [33].

2.4. Data Entry and Management

Missing values from the FFQs (<1% of all dietary data) were imputed using the multiple imputation chained equations method [36] with 5 imputations and 20 iterations. Predictors used were food items, age, sex, education, and living situation (alone or with others). There were no differences in characteristics between participants with and without missing data.
The 4-DFRs were entered into FoodWorks 10 [37] by trained nutritionists and dietitians using a common food assumption list to ensure quality and consistency. Final data entry checks included a single overview of entries by a New Zealand registered dietitian. The original food records were further checked for accuracy when data reports and histograms identified outliers. Any potential errors were checked back to the original food diaries. FoodWorks 10 [37] was used to summarise the mean daily intakes for total energy and macro and micronutrients. The nutrient list for analysis included total energy, protein, carbohydrate, total sugars, dietary fibre, alcohol, total fat, saturated fat, monounsaturated fat, polyunsaturated fat, cholesterol, thiamine, riboflavin, niacin, vitamins A, B6, B12, C, E, folate, beta-carotene, calcium, iron, iodine, magnesium, phosphorus, selenium, potassium, and zinc. Supplement use was excluded from the analysis. The New Zealand food composition database FOODfiles 2016 [32] was prioritised in data entry. For foods not covered by the New Zealand nutrient database, Australian food databases were used, including AusFoods 2017 or AusBrands 2017 [38]. Some foods from the 4-DFR could not be matched to foods in the FFQ. These foods were salt, pepper, other seasonings, lecithin, and artificial and natural sweeteners (all having minimal contributions to micronutrient intake).
A daily intake (g/day) for each food item in both FFQ1 and FFQ2 was calculated using the stated portion size and the frequency options. Nutrient intake from the FFQs’ food item intake was calculated using R [39] by applying a mapping file of selected nutrients to daily intake. The mapping file was based on foods from FOODfiles 2016 [32], AusFoods 2017, or AusBrands [38] that were as representative as possible of each of the 109 food items in the FFQ. For example, the FFQ food item ‘sweetened cereals’ was represented by ‘Nutri-Grain, Kellogg’s fortified’ in the FOODfiles 2016 database [32]. In some instances, a composite of foods was used, e.g., ‘salad vegetables’ was represented by ‘lettuce’, ‘celery’, and ‘cucumber’ in the FOODfiles 2016 database [32]. These decisions were made in collaboration with three members of the research team (AY, KM, KB).
Average daily energy intake was considered implausible when outside the following parameters: 2090–14,640 kJ for women and 3350–16,740 kJ for men [40]. All participants fell within these ranges and therefore no participants were excluded from the analysis.

2.5. Statistical Analysis

All statistical analyses were conducted with the Statistical Package for the Social Sciences (SPSS) software version 27 (IBM SPSS, Inc., Chicago, IL, USA). Firstly, data were checked for normality of distribution visually using Q–Q plots, histograms, and Kolmogorov–Smirnov and Shapiro–Wilk tests. Data were reported as means and standard deviations (SD). Analyses were conducted using both raw and energy-adjusted data where the nutrient intake was adjusted for energy intake (nutrient intake/energy intake (MJ)) [41].
To assess the relative validity of energy and nutrient intakes from the FFQ1 against those from the 4-DFR, a range of statistical methods were conducted. Pearson (or Spearman for non-normally distributed data) correlation coefficients were used to compare energy and nutrient intakes, with the magnitude of the correlation (0 to 1) indicating the strength of the relationship. Correlation coefficients of 0.50 or higher were considered a good outcome, between 0.20 and 0.49 acceptable, and less than 0.20 was a poor outcome [42]. Depending on normality, the paired t-test or Wilcoxon signed rank test was used to compare the mean intake differences between the FFQ1 and 4-DFR. An effect size (Cohen d) was calculated where a large effect was 0.80 or higher, medium between 0.50 and 0.79, and a small effect was between 0.20 and 0.49 [43]. Energy and nutrient intakes were grouped by tertile for cross-classification. It is recommended at least 50% of participants are correctly classified and less than 10% of participants are grossly misclassified into the opposite tertiles for each nutrient [44]. For further level of agreement, the weighted kappa statistic was used alongside cross-classification. The weighted kappa statistic was calculated based on the observed and expected percentage of agreement from the cross-classification table. Values of kappa over 0.60 indicate good agreement, between 0.20 and 0.60 acceptable, and less than 0.20 poor agreement [42]. The Bland–Altman scatterplots were used for visual investigation of level of agreement at an individual level. The difference between the two intake measurements (FFQ1- 4-DFR) for each nutrient was plotted on the vertical axis and the mean of the two intake measurements on the horizontal axis. Limits of agreement (LOA = mean difference ± 2 SD) were calculated [45]. Finally, a linear regression model was used to determine the degree of dependence of the predicting variable (or the slope of bias), in which the difference in nutrient intake was the dependent variable and mean nutrient intake was the independent variable [20]. The same statistical methods used to assess validity were also used to assess reproducibility, e.g., comparing FFQ1 and FFQ2. For all statistical tests, a p-value < 0.05 was considered statistically significant.

3. Results

3.1. Participant Characteristics

A total of 371 participants took part in the REACH study, with 367 participants completing FFQ1. Four weeks later, 320 participants completed FFQ2, and 340 participants completed the 4-DFR in between the administration of FFQ1 and FFQ2. Overall, 294 participants completed all three dietary assessment tools and were included in this study (Figure 1). Their characteristics are presented in Table 1.

3.2. Relative Validity of Energy and Nutrient Intakes Derived from the REACH FFQ

To assess validity, the raw mean energy and nutrient intakes (n = 29) from FFQ1 and the 4-DFR were compared (Table 2). The mean differences for 13 nutrient intakes (including energy) (45% of the 29 nutrients) were lower in FFQ1 compared with the 4-DFR, and the mean differences for nine nutrients (31%) were higher in FFQ1 compared with the 4-DFR (all p < 0.01). After adjusting for energy intake, the number of underestimated nutrient intakes in FFQ1 dropped to 25% (n = 7) but overestimated nutrient intakes in FFQ1 increased to 54% (n = 15) (all p < 0.05) (Table 2). Medium to large effect sizes (Cohen d ≥ 50) were noted for the mean differences in both raw and energy-adjusted intakes for sugars, monounsaturated fatty acids, thiamine, riboflavin, calcium, and selenium (Table 2).
Correlations for mean energy and nutrient intakes ranged from 0.12 (vitamin A) to 0.78 (alcohol) with a mean (SD) of 0.37 (0.11). All correlation coefficients were significant (all p < 0.05). After adjusting for energy, most correlation coefficients improved moderately, were all significant (all p < 0.02), and ranged from 0.14 (selenium) to 0.77 (alcohol) with a mean (SD) of 0.38 (0.13) (Table 2).
The percentage of participants correctly classified into the same tertiles ranged from 32% (vitamin E) to 68% (alcohol), with a mean (SD) of 45% (6%). Two nutrients had 50% or more participants correctly classified into the same tertiles: alcohol (68%) and niacin equivalents (54%) (Table 3). Only alcohol, carbohydrate, magnesium, and niacin equivalents had 10% or fewer participants grossly misclassified into opposite tertiles. After energy adjustment, eight nutrients had 50% or more participants correctly classified into the same tertiles and six nutrients had 10% or fewer participants grossly misclassified (Table 3).
Weighted kappa scores (κw) between FFQ1 and 4-DFR nutrient intakes ranged from 0.12 (folate) to 0.61 (alcohol) with a mean (SD) weighted kappa score of 0.25 (0.09). Eight nutrients showed poor agreement (κw < 0.20), 20 nutrients (including energy intake) had acceptable agreement (κw between 0.21–0.60), and alcohol had good agreement (κw between 0.61–0.80). The weighted kappa scores improved moderately after energy adjustment (mean (SD) = 0.27 (0.10)). Alcohol had good agreement between the methods (κw = 0.62), 22 nutrients had acceptable agreement (0.20 ≤ κw ≤ 0.60), and five nutrients had poor agreement (κw < 0.20): folate, iodine, iron, potassium, and selenium (Table 3).
Bland–Altman analysis was performed to measure the level of agreement between FFQ1 and 4-DFR and to identify outliers. For each nutrient, limits of agreement are reported in Table S2 for both raw and energy-adjusted nutrients. Most nutrients (n = 23, 79%) had 95% of participants within the limits of agreement, and this number reduced after the nutrients were adjusted for energy (n = 20, 71%) (data not shown). Six nutrients (eight energy-adjusted nutrients) had more than 5% of participants outside the limits of agreement. Alcohol (both raw and energy-adjusted) had the most at 8%.
An example of a Bland–Altman plot is provided for energy-adjusted thiamine intake (Figure S1). This plot demonstrates that as the mean intake of thiamine (between FFQ1 and 4-DFR) increases, the difference in intakes becomes greater, indicating a slope of bias (p < 0.001). A slope of the bias was statistically significant (p < 0.05) for most nutrients (n = 19, 66%), indicating higher intakes for these nutrients may affect the differences in intake or level of agreement between the dietary assessment tools. Results improved after energy adjustment, where the number of nutrients with a slope of bias was reduced from 19 to 15 nutrients (Table S2).

3.3. Reproducibility of Energy and Nutrient Intakes Derived from the REACH FFQ

To assess reproducibility, the raw mean energy and nutrient intakes (n = 29) from FFQ1 and FFQ2 were compared (Table S3). The mean difference for most nutrient (including energy) intakes (n = 26, 90%) were higher in FFQ1 compared with FFQ2 (all p < 0.05). The effect size for these differences was minimal to small (Cohen d > 50) (Table S3).
After adjusting for energy intake, the mean differences for four nutrient intakes were significantly higher in FFQ1 compared with FFQ2 (carbohydrate, dietary fibre, potassium, and magnesium), while the mean differences for alcohol and calcium intakes were significantly lower in FFQ1 compared with FFQ2 (p = 0.05) (Table S3). There was a small effect size for the mean difference in calcium intake only (Cohen d = 0.32) (Table S3).
Correlations for mean energy and nutrient intakes ranged from 0.31 (vitamin A) to 0.90 (alcohol), with a mean (SD) of 0.63 (0.10) (all p < 0.001). After adjusting for energy, most correlation coefficients improved moderately and ranged from 0.31 (vitamin A) to 0.90 (alcohol) with a mean (SD) of 0.65 (0.10) (all p < 0.001) (Table S3).
Most nutrients (including energy intake) (n = 28, 97%) had 50% or more participants correctly classified into the same tertiles, only vitamin B12 was the exception (47% of participants) (Table S4). All nutrients had fewer than 10% of participants grossly misclassified into opposite tertiles. After energy adjustment, all nutrients had 50% or more participants correctly classified into the same tertiles and 10% or less participants grossly misclassified (Table S4).
Weighted kappa scores (κw) between FFQ1 and FFQ2 nutrient intakes ranged from 0.39 (selenium) to 0.75 (alcohol) with a mean (SD) score of 0.52 (0.06). No nutrients showed poor agreement (κw < 0.20), 28 nutrients (including energy intake) had acceptable agreement (κw between 0.21–0.60), and alcohol had good agreement (κw between 0.61 and 0.80). After energy adjustment, the mean (SD) weighted kappa score was 0.51 (0.08), with 27 nutrients having acceptable agreement, and alcohol having good agreement (Table S4).
Using Bland–Altman analysis, 11 nutrients (38%) had 95% of participant intakes within the limits of agreement, which improved after adjustment for energy intake (n = 21, 75%) (Table S5). The remaining nutrients (both raw and energy-adjusted) had more than 92% of participant intakes within the limits of agreement.
A Bland–Altman plot example demonstrates the variance of difference in thiamine intake (energy-adjusted) was consistently spread across the energy-adjusted thiamine intake and no slope of bias is apparent (p = 0.91) (Figure S2). The slope of the bias was statistically significant (p < 0.05) for six nutrients (21%). After energy adjustment, seven nutrients (25%) had a significant slope of bias (Table S5).

4. Discussion

This validation study has demonstrated the REACH 109-item FFQ shows acceptable relative validity and good reproducibility for measuring energy and 28 nutrient intakes in older adults living in New Zealand (Table 4). The REACH FFQ was compared against a 4-DFR to assess relative validity, and a second, identical, FFQ that was re-administered four weeks later to assess reproducibility.

4.1. Validity of the FFQ

To determine relative validity of the REACH FFQ, agreement of energy and nutrient intakes between the dietary assessment tools were assessed using various statistical measures. Unadjusted correlation coefficients ranged from 0.12 to 0.78. The correlation coefficients for most energy-adjusted nutrient intakes fell within an acceptable coefficient range of 0.20 to 0.49 (n = 20, 71%) or fell within a good coefficient range of 0.50 or higher (n = 5, 18%), showing acceptable relative validity and the suitability for the nutrient intakes to be ranked. The correlation coefficient range in this study is similar to other FFQ validation studies in New Zealand adults, with raw coefficients for nutrients ranging from 0.11 to 0.59 [23], 0.11 to 0.74 [28], 0.18 to 0.85 [22], and 0.36 to 0.84 [24]. Results were also comparable to similar studies in older populations: 0.01–0.40 [13], 0.09–0.78 [46], 0.17–0.49 [47], 0.34–0.75 [26], 0.38–0.55 [48], 0.56–0.84 [49], and 0.71–0.99 [50]. After adjusting nutrient intakes for energy, validity correlations between the FFQ and the 4-DFR improved moderately for most nutrients in this and other studies [22,23,28,46]. Where studies, in older adults, only reported energy-adjusted nutrients, they also reported similar coefficient ranges: −0.09–0.57 [51], 0.17–0.70 [16], and 0.31–0.67 [52]. These validation studies used similar methodologies and reference methods.
Some nutrients with low correlations, such as vitamin A, are naturally rich in only a few food items (e.g., kumara, carrots, pumpkin), which can result in variability of intake across days. Longer periods of time (more than 4 days) are likely to be needed for accurate measurements of some nutrients, though there is a trade-off between accuracy and practicality when a long data collection period is used [17]. Of interest were the nutrient intakes that were highly correlated here and in other FFQ validation studies. The three most highly correlated nutrient intakes derived from the REACH FFQ were similar to those in other studies: alcohol [16,22,24,28,46,51], carbohydrate [22,26,28,49], and cholesterol [13,22,24,26].
The REACH FFQ under- (n = 17, 59%) and over-estimated (n = 12, 41%) nutrient intake when compared with the 4-DFR. Other FFQ validation studies have reported a variety of outcomes with regards to estimating intakes. With regards to New Zealand FFQs, nutrient intake has generally been overestimated for most nutrients [22,23,28], though one FFQ (compared to a three-day food record) underestimated seven of eleven nutrients in a New Zealand European cohort (40–65 years) [24]. Studies in older populations found FFQs both under- [13,47,50] and over-estimated [16,46,51] energy and nutrient intake when compared with a food diary. In other studies, the FFQs had similar nutrient estimations [16,26,48].
In this current study, cross-classification using tertiles was poor. Ideally, for good agreement of nutrient intakes, correct classification (same tertiles) should occur for 50% or more of participants and gross misclassification (opposite tertiles) in 10% or fewer participants [42,44]. Only two nutrients (7%) had correct classification in more than 50% of participants: niacin equivalents and alcohol. Only four nutrients (14%) had fewer than 10% of participants grossly misclassified: carbohydrate, alcohol, niacin equivalents, and magnesium. Other studies reported a wide range of outcomes. Ten percent or fewer of participants were grossly misclassified in 11% to 100% of nutrient intakes [16,46,50,51]. Fifty percent or more of participants were correctly classified in 0% to 100% of nutrient intakes [16,46,51], or 70% or more of participants were within one classification for 0% to 100% of nutrient intakes [26,46,51]. However, in these studies, quartiles [16,50] and quintiles [26,46,51] were used, rather than tertiles, which makes comparing studies difficult as the number of segments used in cross-classification can affect the proportion of classification. Using quintiles instead of tertiles decreases the proportion of participants who are mis-classified but also decreases the proportion of participants correctly classified. Furthermore, cross-classification can group participants with a large variation of nutrient intakes into the same tertile and participants with very similar intakes into different tertiles depending on the tertile cut-off. Several dietary assessment validation studies in older adults did not perform a cross-classification analysis [13,26,48,49,52].
While the cross-classification analysis had a poor result, the weighted kappa was calculated to overcome agreement that may have occurred by chance in the cross-classification process [44]. Weighted kappa values ranged from 0.12 to 0.61 with a mean (SD) of 0.25 (0.09), indicating acceptable validity (Table 4). These results are comparable to similar validation studies in older adults where weighted kappa values have ranged from 0.00 to 0.77 [51], 0.08 to 0.49 [46], 0.09 to 0.50 [16], 0.16 to 0.32 [47], 0.37 to 0.50 [49], and 0.78 to 0.97 [50].
The Bland–Altman plots and subsequent linear regression analysis identified many nutrients with a slope of bias, indicating that as the mean intake (e.g., the mean of FFQ1 and 4-DFR) increased, the variation in the difference between the two intakes changed. These results suggest there is a systematic difference between the results of the two dietary assessment tools [20]. Few validation studies in older adults are reported using a Bland–Altman analysis. This may be because the analysis aims to assess agreement at a group (or population) level, whereas the correlation coefficient, cross-classification, and weighted kappa statistics are assessing the suitability of ranking the nutrient intakes [42]. However, one study (n 341, 60+ years, Spanish Mediterranean population) reported homogenous dispersions above and below the mean difference in each nutrient plot suggesting consistent agreement in absolute intake between the two dietary assessment tools (FFQ and a 24 h recall) [50].
Sometimes the second administered FFQ is validated (rather than the first FFQ) against the food record [46,47,50]. In this study, using the second FFQ may improve validity because the FFQ2 covers the month the 4-DFR is recorded. However, FFQ1 was chosen to avoid any influence the completion of the 4-DFR may have on responses to the FFQ2 [20]. The use of FFQ1 to assess validity rather than FFQ2 has been reported in other studies [16,51].
Overall, the REACH FFQ demonstrated acceptable relative validity for ranking nutrient intakes when compared to the 4-DFR (Table 4). Caution is required when measuring absolute nutrient intakes as the mean differences between the FFQ1 and 4-DFR were significantly different in 76% of nutrients. In addition, Bland–Altman analysis demonstrated the accuracy of the FFQ to assess intake of several nutrients diminished at higher levels of intake.

4.2. Reproducibility of Nutrients from the FFQ

To determine reproducibility of the REACH FFQ, agreement of energy and nutrient intakes between FFQ1 and FFQ2 were assessed using various statistical measures. Correlation coefficients for energy-adjusted nutrient intakes were good and ranged from 0.31 to 0.90. Results were similar to reproducibility coefficient ranges in older populations: 0.30–0.91 [50], 0.42–0.87 [16], 0.47–0.62 [48], 0.50–0.70 [52], 0.51–0.60 [47], and 0.66–0.96 [46]. In this current study, both the validation and reproducibility correlation coefficients for the intake of alcohol were highly correlated. This was similarly reported in other studies [16,22,24,28,46] and a meta-analysis of reproducibility studies [53].
The percentage difference between energy and nutrient mean intakes from FFQ1 and FFQ2 ranged from −7% to 11% indicating good agreement. Cross-classification between FFQ1 and FFQ2 showed good agreement (Table 4). As recommended by the literature [42,44], at least 50% of participants were correctly classified into the same tertiles for energy and most nutrients (n = 28, 97%). Only vitamin B12 was outside this range with 47% of participants correctly classified. The percentage of participants grossly misclassified into opposite tertiles ranged from 1% to 6% for all nutrients. Another study, assessing reproducibility of an FFQ in older adults, found similar cross-classification results: using quartiles, at least 50% of participants were correctly classified in 48% (n = 12) of nutrients and less than 10% of participants were grossly misclassified in 100% (n = 25) of nutrients [16].
The weighted kappa statistic demonstrated acceptable agreement between the two FFQs ranging from 0.39 to 0.75 (Table 4). The weighted kappa range was similar to other studies assessing reproducibility of FFQs in older adults; 0.24–0.40 [16], 0.37–1.00 [50], and 0.46–0.86 [46].
Bland–Altman plots indicated slopes of bias in few nutrients indicating good agreement (Table 4). This was confirmed with linear regression analysis where 22 energy-adjusted nutrients (79%) demonstrated non-significant results, indicating the difference between FFQ1 and FFQ2 was not significantly dependent on the mean intake.
In this current study, FFQ1 showed good agreement of nutrient intakes when compared with FFQ2. Higher reproducibility is observed in groups with routine and well-established diets [54], particularly noted in older compared to younger populations due to habitual behaviours [13,14]. Furthermore, higher correlations are reported in self-administered FFQs and where the second FFQ is repeated within a six-month period [20,53]. Willet (2012) suggests the ideal method to assess reproducibility is an average of repeated questionnaires over several intervals [19,20,55]; however, this is not always suitable for every research design. In this current study, there was a one-month interval between FFQ administrations, and this may be considered as a limitation. However, this interval was considered short enough to minimise any change in the diet and long enough for participants to forget their initial responses to FFQ1 [20].
Adjusting nutrient intake for energy is recommended to reduce measurement error related to the reported energy intake [14]. The analyses in this study reported both raw and energy-adjusted nutrient intakes. Energy-adjusted intakes improved correlation somewhat, and a substantial improvement was noted in the mean differences of nutrient intakes between the FFQ1 and FFQ2. Here, 26 (90%) raw nutrients had significant mean differences in intakes. This reduced to six (21%) nutrients after adjusting for energy intake, indicating better reproducibility of absolute energy-adjusted values in the REACH FFQ.

4.3. Strengths and Limitations

There were various strengths to the current validation study. Considering the number of challenges in assessing dietary intake and recruiting older adults, the current study was able to obtain a large sample size (n = 294), higher than the 100 participants recommended for validation studies [14,19,20]. All participants were cognitively healthy and as recommended, a wide range of statistical methods were used to assess validity and reproducibility. The study employed several statistical methods to assess validity and reproducibility as fewer methods may be insufficient to provide an in-depth analysis.
There are also limitations to this study. This study did not consider dietary supplement intake. A convenience sample was recruited, so may not represent the general population and participants in the study may have been more motivated to complete the dietary assessments. Participants were relatively lean compared with the New Zealand population; 13% of participants were categorised as obese (BMI > 30 kg/m2) compared with obesity levels in the older New Zealand population overall of 38% [56]. Additionally, the New Zealand population identifies as 70% European, 17% Māori, 8% Pacific peoples, and 15% Asian [57], whereas the majority of study participants were European (95%) meaning the FFQ should be validated prior to use in other ethnic groups.
Under-reporting is common in self-reported dietary assessments [20]. The simplest method to identify mis-reported energy intakes is to examine extreme intakes outside of a proposed energy range; the energy cut-off applied in this study was 2090–14,640 kJ (500–3500 kcal) for females, and 3350–16,740 kJ (800–4000 kcal) for males [40]. Based on this cut-off, all reported energy intakes from the FFQs and food records were within range. However, this crude method does not consider each individual profile and may not identify all under- or over-reported energy intakes.

5. Conclusions

In conclusion, the REACH 109-item FFQ showed acceptable relative validity and good reproducibility for energy and nutrient intakes in older New Zealand adults (65–74 years). The REACH FFQ is considered a valid dietary assessment tool for ranking nutrient intakes rather than assessing absolute intakes. The REACH FFQ can be used in future studies regarding dietary intakes in older New Zealand European adults and associations with health outcomes. Further valid and reproducible FFQs are required in older adults in other ethnic groups living in New Zealand such as Māori, Pacific Island, and Asian groups.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/nu14030519/s1, Figure S1: Bland–Altman plot of agreement for thiamine intake (energy-adjusted) between FFQ1 and 4-DFR, Figure S2: Bland–Altman plot of agreement for thiamine intake (energy-adjusted) between FFQ1 and FFQ2, Table S1: List of the 109 food items in the REACH semi-quantitative FFQ and their serving sizes, Table S2: Bland–Altman analysis of the mean daily nutrient intake from the REACH FFQ1 and the 4-DFR (validation statistics), Table S3: Mean daily energy and nutrient intakes from the REACH FFQ1 and FFQ2 (reproducibility statistics), Table S4: Cross-classification and weighted kappa for energy and nutrient intake from the REACH FFQ1 and FFQ2 (reproducibility statistics), Table S5: Bland–Altman analysis of the mean daily nutrient intake from the REACH FFQ1 and FFQ2 (reproducibility statistics).

Author Contributions

Conceptualization, K.L.B., P.R.v.H., C.A.C., A.-L.H., J.C.; methodology, K.L.B., P.R.v.H., C.A.C., A.-L.H., J.C., A.D.Y.; validation, A.D.Y., K.L.B., K.D.M.; data curation, A.D.Y., N.G., K.D.M.; writing—original draft preparation, A.D.Y., K.D.M.; writing—review and editing, A.D.Y., K.D.M., K.L.B.; supervision, K.L.B.; project administration, K.L.B., C.A.C., P.R.v.H.; funding acquisition, K.L.B., P.R.v.H., C.A.C., A.-L.H., J.C. All authors have read and agreed to the published version of the manuscript.

Funding

Funding is provided by a Health Research Council of New Zealand Emerging Researcher Grant 17/566—Beck: Optimising cognitive function: the role of dietary and lifestyle patterns. The funders have no role in the design of the study; collection, analysis, and interpretation of the data; writing manuscripts or publishing results.

Institutional Review Board Statement

Ethical approval for the REACH study was obtained from the Massey University Human Ethics Committee Southern A, Application 17/69.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Consent was not obtained from study subjects to release data.

Acknowledgments

We thank the REACH team, including Owen Mugridge and Cassie Slade, for managing recruitment and data collection; Anne Hiol and Harriet Guy for assistance with data collection; Harriet Guy, Cassie Slade, Cherise Pendergrast, and Kimberley Brown for entering 4-day food records. Permission has been received from those named in these Acknowledgements.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Fletcher, P.; Lynn, R. Health of Older People in New Zealand: A Statistical Reference; Ministry of Health: Wellington, New Zealand, 2002. Available online: https://www.health.govt.nz/publication/health-older-people-new-zealand-statistical-reference (accessed on 14 August 2021).
  2. Statistics New Zealand. National Population Projections: 2016(Base)–2068. Available online: https://www.stats.govt.nz/information-releases/national-population-projections-2016base2068 (accessed on 28 May 2021).
  3. Ministry of Health. Health Loss in New Zealand 1990–2013: A Report for the New Zealand Burden of Diseases, Injuries and Risk Factor Study; Ministry of Health: Wellington, New Zealand, 2016. Available online: https://www.health.govt.nz/publication/health-loss-new-zealand-1990-2013 (accessed on 14 August 2021).
  4. World Health Organization. World Report on Ageing and Health; World Health Organization: Geneva, Switzerland, 2015; Available online: https://apps.who.int/iris/handle/10665/186463 (accessed on 30 May 2021).
  5. Ter Borg, S.; Verlaan, S.; Hemsworth, J.; Mijnarends, D.M.; Schols, J.M.; Luiking, Y.C.; de Groot, L.C. Micronutrient intakes and potential inadequacies of community-dwelling older adults: A systematic review. Br. J. Nutr. 2015, 113, 1195–1206. [Google Scholar] [CrossRef] [PubMed]
  6. Ter Borg, S.; Verlaan, S.; Mijnarends, D.M.; Schols, J.M.G.A.; de Groot, L.C.P.G.M.; Luiking, Y.C. Macronutrient intake and inadequacies of community-dwelling older adults, a systematic review. Ann. Nutr. Metab. 2015, 66, 242–255. [Google Scholar] [CrossRef] [PubMed]
  7. Dawson-Hughes, B.; Harris, S.S.; Krall, E.A.; Dallal, G.E. Effect of calcium and vitamin D supplementation on bone, density in men and women 65 years of age or older. N. Eng. J. Med. 1997, 337, 670–676. [Google Scholar] [CrossRef]
  8. Lips, P.; Bouillon, R.; van Schoor, N.M.; Vanderschueren, D.; Verschueren, S.; Kuchuk, N.; Milisen, K.; Boonen, S. Reducing fracture risk with calcium and vitamin D. Clin. Endocrinol. 2010, 73, 277–285. [Google Scholar] [CrossRef]
  9. Yang, L.J.; Wu, G.H.; Yang, Y.L.; Wu, Y.H.; Zhang, L.; Wang, M.H.; Mo, L.Y.; Xue, G.; Wang, C.Z.; Weng, X.F. Nutrition, physical exercise, and the prevalence of sarcopenia in elderly residents in nursing homes in China. Med. Sci. Monit. 2019, 25, 4390–4399. [Google Scholar] [CrossRef]
  10. Houston, D.K.; Nicklas, B.J.; Ding, J.; Harris, T.B.; Tylavsky, F.A.; Newman, A.B.; Lee, J.S.; Sahyoun, N.R.; Visser, M.; Kritchevsky, S.B.; et al. Dietary protein intake is associated with lean mass change in older, community-dwelling adults: The Health, Aging, and Body Composition (Health ABC) Study. Am. J. Clin. Nutr. 2008, 87, 150–155. [Google Scholar] [CrossRef] [Green Version]
  11. Schulze, M.B.; Martínez-González, M.A.; Fung, T.T.; Lichtenstein, A.H.; Forouhi, N.G. Food based dietary patterns and chronic disease prevention. BMJ 2018, 361, k2396. [Google Scholar] [CrossRef] [Green Version]
  12. Hooson, J.; Hutchinson, J.; Warthon-Medina, M.; Hancock, N.; Greathead, K.; Knowles, B.; Vargas-Garcia, E.; Gibson, L.E.; Bush, L.A.; Margetts, B.; et al. A systematic review of reviews identifying UK validated dietary assessment tools for inclusion on an interactive guided website for researchers: www.nutritools.org. Crit. Rev. Food Sci. Nutr. 2019, 60, 1265–1289. [Google Scholar] [CrossRef] [Green Version]
  13. Watanabe, D.; Nanri, H.; Yoshida, T.; Yamaguchi, M.; Sugita, M.; Nozawa, Y.; Okabe, Y.; Itoi, A.; Goto, C.; Yamada, Y.; et al. Validation of energy and nutrition intake in Japanese elderly individuals estimated based on a short food frequency questionnaire compared against a 7-day dietary record: The Kyoto-Kameoka study. Nutrients 2019, 11, 688. [Google Scholar] [CrossRef] [Green Version]
  14. Cade, J.E.; Burley, V.J.; Warm, D.L.; Thompson, R.L.; Margetts, B.M. Food-frequency questionnaires: A review of their design, validation and utilisation. Nutr. Res. Rev. 2004, 17, 5–22. [Google Scholar] [CrossRef] [Green Version]
  15. Thompson, F.E.; Subar, A.F. Dietary assessment methodology. In Nutrition in the Prevention and Treatment of Disease, 4th ed.; Coulston, A.M., Boushey, C., Ferruzzi, M.G., Delahanty, L., Eds.; Academic Press: Cambridge, MA, USA, 2017; pp. 5–48. [Google Scholar]
  16. Jia, X.; Craig, L.C.A.; Aucott, L.S.; Milne, A.C.; McNeill, G. Repeatability and validity of a food frequency questionnaire in free-living older people in relation to cognitive function. J. Nutr. Health Aging 2008, 12, 735–741. [Google Scholar] [CrossRef] [PubMed]
  17. Willett, W. 24-hour recall and diet record methods. In Nutritional Epidemiology, 3rd ed.; Oxford Scholarship Online: New York, NY, USA, 2012; pp. 50–70. [Google Scholar]
  18. Willett, W. Food frequency methods. In Nutritional Epidemiology, 3rd ed.; Oxford Scholarship Online: New York, NY, USA, 2012; pp. 71–96. [Google Scholar]
  19. Willett, W. Reproducibility and validity of food frequency questionnaires. In Nutritional Epidemiology, 3rd ed.; Oxford Scholarship Online: New York, NY, USA, 2012; pp. 97–142. [Google Scholar]
  20. Cade, J.; Thompson, R.; Burley, V.; Warm, D. Development, validation and utilisation of food-frequency questionnaires—A review. Public Heath Nutr. 2002, 5, 567–587. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Gibson, R. Principles of Nutritional Assessment, 2nd ed.; Oxford Unversity Press: New York, NY, USA, 2005. [Google Scholar]
  22. Sam, C.H.Y.; Skidmore, P.; Skeaff, S.; Parackal, S.; Wall, C.; Bradbury, K.E. Relative validity and reproducibility of a short food frequency questionnaire to assess nutrient intakes of New Zealand adults. Nutrients 2020, 12, 619. [Google Scholar] [CrossRef] [Green Version]
  23. Beck, K.L.; Houston, Z.L.; McNaughton, S.A.; Kruger, R. Development and evaluation of a food frequency questionnaire to assess nutrient intakes of adult women in New Zealand. Nutr. Diet 2020, 77, 253–259. [Google Scholar] [CrossRef] [Green Version]
  24. Metcalf, P.; Swinburn, B.; Scragg, R.; Dryson, E. Reproducibility and validity of a food frequency questionnaire in European and Polynesian New Zealanders. Ethn. Health 1997, 2, 297–308. [Google Scholar] [CrossRef]
  25. Wilson, P.; Horwath, C. Validation of a short food frequency questionnaire for assessment of dietary calcium intake in women. Eur. J. Clin. Nutr. 1996, 50, 220–228. [Google Scholar]
  26. Horwath, C.C. Validity of a short food frequency questionnaire for estimating nutrient intake in elderly people. Br. J. Nutr. 1993, 70, 3–14. [Google Scholar] [CrossRef] [Green Version]
  27. Ingram, M.A.; Stonehouse, W.; Russell, K.G.; Meyer, B.J.; Kruger, R. The New Zealand PUFA Semiquantitative Food Frequency Questionnaire is a valid and reliable tool to assess PUFA intakes in healthy New Zealand adults. J. Nutr. 2012, 142, 1968–1974. [Google Scholar] [CrossRef]
  28. Sam, C.H.Y.; Skeaff, S.; Skidmore, P.M.L. A comprehensive FFQ developed for use in New Zealand adults: Reliability and validity for nutrient intakes. Public Health Nutr. 2014, 17, 287–296. [Google Scholar] [CrossRef] [Green Version]
  29. Mumme, K.; von Hurst, P.R.; Conlon, C.A.; Jones, B.; Haskell-Ramsay, C.F.; Stonehouse, W.; Heath, A.-L.M.; Coad, J.; Beck, K.L. Study protocol: Associations between dietary patterns, cognitive function and metabolic syndrome in older adults—A cross-sectional study. BMC Public Health 2019, 19, 535. [Google Scholar] [CrossRef] [Green Version]
  30. Mumme, K.; Conlon, C.; von Hurst, P.R.; Jones, B.; de Seymour, J.; Heath, A.-L.; Stonehouse, W.; Coad, J.; Haskell-Ramsay, C.; Beck, K.L. Relative validity and reproducibility of a food frequency questionnaire for assessing dietary patterns and food group intake in older New Zealand adults: The REACH study. J. Acad. Nutr. Diet. 2021, 121, 2389–2400. [Google Scholar] [CrossRef]
  31. Beck, K.L.; Kruger, R.; Conlon, C.A.; Heath, A.-L.M.; Coad, J.; Matthys, C.; Jones, B.; Stonehouse, W. The relative validity and reproducibility of an iron food frequency questionnaire for identifying iron-related dietary patterns in young women. J. Acad. Nutr. Diet. 2012, 112, 1177–1187. [Google Scholar] [CrossRef] [PubMed]
  32. The New Zealand Institute for Plant & Food Research Limited; Ministry of Health. New Zealand Food Composition Database 2017: New Zealand FOODfiles(TM) 2016 Version 01. Available online: https://www.foodcomposition.co.nz/foodfiles (accessed on 2 May 2021).
  33. Momentive Inc. SurveyMonkey. Available online: https://www.momentive.ai (accessed on 2 November 2021).
  34. Marfell-Jones, M.; Stewart, A.; De Ridder, J. International Standards for Anthropometric Assessment; International Society for the Advancement of Kinanthropometry: Wellington, New Zealand, 2012. [Google Scholar]
  35. Nelson, M.; Atkinson, M.; Meyer, J. A Photographic Atlas of Food Portion Sizes; MAFF Publications: London, UK, 1997. [Google Scholar]
  36. Van Buuren, S. Multiple imputation of discrete and continuous data by fully conditional specification. Stat. Methods Med. Res. 2007, 16, 219–242. [Google Scholar] [CrossRef] [PubMed]
  37. FoodWorks 10 Premium, Version 10.0; Xyris Pty Ltd.: Brisbane, Australia, 2019.
  38. Food Standards Australia New Zealand. Australian Food Composition Database—Release 1. FSANZ. Available online: www.foodstandards.gov.au (accessed on 2 May 2021).
  39. R Core Team. R: A Language and Environment for Statistical Computing; R Core Team: Vienna, Austria, 2019; R Version 3.6.1.; Available online: https://www.R-project.org (accessed on 21 January 2021).
  40. Willett, W. Issues in analysis and presentation of dietary data. In Nutritional Epidemiology, 3rd ed.; Oxford Scholarship Online: New York, NY, USA, 2012; pp. 306–333. [Google Scholar]
  41. Willett, W. Implications of total energy intake for epidemiologic analyses. In Nutritional Epidemiology, 3rd ed.; Oxford Scholarship Online: New York, NY, USA, 2012; pp. 261–287. [Google Scholar]
  42. Lombard, M.J.; Steyn, N.P.; Charlton, K.E.; Senekal, M. Application and interpretation of multiple statistical tests to evaluate validity of dietary intake assessment methods. Nutr. J. 2015, 14, 40. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; Routledge: New York, NY, USA, 1988. [Google Scholar]
  44. Masson, L.F.; McNeill, G.; Tomany, J.O.; Simpson, J.A.; Peace, H.S.; Wei, L.; Grubb, D.A.; Bolton-Smith, C. Statistical approaches for assessing the relative validity of a food-frequency questionnaire: Use of correlation coefficients and the kappa statistic. Public Health Nutr. 2003, 6, 313–321. [Google Scholar] [CrossRef]
  45. Bland, J.M.; Altman, D.G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986, 327, 307–310. [Google Scholar] [CrossRef]
  46. Smith, W.; Mitchell, P.; Reay, E.M.; Webb, K.; Harvey, P.W.J. Validity and reproducibility of a self-administered food frequency questionnaire in older people. Aust. N. Z. J. Public Health 1998, 22, 456–463. [Google Scholar] [CrossRef]
  47. Gilsing, A.; Mayhew, A.J.; Payette, H.; Shatenstein, B.; Kirkpatrick, S.I.; Amog, K.; Wolfson, C.; Kirkland, S.; Griffith, L.E.; Raina, P. Validity and reliability of a short diet questionnaire to estimate dietary intake in older adults in a subsample of the Canadian Longitudinal Study on Aging. Nutrients 2018, 10, 1522. [Google Scholar] [CrossRef] [Green Version]
  48. Malekahmadi, M.; Naeini, A.A.; Shab-Bidar, S.; Feizi, A.; Djazayery, A. Development, validity, and reliability of a food frequency questionnaire for antioxidants in elderly Iranian people. J. Res. Med. Sci. 2016, 21, 14. [Google Scholar] [CrossRef]
  49. Corrente, J.E.; Marchioni, D.M.L.; Fisberg, R.M. Validation of a FFQ (food frequency questionnaire) for older people. J. Life Sci. 2013, 7, 878. [Google Scholar] [CrossRef] [Green Version]
  50. Zaragoza-Martí, A.; Ferrer-Cascales, R.; Hurtado-Sánchez, J.A.; Laguna-Pérez, A.; Cabañero-Martínez, M.J. Cross-cultural adaptation, validity, and reproducibility of the Mediterranean Islands Study food frequency questionnaire in the elderly population living in the Spanish Mediterranean. Nutrients 2018, 10, 1206. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Gardener, S.L.; Lyons-Wall, P.; Martins, R.N.; Rainey-Smith, S.R. Validation and reliability of the Alzheimer’s Disease-Commonwealth Scientific and Industrial Research Organisation Food Frequency Questionnaire. Nutrients 2020, 12, 3605. [Google Scholar] [CrossRef] [PubMed]
  52. Morris, M.C.; Tangney, C.C.; Bienias, J.L.; Evans, D.A.; Wilson, R.S. Validity and reproducibility of a food frequency questionnaire by cognition in an older biracial sample. Am. J. Epidemiol. 2003, 158, 1213–1217. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Cui, Q.; Xia, Y.; Wu, Q.; Chang, Q.; Niu, K.; Zhao, Y. A meta-analysis of the reproducibility of food frequency questionnaires in nutritional epidemiological studies. Int. J. Behav. Nutr. Phys. Act. 2021, 18, 12. [Google Scholar] [CrossRef] [PubMed]
  54. Block, G.; Hartman, A.M. Issues in reproducibility and validity of dietary studies. Am. J. Clin. Nutr. 1989, 50, 1133–1138. [Google Scholar] [CrossRef] [PubMed]
  55. Willett, W. Correction for the effects of measurement error. In Nutritional Epidemiology, 3rd ed.; Oxford Scholarship Online: New York, NY, USA, 2012; pp. 288–305. [Google Scholar]
  56. Ministry of Health. Annual Data Explorerer 2019/20: New Zealand Health Survey [Data File]. Available online: https://minhealthnz.shinyapps.io/nz-health-survey-2019-20-annual-data-explorer/ (accessed on 30 August 2021).
  57. Stats NZ. Ethnicity, Culture, and Identity. Available online: https://www.stats.govt.nz/tools/2018-census-place-summaries/new-zealand#ethnicity-culture-and-identity (accessed on 30 August 2021).
Nutrients 14 00519 g001 550
Figure 1. Flow chart of participants in the REACH FFQ evaluation study assessing the relative validity and reproducibility of a semi-quantitative food frequency questionnaire for assessing nutrient intake in 294 older New Zealand adults. Abbreviations: 4-DFR, four-day food record collected between the FFQ administrations; FFQ1, first semi-quantitative food frequency questionnaire; FFQ2, semi-quantitative food frequency questionnaire re-administered four weeks later; REACH, Researching Eating, Activity, and Cognitive Health.
Figure 1. Flow chart of participants in the REACH FFQ evaluation study assessing the relative validity and reproducibility of a semi-quantitative food frequency questionnaire for assessing nutrient intake in 294 older New Zealand adults. Abbreviations: 4-DFR, four-day food record collected between the FFQ administrations; FFQ1, first semi-quantitative food frequency questionnaire; FFQ2, semi-quantitative food frequency questionnaire re-administered four weeks later; REACH, Researching Eating, Activity, and Cognitive Health.
Nutrients 14 00519 g001
Table
Table 1. Characteristics of the participants who completed three dietary assessment tools and were included in the evaluation of the relative validity and reproducibility of the REACH FFQ for assessing nutrient intake in 294 New Zealand adults.
Table 1. Characteristics of the participants who completed three dietary assessment tools and were included in the evaluation of the relative validity and reproducibility of the REACH FFQ for assessing nutrient intake in 294 New Zealand adults.
CharacteristicsMean (SD)n (%)
Age (years)69.8 (2.6)294 (100%)
Female sex 186 (63%)
Ethnicity
European/other 279 (95%)
Māori/Pacific Islander 9 (2%)
Asian 8 (3%)
Education Status
No qualification/Secondary education only 68 (23%)
Post-Secondary 118 (40%)
University 108 (37%)
BMI (kg/m2)26.1 (4.4)294 (100%)
Underweight BMI: <18.5 kg/m2 2 (1%)
Normal BMI 18.5–24.9 kg/m2 124 (42%)
Overweight BMI: 25.0–29.9 kg/m2 129 (44%)
Obese BMI: ≥30.0 kg/m2 39 (13%)
Abbreviations: BMI, body mass index; FFQ, semi-quantitative food frequency questionnaire; REACH, Researching Eating, Activity, and Cognitive Health.
Table
Table 2. Mean daily energy and nutrient intakes from the REACH FFQ1 and the 4-DFR: validation statistics for both raw a and energy-adjusted b dietary data (n = 294).
Table 2. Mean daily energy and nutrient intakes from the REACH FFQ1 and the 4-DFR: validation statistics for both raw a and energy-adjusted b dietary data (n = 294).
NutrientFFQ1 Daily Intake a
Mean (SD)		4-DFR Daily Intake a
Mean (SD)		Mean Difference a,c
(95% CI)		Percentage Difference a,d
(%)		Mean Difference e
p-Value		Effect Size fCorrelation Coefficients gCorrelation
p-Value
Raw aAdj bRaw aAdj bRaw aAdj bRaw aAdj b
Energy (MJ)7.5 (2.2)8.1 (1.9)−0.6 (−0.9, −0.3)−7.3<0.001 0.26 0.37-<0.001-
Protein (g)80.5 (24.5)82.8 (19.9)−2.3 (−5.1, 0.5)−2.80.11<0.0010.090.220.410.52<0.001<0.001
Carbohydrate (g)178.7 (60.7)191.0 (59.3)−12.3 (−19.2, −5.5)−6.5<0.0010.470.210.040.500.58<0.001<0.001
Sugars (g)113.0 (42.6)88.8 (33.5)24.3 (19.6, 28.9)27.3<0.001<0.0010.591.290.440.48<0.001<0.001
Dietary fibre (g)26.2 (9.9)28.4 (10.0)−2.1 (−3.3, −1.0)−7.5<0.0010.800.220.020.510.54<0.001<0.001
Alcohol (g) e,g7.7 (9.1)10.0 (12.8)−2.3 (−3.5, −1.1)−23.20.001 c0.02 c0.220.140.78 e0.77 e<0.001<0.001
Total fat (g)73.3 (24.0)80.4 (25.4)−7.1 (−10.5, −3.7)−8.8<0.0010.180.240.080.290.45<0.001<0.001
SFA (g)31.6 (12.3)29.4 (11.2)2.1 (0.6, 3.73)7.30.008<0.0010.160.550.310.42<0.001<0.001
MUFA (g)23.3 (7.9)29.2 (10.6)−6.0 (−7.3, −4.8)−20.6<0.001<0.0010.550.550.330.39<0.001<0.001
PUFA (g)10.2 (4.0)13.2 (6.0)−3.0 (−3.7, −2.4)−23.0<0.001<0.0010.510.470.350.48<0.001<0.001
Cholesterol (mg)284.9 (134.1)292.7 (121.7)−7.8 (−24.0, 8.4)−2.70.340.250.060.070.400.55<0.001<0.001
Thiamine (mg)1.0 (0.4)1.6 (0.9)−0.5 (−0.6, −0.4)−33.3<0.001<0.0010.630.550.360.29<0.001<0.001
Riboflavin (mg)3.0 (1.4)2.2 (0.8)0.8 (0.7, 1.0)39.1<0.001<0.0010.630.970.360.31<0.001<0.001
Niacin equiv. (mg)38.1 (11.5)37.7 (10.0)0.3 (−0.9, 1.6)0.90.60<0.0010.030.360.480.48<0.001<0.001
Vitamin B6 (mg)3.0 (1.0)2.5 (0.9)0.4 (0.3, 0.6)17.9<0.001<0.0010.480.690.500.32<0.001<0.001
Folate (μg)365.8 (134.2)376.6 (139.8)−10.8 (−30.2, 8.3)−2.90.280.030.060.130.240.19<0.001<0.001
Vitamin B12 (μg)5.2 (4.3)4.3 (3.8)0.9 (0.4, 1.5)22.20.002<0.0010.180.250.180.220.002<0.001
β-carotene (mg)4.5 (2.2)3.7 (2.3)0.9 (0.6, 1.2)23.8<0.001<0.0010.360.500.450.44<0.001<0.001
Vitamin A (mg)1.5 (1.4)1.1 (1.0)0.4 (0.2, 0.6)32.9<0.001<0.0010.230.300.120.17<0.050.004
Vitamin C (mg)133.7 (70.9)125.2 (70.2)8.5 (−0,2, 17.2)6.80.06<0.0010.110.250.420.39<0.001<0.001
Vitamin E (mg)10.2 (3.9)11.1 (4.3)−0.9 (−1.5, −0.4)−8.30.0010.760.200.020.340.49<0.001<0.001
Calcium (mg)1193.2 (552.1)923.7 (341.1)269.5 (209.9, 329.2)29.2<0.001<0.0010.520.850.400.36<0.001<0.001
Iron (mg)10.0 (3.3)12.4 (3.9)−2.3 (−2.8, −1.9)−19.0<0.001<0.0010.570.420.360.36<0.001<0.001
Iodine (μg)87.0 (36.9)97.8 (67.6)−10.8 (−18.8, −2.8)−11.00.0080.140.160.090.220.22<0.001<0.001
Potassium (mg)3965.4 (1172.3)3644.0 (967.2)321.4 (190.5, 452.4)8.8<0.001<0.0010.280.790.450.36<0.001<0.001
Magnesium (mg)340.2 (100.4)381.9 (117.3)−41.8 (−54.9, −28.7)−10.9<0.001<0.0010.370.210.460.49<0.001<0.001
Phosphorus (mg)1476.4 (498.0)1516.5 (383.4)−40.1 (−96.7, 16.5)−2.60.16<0.0010.080.210.400.38<0.001<0.001
Selenium (μg)47.1 (18.6)75.1 (41.0)−28.1 (−32.9, −23.2)−37.4<0.001<0.0010.670.590.170.140.0050.02
Zinc (mg)10.5 (3.4)10.2 (2.8)0.3 (−0.1, 0.7)3.00.14<0.0010.090.400.400.26<0.001<0.001
a Not adjusted for energy intake. b Adjusted for energy intake (nutrient intake/energy intake (MJ)) [41]. c Mean difference = FFQ1 intake − 4-DFR intake. d Mean difference % = (FFQ1 intake − 4-DFR intake)/FFQ1 intake. e The paired t-test or Wilcoxon signed rank test (alcohol) was used to compare the mean differences between FFQ1 and 4-DFR. f Effect size for mean difference. Cohen d (d): small effect 0.20 ≤ d < 0.50; medium effect 0.50 ≤ d < 0.80; large effect d ≤ 0.80. g Pearson or Spearman correlation (alcohol) coefficients. Outcomes: good rho ≥0.50, acceptable rho 0.20 to 0.49, or poor rho <0.20. Significant results, p-value < 0.05. Abbreviations: 4-DFR, four-day food records; CI, confidence interval; FFQ1, first administered semi-quantitative food frequency questionnaire; MUFA, monounsaturated fatty acid; Niacin equiv., niacin equivalents total—the sum of the percentage of niacin, preformed and niacin equivalent from tryptophan; PUFA, polyunsaturated fatty acid; REACH, Researching Eating, Activity, and Cognitive Health; SD, standard deviation; SFA, saturated fatty acid.
Table
Table 3. Cross-classification a and weighted kappa b for energy and nutrient intake from the REACH FFQ1 and the 4-DFR: validation statistics for both raw c and energy-adjusted d data (n = 294).
Table 3. Cross-classification a and weighted kappa b for energy and nutrient intake from the REACH FFQ1 and the 4-DFR: validation statistics for both raw c and energy-adjusted d data (n = 294).
NutrientCorrectly Classified—Same Tertiles (%) aGrossly Misclassified—Opposite Tertiles (%) aWeighted Kappa Statistics b
Raw cAdjusted dRaw cAdjusted dRaw cAdjusted d
Energy43.5 12.2 0.23
Protein47.650.012.211.20.270.31
Carbohydrate45.954.47.88.20.300.39
Sugars44.946.910.912.20.260.26
Dietary fibre44.250.712.29.20.230.34
Alcohol68.068.72.72.00.610.62
Total fat39.148.011.212.60.150.27
SFA42.549.011.214.30.190.26
MUFA39.845.914.613.30.160.24
PUFA46.650.013.37.80.250.35
Cholesterol42.547.315.39.20.180.30
Thiamine45.243.511.212.90.260.22
Riboflavin45.647.312.211.90.250.27
Niacin equiv.54.152.46.510.90.410.34
Vitamin B647.344.612.916.00.260.20
Folate38.440.816.717.00.120.14
Vitamin B1244.250.311.210.90.250.32
β-carotene43.944.213.313.60.220.22
Vitamin A43.245.616.013.60.180.23
Vitamin C44.245.211.611.20.240.26
Vitamin E31.649.014.67.50.180.34
Calcium48.342.511.611.90.290.22
Iron44.240.812.216.30.230.15
Iodine45.641.213.613.90.230.18
Potassium42.940.510.214.60.240.16
Magnesium46.951.78.810.90.300.33
Phosphorus42.942.211.611.60.230.22
Selenium42.944.618.417.30.150.18
Zinc48.044.210.515.60.300.20
a Cross-classification (%) outcomes: good ≥50% in same tertile or ≤10% in opposite tertile; poor <50% in same tertile or >10% in opposite tertile [42]. b Weighted kappa (κw) outcomes: good κw >0.60, acceptable κw 0.20 to 0.60, or poor κw <0.20 [42]. c Not adjusted for energy intake. d Adjusted for energy intake (nutrient intake/energy intake (MJ)) [41]. Abbreviations: 4-DFR, four-day food records; FFQ1, first administered semi-quantitative food frequency questionnaire; MUFA, monounsaturated fatty acid; Niacin equiv., niacin equivalents total is the sum of the percentage of niacin, preformed and niacin equivalent from tryptophan; PUFA, polyunsaturated fatty acid; REACH, Researching Eating, Activity, and Cognitive Health; SFA, saturated fatty acid.
Table
Table 4. Summary of statistical test outcomes and interpretation for validity and reproducibility of energy and nutrient intakes from the REACH FFQ a.
Table 4. Summary of statistical test outcomes and interpretation for validity and reproducibility of energy and nutrient intakes from the REACH FFQ a.
Statistical TestValidity bReproducibility c
Raw dEnergy-Adjusted eRaw dEnergy-Adjusted e
Correlation coefficient f, mean (SD)0.37 (0.11)
Acceptable		0.38 (0.13)
Acceptable		0.63 (0.10)
Good		0.65 (0.10)
Good
Cross-classification g, mean (SD)PoorPoorGoodGood
% in same tertiles45 (6)47 (6)61 (5)61 (5)
% in opposite tertiles12 (3)12 (3)4 (1)4 (2)
Weighted kappa value h, mean (SD)0.25 (0.09)
Acceptable		0.27 (0.10)
Acceptable		0.52 (0.06)
Acceptable		0.51 (0.08)
Acceptable
Percentage difference kAcceptableAcceptableGoodGood
% difference within ±10.9%52439396
% difference between ±11 and 20%102574
% difference beyond ±20%38%32%0%0%
Difference between mean intakes m76%
Poor		78%
Poor		90%
Poor		21%
Good
Bland–Altman nPoorPoorGoodGood
Presence of bias %66542125
an = 294. b FFQ1 (test) and 4-DFR (reference). c FFQ1 (test) and FFQ2 (reference). d Not adjusted for energy intake. e Adjusted for energy intake (nutrient intake/energy intake (MJ)) [41], f Pearson or Spearman correlation (alcohol) coefficients. Interpretation: good ≥0.50; acceptable 0.20–0.49; poor <0.20 [42]. g Calculated using tertiles. Interpretation: % of participants in same tertile, good ≥50%, poor: <50%; % of participants in opposite tertile, good ≤10%, poor >10% [42]. h Interpretation: good ≥0.61; acceptable 0.20–0.59; poor <0.20 [42]. k Mean difference percentage = (FFQ1 intake − reference intake)/FFQ1 intake. Interpretation: good 0–10%; acceptable 11–20%; poor >20% [42]. m Shows % of nutrients where the mean nutrient intakes were significantly different (p < 0.05) using paired t-test or Wilcoxon signed rank test. Interpretation: good p > 0.05; poor: p ≤ 0.05 [42]. n Shows % of nutrients with a slope of bias, calculated using linear regression. Interpretation: good p > 0.05; poor: p ≤ 0.05 [42].
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Yu, A.D.; Mumme, K.D.; Conlon, C.A.; von Hurst, P.R.; Gillies, N.; Heath, A.-L.; Coad, J.; Beck, K.L. Relative Validity and Reproducibility of a Semi-Quantitative Food Frequency Questionnaire for Determining Nutrient Intake in Older Adults in New Zealand: The REACH Study. Nutrients 2022, 14, 519. https://doi.org/10.3390/nu14030519

AMA Style

Yu AD, Mumme KD, Conlon CA, von Hurst PR, Gillies N, Heath A-L, Coad J, Beck KL. Relative Validity and Reproducibility of a Semi-Quantitative Food Frequency Questionnaire for Determining Nutrient Intake in Older Adults in New Zealand: The REACH Study. Nutrients. 2022; 14(3):519. https://doi.org/10.3390/nu14030519

Chicago/Turabian Style

Yu, Angela D., Karen D. Mumme, Cathryn A. Conlon, Pamela R. von Hurst, Nicola Gillies, Anne-Louise Heath, Jane Coad, and Kathryn L. Beck. 2022. "Relative Validity and Reproducibility of a Semi-Quantitative Food Frequency Questionnaire for Determining Nutrient Intake in Older Adults in New Zealand: The REACH Study" Nutrients 14, no. 3: 519. https://doi.org/10.3390/nu14030519

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop