Development of the ‘Healthy Eating Index for Older People’ to Measure Adherence to Dietary Guidelines in Healthy Older New Zealand Adults

Abstract

This study aimed to develop the ‘Healthy Eating Index for Older People’ (the index), based on New Zealand dietary guidelines, and measures the validity and reproducibility of a food frequency questionnaire (FFQ) to derive the index scores in older adults. In Auckland, New Zealand, participants (community-dwelling adults aged 65–74 years, [n = 273, 36% male]) completed a 109-item FFQ administered one month apart (FFQ1, FFQ2), with a four-day food record (4-DFR) collected in between. Adherence to the guidelines was scored using the index, comprising a total score (maximum = 100) and two sub-scores: adequacy (maximum = 60) and moderation (maximum = 40). A comparison of FFQ1 and FFQ2 determined reproducibility, and FFQ1 and 4-DFR determined validity. Higher index scores (from FFQ1) were associated with higher intakes of protein, fibre, vitamins and minerals and lower intakes of alcohol and saturated fats (nutrients from 4-DFR) after adjusting for age and sex (all p < 0.001). Total index and sub-indices correlation coefficients ranged from 0.42 to 0.77 (all p < 0.001); weighted kappa values ranged from 0.35 to 0.67; and mean differences were all <10% of FFQ1 for reproducibility and validity. The Bland–Altman analysis showed no bias for the total index score for reproducibility and validity. However, with validation, the sub-index scores became less reliable as food intake increased (p < 0.05). The index applied to the FFQ demonstrated good construct validity and reproducibility. Relative and absolute validity were acceptable, though caution is required when using the absolute sub-index scores. The index is suitable for measuring total diet quality in older New Zealand adults.

1. Introduction

The population is ageing globally [1]. Health loss in older people is commonly due to chronic diseases such as coronary heart disease, cancers and musculoskeletal disorders. Leading causes of these conditions are diet-related risk factors such as a low fruit and vegetable intake or having a high body mass index [1,2]. In contrast, other older adults are healthy and living with just normal age-related decline in physical and cognitive functioning. In both circumstances, nutritional needs must support age-related changes, with a focus on preventing malnutrition, supporting physical and cognitive function, reducing the risk of chronic disease and preventing disability [1,3,4].

Examining the relationship between dietary intake and health outcomes has traditionally focussed on nutrients, but a dietary pattern approach considers the diet as a whole rather than as singular components and explains more of the effects of dietary intake on non-communicable diseases [5]. Dietary patterns are typically derived empirically (a posteriori) [6], using an index (a priori) [7] or through a hybrid-based approach [8].

Many a priori dietary indices have been created to measure adherence to national dietary guidelines, e.g., the Healthy Eating Index in the United States of America [9], or are specifically associated with disease, e.g., Dietary Approaches to Stop Hypertension (DASH) [10]. In addition, dietary indices can be specific to a population with its own particular food preferences and availability, e.g., the Baltic Sea Diet [11], or to sub-groups within the population, e.g., the Dietary Index for a Child’s Eating [12].

Dietary indices have advantages over empirical data approaches. Indices are easily derived from dietary data, simple and inexpensive to administer, have a low burden on participants and are based on current knowledge and food guidelines. However, indices do not capture the full complexities of the diet and bias can be introduced by foods or food groups included (or not included). Health professionals and researchers can use a diet index to assess adherence to dietary guidelines; examine dietary associations with health outcomes; and observe changes in diet quality over time, at the individual and population level.

Indices, based on New Zealand food and nutrition guidelines [13], have been developed for adults [14], adolescents [15] and children [12]. For healthy older adults 65 years and over, two documents, ‘Eating for Healthy Older People’ [16] and the ‘Food and Nutrition Guidelines for Healthy Older People: a background paper’ [3], are available for use. In 2020, the Ministry of Health updated the current ‘Eating and Activity Guidelines for New Zealand’ adults (18–64-year-olds) [17], which included an update to the recommended servings and serving sizes for all age groups to be in line with Australian guidelines [17]. Further, the booklet ‘Eating for Healthy Older People’ was revised to accommodate changes in the serving recommendations [18]. These updates provide an opportunity to create the first dietary index in the older New Zealand population to measure and monitor adherence to dietary guidelines.

Measuring dietary intake is challenging. To obtain accurate data, data collection tools need to be valid for the population group and reliable. Food frequency questionnaires (FFQs) are commonly used to collect dietary data for deriving an eating index [19]. When used for this purpose, the FFQ should be assessed for its suitability to derive both a valid and reliable dietary index. Validity describes how well a tool, e.g., an FFQ, measures what it claims to measure, e.g., dietary intake. Construct validity can be assessed by examining associations between dietary index scores and food and nutrient intakes, nutritional biomarkers, and socio-demographic, health and behavioural characteristics [14,20,21,22,23]. Validity measures the agreement between two dietary assessment tools with independent errors, e.g., an FFQ and a food record. Relative validity assesses the ability to rank individuals, e.g., were high scorers in the FFQ also high scorers in the food record? Absolute validity assesses the true measure, e.g., do the energy intakes from the FFQ agree with the energy intakes from the food record? The ability of an FFQ to give the same results under similar circumstances but at a different time point provides a measure of reproducibility or reliability.

This article outlines the development of an eating index for healthy older people based on current food and nutrition guidelines and measures the validity (construct, relative and absolute) and reproducibility of an FFQ to derive the eating index scores in an older population living in New Zealand.

2. Materials and Methods

2.1. Study Design, Setting and Participants

This validation study is an adjunct to the cross-sectional REACH (Researching Eating, Activity and Cognitive Health) study designed to identify a posteriori dietary patterns and associations with cognitive function in an older New Zealand population [24]. A protocol of methods has been published [25]. Participants were a convenience sample of males and females aged 65 to 74 years living independently in Auckland, New Zealand. Participants were proficient in English and were excluded if they had a diagnosis of dementia or any other health condition (or taking medication) that could influence cognition. Only one participant per household was accepted. Data were collected between March 2018 and February 2019 during a single appointment at the Human Nutrition Research Unit, Massey University, Auckland, New Zealand. The study was conducted in accordance with the Declaration of Helsinki and approved by the Massey University Human Ethics Committee (Southern A, Application 17/69, approved 15 December 2017). Written informed consent was obtained from all participants.

2.2. Data Collection

2.2.1. Dietary Assessment

109-Item Semi-Quantitative Food Frequency Questionnaire

The 109-item FFQ was adapted from a validated FFQ intending to capture the whole diet, cross-checked and pilot-tested as detailed in Mumme et al. [26]. Serving sizes were based on commonly eaten amounts guided by FOODfiles, the New Zealand food composition database [27]. The 109 food items are provided in Supplementary Table S1. The current FFQ has demonstrated acceptable validity for nutrients [28], food groups [26] and dietary patterns derived using principal component analysis [26]. The FFQ contained extra questions about behaviours regarding whole grains, fat and salt intake, as detailed in Supplementary Table S2.

The FFQ measured dietary intake over the previous month and was completed online using Survey Monkey at the participant’s appointment (FFQ1). A researcher was available to answer questions. The second FFQ (FFQ2) was completed online at the participant’s home one month later—this was the reference instrument for testing the reproducibility of FFQ1. Missing values from the FFQs (<1% of all FFQ food items) were imputed using the multiple imputation by chained equations method [29].

For each participant, a daily intake (g/day) for each food item in both FFQ1 and FFQ2 was calculated using the stated portion size and the frequency options. Energy and nutrient values were obtained from food composition databases [27,30] based on a selected food (or combination of foods) to represent each of the 109 food items in the FFQ. For example, the FFQ item, ‘Cheese, e.g., Cheddar, Colby, Edam, Tasty, blue vein, camembert, parmesan, gouda, feta, mozzarella, brie, processed’ was represented by ‘edam cheese’ in the food composition database.

Four-Day Food Record (4-DFR)

The food record covered four consecutive days including one weekend day (4-DFR) and was the reference instrument for the validation study. This was completed after FFQ1 but before FFQ2, as recommended by Cade et al. [31]. During the participant’s appointment, participants watched an instructional video and received written instructions on completing the 4-DFR. To assist with estimating portion sizes, the instructions included photographs of various servings sizes for some food items.

Trained nutritionists and dietitians entered data from the 4-DFR into FoodWorks 10 [32] and contacted participants if data were unclear or missing. Supplement intake was not included in the analysis as supplements are not included in the New Zealand dietary guidelines. A common food log was used to ensure the consistency of data entry. All food record entries were checked by a New Zealand registered dietitian for accuracy. To obtain nutrient and energy intake data, Foodworks 10 accessed primarily the New Zealand food composition database [27], but also the Australian food composition database [30] when the New Zealand database did not contain the specific food. A final review of data identified data outliers, which were checked back to the original food diaries for accuracy. For both the FFQ and 4-DFR, average daily energy intake was considered implausible if <2100 kJ (500 kcal) or >14,700 kJ (3500 kcal) for females and <3360 kJ (800 kcal) or >16,800 kJ (4000 kcal) for males [33]. Every food entry from the 4-DFR was assigned to an equivalent food item in the 109-item FFQ to allow for similar food groupings for constructing the index scores.

2.3. The Healthy Eating Index for Older People (The Index)

The index is based on the document ‘Eating for Healthy Older People’ last updated in 2021 [18]. Serving sizes for food groups are based on examples specific to healthy older people in this document. The number of daily servings is based on age- and sex-specific recommendations in the revised appendix 3 of the Eating and Activity Guidelines [17]. The index has a maximum score of 100 and comprises two sub-indices (adequacy and moderation). Adequacy (maximum score 60) is based on ‘healthy food choices’, ‘drinking plenty of fluids every day’ and ‘choosing whole grains’. Moderation (maximum score 40) is based on ‘choosing and preparing foods low in fat, salt and sugar’ and ‘reducing alcohol intake.’ Higher scores in both sub-indices represent better adherence to dietary guidelines.

Sections from ‘Eating for Healthy Older People’ which have not been considered in the index were ‘calcium’, ‘preventing constipation’, ‘healthy weight in older people’, ‘being physically active’, ‘vitamin D’ and ‘food safety’. These sections were excluded to keep the index simple and avoid nutrient calculations.

Most food items from the 109-item FFQ were used in the index. Thirteen food items were excluded. Five items (fats and oils) were excluded because they did not fit into the ‘Eating for Healthy Older People’ food groups, but types of fats and oils were considered in the moderation sub-index (Supplementary Table S2). The remaining eight food items were spices, spreads, and sauces or chutneys. The serving sizes were small, did not fit into a category with the adequacy or moderation sub-scores and were not considered material to the index (Supplementary Table S1).

2.3.1. Adequacy

Healthy Food Choices

Five food components were assessed (vegetables; fruit; breads and cereals; milk and milk products; and legumes, nuts, seeds, fish and other seafood, eggs, poultry, or red meat [referred to as ‘protein’]). The scores were scaled between a minimum score of zero (indicating no daily servings) and a maximum score of ten (indicating meeting or exceeding the recommended daily serves) for each component (

Table 1. The eating index, the sub-indices, components, criteria and scoring.

Sub-Index		Component and Guideline	Maximum Points	Criteria for Maximum Score 1,2	Criteria for Minimum Score 1,2
Adequacy			60
	Healthy food choices
		Plenty of vegetables–servings of vegetables per day	10	Daily serves M: 51–70 y ≥ 5½ serves M: 70+ y ≥ 5 serves F: 51–70 y ≥ 5 serves F: 70+ y ≥ 5 serves	No vegetables
		Plenty of fruit–servings of fruit per day	10	Daily serves M: 51–70 y ≥ 2 serves M: 70+ y ≥ 2 serves F: 51–70 y ≥ 2 serves F: 70+ y ≥ 2 serves	No fruit
		Bread and cereals, including rice, pasta, breakfast cereals and other grain products.	10	Daily serves M: 51–70 y ≥ 6 serves M: 70+ y ≥ 4½ serves F: 51–70 y ≥ 4 serves F: 70+ y ≥ 3 serves	No bread or cereal foods
		Milk and milk products Servings of milk and milk products per day	10	Daily serves M: 51–70 y ≥ 2½ serves M: 70+ y ≥ 3½ serves F: 51–70 y ≥ 4 serves F: 70+ y ≥ 4 serves	No milk, milk products or alternative
		Legumes, nuts, seeds, fish and other seafood, eggs, poultry, or red meat with the fat removed, some legumes, nuts, seeds, fish and other seafood, eggs, poultry and/or red meat with the fat removed	10	Daily serves M: 51–70 y ≥ 2½ serves M: 70+ y ≥ 2½ serves F: 51–70 y ≥ 2 serves F: 70+ y ≥ 2 serves	No protein foods
	Choosing whole grains
		Grain foods, mostly whole grain and those naturally high in fibre	5	Always chose whole-grain breads and cereals	Never chose whole-grain breads and cereals
	Drink plenty of fluids every day
		Fluids—make plain water your first choice over other drinks	5	Daily serves M: 10 cups F: 8 cups but limits on coffee (≤4 cups), tea (≤6 cups), and fruit/vegetable juice or low-calorie cordial or diet soft drinks (≤1 cup)	No fluid intake
Moderation			40
	Choose and prepare foods low in fat, salt and sugar
		with unsaturated fats instead of saturated fats
		Lean cuts of meat	5	Fat and skin always removed from meat	Fat and skin never removed from meat
		Choice of cooking fat	2.5	No saturated fats used	Saturated fat used
		Choice of spread	2.5	No saturated fats used	Saturated fat used
		with little or no added sugar
		Sugar drinks–servings per week	5	No sugar drinks	≥1 serving per week
		Sweet treats–servings per week	5	No sweet treats	≥1 serving per week
		that are mostly ‘whole’ and less processed
		Processed foods	5	No processed foods	≥1 serving per week
		that are low in salt (sodium)
		Salt used in cooking	2.5	Salt never added	Salt always added
		Salt added at the table	2.5	Salt never added	Salt always added
	Alcohol and your health
		If you drink alcohol, keep your intake low–standard drinks per week	10	No alcohol	M: ≥15 drinks F: ≥10 drinks

¹ for the adequacy component, score criteria are based on recommended serves per day [17]. ² scores are scaled between maximum and minimum criteria except choice of cooking fat or spread, which is maximum score if no saturated fat used and zero if saturated fat used. M, male; F, female.

). Fifty-nine food items from the 109-item FFQ were assigned to one of the five food components (see Supplementary Table S1).

To avoid under-scoring plant-based eaters, any legumes consumed over and above the recommended protein servings were assigned to the vegetable group, except soy products, which are usually used as a meat substitute. This is similar to the approach used by the Healthy Eating Index 2015 [9] and the Healthy Eating Index for Australian Adults [34].

Choosing Whole Grains

Behaviours about choosing whole grains were scaled between five points (maximum) for always choosing whole grains and cereals to zero points where whole grains were never consumed (Supplementary Tables S1 and S2).

Drink Plenty of Fluids Every Day

A maximum score of five was achieved where eight (female) or ten (male) cups of suitable fluids were consumed daily. Eleven food items from the 109-item FFQ were assigned to the fluid’s component including up to one glass of juice (Supplementary Table S1). Cordials, sport drinks, energy drinks, and fizzy drinks were excluded. For tea and coffee, up to six or four cups per day were counted, respectively, as indicated by the recommendations. Any juice surplus to fluid requirements was transferred to sugar drinks in the moderation sub-index.

2.3.2. Moderation

Choose and Prepare Foods Low in Fat, Salt, and Sugar

It is recommended that intakes of fast food, takeaways, processed snacks, cakes, confectionery, cordials, and soft drinks are only occasional i.e., less than once a week due to containing high amounts of fat, salt and sugar.

The questions used to measure behaviour around fat and salt intake are in Supplementary Table S2. Behaviours about consuming saturated fats were scaled between ten points (maximum), where all saturated fats were excluded from the diet, and zero points, where saturated fats were consistently part of the diet (Table 1). This concept was the same for salt intake, except the maximum score was five (Table 1).

The scoring of sugary foods was based on two components: sugar drinks and sweet treats. Fifteen food items from the 109-item FFQ were assigned to one of these two components (Supplementary Table S1). A maximum score of five for sugar drinks and five for sweet treats was achieved when no servings were consumed and scaled to zero where more than one serving per week was consumed (Table 1).

Processed Food

The scoring for the intake of processed foods was based on seven food items from the 109-item FFQ. Like sugar foods, a maximum score of five was achieved for no servings of processed foods scaled to zero for more than one serving per week (Table 1).

Alcohol and Your Health

A low alcohol intake is recommended—no more than ten (females) or fifteen (males) standard drinks per week. Six food items from the 109-item FFQ were used to calculate weekly standard drinks (Supplementary Table S1). A maximum score of ten was achieved for consuming no alcohol and scaled to zero for ten (female) or fifteen (male) or more standard drinks per week (Table 1).

2.4. Statistical Analysis

Statistical analyses were performed using R (version 4.1.3) [35]. Normality of data was assumed based on the central limit theorem. Homogeneity of variance was checked using graphs. For descriptive statistics, means (standard deviation) were used for continuous variables and count and percentages were used for categorical variables. Welch two-sample t-tests and Chi-squared tests (or Fisher’s exact test if assumptions were not met) examined differences between the sexes. To test the construct validity of the index as a measure of diet quality, the linear relationship between the index scores from the FFQ1 and nutrients from the 4-DFR was assessed after adjusting for age, sex, and energy intake. As there were multiple tests, a Benjamini–Hochberg adjustment was applied, and an adjusted p-value ≤ 0.05 was significant when considering construct validity.

The 4-DFR (reference) tested the validity (absolute and relative) of FFQ1 (test) and the derived index scores. FFQ2 (reference) tested the reproducibility of FFQ1 (test) and the derived index scores. There were no sex differences in any index score derived from the test (FFQ1) or reference (FFQ2 or 4-DFR) dietary assessment tools (

Table 2. Characteristics of participants evaluating construct validity, relative validity and reproducibility of a food frequency questionnaire for assessing an a priori dietary pattern in 273 older New Zealand adults.

Characteristic	Total a		Male a		Female a		p-Value b†
n	273		100		173
Age (years)	69.8	(2.6)	70.4	(2.4)	69.5	(2.6)	0.006
Education							0.009
Secondary	62	(23)	14	(14)	48	(28)
Post-secondary	112	(41)	40	(40)	72	(42)
University	99	(36)	46	(46)	53	(31)
Ethnicity							0.72
Asian	7	(3)	2	(2)	5	(3)
European/Other	260	(95)	95	(95)	165	(95)
Māori/Pacific	6	(2)	3	(3)	3	(2)
Index of Multiple Deprivation c	2091	(1433)	1899	(1489)	2203	(1391)	0.10
Daily energy intake (kJ)
FFQ1	7569	(2143)	8073	(2341)	7278	(1969)	0.005
FFQ2	7184	(2140)	7650	(2069)	6915	(2139)	0.006
4-DFR	8135	(1975)	9425	(2036)	7389	(1504)	<0.001
Eating index scores
FFQ1
Adequacy (max 60)	42.7	(7.5)	41.6	(7.8)	43.4	(7.3)	0.06
Moderation (max 40)	24.2	(6.1)	24.0	(6.1)	24.3	(6.2)	0.66
Total (max 100)	66.9	(9.8)	65.6	(9.8)	67.7	(9.8)	0.09
FFQ2
Adequacy (max 60)	41.2	(7.3)	40.8	(7.5)	41.5	(7.2)	0.43
Moderation (max 40)	24.6	(6.0)	24.7	(5.8)	24.5	(6.2)	0.80
Total (max 100)	65.8	(9.7)	65.5	(9.7)	65.9	(9.7)	0.65
4-DFR d
Adequacy (max 55)	37.3	(7.3)	38.2	(6.6)	36.8	(6.3)	0.07
Moderation (max 25)	11.3	(5.4)	10.9	(5.4)	11.6	(5.5)	0.34
Total (max 80)	48.6	(8.8)	49.2	(8.5)	48.4	(9.0)	0.46

BMI, body mass index; FFQ1, initial food frequency questionnaire (test); FFQ2 = second food frequency questionnaire one month later (reference); 4-DFR = four-day food record (four consecutive days, including one weekend day) completed in the month after FFQ1 completion (reference). ^a Mean (SD) or n (%). ^b Welch two-sample t-test; Fisher’s exact test; Pearson’s Chi-squared test. ^c Cohort score range 11–5555, Index of Multiple Deprivation score range 1–5958, higher number is a higher level of deprivation. ^d the scores from the 4-DFR excluded fat, salt and whole grain components. ^† Difference between male and female.

); therefore, the analyses for the validation and reproducibility of the index scores were performed as one data set. However, there were sex differences in the mean differences between the FFQ1 and 4-DFR index scores, and additional analyses were performed by sex.

Several tools were used to assess the different facets of validity [36]. First, a Spearman correlation coefficient tested the relative linear association between test and reference scores. A Spearman correlation coefficient rho > 0.50 indicated good correlation; between 0.20 and 0.49 was acceptable, and <0.20 was a poor outcome [36]. Second, a weighted Cohen’s kappa score measured cross-classification while considering chance. To achieve this, the total index and sub-indices scores were grouped into tertiles for test and reference methods. The weight applied was 1 for correctly classified participants, 0.5 for adjacent classification, and 0 weighting for gross misclassification. A weighted kappa statistic > 0.60 indicated good agreement, while between 0.20 and 0.60 was acceptable, and <0.20 was poor agreement [36]. Thirdly, the method recommended by Bland and Altman [37] and used in nutrition validation studies [38,39] examined the absolute agreement between the test and reference index scores. Mean differences between the test and reference scores were calculated and plotted against the average of the test and reference scores. Acceptable agreement was considered where (1) no slope of bias was detected using linear regression and (2) the limits of agreement were between 50 and 200% (or one half and twice) of the test and reference tool [37,38]. Limits of agreement were +/− 1.96 of the mean difference in standard deviation and provided an interval where 95% of all individual differences are expected to fall. The limits of agreement were calculated with log-transformed data to provide a percentage of agreement where 100% is exact agreement. For further comparisons between studies, the width of the limits of agreement (as a percentage of the maximum score) were calculated. Fourth, a one-sample t test examined whether the mean difference (e.g., test−reference) for the index scores was significantly different to zero. Effect sizes, from the mean difference test, were calculated using Cohen’s d [40]. An effect was small where 0.20 ≤ d < 0.50, medium where 0.50 ≤ d < 0.80, and large where d ≥ 0.80 [40].

The fat and salt moderation and whole grain adequacy components were not measured with the 4-DFR and were omitted from the validation analysis. This meant the index scores for validity were out of 80 rather than 100. However, a separate analysis was performed with the FFQ1 fat score and the saturated fat intake (as a percent of energy intake) to examine relative validity.

3. Results

3.1. Participant Characteristics

The FFQ1, FFQ2 and 4-DFR were completed by 273 participants (Figure 1). Participants had a mean age of 69.8 years, and 95% identified as being of European origin (Table 2). Males were older and had a higher education and higher energy intake than females (Table 2). All participants had plausible energy intake and were therefore included in the analysis.

3.2. Construct Validity of the Index

After adjusting for age, sex, energy intake and multiple comparisons, higher total index scores (from FFQ1) were associated with higher intakes of protein, fibre, polyunsaturated fat (PUFA), riboflavin, vitamin B6, folate, beta-carotene equivalents, vitamin C, vitamin E, calcium, iron, potassium, magnesium, phosphorus and zinc and lower intakes of alcohol, saturated fat, and sodium, as well as a lower PUFA/saturated fat ratio (from the 4-DFR) (all p ≤ 0.01) (Supplementary Table S3).

3.3. Validation of the Index Scores

The validation analysis examined similarities and agreements between the index scores derived from the FFQ1 and 4-DFR, and the results are reported in

Table 3. Validation summary statistics and interpretation for FFQ1 and 4-DFR ^a and the total and sub-index scores of the eating index (n = 273).

Statistical Test	Adequacy (55)	Moderation (25)	Index Score (80 a)
FFQ1 score a, mean (SD) †	38.6 (7.3) †*	10.7 (4.6)	49.3 (8.4) †**
4-DFR score a, mean (SD) †	37.3 (6.4)	11.3 (5.4)	48.6 (8.8)
Correlation coefficient b	0.42 *** Acceptable	0.62 *** Good	0.47 *** Acceptable
Weighted kappa value (95% CI) c	0.35 (0.24, 0.45) Acceptable	0.54 (0.46, 0.62) Good	0.36 (0.25, 0.46) Acceptable
Mean difference (95% CI) d Cohen’s d e Difference between male and female †	1.3 (0.4, 2.2) ** d = 0.18 ***	−0.7 (−1.2, −0.1) * d = 0.15 ns	0.6 (−0.4, 1.6) d = 0.08 ***
Mean % difference f	3.4 Good	−6.4 Good	1.3 Good
LoA as score values g LoA width % (range/maximum score)	−13.3, 16.0 53%	−9.5, 8.2 71%	−16.0, 17.2 42%
LoA as % h	67, 159	24, 379	70, 148
Presence of bias, y = intercept + βx (95% CI β) i	−5.44 + 0.18x (0.02, 0.33) *	+1.46 − 0.20x (−0.31, −0.08) **	+3.92 − 0.07x (−0.20, 0.07)
Bland–Altman agreement j	No agreement	No agreement	Agree

^a 4-DFR did not measure whole grain choice, fat moderation and salt moderation. FFQ1 scores match the 4-DFR, and maximum index score is 80. ^b Spearman correlation. Interpretation: good ≥ 0.50; acceptable 0.20–0.49; poor < 0.20 [36]. ^c Weighted kappa (Κ_w) outcomes: good Κ_w > 0.60, acceptable Κ_w 0.20 to 0.60, or poor Κ_w < 0.20 [36]. Cross-classification interpretation: % of participants in same tertile, good ≥ 50%, poor: <50%; % of participants in opposite tertile, good ≤ 10%, poor > 10% [36]. ^d Mean difference (FFQ1 score–4-DFR score), 95% CI; Asterix indicates mean difference is significantly different to zero (t-test). Interpretation: good p > 0.05; poor: p < 0.05 [36]. ^e Effect size for mean difference. Cohen’s d (d): small effect 0.20 ≤ d < 0.50; medium effect 0.50 ≤ d < 0.80; large effect d ≤ 0.80. ^f Mean difference percentage = (FFQ1 score–4-DFR score)/FFQ1 score. Interpretation: good 0–10%; acceptable 11–20%; poor > 20% [36]. ^g LoA expressed as 1.96 SD above and below the mean difference. ^h LoA expressed as the upper and lower limit of difference between the FFQ1 and 4-DFR, e.g., FFQ1 total index score is between 70% and 148% of the corresponding 4-DFR score (with 95% confidence). Acceptable is between 50% and 200%. ⁱ Fitted regression line calculated using simple regression to show linear trend or presence of bias. ^j Bland–Altman analysis agreement when no presence of bias and LoA % are between 0.50 and 200. * p < 0.05, ** p < 0.01, *** p < 0.001. ^† Difference between male and female. 4-DFR = four-day food record (four consecutive days, including one weekend day) completed in the month after FFQ1 completion (reference); CI, confidence interval; FFQ1, initial food frequency questionnaire (test); LoA, limits of agreement; SD, standard deviation.

and Supplementary Tables S4 and S5. The Spearman correlation coefficients for adequacy, moderation and total index scores were 0.42, 0.62 and 0.47, respectively (all p < 0.001), indicating acceptable to good relative validity (Table 3). The weighted kappa scores for the adequacy, moderation and total index ranged from 0.35 to 0.54, indicating acceptable to good relative validity (Table 3).

The mean score differences (FFQ1–4-DFR) for the total index score were not significantly different to zero (Table 3). The mean score differences for both adequacy (1.3 [95% confidence interval (CI) 0.4, 2.2] p < 0.01) and moderation (−0.7 [95% CI −1.2, −0.1] p < 0.05) were significantly different to zero, though the effect sizes, or Cohen’s d values, were very small (d < 0.20), and the mean difference, as a percentage ((FFQ1–4-DFR)/FFQ1), was below 10% (Table 3). More than 93% of participants fell between the limits of agreement of the mean difference for adequacy, moderation and total index scores. The width of the limits of agreement, as a percentage of the maximum score, ranged from 42% to 71% (Table 3). The limits of agreement showed the FFQ1 total index score was between 70% and 148% of the corresponding 4-DFR score (with 95% confidence). Likewise, the adequacy score was within acceptable limits (67% and 159%), but the moderation score was outside the acceptable limits (24% and 379%) of the corresponding 4-DFR score. Simple regression showed the total index score was not subject to bias (p = 0.33), though both adequacy and moderation scores became less reliable as the scores increased (β = 0.18 [95% CI −0.02, 0.33] p = 0.02; β = −0.20 [95% CI −0.31, 0.08] p = 0.001) (Table 3). The relative validity of fat intake was acceptable, with a significant Spearman correlation (rho = −0.36, p < 0.001) and acceptable kappa statistics (κ_w = 0.33 [95% CI 0.23, 0.43]) between the FFQ1 fat score (higher score better) and the 4-DFR % energy intake from saturated fat (Supplementary Table S4).

During the Bland–Altman analysis, it was observed that males scored higher from the 4-DFR than the FFQ1, which contrasted with females scoring higher from the FFQ1 than the 4-DFR for adequacy and total index scores. This resulted in mean differences (FFQ1–4-DFR) being significantly different for males and females (males v females, mean difference [FFQ1 score–4-DFR score], [95% CI]; adequacy −0.9 [−2.5, 0.6] v 2.6 [1.6, 3.7] p < 0.001; moderation −0.8 [−1.6, −0.1] v −0.6 [−1.3, 0.1] p = 0.71; total index −1.7 [−3.3, −0.1] v 2.0 [0.7, 3.3] p < 0.001) (Figure 2, Supplementary Table S5).

3.4. Reproducibility of the Index Scores

The reproducibility analysis examined similarities and agreements between the index scores derived from the FFQ1 and FFQ2. The results are reported in

Table 4. Reproducibility summary statistics for FFQ1 and FFQ2 and the total and sub-index scores of the eating index (n = 273).

Statistical Test	Adequacy (60)	Moderation (40)	Index Score (100)
FFQ1 score, mean (SD) †	42.7 (7.5)	24.2 (6.1)	66.9 (9.8)
FFQ2 score, mean (SD) †	41.2 (7.3)	24.6 (6.0)	65.8 (9.7)
Correlation coefficient a	0.69 *** Good	0.77 *** Good	0.76 *** Good
Weighted kappa value (95% CI) b	0.63 (0.55, 0.71) Good	0.67 (0.59, 0.74) Good	0.67 (0.60, 0.74) Good
Mean difference (95% CI) c† Cohen’s d d Difference between male and female †	1.5 (0.8, 2.2) *** d = 0.26 ***	−0.4 (−0.8, 0.1) d = 0.10 ns	1.1 (0.3, 1.9) d = 0.17 ns
Mean % difference e	3.5% Good	−1.5% Good	1.7% Good
LoA (as values) f LoA width % (range/maximum score)	−9.5, 12.5 37%	−8.0, 7.2 38%	−11.8, 14.0 26%
LoA as % g	78, 138	69, 140	82, 125
Presence of bias, y = intercept + βx (95% CI β) h	+0.37 + 0.03x (−0.07, 0.13)	−0.87 + 0.02x (−0.06, 0.10)	+0.25 + 0.01x (−0.07, 0.10)
Bland–Altman agreement i	Agree	Agree	Agree

^a Spearman correlation. Interpretation: good ≥ 0.50; acceptable 0.20–0.49; poor < 0.20 [36]. ^b Weighted kappa (Κ_w) outcomes: good Κ_w > 0.60, acceptable Κ_w 0.20 to 0.60, or poor Κ_w < 0.20 [36]. Cross classification. Interpretation: % of participants in same tertile, good ≥ 50%, poor: <50%; % of participants in opposite tertile, good ≤ 10%, poor >10% [36]. ^c Mean difference (FFQ1 score–FFQ2 score), 95% CI; Asterix indicates mean difference is significantly different to zero (t-test). Interpretation: good p > 0.05; poor: p < 0.05 [36]. ^d Effect size for mean difference. Cohen’s d (d): small effect 0.20 ≤ d < 0.50; medium effect 0.50 ≤ d < 0.80; large effect d ≤ 0.80. ^e Mean difference percentage = (FFQ1 score–FFQ2 score)/FFQ1 score. Interpretation: good 0–10%; acceptable 11–20%; poor > 20% [36]. ^f LoA expressed as 1.96 SD above and below the mean difference. ^g LoA expressed as the upper and lower limit of difference between the FFQ1 and FFQ2, e.g., FFQ1 total index score is between 70% and 148% of the corresponding 4-DFR score (with 95% confidence). Acceptable is between 50% and 200%. ^h Fitted regression line calculated using simple regression to show linear trend or presence of bias. ⁱ Bland–Altman analysis agreement when no presence of bias and LoA % are between 0.50 and 200. *** p < 0.001. ^† Difference between male and female. CI, confidence interval; FFQ1, initial food frequency questionnaire (test); FFQ2, second food frequency questionnaire (reference); LoA, limits of agreement; SD, standard deviation.

and Supplementary Table S6. The Spearman correlation coefficients for adequacy, moderation and total index scores were 0.69, 0.77 and 0.76, respectively (all (p < 0.001), indicating good reproducibility (Table 4). The weighted kappa scores for total and sub-indices scores ranged from 0.63 to 0.67 (Table 4).

Only the adequacy score had a significant mean score difference from zero, with the FFQ1 score being higher than the FFQ2 score (score [SD]; FFQ1 42.7 [7.5]; FFQ2 41.2 [7.3]; p < 0.001 Cohen’s d = 0.26) (Table 4). For adequacy, moderation and total index, the mean percentage difference ((FFQ1–FFQ2)/FFQ1) was less than 5%; more than 93% of participants fell between the limits of agreement of the mean difference; the proportion of the limits of agreement maximum score ranged from 26% to 38%; the limits of agreement showed the FFQ1 scores were within an acceptable range of 50% and 200% of the corresponding FFQ2 score; and no slope of bias was observed (Table 4). The mean differences (FFQ1–FFQ2) in the Bland–Altman analysis did not show any significant differences between male and female scores.

4. Discussion

This study describes the development of the first eating index in healthy older New Zealanders, to reflect adherence to the New Zealand Ministry of Health dietary guidelines. This study further evaluates the construct validity of the index and the ability of an FFQ to produce reproducible and valid (relative and absolute) index scores (both total index and sub-index scores).

The index is based on published, evidence-based dietary guidelines (including updated serving recommendations and sizes) provided by the Ministry of Health, New Zealand [17,18], and uses food groups to consider the whole food matrix rather than nutrients alone. The index has a total score of 100 and adopts a nested structure based around two of the four aspects of diet quality, as outlined by Burggraf et al. [41]: adequate intakes of recommended food groups (adequacy score, 60 points) and moderate intakes of foods (saturated fat, salt, sugar and alcohol) that may increase the risk of chronic disease (moderation score, 40 points). The other two components from Burggraf, Teuber, Brosig and Meier [41] are balance and diversity. Balance, in this case, considers the percentage of energy intake from each macronutrient and was therefore not considered a suitable component for the index as people consume food groups rather than macronutrients. Diversity considers food variety and was not considered in the index due to the possibility of double counting, as diversity and adequacy may be correlated.

The index has many strong points. Firstly, having an index based on foods rather than nutrients increases usability and minimises measurement error through the process of converting FFQ food items to nutrients. However, excluding nutrients may fail to recognise lower than ideal nutrient intakes. Secondly, the index goes beyond quantitative measures of food intake and incorporates behaviours in choosing and preparing foods. Thirdly, the number of servings recommended by the Eating and Activity Guidelines are categorised by age and sex, thus considering the energy intake required by an individual. Finally, the index is constructed with sub-indices (adequacy and moderation), allowing for a more in-depth analysis of adherence to dietary guidelines in individuals or populations. Including sub-indices avoids offsetting in the total score. For example, a low score may be due to a high intake of moderation foods and ignore a high fruit and vegetable intake (adequacy foods). This nested approach has been followed by other dietary indices in Australia and New Zealand [19].

The weighting of index scores has been discussed by Burggraf, Teuber, Brosig and Meier [41]. The most common technique is linear aggregation, where components of the index are given a similar weighting. This is seen in the Alternative Healthy Eating Index 2010 [42], Dietary Approaches to Stop Hypertension (DASH) [43], the Dietary Guideline Index [22] and others [19,41]. Some indices have small variations, where there were different (unequal weighted) scores for components depending on multivitamin use [44], low-fat milk use [23,45] or trimming fat from meat [23,45]. The index followed equal weighting (10 points) for core food groups in ‘Eating for Healthy Older People’ and moderation foods (Table 1) but reduced the score for ‘choosing whole grains’ and ‘drinking plenty of fluids’ to five points each. The weightings are unlikely to affect our results of comparisons between the FFQ and four-day food record.

As the index was derived specifically for an older population, decisions regarding the inclusion/exclusion of components and food items were guided by the education booklet, ‘Eating for Healthy Older People’ [18], which may have differed from guidelines for the development of indices available in the literature. For example, the bread and cereals adequacy component included both refined and whole grain food items as it is important for older people to obtain an adequate energy intake. However, an additional score (of up to five points) was awarded if the grains chosen were always whole grains, due to the importance of whole grains contributing to a healthy diet [46]. This decision is despite Waijers, Feskens and Ocké [7] recommending that whole grains should be measured separately from refined grain food items.

Older adults have unique needs compared with the general population, and these needs were considered in the development of the index. While low-fat dairy is often recommended for the general population, the ‘Eating for Healthy Older People’ booklet states people needing to lose weight should choose low-fat options and people needing to gain weight or with low energy should choose whole milk. Therefore, low- and high-fat milks were not separate in the FFQ used and this subsequent index. Additionally, calcium is an important nutrient for older adults, and ice cream was considered a suitable source of dairy. The fluid adequacy component included many fluid sources because older people have an increased risk of dehydration due to decreased/impaired thirst response, polypharmacy and reduced kidney function [47]. In New Zealand, Borkent et al. [48] and Wham et al. [49] observed low fluid intake and impending dehydration in 30% to 43% of older adults.

However the scoring is determined, it is the role of the construct validation to consider whether the index measures its intended measure. Our results suggest the index has good construct validity, as macro- and micronutrients were in line with the associated food groups of Eating Statement 1 (Appendix 4, [17]). Higher total scores were associated with increased intakes of fibre and beneficial micronutrients and decreased intakes of saturated fats and alcohol. However, a higher index score was also associated with a lower PUFA/saturated fat ratio. The effect was small and equates to an increase in the ratio of 1.0, resulting in a decrease in the index score of 1.9 (95% CI −2.8, −0.9). The Australian Dietary Guideline Index measured this same ratio in their construct validity and found a positive association between the ratio and a higher index score [22], though other nutrients trend in similar ways to this validation study [15,22]. Overall, our results suggest the index reflects a quality diet based on the New Zealand guidelines for older adults.

The adequacy, moderation and total index scores derived from the FFQ1 reflect acceptable relative validity (thus, the scores are suitable for ranking) and good reproducibility. Correlation coefficients and weighted kappa scores between the test and reference dietary assessment tools had acceptable to good outcomes, and the adequacy, moderation and total index score derived from the FFQ1 reflect acceptable facets of validity and are comparable to other studies validating an eating index in older adults [39,50,51]. Also, similar to other studies [39,51,52], the test tool (FFQ1) had total index scores 1–2% higher than the reference tools.

While the Bland–Altman analysis showed good absolute agreement with the total index score, there were significant differences between males and females. For example, females reported a 4% higher and males a 4% lower total index score on the FFQ1 index when compared to the 4-DFR total index score. The differences were minor: the female adequacy score showed that FFQ1 estimated a score of 2.6 points higher than the 4-DFR, which equates to approximately one vegetable serving or half a fruit or protein serving.

Males and females eat differently. Generally, but not always, older females are likely to have a healthier diet than older males by eating more fruit and vegetables [53,54,55]. This was seen in principal component analysis-derived dietary patterns from this same study, and in data where females scored higher on ‘Mediterranean-style’ and lower on ‘Western’ dietary patterns than males [56]. Therefore, it was not surprising that differences in the validation were observed.

Despite the mean differences offsetting in the males and females, the total index score showed agreement for reproducibility and validity in the full cohort and in male and female sub-cohorts. Further, no sex differences were observed when comparing the FFQ1 and FFQ2.

Further offsetting was observed in the Bland–Altman analysis for validation only (full cohort). The bias present in the adequacy score was offset by bias in the moderation score, resulting in no bias in the total index score. As the average intake of food groups in the adequacy or moderation component increased, the adequacy score for the FFQ1 increased and the moderation score decreased. Of course, this offset could be expected as the desire to over-report consuming healthy foods would be balanced by the desire to under-report consuming less healthy foods.

The limits of agreement from Bland–Altman analyses are reported in validation studies; however, the clinical importance of these widths is rarely discussed [36]. Establishing the clinical importance of an eating index score may not be possible; therefore, the limits of agreement (where 95% of differences will be included) have been described as (1) a percentage of the maximum score and (2) as a percentage difference between a test and reference instrument, where 100% is complete agreement (e.g., FFQ1–4-DFR = 0). These descriptions allow for comparisons between other studies. The first measure showed the width of our limits of agreement (as a % of the total score) ranged from 26% to 38% (reproducibility) and 42% to 71% (validity). This is comparable to other studies, where width percentages ranged from 34% to 44% [34,39,52]. The second measure revealed the FFQ1 total index score was between 70% and 148% of the corresponding 4-DFR score. This is comparable to other studies where scores ranged between 57% and 153% [39,51].

Strengths of this study include using a validated FFQ to measure the usual intake of food items; investigating the construct validity by comparing nutrient intakes from the 4-DFR; using a Benjamini–Hochberg adjustment when multiple analyses were used in the construct validity analysis; and having a large sample size (74% response rate) to assess males and females separately, where necessary [31]. This study also has limitations. The index was validated in adults aged 65 to 74 years but is intended to cover ages beyond 75 years. Measures for saturated fat and salt intake were limited to questions on behaviours; however, this is a method similar to that used with other indices [34,57,58]. Bias may have been introduced with the choice of food groups included and excluded in the FFQ. Our findings could be a result of chance. Biomarkers were not used in this validation, though they would add more depth to this study.

Further research can utilise the index to investigate relationships between adherence to the New Zealand dietary guidelines and health outcomes in the New Zealand population. The FFQ and index have been validated in a largely New Zealand European population and should be validated in other ethnic groups, e.g., Māori, Pacific people and Asian groups.

In conclusion, the index demonstrated good construct validity and reflected nutrient intakes associated with a healthy diet. As the index has shown good reproducibility and acceptable relative and absolute validity, it can be used to assess and monitor the diet quality of older adults. Further, the total and sub-index scores can be used further in health and other research. Caution is required where absolute agreement is required, as scores may offset between males and females and between adequacy and moderation scores. The FFQ, used in this study, is suitable to collect dietary data for creating valid and reproducible scores in this eating index in healthy older people based on New Zealand dietary guidelines.