Leveraging Observations of Untrained Panelists to Screen for Quality of Fresh-Cut Romaine Lettuce

Abstract

Fresh-cut romaine lettuce’s high perishability challenges ready-to-eat (RTE) salad production. Selecting cultivars less prone to browning and decay is crucial for extending shelf life. Traditional quality evaluation methods using instrumentation and trained panelists are time-consuming and logistically complex. This study investigated the effectiveness of untrained volunteers in assessing fresh-cut romaine lettuce quality. Given that the average consumer in the USA is familiar with the flavor characteristics of romaine lettuce, this study proposed to investigate the value of having untrained volunteers discern the quality of fresh-cut romaine lettuce. Therefore, six romaine lettuce accessions (Green Forest, King Henry, Parris Island Cos, PI 491224, SM13-R2, and Sun Valley) were assessed for sensory quality attributes (browning, green color, decay, and overall quality) and compared with instrumentation analyses (gas composition including O₂ and CO₂, electrolyte leakage, and color). The results showed significant quality differences (p < 0.05) among the accessions, with some seasonal variability. Very importantly, the consumers’ (n = 159) assessments revealed similar results to those produced by either instrumentation or a trained panel. The consumers provided sensory scores that allowed for the grouping of accessions based on their postharvest quality, which efficiently matched their pedigree relationship. In conclusion, ad hoc consumer panels can be an effective way to characterize the quality of romaine lettuce for RTE salads.

1. Introduction

Lettuce is an important vegetable for the fresh-cut industry, commonly used as a main ingredient in ready-to-eat (RTE) vegetable salads [1,2,3]. Until the 1970s, iceberg lettuce had dominated the produce aisle in the USA [4]. Since then, romaine lettuce, which was broadly unknown at the time of introduction, has slowly gained traction in the US market, to the point that in 2021 it accounted for 88.9% of all lettuce produced and 80% of all lettuce consumed [5].

RTE salads have steadily increased in popularity since the 1990s, in a trend that is partially associated with health-conscious consumers that demand nutrient dense, on-the-go food products [1,2,3]. RTE salad sales grew to nearly USD 10.7 billion annually in 2021 [6]. As these sales grew, so did the research efforts to investigate the optimal breeding, growing, harvesting, and processing practices that yield high quality fresh-cut salads and reduce their food loss and waste.

RTE salads are highly susceptible to quality deterioration. Common production practices, such as trimming and cutting, pose additional stress to what harvest already does to the plant tissue. Lettuce is prone to tissue browning and decay [7]. Upon cutting of the lettuce, cell integrity is disrupted, prompting physiological and biochemical reactions that lead to faster senescence [8,9,10].

Interventions to delay the deterioration rate of fresh-cut lettuce span across the cultivation to processing chain, with coordinated efforts among seed producers, growers, processors, shippers, and retailers [11,12,13]. The final quality of fresh-cut produce can be prolonged by (i) optimizing agricultural practices, (ii) applying effective postharvest treatments, and (iii) developing cultivars through breeding or genetic selection that can ensure adaptability to both growing conditions and stress caused during postharvest handling and processing.

The selection of a romaine lettuce accession that is suitable for fresh-cut applications is a fundamental step to overcome quality deterioration challenges of RTE salads. Lettuce breeding aims to select and develop cultivars that are better suited for fresh-cut processing, especially those that are slower to develop discoloration (pinking or browning) after cutting or during storage [2,3,9,14,15,16]. One recent example that targeted postharvest performance is the SM13-R2 accession, which was developed by the USDA lettuce breeding program, precisely, to introduce desirable traits from Parris Island Cos and PI 491224 [17,18].

To determine the performance of lettuce accessions as source of product for RTE salads, trained panelist evaluations, in conjunction with instrumental measurements, are often utilized. However, this is not always reliable, as the panelists commonly represent a narrow background/perspective of that of the general consumer population and, in general, consumers are more inclined to assess food products with an integrated approach rather than looking at individual characteristics or parameters [19,20]. On the other hand, consumer panels, even when formed in an ad hoc manner, occasionally have shown remarkable efficiency in predicting food quality perception by the general consumer [21,22].

Multiple studies have evaluated the sensorial quality of lettuce with trained panels [23,24,25], but there have been only a few studies that used untrained consumer panels, and the results indicate that untrained panels could be very informative [26]. For example, McWatters et al. [27] recruited 80 panelists to evaluate the consumer acceptance of fresh-cut iceberg lettuce treated with 2% hydrogen peroxide and mild heat. They concluded that the treatment resulted in the retention of good sensory quality and high product acceptance.

Several food scientists have questioned the over-reliance on trained panelists for quality assessments, arguing they may be able to discern some sensory traits better than the average volunteer and consumer [21,22]. Ad hoc panels could perform quality assessments in an efficient way when several factors are prevalent and/or avoided, including a good familiarity with the product and the experimental site, no evaluation of specific quality traits (e.g., aroma), and no request for in-depth descriptions of the product [28]. Therefore, we hypothesized that a well-known food product, which is very popular today, such as the romaine lettuce, could be efficiently evaluated for quality by local/in-house consumer panels. Hence, our objective was to determine the value of leveraging untrained consumer panels to categorize and compare the quality of six romaine lettuce accessions as sources for RTE salads. Thus, this study compared results obtained with instrumentation and trained panelists. Moreover, an additional objective was to determine whether SM13-R2, a relatively new inbred lettuce accession, had potential for fresh-cut applications.

2. Materials and Methods

2.1. Processing and Packaging

Six accessions of romaine lettuce (Lactuca sativa var. longifolia) were evaluated for sensory quality parameters; four commercial cultivars (Green Forest, King Henry, Parris Island Cos, Sun Valley), a plant introduction (PI 491224), and a breeding line (SM13-R2). SM13-R2 was developed from a cross between PI 491224 (resistant to lettuce dieback but deteriorating rapidly after processing for salad) and Parris Island Cos (good postharvest quality but susceptible to lettuce dieback) [17,18]. The lettuce plants were grown in the U.S. Department of Agriculture (USDA), the Agricultural Research Service (ARS), the Crop Improvement and Protection Research Unit, Salinas, CA, and harvested in July 2018 (first trial). The trial was repeated in November 2018 (Second trial) and in July 2021 (Third trial). The lettuce heads were harvested at market maturity, packed into carboard boxes, and immediately transported to a commercial distribution center. The boxes of lettuce were cooled to 4 °C within less than 12 h and were transported in a refrigerated truck to the USDA-ARS Food Quality Laboratory (FQL) in Beltsville, MD, USA [9]. Upon arrival, the products were transferred to a 5 °C walk-in cooler and were processed within 1 day after arrival (~5 days after harvest). Approximately 5 to 6 heads for each variety of lettuce were processed by first removing the outer leaves and trimming off the stems (cores) and the top of the leaves. After trimming, the lettuce leaves were cut into approximately 2.5 cm strips (First and third trials) or squares (Second trial) using a pilot-scale cutter (Nichimo Seven Chefs ECD-302, Tokyo, Japan). Fresh-cut lettuce was then washed in 100 mg$·$mL⁻¹ chlorine solution (NaOCl) adjusted to pH 6.5 with the washing aid, T-128 (New Leaf Food Safety Solutions, LLC, Salinas, CA, USA) for 30 s and dried using a pilot-scale centrifugal dryer (Model T-304, Meyer Machine Co, Watsonville, CA, USA) at 11.1 Hz for 2 min. Fresh-cut lettuce (128 g) was packaged in polypropylene bags (22.9 cm × 15.3 cm) with an oxygen transmission rate of 16.6 pmol$·$s⁻¹$·$m⁻²$·$Pa⁻¹ and stored at 5 °C for up to 12 days. Sensory and quality evaluations were performed periodically during cold storage, as described below.

2.2. Sensory Evaluation Using Consumer and Trained Panels

Sensorial quality attributes of fresh-cut lettuce were evaluated by a total of 159 panelists (untrained panelists) in the two trials conducted in 2018. The consumer panelists were recruited for visual quality of lettuce from among the ARS Beltsville staff who indicated enjoying eating lettuce on a frequent basis. The panelists did not receive any information about this study prior to the recruitment.

For the third trial, five members of Beltsville ARS were selected after adequate training, in part due to the COVID-19 pandemic-related constraints, but also to determine any potential deviation of trends (consumers vs. trained panelists) in relation to instrumentation analysis. The samples were evaluated after 6 and 10 days of storage in the first trial and after 6 and 12 days of storage in the second and third trials (the variability with the days selected for evaluation was due to logistical matters). For each trial with the consumer panels, eight sensory sessions were held, with nine or ten panelists in each session. Prior to evaluating the samples, the panelists were provided with descriptive explanations of the definitions of each quality attribute. For example, each panelist was provided with a pictorial intensity scale reference for browning intensity, which had been developed by a previously formed in-house trained panel at the USDA-ARS FQL.

The panelists evaluated the samples using a digitized ballot, with computers connected through Compusense Five^® Software program (version 5.6, Compusense Inc., Guelph, ON, Canada), in individual booths, with a controlled temperature set at 21 °C, under moderate incandescent light, to provide comfort during the assessments [29]. The panelists rated the lettuce samples according to four appearance attributes: (i) incidence of green (0 to 100% green); (ii) incidence of browning at edge (0 to 100% browning); (iii) decay (0 to 100% decay), defined as decomposition caused by microbial activity or tissue physiological processes; (iv) overall visual quality (OVQ) (same scale, 0 to 100%, with 100 being the highest perceived quality).

All attributes were rated on unstructured 15 cm line scales with bipolar words or general anchors for each attribute. The distance between each extreme point of the line was converted to 0–100 to obtain an intensity scale. Panelists were presented with six samples per session. The samples were arranged in white trays labeled with three-digit random number codes. Each sample weighed approximately 20 g. After the evaluation, the panelists answered a demographic questionnaire regarding gender, age, and ethnicity. The demographic information showed a gender distribution of 104 (64%) females and 55 (35%) males in two trials (First and second trial). The age distribution was as follows: 20 years or younger (4%), 21–30 years (17%), 31–40 years (19%), 41–50 years (25%), 51–60 years (19%), and over 60 years (16%). The panelists consisted of 57% Caucasian, non-Hispanic/Latinx; 24% Asian; 10% African American; 6% Latinx; 3% Native Hawaiian or other Pacific Islands. The trained panel in the third trial consisted of three males (one Caucasian, two Asians) and two females (one Caucasian, one African-Asian American), in the age range of 28 to 45 years.

2.3. Gas Analysis of the Headspace

The gas composition was measured in triplicate after 0, 3, 6, and 10 days of storage for the first trial. For the second and third trials, the measurements were taken at 0, 2, 6, 9–10, and 12 days after the samples were placed in refrigerated storage. The concentration of oxygen gas (O₂) and carbon dioxide (CO₂) within the package headspace was measured with a headspace gas analyzer system (Model Combi Check 9800-1, PBI Dansensor Co., Rinsted, Denmark) by inserting a gas-tight needle through a septum placed on the package surface [30,31,32].

2.4. Electrolyte Leakage

Electrolyte leakage percentage (EC) was calculated by electrical conductivity measurements. Samples of fresh-cut lettuce (30 g), after measuring gas composition, were immersed in 300 mL of distilled water, and the electrical conductivity was measured after 30 min. EC in microsiemens (μS) was measured using a conductivity meter (Model 135A, Orion Research Inc., Beverly, MA, USA), by inserting the probe into the sample solution and waiting for the reading to stabilize. In the first trial, the EC of the samples was measured at room temperature (around 24 °C) [33,34], while the EC of the samples from the second and third trial was measured at cold temperature (5 °C) [30,31,32]. After the initial measurements of EC were performed, samples were exposed to three freeze–thaw (−20 °C) cycles for 24 h, followed by a final measurement of EC (EC_F). EC (after 30 min: EC₃₀) was expressed as a percentage of the total electrolytes produced following the freeze–thaw cycles using the equation EC (%) = [EC₃₀/(EC_F)] × 100.

2.5. Color Quantification by Image Analysis

The surface color of the sample was quantified with a computer vision system, which was composed of a digital camera, a computer, and a portable shooting tent with controlled lighting [9,10,23]. Digital images of lettuce samples were captured using a Nikon D 800 digital camera, with a 60 mm lens (Nikon Inc., Melville, NY, USA), under the portable shooting tent (63.5 mm by 76.2 mm × 63.5 mm). The tent (Amazon Basics Portable Photo Studio, Amazon, Seattle, WA, USA) has attached white panels with a 5600 k daylight by two light-emitting diodes (LED) inside. The camera settings were an F (aperture value) of 20, shutter speed of 1/30 s, and ISO (International Standard Organization) sensitivity of 640. The images were produced and acquired with both camera software (Nikon Camera Capture 2.0, Nikon Inc., Melville, NY, USA) and the Adobe Lightroom program (version 6.3, Adobe Systems Inc., San Jose, CA, USA). Colors in the images of the samples were corrected with a 24-color reference card (X-rite ColorChecker^® Passport, X-rite Inc., Grand Rapids, MI, USA). RGB measurements that represent red, green, and blue light sources were analyzed using Image Pro Plus software (version 9.3b, Media Cybernetics, Inc., Rockville, MD, USA) and converted to L*, a*, and b* (CIELAB 1976) in the Image Processing Toolbox in MATLAB (version R2017b, MathWorks, Natick, MA, USA) [35]. The values of hue angle and chroma were calculated from a* and b* with standard equations (hue angle = tan⁻¹ [(b*) (a*)⁻¹] and chroma = [(a*)² + (b*)²]^0.5. The threshold image segmentation method developed by Otsu [36] to isolate lettuce pieces from background in RGB was used and quantified using algorithms [37]. “Brown Pixel”, as an indicator of visual browning in lettuce, was defined as the percentage of brown-colored pixels for each lettuce sample image, combining all three browning pixels from the segmentation (light, medium, dark). The method was developed in previous studies [9,10].

2.6. Statistical Analysis

The sensory and instrumental data (i.e., gas composition, electrolyte leakage, and color parameters) were subjected to an analysis of variance (ANOVA) to compare the different accessions through the storage evaluation times, both within each trial and across the two initial trials, using SAS (Statistical Analysis Systems Inc., version 9.4, Cary, NC, USA), Proc GLM (general lineal model). Mean comparisons were performed with a Tukey–Kramer least significant difference test (p < 0.05). The coefficients of determination (Pearson) were obtained for visual sensorial attributes and for all pairs of sensorial and instrumental parameters [O₂ values, CO₂ value, electrolyte leakage percentage (EC), L*, a*, b*, hue angle, chroma, percent of brown areas/colored pixels (“Brown Pixel”)]. In addition, a component analysis using Proc FACTOR (factor analysis) was performed to characterize the relationship among the parameters by latent factors. After the data matrix was normalized, the means were centered for each parameter, followed by each value disintegration; the factors were nominated by the Scree test. Bi-plots were constructed to illustrate relationships among lettuce traits and accessions using SigmaPlot (version 13.0, Systat Software, Inc., San Jose, CA, USA) [29].

3. Results and Discussion

3.1. Color Quantification

The color parameters (L*, a*, b*, hue angle, chroma) of most samples decreased during storage time, except for the a* value, which increased over time. As expected, the increased incidence of brown lettuce tissue coincided with color measurements, revealing that the samples became darker (lower L*), with more red and less green (higher a*) color, as well as more yellow and less blue (higher b*). Among the accessions, King Henry and Sun Valley showed the highest amount of brown tissue in the first trial and second trial, respectively. Conversely, the accessions with the lowest levels of brown color were SM13-R2 in the first trial, and its parental lines, Parris Island Cos and PI 491224, in the second trial (Figure 1). This better performance of the Parris Island Cos pedigree accession over King Henry was also observed in another study [10].

3.2. Electrolyte Leakage

The calculated electrolyte leakage in the first trial was relatively high immediately after fresh-cut processing and decreased rapidly during the first 3 days, then gradually increased during the next 3 days and remained stable afterward. The only exception was PI 491224, whose percentage increased rapidly after day 6. A similar result has been observed previously [2,3]. In the second trial, the rapid increase in EL after day 6 was marked for all accessions.

When cell membranes, which protect the content of a cell and control the movement of particles in and out of the cell, lose integrity, electrolytes, such as K⁺ ions, leak out of the cell. Thus, the amount of the leaked electrolytes from a tissue is used as an indirect measurement for the extent of cell damage, injury, or death [30,31,32,33]. The initial high level of the electrolyte leakage might be explained due to damage from the fresh-cut processing, while the reduction during the beginning of storage (usually day 2–6) is due to the healing of damaged tissue. The flattening or rapid increase during further storage time may be the result of poor tissue healing and subsequent increase in ruptured tissue and overall deterioration [30,31,32,33]. The latter is reflected in Figure 2.

In both the second and third trials, the EC increased after flattening began, on the 6th day in storage. This was not observed in the first trial. This difference could be explained by pre-harvest growing conditions (e.g., temperatures, water scheduling), which dissimilarly affect the seasonal quality of the samples at harvest. The perception of lower quality sensory attributes was confirmed by most instrumental and sensory evaluations. Lettuce is a cool weather crop that grows better when both day and night temperatures are low, and the cumulative degree days during the crop growth cycle are not high [38]. In the Salinas area, lettuce plants harvested in the fall are cultivated in relatively higher temperatures and cumulative degrees days compared to those harvested in the spring or summer. Higher temperatures at the early stages of plant development can result in the rapid elongation of plant tissue, impeding optimum translocation of key nutrients, such as calcium and nitrogen [39,40,41]. The latter has been associated with increased tendency to undergo browning in postharvest storage [25,41]. The temperatures in Salinas, CA, USA, during the growing season of the second trial were unusually high [42].

The initial amount of electrolyte leakage in the first trial averaged over one and a half to two times higher than that observed in the second or third trial. This was somewhat expected, considering that the first trial samples were measured at room temperature, whereas the second trial and the third trial samples were measured at 5 °C. The electrolyte leakage values obtained were consistent with previous research [32].

When observing the individual accessions across the three trials, the graph patterns of electrolyte leakage in storage evidenced similar trends for Parris Island Cos (parental accession) and SM13-R2 (descendant accession), with a high correlation (r = 0.93; p-value < 0.0001) between them in Figure 2. On the other hand, the correlation between PI 491224 (another parental accession) and SM13-R2 was low (r = 0.26; p-value = 0.369).

3.3. Gas Composition

The physiology of lettuce during postharvest stages resembles that of a living organism, with a respiration that involves the uptake of oxygen and the release of carbon dioxide. The stress deriving from harvest and fresh-cut processing triggers the acceleration of respiration, in a response often called ‘wound-induced respiration’ [43,44,45]. The greater the degree of processing, the higher the induced respiration. For example, the respiration rates of shredded lettuce were 200–300% greater than those of the intact heads [46]. This explains why it has been common to associate the total amount of oxygen consumption by fresh produce with the quality deterioration rate [45,47]. In all three trials, the O₂ concentration decreased rapidly at the beginning of the storage time and remained relatively stable afterward. Among the accessions, PI 491224 had significantly less O₂ concentration and higher CO₂ concentration in all trials, indicating a higher respiration rate. The oxygen concentration for the sample in the second trial was lower than that of the first and third summer trials, suggesting a lower respiration rate (Figure 3), which agrees with the slower deterioration implied by the electrolyte leakage results (Figure 2). In the Salinas region, romaine lettuce harvested during the fall usually has a more rapid deterioration at MAP conditions compared to lettuce harvested in spring or summer [48].

3.4. Sensory Quality

Four sensory attributes were evaluated: browning, decay, green, and overall visual quality (OVQ). Sensory scores were rated on scale from 0 to 100, where 0 is bad and 100 is good, while the anchor terms for incidence of green, browning at the edge, and decay were expressed as a percentage. Generally, scores of 40 to 70 OVQ are considered within the salability range, while scores > 70 were defined as highly acceptable [49]. The ‘browning’ and ‘decay’ scores increased, while the ‘green’ and OVQ decreased with longer storage time.

3.4.1. Browning

Brown (discolored) tissue is a common problem in the fresh-cut lettuce industry. It is preceded by the production and subsequent oxidation of selective metabolites (e.g., phenolics) involved in the plant’s defense mechanism in response to environmental/physical stress [14,43,44]. Consumers find the browning of lettuce cuts unappealing, resulting in one of the main reasons for not purchasing fresh-cut products [50]. Thus, delaying the onset of the discoloration process is a key challenge of the RTE salad processors.

After 6 days of storage in the first trial, the browning scores of King Henry (18.9 ± 2.7, 24.2 ± 6.1) were significantly higher than those of Parris Island Cos (7.5 ± 1.3, 9.1 ± 2.7), Sun Valley (9.6 ± 2.5, 7.7 ± 2.0), Green Forest (10.5 ± 2.0, 7.9 ± 2.4), and SM13-R2 (7.5 ± 1.9, 8.2 ± 2.7). Between days 6 and 10, King Henry, Green Forest, and SM13-R2 deteriorated at a similar rate, as they did during the 0–6 days period, whereas Parris Island Cos and Sun Valley deteriorated more slowly. After 10 days of storage in the first trial, Parris Island Cos had the lowest browning score, followed by Sun Valley, while King Henry showed again the highest browning score (Figure 4A). The browning scores were much higher in the second trial than in the two summer trials (first and third), likely due to seasonal variations and a lower quality raw product in the fall. While the difference among accessions were minor toward the end of the fall trial, in the two summer trials, King Henry was found more browned than the other accessions (Figure 4). Overall, the scores provided by the consumers reflected well what was calculated with the image analysis (brown pixels).

3.4.2. Green Color

The green scores (%) ranged from 66 to 87 on day 6 and from 52 to 72 on day 10 in the first trial. In the second trial, the scores ranged from 63 to 75 on day 6 and from 51 to 69 on day 12. In trial 3, the green scores ranged from 64 to 86 on day 6 and from 70 to 88 on day 10. There was no significant difference in green ratings among accessions. Given that the green color results are not significantly different from each other, green incidence may not be useful for differentiating quality among different accessions of romaine lettuce (Figure 5). Despite the latter, the green score showed a high positive correlation (r = 0.87; p-value < 0.0001) with OVQ. Overall, based on the green and browning scores, it appeared that browning is an easier way to differentiate the visual quality of fresh-cut romaine lettuce.

3.4.3. Decay

‘Decay’, defined as deterioration caused by microbial activity and/or tissue physiological changes, is perceived in lettuce cuts as darkening, water logging, and decomposition [2,3,51]. It shortens the shelf-life and eventually makes the product unacceptable to consumers. The decay score had a strong negative correlation with OVQ (r = −0.80; p-value = 0.0019).

In the summer trials, the decay scores ranged from 4.3 to 10.9 (first trial) and from 0.4 to 11.0 (third trial) on day 6. Scores ranging from 7.9 to 16.0 (first trial) and from 2.4 to 11.0 (third trial) were reported on day 10. However, there were no statistically significant differences among the accessions in any of the evaluation days. In the second trial, the decay scores among accessions followed the same pattern as in the first trial, although the scores were higher. The only exception was PI 491224, whose decay score jumped from 11.0 (day 6) to 47.6 (day 12) during the second trial, suggesting a rapid deterioration rate (Figure 6A,B).

3.4.4. Overall Visual Quality

Overall visual quality (OVQ) reflects the consumer’s final judgement of the acceptability of the product, after integrating all sensory inputs [52]. It is the most important factor in determining consumer preference and leads consumers to purchasing behavior.

On day 6 of the first trial, the OVQ scores of Parris Island Cos (83.9 ± 2.5), Green Forest (84.1 ± 2.6), and SM13-R2 (81 ± 3.3) were significantly higher than those of PI 491224 (64.8 ± 3.9) and King Henry (62.6 ± 3.8). On day 10, their scores decreased by 10 points for Parris Island Cos, PI 491224, and Sun Valley, by 20 points for King Henry and SM13-R2, and by 25 points for Green Forest. On the same storage day, the OVQ scores of Parris Island Cos (73.7 ± 3.5), Sun Valley (69.2 ± 3.8), and SM13-R2 (62.2 ± 4.1) were significantly higher than those of King Henry (42.7 ± 3.0).

In the second trial, the OVQ scores were significantly lower than those in the summer trials (first and third trials). On day 6, PI 491224 (71.1 ± 3.7) and Parris Island Cos (60.9 ± 3.7) scored significantly higher than King Henry (45.8 ± 3.3). PI 491224 scores fell markedly, reaching the lower numbers in this study. The OVQ scores of Parris Island Cos (57.8 ± 4.0) and King Henry (48.3 ± 4.1) on day 10 remained similar to the values on day 6, as depicted in Figure 7.

The OVQ scores in the third trial followed a similar pattern to that observed in the first trial. On day 6, King Henry (69.9 ± 4.1) had a significantly lower score compared to the other accessions. Four days later, SM13-R2 dropped 20 points, as the accession with the second lowest score.

In all trials, the OVQ scores of Parris Island Cos (78.2 ± 3.0) on day 10 were similar to those on day 6, indicating it is a cultivar with a good shelf-life performance. King Henry showed the lowest OVQ score, suggesting it has a short self-life (Figure 7). Based solely on the OVQ scores, the SM13-R2 breeding line and its parent, Parris Island Cos (and to some degree PI 491224), followed a similar pattern, with initial high scores (day 6) but then dropping substantially (day 10).

3.5. Correlation between Sensory Assessments and Instrumentation Measurement

To achieve main purpose of this study, that is, to determine the feasibility of relying on ad hoc consumer panels for the analysis of romaine fresh-cut lettuce as a source of RTE salads, the next step was to determine to what extent consumer sensory results could resemble the instrumentation measurements.

The bi-plot in Figure 8 shows the relationship between the instrumental parameters (L*, a*, b*, hue angle, chroma, brown pixels, O₂, CO₂, and electrolyte leakage values) and sensory attributes (green, browning, decay, and OVQ). Factor analysis was also conducted for the correlation matrix. The first factor and the second factor accounted for 53.8% and 24.4% of the variation, respectively. The visual scores provided by consumers for browning were highly correlated with the hue angle (r = −0.83; p-value < 0.0001), a* value (r = 0.72; p-value < 0.0001), and brown pixels (r = 0.76; p-value < 0.0001), which were also commonly associated to hue angle and the a* value in previous work [53]. This suggests that color changes due to the darkening of the tissue during the storage of the produce, usually associated with physiological oxidation and/or microbial decay, were well captured by the consumers. In fact, when plotting all the results from the sensory attributes and instrumental parameters, we observed two distinct groups. The first group included hue angle, green sensory scores, and OVQ, which were highly correlated with each other. The second group included decay, browning score, percent brown color, the a* value, and percent electrolyte leakage (EC), all of which were also highly correlated with each other.

3.6. Comparing Accession Quality

The bi-plot in Figure 9A summarizes the results of comparing accessions using factor analysis. It reveals that Parris Island Cos was the most desirable cultivar, as it was closer to the consumer-preferred attributes. SM13-R2 was also in the same quadrant as Parris Island Cos, near the consumer-preferred attributes (Figure 9A). King Henry, Sun Valley, and Green Forest were located closer to the parameters that the consumers disliked (higher browning, decay). The bi-plot showed that PI 491224 had less browning, but was more prone to decay.

3.7. Sensory Evaluation and Accession Pedigree

Although consumer panels have the advantage of providing valuable information on potential buyer preferences, the use of these untrained panelists can result in large variations in responses, both between and within consumers. In general, trained panelists are preferred when generating intensity scores, given that they are better trained to discriminate during the evaluation of each quality variable [54,55,56,57,58].

Traditionally, sensory researchers have doubted the ability of consumers to effectively discern the intensity of sensory attributes. Heymann and Lawless [20] have argued that human beings are very poor at defining absolute measurements, even though they are good at relative measurements. Thus, they hypothesized that consumers can reliably assess well only what they like or dislike. Therefore, they concluded that asking consumer panelists (without any training) to generate intensity scores was akin to obtaining measurements from instruments without calibration. Stone and Sidel [58] indicated that, in general, there was no assurance that the responses of consumers were reliable or valid. One important issue also discussed was that the consumer was more prone to a cognitive bias known as the halo effect, causing the evaluation of one attribute to influence the evaluation of another attribute [20].

However, some researchers challenged the traditional notion of consumer panels. Moskowitz [21] claimed that untrained consumers were easily able to rate the intensity of the sensory attributes (flavor) of products (sausage) and provide results that were similar to those of trained panels. In the same way, sensory studies with perfumes showed that both consumer and trained panels performed similarly in terms of discriminatory ability and reproducibility [22]. Wheeler and Koohmaraie [59] suggested in their study of beef that consumer panels could accurately and repeatedly detect differences in beef tenderness.

The statistical analysis and the graphing of the data in our study showed that consumers had essential differentiating abilities, which many traditional researchers have questioned. The sensory scores by consumers and the instrumental data were able to group accessions according to their postharvest quality, closely matching the known pedigree relationships. As seen in Figure 9, the bi-plots by factor analysis of the data generated by the consumer panel and the instrumental analysis in this study (Figure 9A) matched the map of pedigree lines of each accession (Figure 9B). The disputable common argument that a consumer does not have a keen sense of detecting differences appears weak when presented with the “collective intelligence” of consumers, especially with a well-known and desirable product (romaine lettuce) and a sufficiently large sample/panelist group.

PI 491224 is a primitive romaine-type lettuce that has been shown in past studies to be highly perishable when processed for fresh-cut salad [2,3]; the bi-plot of this study also indicated that PI 491224 had a high amount of decay. SM13-R2 originates from a cross between PI 491224 and Parris Island Cos. This breeding line was developed to combine the desirable production and postharvest qualities from Parris Island Cos (particularly good yield and a slower rate of decay) with resistance to lettuce dieback introgressed from PI 491224 [17]. Such a combination of traits was achieved through extensive phenotyping and marker-assisted selection [60]. The loading map of the sensory data also shows that Sun Valley and Green Forest are closely related phenotypically, which agrees with their pedigree. Sun Valley [61] was developed by Central Valley Seeds, Inc. (Salinas, CA, USA) from a cross between the cvs. Green Forest (female parent) and Caesar (male parent). The F₁ seeds were backcrossed twice to the cv. Green Forest [62] as a recurrent parent; thus, there is a high genetic similarity between the cvs. Green Forest and Sun Valley. King Henry is positioned farther away from the other accessions in the sensory loading map (Figure 9A). This cv. [63] was developed by Progeny Advanced Genetics, Inc. (Yuma, AZ, USA) as a single-seed selection from cv. Tall Guzmaine; this cultivar is not closely related [63] to any other accessions tested in the present study.

This study did not have the intention of determining the superiority of consumer panels over trained panels or vice versa. Each of the panels has its advantages and disadvantages and can be complementary to each other. This study did demonstrate that, in the case of a well-known salad vegetable, the average consumer has an inherent differentiating ability that is good enough to group the accessions based on their postharvest quality, which, in this study, also relates to their pedigree. The OVQ scores indicated that both the cultivars King Henry and PI 491224 generally had less desirable postharvest quality than the other tested accessions. However, it needs to be pointed out that the visual evaluation of individual traits determined a substantial difference in the poor postharvest quality of these genotypes: while the cv. King Henry was more likely to display (dry) browning of the tissue, the poor quality of PI 491224 was due to rapid tissue decay. A more sophisticated statistical analysis (ongoing study) could explain why or how untrained consumers were able to detect the relationship among the accessions.

It is important to emphasize that the close resemblance of the results obtained with untrained consumers, in comparison with those of trained panelists and instrumental measurements, may or may not be obtained with non-familiar flavors of fresh produce. Life experiences could influence sensory perception, especially with untrained panelists, which is probably enhanced when tasting the unfamiliar flavors of fresh produce. The latter issue has not been addressed formally in the literature, which warrants further research. In fact, a recent thorough study using black tea with added flavors as a model has already shed light on this topic, as the authors concluded that product involvement (relevance or familiarity with the food product) and food neophobia (reluctance to try new foods) in sensory panelists influence the degree to which they can discern sensory traits [64].

Our study highlighted the importance of not overlooking the potential ability of the consumers to assess fresh produce and identified what volunteered consumers can achieve. This approach can save time and money in the sensory community. This becomes more relevant in the current society, in which a higher proportion of the population is making decisions about food acquisition and preparation in an increasingly consumer-oriented market, showing that quality demands are more specific and rapidly changing. Modern technology, such as high-quality cameras in smart phones, is opening the possibility for consumers to participate remotely. A recent study revealed that by examining a digital image of a fruit, consumers were able to qualitatively determine the actual, real physical appearance of the same fruit, with only a small quantitative score difference between perception through the image vs. the real produce [65]. These types of results will likely unfold a new, efficient way to conduct sensory evaluation with ad hoc consumer panels.

4. Conclusions

One of the major challenges with RTE salads containing romaine lettuce is the susceptibility of the latter to injury-induced discoloration. In response to this, multiple accessions are being evaluated, and new lettuce cultivars, specifically suited for fresh-cut processing, are being developed. To potentially gain time and save resources, this study revisited the value of using untrained panelists for assessing the visual quality of several accessions of romaine lettuce. The ad hoc panels were able to distinguish differences among accessions, even grouping them by hard-to-distinguish traits such as browning and decaying, in a similar way to empirical measurements with instrumentation and a single trial with trained panelists. In that regard, with the exception of King Henry and PI 491224, all accessions, including the novel SM13-R2 breeding line, performed relatively well as fresh-cut raw materials. Two factors, an adequate consumer sample size and embedded knowledge of the targeted product, were considered key for efficient quality evaluation by volunteers. Our results encourage further studies with ad hoc consumer panels to assess other quality traits of products that are familiar to the general public, in both research and commercial settings.