Citizen Science in Vegetable Garden Cultivar Evaluation in Tennessee

Abstract

Edible food production is a growing area of horticultural interest that can engage multiple generations of rural to urban residents with varying levels of experience. Residential or community garden food production can provide many benefits, including the production of healthy produce, establishment of community or social connections, and increased physical activity. Regardless of experience, food gardeners are interested in growing crops and cultivars well-suited to their region and which provide both productivity and crop quality. This means that cultivar selection is a common question for gardeners. However, formal cultivar evaluation is relatively rare in the non-commercial food production sector due to the number of cultivars, the challenges of replicated trial management, and the scarcity of public researchers focused on consumer horticulture. This limits the information available to support new gardeners, which lowers the chances of overall success including high-quality harvests. Such crop and variety selection questions are common for Extension personnel in the United States as well as many others who work with gardeners. Even with this high level of interest, funding for consumer garden trials is limited and the cost of replicated trials across various geographical sites is high. To fill this gap in research and address the need for high-quality data to support education, University of Tennessee Extension and research faculty have developed a citizen science approach called the Home Garden Variety Trial (HGVT) program. The HGVT is a collaborative effort between Extension and research faculty and educators, who select trials, provide seeds, and compile data, and citizen scientists around the state, who conduct the trials using their usual gardening practices in their own home or community gardens. Beginning in 2017, the collaborators have conducted five years of research involving over 450 individual gardeners in more than half of the counties in Tennessee. The HGVT is a novel and effective tool to introduce gardeners to new crops and cultivars while providing previously unavailable data to researchers. Together, researchers and home gardeners collect and compile data that supports residential and community food production success while engaging new and experienced gardeners in participatory science research.

1. Introduction

Interest in residential food production has risen in recent years, with an estimated 35% of U.S. households gardening for food [1]. Increased interest from younger audiences and new gardeners has contributed to the popularity of vegetable and fruit production in recent years [1,2]. This influx of new gardeners is an exciting opportunity for consumer horticulture retailers, researchers, and educators. However, facilitating gardeners’ success in edible crop production has created an increased demand for information about the selection of crops and cultivars and their production in residential scale growing systems, such as raised beds and containers.

In the United States, land-grant Extension systems serve the vital function of delivering research-based information to the public and meeting the needs of existing and new gardeners. Even with the crowded digital environment and competition within the information sector, Extension is seen as a trusted source of information by horticulture consumers [3]. However, research-based information is lacking for many areas of home food production, including cultivar selection for residential-scale vegetable growing [4]. The number of garden crops and cultivars along with the expense of vegetable crop trials and the small number of consumer horticulture vegetable researchers has resulted in a scarcity of state- or regional-specific vegetable variety performance and selection information. Regionally specific cultivar data are valuable because they can support new gardeners during this time of high interest as well as assist experienced gardeners in utilizing newer disease-resistant cultivars to enhance sustainable management practices and improve crop yield and quality.

It is not just cultivar trials that are needed to support new horticulture practitioners. Research within the area of consumer horticulture overall is challenged by a current personnel focus on consumer education and a lack of funding targeting foundational research in the areas of plant function, performance, and human benefits [5]. A unique solution to funding and personnel challenges in the consumer horticulture research area, and specifically for plant selection and performance evaluation, is the opportunity to engage with stakeholders through citizen science programs. Citizen science involves public participation in research projects where engaged and interested volunteers with a range of expertise take part in data collection or analysis [6,7]. Citizen science has been documented to help study ecosystems in the context of environmental monitoring [6], conservation [8], urban farming [9], and agronomic crop variety selection [10]. In the broader context, citizen science can also contribute to the larger goals of scientific trust and literacy and supporting the resilience of our food systems [11,12]. It has also been studied in the context of behavior change and impact on participants [13], so it can benefit both scientific and individual knowledge.

To address these needs in applied horticulture research, the Home Garden Variety Trial (HGVT) program was begun at the University of Tennessee in 2017. By involving Tennessee home gardeners in a citizen science approach to variety evaluation, variety performance has been collected across many locations. Important data has been gathered on crucial attributes for home gardeners, such as germination, plant health, yield, attractiveness, and flavor. The goal of the Tennessee HGVT program is to answer the question, “which varieties are best suited to Tennessee gardens?” Growing a well-adapted variety can result in higher yields, more flavorful produce, and more success in pest and disease control. In addition to these applied research and outreach objectives, the home garden variety trial program also enables participatory science across experience levels and ages. This engagement builds new avenues for connecting the land-grant university Extension system and stakeholders, while building understanding and valuation of applied horticulture research. This opportunity for Extension and research faculty and educators to collaborate with Tennessee residents and gardeners has been valuable in supporting several key objectives:

• Collection of variety performance data that is analyzed and compiled to identify varieties with superior performance in Tennessee;
• Introduction of newer disease resistant varieties to gardeners, which can lead to more productive gardens;
• Introduction of productive garden crops and cultivars to novice gardeners and youth;
• Engagement of gardeners in the scientific process through citizen science project participation.

This paper will provide an overview of the University of Tennessee HGVT program from its inception in 2017 through recent pandemic challenges and opportunities. Time at home and food system interruption by the COVID-19 pandemic have further enhanced recent trends in home food production and demand for high-quality gardening information. The year 2020 brought in an estimated 18.3 million new gardeners, while two-thirds of gardeners tried new crops or techniques [2]. This interest, and thus the need for vegetable gardening information, is expected to remain strong with recent surveys showing that most (86–89%) gardeners plan to maintain or increase their gardening efforts in the future [2,14]. The pandemic increased demand for the applied research developed through this program and enhanced the justification and need for such projects. Therefore, the central goal of this paper is to explain the methods of program design, development, data collection, and analysis in order to enable others to understand the potential for citizen science. This paper will focus on the quality and value of citizen science as an applied research tool that can be used to support related Extension outreach. However, the concentration here will be on the participatory cultivar evaluation research rather than the Extension outreach. This approach is intended to support others in developing and deploying similar programs to enhance consumer horticulture research and support residential and community food production success in their regions.

2. Materials and Methods

2.1. Trial Design

The first step in initiating the HGVT was determining the criteria on which cultivars would be trialed and developing the framework of cultivar comparison. Insight on crop and cultivar selection and areas of evaluation was gathered from university faculty conducting similar efforts (Kalb, pers. comm). It was determined that the HGVT would focus on direct-seeded vegetable crops for logistical simplicity compared to transplanted crops. Trials were designed to be paired comparisons within crops, focused on comparing plant performance and quality of the two cultivars across various factors including germination, health, appearance, yield, and flavor. Each year, trial cultivars were selected to understand the performance of well-known and established cultivars compared to new cultivars, disease-resistant selections, and heirlooms. This combination of common garden varieties trialed against promising newly-released varieties that offered disease resistance, flavor, or novel traits was designed to support data collection and to introduce new cultivars that may offer performance benefits to citizen science participants. Selections offered in the HGVT over the past few years were from sources available to home gardeners, including Southern Exposure Seeds, Burpee Seeds, Stokes Seed, Kitchen Garden Seeds, Seed Savers Exchange, HPS Seeds, Gurneys Seed, Outside Pride Seeds, Seeds N Such, Harris Seeds, Johnny’s Selected Seeds, Park Seeds, Territorial Seeds, and Holmes Seed. Industry breeders were allowed to submit entries for recently released or pre-release material, but project PIs selected most trials from available seed sources.

The main crop categories included beans, cucumbers, corn, muskmelons, watermelons, summer squash, winter squash, pumpkins, okra, leafy greens, as well as herbs and flowers, which were added in 2018 and 2019. These were further divided into sub-categories (e.g., bean—green pole, compact green bush), and two cultivars each year were selected to be compared in the trial. Roughly 20 trial categories were selected each year. Trial options have changed year to year based on total participation in each trial type and participant feedback. Trials with lower participation or higher failure rates were reduced or removed and replaced with more promising crops for Tennessee home growers. As more new gardeners utilized raised beds and containers, smaller space options have been an increasing focus in the trial. Additionally, an online roundtable discussion with all interested HGVT participants occurred at the end of each trial season. Feedback from these discussions has contributed to the addition of several vegetable cultivars as well as flower (2019) and herb (2018) trials to support culinary uses as well as pollinator support.

Crop and cultivar selection was designed to provide replication within years, through the broad range of sites represented by individual citizen scientists and across time, by repeating the trialing of cultivars in multiple trial years. Cultivars that performed well in the trial were repeated in later years to ensure their performance was consistent and as comparisons for new cultivars in the trial. Another trial design element to evaluate the reliability of the data was the inclusion of blind trials in 2019 and 2020. Blind trials consisted of removing the cultivar names of several popular trials in order to determine if recognizability influences participant bias when rating the named trial.

2.2. Participant Recruitment and Engagement

Recruitment has been a key focus in the winter to early spring of each trial year following selection of trial crops and cultivars. In 2017, the trial began with Extension Master Gardener (EMG) volunteers as citizen science participants. This enabled the program to be initially evaluated with feedback from experienced gardeners who were familiar with agricultural research and the Extension system. Participants were recruited from the broad population of Tennessee residents in all subsequent years without any knowledge or experience prerequisites. A variety of publicity methods have been used to reach out to current and potential Extension audiences. County Extension offices were used as a site for both promotion and collection of order forms and fees. Promotional videos and flyers for print and social media outreach have been used.

Seed for the trials was purchased prior to opening the program to trial requests. Trial requests were filled on a first come/first served basis, until seed supplies were depleted or seasonal timing was no longer appropriate for the selected trial. On occasion, for trials that quickly depleted supplies early in the request period, additional seeds were purchased. A mid-March deadline has been used to enable the team to fill and ship orders in time for proper planting. A trial fee of USD 2 per trial has been collected to encourage engagement and commitment. Participants can mail in cash or checks and pay fees by credit card through an online ordering system. Families with children under 18 may request one free youth trial per year to encourage young gardeners to gain experience growing vegetable crops and introduce them to new crops. Youth trials have typically focused on crops with greater appeal to younger gardeners, such as yard long beans or Tennessee dancing gourds. Youth trials contain an additional handout explaining the importance and logistics of the trialing process in simplified language. Youth trials are available irrespective of participation in other fee-based trials. Participants are able to request a maximum of five trials each year (excluding blind and youth trials).

All trial orders have been filled, packed, and shipped by Extension faculty, staff, and students at the University of Tennessee-Knoxville campus. Bulk seed is repackaged in coin envelopes to prevent seed company information from influencing participants. Each participant receives a kit containing seed (enough for a 10-foot row) for two varieties for each trial requested, a blank garden map to plan for planting, information specific to the crop species selected (including planting date, plant spacing, nutrient requirements, and other care instructions.), small garden stakes, and an evaluation sheet for each trial.

Individual participants have grown the trials at their residence, a local community garden, school garden, or Extension Master Gardener demonstration garden. All trials have taken place within the state of Tennessee throughout the spring to fall growing season as appropriate for the crops being trialed. Tennessee provides a range of different growing conditions, so participants are advised to plant their trials according to local climate trends. Results are presented on a statewide basis and are not broken down regionally, as this would reduce sample size below the threshold with which meaningful analysis could be conducted. However, this may be a focus in the future for some of the trials with larger participation rates.

Participants manage their garden trials, evaluate characteristics identified in the evaluation sheet, and return data upon completion of the trial. All files and instructions permanently reside on the HGVT web page (tiny.utk.edu/hgvt, accessed on 18 October 2021). When evaluating trials, participants mark which of the two varieties performed better in germination, plant health, first fruit, yield, appearance, and flavor. They also give a performance rating on a scale of 1 to 10, with number one indicating poor performance and number 10 indicating excellent performance, along with an open-ended question about their perception of the performance and desirability of each cultivar. Participants are asked to select whether they would recommend either variety to other home gardeners (Figure 1). Early in the project, evaluations were completed via paper surveys, while in 2019–2021, there was the additional option of online or paper evaluation. Communication has taken place throughout the growing season, with participants being contacted approximately once a month. These email communications have included educational Extension materials, videos, suggestions for successful home gardening, and information and links to the trial resources.

2.3. Data Collection and Analysis

Data has been collected in the fall as participants either mail back paper copies of their trial evaluation forms or fill out an online survey version of the evaluation form (UTK IRB-17-04195). Evaluation forms consisted of 14 questions per trial, including multiple-choice, open-ended, and rating scales. Over the years, online evaluations have increased in popularity while fewer participants mail back paper forms. Participant responses from the paper forms were entered into spreadsheets, and online results downloaded and moved into spreadsheets for analysis. JMP Pro 16 and SAS 9.4 (Cary, NC, USA) were used to analyze data and determine which cultivar participants preferred for each trial. Data were evaluated using a two-tailed chi-square test for categorical variables (germination, health, first fruit, yield, appearance, flavor) and one-way analysis of variance for continuous variables (performance). Both tests used an alpha level of 0.1 to determine significance.

This data was compiled each year into an Extension bulletin detailing which garden cultivars are recommended by home gardeners across the state and complete results from all crops in the trial. Graphs showed the percentage of evaluations in which a variety was marked as superior for a given trait and reported the mean performance score for each variety. Results provided the percentage of gardeners that recommended each cultivar, lists describing why they recommend it, photos of the cultivars, and select comments from growers describing their experience growing each cultivar (Figure 2). These publications can be found online and are the UT Extension document W657 [15,16,17,18].

Data on requested trials, returned data, failure rates, the participant reported methods, and experience level were compiled from 2017 through 2021. Categorical data analyses were performed using JMP (Pro 6.0; Cary, NC, USA), and analyses of variance were performed using SAS (v. 9.4; Cary, NC, USA). An alpha level of 0.1 was used to determine statistical significance. Mean separation was performed using Fisher’s protected LSD.

3. Results and Discussion

3.1. Participation, Management Methods, and Experience of Citizen Scientists

Between 2017 and 2021, a total of 4408 trials were requested by citizen scientist participants (

Table 1. Trials requested by crop by year from 2017 to 2021. Data are presented in descending order from the highest yearly average number requested. Crops that do not have data for a specific year were not offered as trials in that year.

Crop	Sum across Years (2017–2021)	Yearly Avg.	By Year
			2017	2018	2019	2020	2021
Flower	632	211	-	-	265	96	271
Summer Squash	757	151	60	102	222	110	263
Cucumber	642	128	43	110	168	96	225
Tomato	126	126	-	-	-	-	126
Bean	612	122	60	128	138	101	185
Blind	172	86	-	-	107	65	-
Herbs	273	68	-	52	61	78	82
Youth	256	64	-	40	76	57	83
Winter Squash	205	51	-	59	36	24	86
Melon	204	51	11	81	70	42	-
Okra	202	51	-	40	56	33	73
Pumpkin	125	42	31	63	31	-	-
Leafy Greens	82	41	-	-	-	17	65
Corn	91	30	13	43	35	-	-
Carrot	29	29	29	-	-	-	-
Sum	4408	1252	247	718	1265	719	1459

). The most popular trials, averaging over 100 requests annually, were flowers (211), summer squash (151), tomato (126), and bean (122). The least popular trials, averaging less than 50 requests annually, were carrot (29), corn (3), leafy greens (41), and pumpkin (42). A total of 478 unique individuals have participated in the trial program. Of these, 124 participated in two or more years, while 354 participated in a single year. Participation was capped at 45 individuals in the pilot year (2017). Since 2017 was a pilot year designed to evaluate the program methods, experiences and general program evaluation that helped guide the project are included. However, individual cultivar evaluation data is only presented from the full project years of 2018–2020. The number of participants grew to 135 in 2018 and 171 in 2019. Participation dipped slightly in 2020 due to a recruitment window reduced by COVID-19 restrictions (139 participants) but recovered and was at its highest level in 2021 (186 participants).

On average, 41% of participants returned trial data at the end of the season (

Table 2. Data return rate by crop by year from 2017 to 2020. Data are presented in descending order from the highest yearly average data return rate. Crops that do not have data for a specific year were not offered as trials in that year. Data for youth trials are not presented as these trials were primarily for outreach purposes, and data was not requested in return. Tomato data is also not presented as these trials were first offered in 2021, and data collection and data are incomplete.

Crop	Yearly Avg.	By Year
		2017	2018	2019	2020
Carrot	55%	55%	-	-	-
Corn	54%	69%	58%	34%	-
Leafy Greens	53%	-	-	-	53%
Okra	46%	-	65%	38%	36%
Melon	46%	64%	52%	36%	31%
Cucumber	43%	67%	45%	27%	32%
Pumpkin	43%	52%	44%	32%	-
Bean	42%	53%	47%	34%	33%
Summer Squash	42%	57%	54%	25%	31%
Herbs	39%	-	46%	36%	35%
Winter Squash	37%	-	46%	39%	25%
Flower	37%	-	-	32%	41%
Blind	32%	-	-	31%	34%
Average	41%	60%	46%	30%	32%

). Return rate ranged from 32% to 55%, depending on the crop. Excluding 2017, which was the program evaluation year, first-time participants tended to have lower average return (33%) rates compared to those who had participated in the program for more than one year (53%) (

Table 3. Trials requested and data return rate by previous trial experience of participant (new vs. experienced).

Trial Year	Trials Requested					Data Return Rate (%)
	Sum			Proportion (%)
	Total	New	Exp.	New	Exp.	Avg.	New	Exp.
2017	247	247	0	100%	0%	58%	58%
2018	718	621	97	76%	24%	47%	45%	82%
2019	1265	921	344	57%	43%	29%	27%	39%
2020	719	390	329	37%	63%	31%	26%	39%
2021	1459	783	676	37%	63%	-	-	-

). Average return rates were lower in years three and four of the program compared to the preceding years. This is likely due to advertising to a broader demographic, resulting in participants representing a wider range of gardening experience.

Participation numbers and return rates illustrate both the opportunity and the challenge of using a citizen science project to educate and engage participants and efficiently collect trial data and evaluations. Following the 2017 program evaluation year, when participation was limited to EMG volunteers, no set requirements have been placed upon participants. Those interested have been able to request on a first-come, first-served basis, and the only limits have been seed quantities and the number of orders that could be filled and shipped in the early spring to enable arrival before ideal planting dates. This means that as the program has become more well-known and interest in food gardening has increased, new participants and those with less experience have been able to participate.

While the proportion of first-time participants has varied throughout the trial years, the return rate for new participants has lagged those of returning participants (Table 3). This may be due to a lack of complete understanding of the project’s scientific goals or lack of commitment by new gardeners or those less connected to land-grant universities. This fits with similar citizen science programs that have found participant knowledge has increased through citizen science engagement, but participation alone is likely insufficient to provide a foundation for understanding of scientific research and efforts overall [13]. In the 2019, 2020, and 2021 trial years, there has been an increased focus on communicating with participants and providing educational resources through email, video, and social media engagement throughout the growing season. These efforts were designed to support knowledge gain while also explaining the value of the participant role and data in the project. More concerted efforts and formalized processes are needed to introduce new gardeners to both key management practices in the garden and the importance of their follow-through in returning data on both crop successes and failures.

Beginning in 2019, participants were asked to report crop failure that prevented the complete evaluation of a trial. This was done to increase data return rates as it was anticipated that participants might not consider data return relevant to the trial program if a crop had failed. While this did not provide the expected increase in data return rates, it did provide information on which crops were more challenging for citizen scientists and why. Reported crop failure rates varied widely by crop (

Table 4. Crop failure rate (self-reported) by crop by year from 2019 to 2020 and by reason for crop failure (self-reported). Data are presented in descending order from the highest yearly average data failure rate. Crops that do not have data for a specific year were not offered as trials in that year. Data are not presented for youth trials, as these were for outreach purposes, and data was not collected for trials offered only in years before 2019.

Crop	Yearly Avg.	By Year		By Reason for Crop Failure
		2019	2020	Germ.	Human	Insect/Disease	Wildlife	Unknown
Leafy Greens	44%	-	44%	11%	0%	0%	25%	25%
Pumpkin	40%	40%	-	0%	0%	0%	75%	0%
Corn	33%	33%	-	0%	17%	0%	25%	25%
Flower	29%	24%	33%	9%	12%	3%	9%	53%
Melon	28%	32%	23%	13%	4%	8%	0%	73%
Okra	25%	0%	50%	3%	5%	0%	17%	100%
Blind	16%	9%	23%	2%	6%	14%	13%	25%
Herbs	15%	5%	26%	10%	14%	0%	0%	0%
Cucumber	15%	11%	19%	5%	4%	0%	27%	36%
Summer Squash	14%	16%	12%	3%	4%	9%	15%	46%
Bean	14%	15%	12%	1%	2%	6%	27%	73%
Winter Squash	4%	7%	0%	0%	0%	17%	0%	0%
Average	23%	18%	24%	5%	7%	5%	16%	44%

). Leafy greens had the highest reported failure (44%), while winter squash had the lowest (2%). While many crop failures were equally spread across germination, human error, insect/disease, and wildlife issues, certain crops exhibited higher rates within specific categories. Pumpkins and corn appeared to be more prone to wildlife issues, corn and flowers to human error, flowers, herbs, and melons to germination issues, and squash and pumpkin to insects/diseases.

These rates and reasons of failure have been used to guide crop and cultivar selection in subsequent years. Over time, pumpkin and sweet corn trials have been replaced by those with higher levels of success and participation. Conversely, flower and herb trials have experienced high enough participation rates to continue to be included despite germination issues. However, as discussed above, more time and communication efforts have been spent in recent years to educate and support gardeners in starting transplants and seeding practices and timing to support better success in trial establishment. It may also be helpful in the future to assess citizen science participant motivation and focus recruitment and engagement efforts based on key motivations, as has been investigated in other citizen science projects [19]. While this project has been designed primarily based on techniques to enhance data gathering and quality, it may be valuable in the future to focus more on the social dimensions.

As the HGVT program grew and expanded, new questions were added to the evaluation to better understand participant techniques and gardening experience and practices. Site and nutrition management data collection began in 2019. These questions included fertility type, site type, and years of gardening experience. Most participants grew their trials in-ground (58%), followed by raised beds (31%) (

Table 5. Crop failure rate (N = no, Y = yes) by trial site as self-reported by trial participants in 2019 and 2020. Categorical analysis of crop failure by site was not significant at an alpha level of 0.1 (p = 0.8090).

Site	Crop Failure		Trials		Failure Rate
	N	Y	Sum	Proportion
Container	36	12	48	9%	25%
In-Ground	253	60	313	58%	19%
Raised Bed	148	34	182	34%	19%
Total	437	106	543	100%	20%

). Only 9% of trials were grown in containers. Categorical analysis of crop failure rates by site was not significant (p = 0.8090), with sites averaging a 20% failure rate across all the growing systems. The fact that the failure rate is not significantly different between garden sites supports the inclusion of a range of sites in the trial. The practice of sharing data and cultivar suggestions with gardeners across growing systems is also supported by the results of Crane et al. [4], who found that in-ground performance of cucumber cultivars in trials was similar enough to be useful in suggesting cultivars to gardeners who grow in containers.

Fertility methods were close to equally distributed among compost, compost + organic, synthetic, and none, averaging 22% to 29% of trials (

Table 6. Crop failure rate (N = no, Y = yes) by fertility regime as self-reported by trial participants in 2019 and 2020. Categorical analysis of crop failure by site was not significant at an alpha level of 0.1 (p = 0.1498).

Fertility Method	Crop Failure		Trials		Failure Rate
	N	Y	Sum	Proportion
Compost	123	34	157	29%	22%
Compost + Organic	103	24	127	24%	19%
Compost + Synthetic	17	2	19	4%	11%
None	87	29	116	22%	25%
Synthetic	103	16	119	22%	13%
Total	433	105	538	100%	20%

). Compost + synthetic was a much smaller proportion of trials (4%). Crop failure rates by fertility were not significant (p = 0.1498). While failure rates for compost + synthetic or synthetic averaged 11 to 13%, failure rates were nearly double (19% to 25%) for the remaining categories, which did not include synthetic fertilizer. Questions targeted at better understanding participant fertility use could provide greater insight into impact on crop failure rate as well as indicate if education materials are needed on how to optimally combine compost use with fertilizer to best meet crop nutrient needs.

The majority of participants had 20 to 30 years of gardening experience (27%) (

Table 7. Crop failure rate (N = no, Y = yes) by participant’s years of gardening experience as self-reported by trial participants in 2019 and 2020. Categorical analysis of crop failure by years of gardening experience was significant at an alpha level of 0.1 (p = 0.0795).

Years Gardening Experience	Crop Failure		Trials		Failure Rate
	N	Y	Sum	Proportion
0–5	11	9	20	8%	45%
5–10	22	9	31	13%	29%
10–20	27	4	31	13%	13%
20–30	53	10	63	27%	16%
30–40	13	2	15	6%	13%
40–50	34	13	47	20%	28%
50+	25	5	30	13%	17%
Total	185	52	237	100%	22%

). Only 8% of participants had less than five years of gardening experience. Crop failure by years of gardening experience was significant (p = 0.0795). Trials executed by participants with less than five years of gardening experience had a failure rate of 45%. This was three times higher than trials executed by those with 10 to 40 or greater than 50 years of gardening experience. While not surprising, the higher level of failure among participants with less gardening experience indicates that more education may be warranted in order to build success among participants and produce a more extensive data set. In future trial years, first-time participants may need to be engaged earlier in the trial and more directly, potentially with an introductory visit or training to best support their first-year experience.

3.2. Crop and Cultivar Evaluations

In the past four years (2017 to 2020), 158 cultivars have been evaluated through the Tennessee Home Garden Variety Trial program. Performance ratings differed (p < 0.0001) among cultivars.

Table 8. Mean performance rating by variety for 2018–2020. Performance was rated on a scale of 1 to 10, with 1 indicating poor performance and 10 indicating excellent performance. The number of participants who returned usable data for each cultivar is reported as “n”.

Cultivar	Crop	n	Mean	SE	Recommend (%)
Ambrosia	Melon	8	9.5	1.0	100%
Desert	Summer Squash	8	9.4	0.9	88%
Golden Star	Summer Squash	12	9.3	0.8	100%
Maxibel	Bean	9	9.0	0.9	100%
Sweetie Pie	Melon	6	8.8	1.0	100%
Easy Pick Gold II	Summer Squash	12	8.6	0.8	100%
Rattlesnake	Bean	14	8.6	0.7	100%
Provider	Bean	11	8.6	0.8	100%
Grandprize	Summer Squash	12	8.5	0.8	92%
Marketmore 76	Cucumber	23	8.5	0.5	100%
Amethyst	Bean	13	8.4	0.7	92%
Everleaf	Basil	21	8.4	0.6	95%
Giesha Garlic	Chives	17	8.4	0.8	94%
Jambalaya	Okra	41	8.3	0.5	91%
Obsession	Basil	21	8.3	0.6	100%
Candle Fire	Okra	26	8.3	0.5	80%
Metro	Winter Squash	11	8.3	0.8	91%
Golden Glory	Summer Squash	13	8.2	0.7	85%
Tankuro	Bean	9	8.2	1.0	100%
Sugar Cube	Melon	25	8.1	0.5	82%
Aroma II	Basil	22	8.0	0.5	86%
Blaze	Pumpkin	11	8.0	0.9	86%
Bristol	Cucumber	23	8.0	0.5	90%
Diva	Cucumber	14	8.0	0.7	75%
Green Light	Cucumber	12	8.0	0.8	91%
Knucklehead	Pumpkin	6	8.0	1.1	100%
Volunteer	Bean	4	8.0	1.4	67%
Tempest	Summer Squash	32	8.0	0.4	93%
Double Click Pink	Flower	34	7.9	0.5	91%
Goldmine	Summer Squash	13	7.9	0.7	85%
Cool Customer	Cucumber	8	7.8	1.0	67%
Flash	Cool Season	13	7.8	1.0	92%
Clemson Spineless	Okra	45	7.8	0.4	90%
Success	Summer Squash	12	7.8	0.8	83%
Blue Lake Bush 274	Blind	24	7.7	0.5	79%
Spacemaster	Cucumber	32	7.7	0.5	90%
Champion	Cool Season	13	7.7	1.0	92%
Sensation Mix	Flower	34	7.7	0.5	88%
Bossa Nova	Summer Squash	11	7.6	0.8	80%
Dolce Fresca	Basil	46	7.6	0.4	76%
Kentucky Wonder	Bean	9	7.6	0.9	100%
Zephyr	Summer Squash	13	7.5	0.8	100%
Sunburst	Summer Squash	12	7.5	0.8	80%
Brulee	Winter Squash	11	7.5	0.8	91%
G-Star	Summer Squash	9	7.5	0.8	89%
Classic Magic	Flower	22	7.4	0.8	100%
General Lee	Cucumber	24	7.4	0.5	74%
Moonshine	Pumpkin	10	7.4	0.9	100%
Flamino	Summer Squash	11	7.4	0.8	60%
Saladmore	Cucumber	38	7.4	0.4	70%
Temptress	Corn	12	7.4	0.8	75%
Calima	Bean	17	7.3	0.6	80%
State	Bean	4	7.3	1.4	33%
Candy Mix	Flower	33	7.3	0.5	76%
Bennings Green Tint	Summer Squash	9	7.3	0.8	89%
Raven	Summer Squash	21	7.2	0.5	70%
Tavera	Blind	22	7.2	0.5	67%
Sweetness	Corn	12	7.1	0.9	75%
Annihilator	Bean	11	7.1	0.8	73%
Janue et Verte	Summer Squash	12	7.1	0.8	80%
Jade II	Bean	49	7.1	0.4	75%
Tigress	Summer Squash	21	7.1	0.5	55%
Hill Country Red	Okra	22	7.0	0.5	67%
Honey Bear	Winter Squash	15	7.0	0.7	55%
Mambo	Melon	6	7.0	1.0	80%
Mascotte	Bean	17	7.0	0.6	80%
Easy Pick Green	Summer Squash	12	7.0	0.9	75%
Cinnamon Girl	Pumpkin	25	6.9	0.7	85%
Choice Mix	Flower	22	6.9	0.8	90%
Olympian	Cucumber	11	6.9	0.8	50%
Roma II	Bean	9	6.9	0.8	56%
Sunlight	Pumpkin	10	6.9	0.9	86%
Seychelles	Bean	9	6.8	1.0	43%
Crockett	Bean	25	6.8	0.5	55%
Amiga	Cucumber	14	6.8	0.7	25%
Itachi	Cucumber	9	6.8	0.8	67%
Queen Lime Red	Flower	33	6.7	0.5	72%
Muncher	Cucumber	12	6.7	0.8	64%
Chiba Green	Bean	9	6.7	1.0	88%
Desi	Summer Squash	6	6.7	1.0	67%
Patio Snacker	Cucumber	23	6.7	0.6	74%
Kentucky Blue	Bean	6	6.7	1.0	83%
Honeynut	Winter Squash	14	6.6	0.7	75%
Fordhook 242	Bean	9	6.6	0.8	57%
Pantheon	Summer Squash	12	6.6	0.9	75%
Slik Pik YS 26	Summer Squash	19	6.6	0.6	53%
Trailing Mix	Flower	27	6.6	0.6	61%
American Dream	Corn	12	6.5	0.8	40%
Savor	Melon	8	6.5	1.2	86%
Starlight	Melon	4	6.5	1.2	25%
Turkey Craw	Bean	12	6.4	0.9	86%
Green Machine	Summer Squash	8	6.4	0.9	75%
Miniature Popcorn	Corn	11	6.4	0.8	70%
Eleonora	Basil	24	6.4	0.6	63%
Starry Night	Winter Squash	15	6.4	0.7	73%
Jumbo	Bean	9	6.3	0.8	67%
Butterscotch	Winter Squash	31	6.2	0.5	61%
Lemon	Summer Squash	6	6.2	1.0	67%
Bush Crop	Cucumber	17	6.1	0.6	87%
Polyvert	Chives	17	6.1	0.8	88%
Winter Luxury	Blind	15	6.0	1.1	85%
Alaska Mix	Flower	27	5.9	0.6	56%
Butterbush	Winter Squash	17	5.9	0.6	38%
Bush Delicata	Winter Squash	10	5.9	0.8	50%
Sugar Pot	Melon	19	5.9	0.8	50%
Baby Bear	Pumpkin	11	5.9	0.9	29%
H-19 Little Leaf	Cucumber	8	5.8	1.0	33%
Baby Doll	Melon	16	5.8	0.7	83%
DMR 401	Cucumber	13	5.8	0.7	46%
White Nite	Flower	27	5.8	0.6	60%
New Orchid	Melon	4	5.8	1.2	50%
Minnesota Midget	Melon	17	5.7	0.6	13%
Sunshine	Winter Squash	10	5.7	0.8	50%
Gumdrop	Pumpkin	6	5.6	1.1	60%
Honey n Pearl	Corn	12	5.6	0.8	40%
Glass Gem	Corn	11	5.6	0.8	50%
Sonja	Flower	27	5.6	0.6	71%
Mini Love	Melon	20	5.5	0.8	25%
Speckled Dixie	Bean	9	5.5	0.8	57%
Suyo Long	Cucumber	9	5.5	0.8	78%
Trifecta	Melon	8	5.5	1.0	17%
Hill Family Greasy	Bean	12	5.4	0.9	57%
Lazy Housewife	Bean	14	5.4	0.7	64%
Sugar Baby Bush	Melon	16	5.1	0.8	55%
Malibu	Bean	6	4.8	1.0	67%
Pepitas	Pumpkin	10	4.1	0.9	33%

provides a summary of cultivar trials as well as statistical analysis for years 2018–2020. Individual cultivar evaluations from program evaluation year 2017 have not been included in the tables below. Performance ratings were on a 1 to 10 scale in all years of the trial, with 1 indicating poor performance and 10 indicating excellent performance. The most highly ranked cultivars across all years, with ratings of 9.0 or greater, were ‘Ambrosia’ (melon), ‘Desert’ (summer squash), and ‘Golden Star’ (summer squash), and ‘Maxibel’ (bean). The least popular cultivars, with ratings of 5.1 or less, were ‘Sugar Baby’ (watermelon), ‘Malibu’ (bean), and ‘Pepitas’ (pumpkin).

The proportion of respondents who stated they would recommend a cultivar to other home gardeners is also presented in Table 8. While this number tended to follow the same trend as performance ratings, a few high-performance scores were matched with a low recommendation and vice versa. This is very interesting and the reason for these dual questions in the evaluation. It is a goal of this evaluation to separate personal preferences for cultivars with overall performance. While it is likely that overall rating and recommendation for others will be similar, it is interesting that some cultivars were rated as better for others than the participant rated their own approval and satisfaction.

Participants were asked to specify qualities that influenced that recommendation in order to provide more insight into why participants preferred or did not prefer cultivars for themselves or others. Reasons listed by the highest proportion of respondents included yield (58%), flavor (55%), appearance (53%), health (51%), and size (42%). Less than 5% of respondents noted other reasons like texture, shape, early maturity, germination, shelf life, canning/cooking, production season length, and vigor. While non-commercial food producers use a range of factors in evaluating cultivars, it is interesting to see that appearance, health, yield, and flavor were all quite similar in the percentage of influence on ratings. These percentages illustrate the importance of trials that evaluate attributes other than yield, the most common data gathered in traditional cultivar trials. These reasons also line up with national surveys of gardeners who often rank taste and quality as high or higher in importance than productivity or money savings related to crop yield [1]. In many ways, these data on appearance, health, and flavor, as rated by real home gardeners, would not be accessible using traditional trial techniques.

Performance ratings by crop also exhibited significant differences (p < 0.0001) (

Table 9. Mean performance rating by crop for 2018–2020. Performance was rated on a scale of 1 to 10, with 1 indicating poor performance and 10 indicating excellent performance. The number of participants who returned usable data for each cultivar is reported as “n”. Mean separation was performed using Fisher’s Protected LSD with an alpha level of p < 0.1.

Crop	Performance Rating (1–10)				Recommend (%)
	n	Mean	Standard Error	Mean Separation
Okra	134	7.9	0.2	A	84%
Leafy Greens	26	7.8	0.7	ABCDEF	92%
Herb	168	7.6	0.2	AB	85%
Summer Squash	298	7.5	0.2	ABC	79%
Cucumber	290	7.2	0.2	BCD	74%
Bean	342	7.1	0.1	CDE	75%
Flower	286	6.9	0.2	DEF	79%
Melon	158	6.6	0.2	EF	58%
Pumpkin	105	6.6	0.3	DEF	77%
Winter Squash	134	6.5	0.2	F	63%
Corn	71	6.4	0.3	F	57%

). Okra, herbs, and summer squash had higher performance ratings (7.5 to 7.9) than flowers, pumpkin, melons, winter squash, and corn (6.4 to 6.9). Cucumber (7.2) and bean (7.1) fell in the middle of these two groups (7.2). The proportion of participants that would recommend cultivars within these groups closely followed rankings of performance scores. Some crops that did not perform well in the program evaluation year were not trialed again. For example, carrots were trialed only in 2017, and there were timing challenges in getting seeds to participants in the spring early enough for good production. As a general trend, longer season and more space consumptive crops, such as pumpkin, winter squash, and corn, tended towards lower ratings overall.

Of the cultivars evaluated, 11 were evaluated over two or more years in 2018–2020 (

Table 10. Mean performance rating by year within cultivars evaluated over multiple years (2018–2020 trials). Performance was rated on a scale of 1 to 10, with 1 indicating poor performance and 10 indicating excellent performance. The number of participants who returned usable data for each cultivar is reported as “n”. Mean separation was performed using Fisher’s Protected LSD with an alpha level of p < 0.1.

Cultivar	Crop	Year	n	Est.	SE	p Value
Butterscotch	Winter Squash	2018	17	6.8	0.8	N.S.
		2019	14	5.8	0.6
Cinnamon Girl	Pumpkin	2018	10	8.2	1.0	N.S.
		2020	15	6.0	0.8
Clemson Spineless	Okra	2018	26	8.7	0.8	N.S.
		2020	19	7.6	0.4
Dolce Fresca	Basil	2018	24	7.9	0.6	N.S.
		2019	22	7.2	0.6
General Lee	Cucumber	2018	11	7.5	0.8	N.S.
		2020	13	7.4	0.8
Jade II	Bean	2018	25	7.2	0.6	N.S.
		2019	24	7.0	0.5
Jambalaya	Okra	2019	22	8.7	0.7	N.S.
		2020	19	8.2	0.4
Saladmore	Cucumber	2018	23	6.8	0.8	N.S.
		2020	15	6.5	0.7
Spacemaster	Cucumber	2019	17	7.7	0.5	N.S.
		2020	15	7.6	0.6
Sugar Cube	Melon	2018	17	8.8	0.9	N.S.
		2020	8	8.0	0.5
Tempest	Summer Squash	2018	19	8.0	0.5	N.S.
		2019	13	7.9	0.4

). Performance ratings between years within these cultivars were very similar with none of the cultivar rating showing significantly different results in different trial years. Differences between years ranged from 0.1 to 2.2 points, averaging 0.7 points. The strong internal consistency of the data for cultivars across years suggests reliable evaluations within our trial participants even with fluctuations in the individuals participating and the number of evaluations completed in a given trial.

Six cultivars were evaluated in both named and blind trials to determine if cultivar name biased participants evaluations (

Table 11. Mean performance rating by trial (blind, named) within the cultivar in 2019 and 2020 trials. In named trials, seeds were labeled with cultivar names, while blind trials were assigned a code Performance was rated on a scale of 1 to 10, with 1 indicating poor performance and 10 indicating excellent performance. The number of participants who returned usable data for each cultivar is reported as “n”. Mean separation was performed using Fisher’s Protected LSD with an alpha level of p < 0.05.

Cultivar	Crop	Trial	n	Est.	SE	p Value
Blue Lake Bush	Bean	Blind	7	8.4	0.6	N.S.
		Named	12	7.3	0.5
Bristol	Cucumber	Blind	10	8.0	0.6	N.S.
		Named	10	8.0	0.6
Cinnamon Girl	Pumpkin	Blind	5	8.2	1.0	N.S.
		Named	7	6.0	0.8
Jade II	Bean	Blind	7	7.6	0.9	N.S.
		Named	32	7.0	0.4
Marketmore 76	Cucumber	Blind	11	9.1	0.4	0.0573
		Named	10	7.8	0.5
Tavera	Bean	Blind	12	7.0	0.9	N.S.
		Named	7	7.6	1.1

). Only Marketmore 76 cucumber exhibited a significant difference in performance rating between the named and blind trial. This cultivar ranked higher (9.1) in the blind trial compared with the named trial (7.8). Among most cultivars, the difference in performance rating between blind and named trials did not exceed 1.3 points. The pumpkin ‘Cinnamon Girl’ exhibited the largest difference, with the blind trial rated at an 8.2 for performance and the named trial rated at a 6. These results indicate name recognition bias appears to be a minor factor.

4. Conclusions

This HGVT project has provided valuable data and information to support home gardeners in Tennessee while also illustrating the effectiveness of novel applied research methods. Internal consistency of cultivar evaluations across years and the lack of significant differences in most named and blind cultivar evaluations do not reveal indications of bias or variability across years. These results build confidence in the evaluations as a tool for consumer horticulture research. These analyses support the evaluations and illustrate the unique aspects of cultivar trialing with a participatory approach. The evaluations go beyond yield and traditional field measures to provide feedback on flavor, quality, appearance, health, preservation, and other aspects of value to gardeners that would never be able to be assessed outside of citizen science systems.

As citizen science methodology becomes better understood, there are likely many methods that could be used to enhance such efforts for the benefit of applied scientific research, participant education, and quality of life. Internet applications and social media tools continue to evolve and provide creative methods to add value to scientific efforts for researchers while expanding the role of citizen scientists beyond data collection to a more integrated role as research collaborators [20]. This continued evolution of the role of the consumer in consumer horticulture research holds the potential for beneficial returns to academics, non-profits, horticulture retailers, and consumers.