A Study on Modular Smart Plant Factory Using Morphological Image Processing

Abstract

This paper is a study on a modular smart plant factory integrating intelligent solar module, LED module with high efficiency for plant growth, IoT module control system and image processing technology. The intelligent sun and modules have a corrugated structure, and the angle of the module can be adjusted to obtain a large amount of power generation. It is fully foldable for wider angles during the day and module protection at night. The LED module is designed and manufactured to distribute energy evenly over the entire wavelength range so that high efficiency can be obtained. The control system with IoT convergence technology enables control of all parts related to plant growth such as angle control of solar modules, LED lighting control, temperature/humidity control, and fan control. In particular, the control method is programmed to be controlled by a computer monitoring system and a smartphone app, so there are few places. In addition, this paper developed an image processing algorithm to extract the growth information of lettuce grown in the plant factory. The acquired images were separated into R, G, and B images using Matlab software. The applied algorithms are k-mean and improved morphological image processing. By applying this method, we can determine the area calculation and shipping of lettuce seedlings. As a result of the fusion and application of solar modules, LED modules, and IoT modules, information on plant growth and status was confirmed.

1. Introduction

Since agriculture is affected by climate, land, and environment, production is unstable and cultivation is restricted by season. In recent years, house farming has been so popular that it can be grown all year round, but winter production is low due to lack of sunshine. In addition, the generation of plant-based plant cultivation increases, and the use of pesticides is rapidly increasing, which threatens the health of consumers. The global agrifood-related industry was $5.3 trillion in 2014, three times the $ 1.6 trillion in the automotive industry. Stable food supply is important due to drought and flooding due to climate warming, and demand for high-quality food materials in China and Africa is expected to increase the global agricultural and food industry to US $6.4 trillion by 2020.

The plant factory originated from the production of sprout vegetables using artificial light sources in Denmark in 1957 to supplement the lack of sunshine. Since 1999, the concept of vertical plant farms using high-rise buildings has been developed. In recent years, development and research of enclosed plant plants have been actively conducted for use in extreme environments such as Antarctica, the desert, ocean, and even space [1,2,3,4,5].

The plant factory is a new business area that combines agriculture, technology such as IT, BT, ET, etc. In the past, lighting equipment and mechanical equipment companies participated. Recently, IT companies such as Panasonic, Fujitsu, Sharp, etc. are operating. In this paper, the shortcomings of the existing plant factory system are complemented, the integrated LED generating the optimal wavelength required for plant photosynthesis, the efficiency improvement of more than 10% of the solar power generation system, the solar module with the protection function of the intelligent wrinkle structure, a modular smart plant factory system using an intelligent solar power generation system that has the function of determining the shipping date of plants through image processing was studied.

In this paper, the following three fields were studied and converged. First is the energy supply sector of plant factories. Plant growth in plants is achieved through LEDs. In this paper, we have developed an integrated LED that generates the wavelengths needed for optimal plant photosynthesis growth. In addition, an intelligent photovoltaic power generation system was applied as a power source for the LED. As a result, it was confirmed that there are more than 10% efficiency improvement and protection function of the module.

Second is the plant control system. In-plant plants, the system is lighting, temperature, humidity, and image storage. IoT module control is applied to temperature control using a carbon fiber planar heating element, environmental control such as temperature, humidity, and air quality and image storage. The third is the big data image processing field of plant factories. The images taken at the plant factory are transmitted to the computer via the Internet, and then the image processing program is automatically informed of the shipment of the plant.

The rest of this paper is organized as follows. Section 2: Energy supply of plant factory, Section 3: System control of plant factory, Section 4: Image processing of plant factory, Section 5: System convergence and evaluation, Section 6: Discussion. Finally, conclusions are given in Section 7.

2. The Energy Supply of the Plant Factory

2.1. Intelligent Solar Module

Photovoltaic modules used in conventional flat photovoltaic power generation is a form in which solar cells are arranged in a plane. It is a photovoltaic module that makes a certain number of solar cells to form a corrugated structure in a planar photovoltaic module in a longitudinal direction or a width direction. The solar cell is Sunpower’s Maxeon Gen II Cell and the specifications are shown in Figure 1 [6].

The intelligent photovoltaic module is an automatic folding photovoltaic module frame structure designed to have a corrugated structure to protect climate change and day and night photovoltaic modules [7,8,9,10,11,12]. The brackets were used to fold or unfold the sun and the module, which are the characteristics of this structure. The operation of the solar module determines power generation according to the value of the sunshine sensor attached to the control box. If the solar power generation performance is insufficient, the motor is driven to fold and fold the solar module to protect the sun and the panel from the external environment. Figure 2 shows the design of the intelligent solar module, and Figure 3 shows the solar module in the folded state.

The intelligent photovoltaic module utilizes a microprocessor, stepper motor, and CDS (Cadmiumsulfide) sensor to program the amount of sunshine. Figure 4 shows a block diagram of the hardware configuration of an intelligent solar module.

The photovoltaic module used in the plant factory has a wrinkle structure as shown in Figure 5 and can control the angle of wrinkles according to the amount of sunshine.

2.2. Plant Growth LED Module

Plant photosynthesis grows by synthesizing glucose and oxygen from carbon dioxide and water using sunlight energy (wavelength) in the plant’s chloroplasts. Plant systems grow plants using LED light as an artificial light source instead of the sunlight needed for photosynthesis.

When plants do photosynthesis, they absorb all the wavelengths of visible light and are not used for photosynthesis. Green plants absorb blue and red wavelengths. The wavelengths absorbed by plants are 440 and 655 nm for chlorophyll, and 430 and 670 nm for photosynthesis. Figure 6 is a graph showing the effect of light wavelength on plants.

As artificial light source necessary for photosynthesis, we select a low power single LED chip (Allix Plant Grow Led) mixed with blue light of 455 nm band and red light of 660 nm band, and it is used for leafy vegetables such as lettuce, bok choy, and green vegetables such as cherry tomatoes, paprika, and red pepper. The required LED module was designed. Two types of LED chips were used and the specifications are as in

Table 1. Specification of plant growth LED chip.

AT56SNP-(PGRB)	Size: 5.6(L) X 3.0(W) X 0.9(T) mm Viewing Angle: 2θ1/2 = 120deg Color: Red + BLUE PeakWavelength: Red (645–655 nm), Blue (450–460 nm) Luminous Intensity: typ. 3000 (mcd) Forward Voltage (V): 2.9–3.4 V Forward Current (mA): 65 mA
AT56SNW-(PGWH)	Package: Top View LED in Slug Type Size: 5.6(L) X 3.0(W) X 0.9(T) mm Viewing Angle: 2θ1/2 = 120deg Color: White ( Plant Growth ) CRI: Min. 80 Forward Voltage (V): 2.9–3.4 V Forward Current (mA): 65 mA

[13].

The AT56SNP-(PGRB) LED chip has the maximum wavelength in the red (645–655 nm) and blue (450–460 nm) bands and the AT56SNW-(PGWH) shows the maximum wavelength in the red and blue bands. The plant growth LED 5630 chip is designed for circuit design by selecting the optimum resistance by simulating the output value according to the change of resistance value by connecting 7 and variable resistors with 2.9 V, 65 mA specification.

Figure 7 is a single circuit diagram with a variable resistor and 7 LEDs connected in series.

The circuit design of the whole plant growth LED module was designed and manufactured by connecting six single circuits in Figure 7 in parallel. Figure 8 shows the plant growth LED module applied in this paper.

3. System Control of Plant Factory

Developed Plant Growth IoT Control Monitoring System

The IoT control monitoring system regulates the plant growth environment for plant cultivation [14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29].

The system consists of various control modules, hardware (HW) such as temperature and humidity sensors, and a software program (SW) that can be controlled. In addition, for more efficient production and management, it is necessary to build a server system that can be controlled from the Internet and mobile phones. Figure 9 shows that hardware consists of control modules, the software is written in C, and Internet web programs and mobile apps can be implemented using cloud servers.

The system control method utilizes the WiFi LINK module and can be controlled by an enhanced computer and a smartphone when the Internet is connected. The reason why it can be controlled at a remote distance is that it is implemented through a cloud server.

The interworking process between the internet web and mobile app is as follows:

1. Connect the power to the IoT control module and set up the WiFi Link on the baseboard.

2. Install WiFi “WiFi Connection Manager” app from Google PlayStore and select WiFi from the list.

3. Run WiFi Connection Manager to connect to the router and server.

4.1 Go to http://iot.vitcon.co.kr to use the cloud server.

4.2 Log in after joining the site

4.3 Add New Device-Setup-Copy device ID.

4.4 The copied device ID is connected to the IoT control module by upload after the program compilation.

5. Check internet web and mobile apps.

5.1 Connect to http://iot.vitcon.co.kr to use the cloud server

5.2 Register as a member and log in.

5.3 After checking the connection status, click View to check the web program.

The web programming design method for the IoT control module is as follows:

1. Access http://iot.vitcon.co.kr which is a cloud server.

2. After signing up for the site, log in, select the “Project List” at the top and press the “Add New Project” button.

3. Select the project name and preset.

4. Set the horizontal view grid row/column and resize the screen.

5. When the newly created project appears, click the edit icon.

6. The windows and menus for configuring the screen appear.

7. Configure the screen using the menu to select the widget icon.

Figure 10 is the screen composition of the monitoring system. The contents described above were reflected, and status information was checked and controlled on the user’s computer and smartphone based on a web server.

Figure 11 shows the interface block diagram of the IoT control module and peripherals. The air quality sensor is connected to analog input A2 and the temperature and humidity sensor is connected to digital inputs D6 and D7. Vent fans, carbon fiber planar heating elements, feed water motors, and plant growth LEDs were connected to D2, D3, D4, and D5, respectively.

Figure 12 is a flowchart of web programming. The default action is to select the auto/manual state. If you select Auto, the fan, carbon fiber surface heating element, nutrient supply motor, and plant growth LED will be automatically operated according to the temperature, humidity, and time settings. The screen configuration is displayed including soil temperature and humidity, internal temperature and humidity, and air quality.

4. Image Processing of Plant Factory

4.1. Plant Growth Information Extraction Image Recognition

The image recognition algorithm of plant growth information extraction is an algorithm that extracts only plant parts from the original image. R, G, B image separation, binarization and filtering, background and object segmentation, morphological image processing [30,31], color image conversion, plant image recognition algorithm through the numerical steps. The image processing flowchart is shown in Figure 13.

The acquired images were separated into R, G, and B images using Matlab software. After converting to the HIS (Hue, Saturation, Intensity) color model, preprocessing was performed by applying the k-mean algorithm, and only lettuce was separated from the seedling plate by applying the proposed morphological image processing. Information about lettuce can be obtained by calculating the area of the image.

4.2. Image Acquisition

Acquiring image information is the first step for image processing. It is important to note that at this stage, images should be taken at regular intervals in the same place. In this paper, a video camera was installed on the Arduino board and stored on an SD card, and the shooting time was taken every 6 hours. By doing so, it was possible to acquire images at a certain place and time on the SD card. Figure 14 shows the acquired images.

4.3. Filtering and Morphological Multimedia Image Processing

The original image in Figure 14 can be read and separated into R, G, and B images through Matlab software. Figure 15 shows a separate image.

Morphological Algorithm

The basic operations of morphology include dilation operation and erosion operation, and the definitions for these are as follows:

$$\mathrm{dilation}\mathrm{operation}:\partial \left(\mathrm{f}\right)=\max \left\{f\left(x-k\right),k\in B\right\}=f\oplus B,$$

(1)

$$\mathrm{erosion}\mathrm{operation}:\mathsf{\epsilon }\left(\mathrm{f}\right)=\min \left\{f\left(x+k\right),k\in B\right\}=f\ominus B,$$

(2)

where $B$ denotes a structural element and $f\left(x\right)$ denotes an input signal. The opening operation and closing operation are as follows:

$$\mathrm{opening}\mathrm{operation}:\mathsf{\gamma }\left(\mathrm{f}\right)=\partial \left(ϵ\left(\mathrm{f}\right)\right)=\left(f\ominus B\right)\oplus B=f\circ B,$$

(3)

$$\mathrm{closing}\mathrm{operation}:\mathsf{\phi }\left(\mathrm{f}\right)=ϵ\left(\mathsf{\gamma }\left(\mathrm{f}\right)\right)=\left(f\oplus B\right)\ominus B=f·B.$$

(4)

When lettuce seedlings are attached to the seedling plate, only lettuce cannot be separated. In this case, the lettuce seedling can be separated through erosion and expansion operations, and then the original image is restored through the open operation and the close operation.

In Figure 15, the green image of the separated image is visible in the vegetation part, and the remaining part is shown in the dark part. In this paper, we binarized the green image and then filtered it. Figure 16 shows the binarization and filtering image.

At the top of the filtered image in Figure 16, you can see that some of the lettuce seedlings are included. There are several ways to remove this part, but in this paper, morphological algorithm processing was performed using the image separated from the original image. As a result, it was possible to remove some of the stony leaves in the filtered image of Figure 16. Figure 17 shows the images undergoing morphological processing.

Color images can be obtained by adding red and blue images to the morphologically processed images. Figure 18 shows only the lettuce image extracted from the original image and image processing.

5. System Convergence and Evaluation

In this paper, we proposed a convergence method for three technologies to control plant plants from web and apps. Figure 19 shows the overall schematic diagram of a modular smart plant factory system using intelligent solar modules.

The intelligent solar module is applied to protect the module according to climate conditions, and the generated power is stored in the battery or supplied to the power of the plant light source LED through a DC/DC converter, and the IoT monitoring system is the temperature that is the plant’s growth environment. Humidity and air quality can be measured and controlled by the Internet web and mobile apps, and the plant growth recognition system reads the stored plant image, and an image processing algorithm is applied to determine the shipping date.

First, the intelligent solar module was to adjust the angle according to the situation, as a result, it can be seen that the energy efficiency is improved.

Table 2. Solar module performance measurement results.

Key Performance Indicator	Unit	Conventional (flat)	$\mathbf{Conventional}\left({100}^{\circ }\right)$	Intelligent Solar Modules
PV Module Conversion Efficiency	%	15.38	13.95	20.04
Maximum Power	w	86.745	87.673	119.867

shows the energy efficiency.

Plant growth LED performance is shown in Figure 20. Plants receive the intensity of light (lx) differently from humans as a force by the number of photons in the light. Here, a plant responds to the intensity of light, which means that it is based on how much photosynthesis receives light.

Table 3. Photosynthetic photon flux density (PPFD) measurement results.

LED Module	PPFD (μmol/m−2s−1)
I-PGRB_56(n1)	31.3
I-PGWH_56(n2)	35.1
I-PGRB_51(n3)	33.2
I-PGWH_51(n4)	36.7

shows the measurement results of photosynthetic photon flux density (PPFD).

In the developed LED chip, higher results were obtained at PGWH than PGRB. This resulted in high results as the module using the PGWH chip was evenly radiated in the whole wavelength range.

Fruits such as cherry tomatoes, paprika, and red peppers were cultivated and managed using the Internet and Web app using the IoT control module.

The temperature required for cultivation was set at 23 ± 3 °C and humidity at 95 ± 5%, and the plant growth LED module was operated for 24 hrs. The nutrient solution was used for fruit vegetable ash No.2 and was diluted to 33 ml (600 times) per 20 liters of water. It is effective in growing seedlings, developing new roots, increasing swelling, increasing yields, and increasing size.

Figure 21 shows a plant factory incorporating solar modules, LED, and IoT technologies. For the multimedia image processing in the plant factory, the proposed image processing algorithm was applied to extract the plant image from the seedlings of the leafy vegetables and the overhaul. Figure 22 shows the result of the image processing of the plant factory.

Table 4. Plant growth image processing results.

Assortment	Plant Area	Shipment	Recognition	Remarks
Day 7	66,762	No	Yes	-
Day 9	718,333	No	Yes	-
Day 11	913,748	No	Yes	-
Day 13	2,123,174	Yes	Yes	-
Day 15	2,852,479	Yes	Yes	-
Day 17	6,682,061	Yes	Yes	-

shows the results of image processing by reading a total of six plant photos from the 7th day to the 17th day of the plant growth.

6. Discussion

This paper proposed a power supply, LED lighting, IoT control system, and image processing system for plant factories. The solar module conversion efficiency was improved to 20.05%, which is higher than the existing 15.38% by using an intelligent module. The LED lamp was made to be 36.7 as a result of PPFD, and the wavelength suitable for plant growth was emitted, which helped the plant to grow quickly. The IoT system has developed an integrated monitoring system to measure and control plant growth conditions, and this technology can be used in computers and smartphones.

The image processing system was able to calculate the plant division and area by applying the improved morphological multimedia imaging technique to the growth of plants and to inform the shipment of plants. Such a plant factory system was constructed and applied, and in the future, research combining multiple plant factories is needed.

7. Conclusions

In this paper, we proposed a modular smart plant factory system integrating intelligent photovoltaic module, high efficiency LED, IoT control system, and multimedia image processing.

The intelligent solar module aims to protect the module according to the climatic condition, and the operation is judged to be a poor climate when the power generation is significantly lower than usual, and the module is automatically folded.

The plant growth LED module was designed and manufactured to have a maximum photosynthetic photon density (PPFD) of 30 μmol/m⁻²s⁻¹ or more. The plant growth IoT control monitoring system was applied directly at the developed modular plant factory. The system measures the temperature, humidity, and air quality of the plant’s growing environment and can be controlled by the Internet web and mobile apps. In addition, through the embedded system and the camera module, it was able to transmit and store the growth image of the plant. The stored image is read by the developed plant growth recognition system, and the image processing algorithm can be used to determine the shipment. The modular smart plant factory system using intelligent solar modules has higher efficiency compared to the existing system by improving energy efficiency, rapid growth of plants, and determining the shipping timing through image processing. The cost aspect is at the same level as the existing system construction cost, and it is considered to be of high economic value as a highly efficient system. Further research is expected to greatly contribute to the growth of agriculture if standardization and big data are established.