Next Article in Journal
Enforcing a Force–Displacement Curve of a Nonlinear Structure Using Topology Optimization with Slope Constraints
Previous Article in Journal
A Survey of Optimal Hardware and Software Mapping for Distributed Integrated Modular Avionics Systems
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 10
Issue 8
10.3390/app10082679
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Encapsulation and Release Control of Fish Pathogen Utilizing Cross-Linked Alginate Networks and Clay Nanoparticles for Use with a Potential Oral Vaccination

by
Su-Bin Lee
1,
Ji-Yeon Kim
2,
Kyusik Kim
3,
Kyoung-Jin Ahn
4,
Tae-il Kim
3,5,* and
Jae-Min Oh
1,*
1
Department of Energy and Materials Engineering, Dongguk University-Seoul, Seoul 04620, Korea
2
Department of Chemistry and Medical Chemistry, College of Science and Technology, Yonsei University, Wonju, Gangwondo 26493, Korea
3
Department of Biosystems & Biomaterials Science and Engineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 08826, Korea
4
JUNWON GBI Co. Ltd., 55, Daetongdong-gil, Jeju-si, Jeju-do 63074, Korea
5
Research Institute of Agriculture and Life Sciences, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 08826, Korea
*
Authors to whom correspondence should be addressed.
Appl. Sci. 2020, 10(8), 2679; https://doi.org/10.3390/app10082679
Submission received: 31 January 2020 / Revised: 6 April 2020 / Accepted: 10 April 2020 / Published: 13 April 2020
Browse Figures
Versions Notes

Abstract

:
Streptococcus parauberis is utilized as an oral vaccine by first inactivating the cells with formalin to produce formalin- killed cells (FKC) and then encapsulating them with polymer beads consisting of a cross-linked alginate-Ca2+ network. The encapsulation efficiency and media-dependent release are controlled by pre-treating the FKC with two types of clay nanoparticles: kaolinite (KA) and layered double hydroxide (LDH). The addition of LDH induced large agglomerates of FKC, and the KA enhanced the dispersion of FKC. The differences in the dispersibility of the FKC upon the use of clay nanoparticles was determined to strongly affect the encapsulation efficiency and release properties. The FKC + LDH mixture exhibited a slightly reduced encapsulation efficiency compared to the FKC alone. However, FKC + KA exhibited a dramatically improved encapsulation efficiency. In terms of the media-dependent release, the alginate beads were found to be fairly stable under gastric conditions and in deionized water with or without clay nanoparticles, preserving most of the encapsulated FKC. The intestine was the final target organ for FKC vaccination, and release at the site varied according to the use of clay nanoparticles. Both clays seemed to enhance the release of FKC, the cumulative amount being 3.6 times and 1.3 times larger for LDH and KA, respectively, than was shown with only FKC encapsulated beads.

Graphical Abstract

1. Introduction

Aquaculture is one of the major food production system in modern society. In recent decades, the amount of aqua culture production has increased almost two-fold while capture fishery has not increased significantly in that time [1]. A major issue that influences aquaculture productivity is the prevention of disease in the fish populations which can occur by the introduction of a bacterial infection into the fish population. For example, Renibacterium salmoninarum can cause kidney disease in Chinook Salmon (Oncorhynchus tshawytscha) [2]. Kim et al. [3] also reported that Vibrio ichthyoenteri can cause devastating bacterial disease in olive flounder (Paralichthys olivaceus). To treat bacterial infections in fish, aquaculturists usually administer antibiotics to the fish by mixing antibiotics in the fish species. This method is cost-effective and has high efficiency in producing healthy fish, [4] so it has been widely applied. However, there exist several problems inherent to the use of antibiotic feed at fish farms. It is not easy to administer an ideal dose of antibiotics, the Food and Agriculture Organization (FAO) of the United Nations has warned that antimicrobial residues in fish species could have an impact on human health [1]. Ferdous et al. [5] reported in 2015 serious issues with remnants of antimicrobial substances in Bangladesh. In their research, high concentration of antimicrobial substances, such as amoxicillin and oxytetracycline, remained in climbing perch (Anabas testudineus) and shrimp species (Penaeus). In addition, extra-label use of antibiotics [6] can produce environmental and ecological problems that are detrimental to human health. The European Commission warned that antimicrobial residues can affect fish and humans. They also reported that antibiotic residues, such as amoxicillin and clarithromycin, are present in waste water or water used in agriculture, possibly contributing to bacterial disease in humans who use food created from these water resources. Consequently, resistance in bacteria can represent a potential risk to human health, and this is also a problem that has been noted and identified as an issue in many waterbodies in Europe [7].
The use of vaccinations is a powerful approach to solve the problems resulting from extra-label use of antibiotics. Several commercial fish vaccines have been developed [8] that can be administered to fish through injection and immersion [9,10]. However, applying the vaccination through injection or immersion is fairly limiting in aquaculture systems, since it is a time and labor-intensive job. The ideal way would be to have oral administration of the fish vaccination considering the ease of administration, less stress to the fish, unlimited applications according to fish size, and economical aquaculture management. There have been limited attempts to develop oral fish vaccines. Li et al. [11] developed an attenuated oral vaccine strain of tilapia group B streptococci. Mostafa et al. [12] tried to produce an oral vaccine utilizing Lactococcus garvieae with chitosan/alginate ingredients. In most oral vaccine studies, the most important point noted is the stable storage of antigens before release in the intestine, which was the target organ [13]. According to Mutoloki et al. [14], the presence of a hostile stomach environment is the first barrier for orally delivered fish vaccines. Recent reports, noted that the digestive fluid of a fish lacks an alginate-specific gradation enzyme [15,16]. Because of the aforementioned stability, alginates have even been utilized to adopt exogeneous digestive enzymes or probiotics to increase the fishes’ digestive ability [17,18,19]. Clays are also inert toward enzymes, and may be considered an important ingredient that can be used in this process. Because of their inertness and stability toward enzymes, clay minerals are often utilized to immobilize enzymes much like lipases [20,21,22]. On the other hand, both alginate and clays are fairly sensitive to the pH conditions since their surface charge, swelling property and hydrolysis are dependent upon the proton concentration. In this regard, our organ-simulation was carried out with pH adjusted media. Therefore, we focused on storage under gastric condition and release under intestinal conditions for the main experiments.
The alginate that forms cross-linked hydrogel beads under the existence of Ca2+ [23,24] was selected as the major encapsulating material. Alginate has been widely known to be a biocompatible polymer, and it can be utilized as a drug reservoir with controlled release [25,26,27,28]. The alginate beads were shown to easily swell in neutral pH by absorbing water. On the other hand, the mannuronic side chains became hydrophobic at a low pH, prohibiting swelling in an acidic condition. In this work, two biocompatible ingredients, kaolinite (KA) and layered double hydroxide (LDH) were selected to control the encapsulation efficiency and intestinal delivery rate. Both of these ingredients are clay nanoparticles that are registered in the European pharmacopoeia. Notably, they are similar in terms of having a layered structure and plate-like morphology [29,30], but they are different in their surface charge. KA and LDH are negatively and positively charged, respectively, under a neutral pH. We hypothesized that the interface interaction between the clay nanoparticles and pathogenic microorganisms would modify the encapsulation efficiency and release behaviors of the alginate beads. We selected Streptococcus parauberis as the model antigen for starry flounder (Platichthys stellatus). In this case, the microbe cells were inactivated to obtain formalin-killed cells (FKC). Both the FKC and FKC + clay mixtures were thoroughly characterized before and after the alginate bead encapsulation. The encapsulation efficiency and release at different simulated biological fluids will be discussed in further detail later in this report.

2. Materials and Methods

To confirm the pathogenicity of Streptococcus parauberis against starry flounder, an in vivo experiment was carried out following the guidelines of the National Institute of Fisheries Science, of Korea. The fish were obtained from a fish farm in Jeju Island, Korea and were acclimated in a fish tank before the experiment. Fish without any physical malformations or abnormal swimming behavior were used in this study. A total of 100 fish (720 ± 25 g) were kept for 4 weeks (17 ± 0.5 °C) and fed daily with compound fish feed. The fish were then divided into two groups (50 heads per group) and the pathogenicity experiments were duplicated. The Streptococcus parauberis was prepared in suspension at a concentration of 1 × 109 cfu/mL, and 100 μL of the suspension was intraperitoneally injected to each fish. Before injection, the fish were anesthetized to MS-222 (250 mg/L) for 20 min and washed for sedation. The FKC was obtained by treating Streptococcus parauberis in 1% formalin solution. Then, the FKC was suspended in phosphate buffered saline (PBS) to have an optical density (O.D.) of 0.1 for further treatment. The kaolinite (KA, EP grade) was purchased from Fisher Chemical, Fisher Scientific (Loughborough, UK). Another ingredient, layered double hydroxide (LDH) was synthesized to produce a size and morphology of LDH similar to KA. Typically, mixed metal solution containing Mg(NO3)2 6H2O (0.3 M) and Al(NO3)3 9H2O (0.15 M) was titrated with alkaline solution (0.5 M NaOH and 0.5 M NaHCO3) up to pH 9.5. Then the white suspension was subjected to hydrothermal treatment at 150 °C for 24 h. Finally, the precipitates were obtained through centrifugation, and were washed with deionized water and lyophilized. To obtain the FKC + clay mixture, FKC suspension (OD = 0.1) and an equivalent volume of aqueous clay suspension (1 mg/mL) were mixed and gently stirred under room temperature for 1 h. Next, the agglomeration or dispersion of FKC in the presence of clay particles was investigated by either measuring the hydrodynamic radius (dynamic light scattering measurement by ELSZ-1000, Otsuka, Kyoto, Japan) or by observing via scanning electron microscopy (FE-SEM, FEI Quanta FEG250, Hillsboro, OR, USA). For the SEM specimen, FKC or FKC + clay suspension were drop-cast on a piece of silicon wafer (~0.5 mm × 0.5 mm) that was pre-cleaned with piranha solution (sulfuric acid: hydrogen peroxide = 3:1). After drying the specimen at room temperature, the specimen was coated with Pt/Pd sputtering (60 s) and was subjected to SEM measurement under 30 kV of accelerating voltage.
To begin, the alginate was obtained from local seaweed. Typically, 5 g of dried seaweed was treated with 50 mL of 0.4% HCl and was washed with deionized water. Then, the precipitate was treated with 125 mL of 0.4% NaOH, and the solution was heated-up to 60 °C for 2.5 h. After the solution had cooled down, 150 mL of deionized water were added for dilution. Furthermore, the solution was separated by filtration and was treated with a small amount of 1 M HCl for gelation. The alginate gel was then separated by filtration, and 0.4% NaOH (200 mL) was added for neutralization. The final product was washed with 200 mL of EtOH then was then dried at 60 °C for 24 h.
To conduct a quantitative analysis of the FKC, fluorescent dye, fluorescein isothiocyanate (FITC) (Sigma-Aldrich, Co. Inc., St. Louis, MO, USA, CAS No. 27072-45-3) was labelled on FKC. First, 1 mL of FKC suspension (OD = 0.1) was mixed with 1 mL of FITC solution (20 ppm). The mixture was vigorously stirred to induce thiourea bond between the amine terminal of the FKC surface and the isothiocyanate group of FITC. A reaction was carried out under dark conditions to avoid photodegradation. After 3 h of reaction, the FITC labelled FKC was washed with PBS and was collected via centrifugation. Next, the precipitate was re-suspended in PBS for further encapsulation by alginate beads. Then, the encapsulation state of the FKC inside the alginate bead was investigated by either SEM (FE-SEM, FEI Quanta FEG250, Hillsboro, OR, USA), fluorescence microscopy (iRis Digital Cell Imaging System, Logos biosystems, Anyang-Si, Korea) or the use of confocal microscopy (Confocal laser scanning microscope SP8 X, Leica, Wetzlar, Germany). For the specimen of the SEM measurement, an alginate bead was dried under ambient air and was then crushed into pieces. Next, the pieces were placed onto a sticky carbon tape and were sputtered with Pt/Pd plasma for 60 s. Before fluorescence and confocal microscopy, wet alginate beads containing FITC-FKC were pressed on a slide glass utilizing a coverslip and were subjected to a microscopic analysis. To confirm the possibility of FKC quantification by green fluorescent marker, we mixed 1 mL of FKC suspension with 1 × 109 cells/mL concentration. Then, the suspension was diluted with PBS obtained from the FKC-FITC concentrations of 5 × 104, 1 × 105, 2 × 105, 4 × 105, 6 × 105, 8 × 105 cells/mL. We next prepared six standard samples that were subjected to fluorescence quantification under an excitation wavelength at 492 nm and emission wavelength at 520 nm utilizing a microplate reader (Varioskan™ LUX multimode microplate reader, Thermo Fisher Scientific™, Waltham, MA, USA).
To encapsulate the FKC into alginate bead, the following process was utilized. First, an alginate solution was prepared in 2% (wt/v) concentration. The FKC suspension either with or without clay moiety was prepared as follows: the FKC suspension with OD 0.2 was mixed with an equivalent volume of PBS, KA-PBS suspension (1 mg/mL) and the LDH-PBS suspension (1 mg/mL), respectively. At this time, the alginate solution and FKC suspension (with or without clay) were gently mixed and the mixture was dropped into CaCl2 solution (0.1 M) utilizing a 20-gauge needle. Finally, alginate beads were readily formed through cross-linking of the alginate chains by Ca2+ coordination.
The encapsulation of the FKC into alginate with or without clays was quantified utilizing FITC-FKC. After the encapsulation process, the supernatant was subjected to a fluorophotometer analysis under an excitation wavelength of 492 nm and emission wavelength of 520 nm. Here, the encapsulation efficiency (EE) and loading capacity (LC) was calculated by using the equation below.
E E   ( % ) = ( t o t a l   F K C   u t i l i z e d   i n   e n c a p s u l a t i o n ) − F K C   i n   s u p e r n a t a n t t o t a l   F K C   u t i l i z e d   i n   e n c a p s u l a t i o n × 100
L C = n u m b e r   o f   F K C   i n s i d e   t h e   b e a d t o t a l   d r i e d   w e i g h t   o f   a l g i n a t e   b e a d   w i t h   F K C
In this study, FKC release was quantified under three different solutions: deionized water (DW), simulated gastric solution, and simulated intestinal solution. The simulated gastric solution was prepared by adjusting the acidity of the aqueous NaCl solution (0.002%) to pH ~1.2 utilizing hydrochloric acid, and the simulated intestinal solution was prepared by mixing 250 mL KH2PO4 (0.2 M), 76 mL NaOH (0.2 M), and 674 mL DW. The pH of the intestinal solution was ~6.5. The alginate bead (~0.1 g) was put into each solution (~20 mL) and was stirred gently at 100 rpm under darkness. At a designated time point (2 h for DW, 4 h for gastric condition and 2 h for the intestinal condition), the supernatant was collected and the predetermined amount of released FKC was quantified by a fluorophotometer (excitation wavelength of 492 nm and emission wavelength of 520 nm). The time point for DW, gastric, and intestinal condition was determined considering the residence time of the alginate in water before feeding, in the stomach and in the intestine of the fish after feeding.

3. Results

The pathogenicity experiments confirmed that Streptococcus parauberis could be utilized as an antigen to stimulate the immune system of starry flounder. Thus, we tried to estimate the potential of Streptococcus parauberis as an oral vaccine after appropriate formalin inactivation and encapsulation. Upon review, the colloidal property of FKC with or without clay nanoparticles in an aqueous system was also tested and monitored. As shown in Figure 1a, the hydrodynamic radius of the FKC + LDH mixture was much larger than that of FKC or LDH itself before mixing, suggesting the formation of agglomeration between FKC and LDH nanoparticles. On the other hand, the reviewed FKC + KA mixture showed a similar hydrodynamic radius as compared to the FKC or KA itself (Figure 1b).
As we have seen, the quantitative colloidal parameters are summarized in Table 1, showing the different effects of LDH and KA on the colloidal stability of FKC. The Z-average of FKC + LDH was much larger than those of the FKC and LDH. Furthermore, the PdI values, which indicate the degrees of homogeneity with lower values, dramatically increased upon mixing. On the other hand, the FKC + KA mixture showed still a comparable Z-average with FKC or KA, and the PdI value of the mixture even decreased upon mixing. These results strongly suggest that LDH and KA respectively acted as an agglomerating agent and dispersant for FKC under the reviewed aqueous system.
The formation of agglomerates or dispersion was also visualized using SEM measurements (Figure 2). As shown in Figure 2a, the FKC of Streptococcus parauberis exhibited round shaped cells of more or less 1 μm in size. Both clays showed particle dimension of hundreds of nanometers with a characteristic plate-like morphology (Figure 2b, d). The SEM images of the mixture were fairly different from each other depending on the type of clay nanoparticles that were used. We could not observe any FKC cells on the FKC + LDH mixture (Figure 2c). It seems that smaller LDH particles cover all FKC cells to hide them. On the other hand, the SEM image (Figure 2e) of the FKC + KA showed plate-like KA particles, as well as round FKCs (white dotted circles in Figure 2e). The SEM images also showed an agglomeration and dispersion of the FKCs under the existence of LDH and KA, respectively.
To confirm the possibility of FKC encapsulation by alginate bead, we carried out a SEM measurement on the dried alginate beads containing FKC. As shown in Figure 3a, a half-crushed bead showed hollow spaces that could be attributed to the dehydration process. In the area of the wall inside the hollow (Figure 3b), we could find homogeneously patterned texture at the background, where several assemblies of spheres (white dashed line in Figure 3b,c) were observed. The patterned texture was considered as the alginate-Ca2+ cross-linked network and the assemblies were linked FKCs. This result showed that FKC could be encapsulated within the alginate bead while several cells were linked together.
The location of the FKC inside the alginate beads in the hydrogel state was visualized by the use of fluorescence and confocal microscopy, respectively. For this process, FKC was previously labelled with green fluorophore, FITC (Figure 4). Upon review, the fluorescence microscopy image showed a green fluorescence that spread throughout the alginate bead, suggesting that the FKCs could be homogeneously encapsulated in the beads. Furthermore, the green fluorescence in the microscopic images was seen to be slightly blurred at some points. Therefore, to confirm the existence of FITC-labelled FKC in detail, confocal microscopy was carried out. When we focused on a certain plane along the z-axis of the alginate bead, we could clearly observe green dots that indicate the location of the FITC-labelled FKC. Notably, the green dots were found to be linked to each other forming assemblies of ~10 FKCs, which corresponded to the findings of the SEM observation (Figure 3).
We further investigated the inside of the dried alginate beads containing either FKC + LDH or FKC + KA mixture (Figure 5). When compared to alginate beads with FKC only, the bead wall was shown to be less homogeneously patterned. However, we could clearly observe a difference between the FKC + LDH containing alginate bead and the one containing FKC + KA. The former showed several points that were characteristic of intensive small dots (white dotted circle in Figure 5a), which were attributed to the FKC + LDH agglomerates. On the other hand, the latter showed a more homogenous aspect, which indicated that the FKC cells and KA were uniformly distributed throughout the alginate beads.
As shown in Figure 6, the regression factor, R2 was larger than 0.999, suggesting the possibility of FKC quantification with FITC measurement. In the calibration, we prepared six standard solutions with concentrations of 5 × 104, 1 × 105, 2 × 105, 4 × 105, 6 × 105, 8 × 105 cells/mL. This quantification method showed the presence of a determined limit of detection (LOD) and limit of quantification (LOQ) of 3.2 × 104 cells/mL and 9.6 × 104 cells/mL, respectively. Incidentally, these values corresponded to OD = 0.0032 and OD = 0.0096 for conventional spectrophotometric quantification, and thus the current fluorescence quantification was considered to be more sensitive than the conventional method.
Based on this quantification method, the encapsulation efficiency and loading capacity of the FKC in the alginate beads with or without clay nanoparticles were analyzed. As summarized in Table 2, the encapsulation efficiency and loading capacity were noted to be highly dependent on the type of clay nanoparticles used as ingredients. Here, the existence of LDH was shown to reduce the encapsulation efficiency while KA dramatically increased the encapsulation efficiency. As a result, 1 g of alginate beads were determined to contain from 1.25 × 105 to 1.89 × 105 cells depending on the type of clay nanoparticles as ingredients.
The FKC release in various situation was investigated utilizing DW, simulated a gastric solution (pH 1.2) and an intestinal solution (pH 6.5), respectively. The cumulative release in DW at 2 h was observed only in FKC alone-encapsulated alginate beads. No significant FKC release was noted in DW in FKC + LDH or FKC + KA mixture-encapsulated alginate beads. In the gastric condition, all of the alginate beads did not show a measurable FKC release within 4 h. The FKC release was considerable in the intestinal condition showing 17.06%, 61.12%, and 22.26% for FKC alone, FKC + LDH, and FKC + KA-encapsulated alginate beads, respectively, within 2 h.

4. Discussion

In terms of oral vaccination, pathogens should be stably delivered to the intestine since the intestinal mucous membranes are point of stimulus for the immune system [31,32,33]. Therefore, a strategy for oral vaccination should satisfy the following conditions: (i) Provide for the selection of an appropriate pathogen, (ii) apply a reservoir of pathogen, (iii) provide for the adequate formulation of the reservoir with fish feed, (iv) offer stable storage of the pathogen in the formulation and in the gastric condition, and (v) provide for a bursting out of the pathogen in an intestinal condition during the residence time.
In this regard, Streptococcus parauberis, which was found to have pathogenic effect on the starry flounder, was first inactivated by formalin and was encapsulated with alginate hydrogel beads. Furthermore, the alginate was found to form a cross-link with metal cations like Ca2+ [34,35], and they easily formed hydrogel beads when the alginate solution droplet meets the CaCl2 solution [36]. We also confirmed that the alginate beads could be formed with a high water content, and that the cross-linked alginate could contain streptococcus-originated FKCs (Figure 3). Furthermore, the streptococci were found to be homogeneously distributed throughout the hydrogel, thus preserving their pair or chain morphologies (Figure 4).
Generally speaking, since FKC encapsulation by the alginate beads is a passive process, i.e., FKC suspension in the aqueous system is surrounded by alginate polymers whose cross-linking with Ca2+ happen to encapsulate the FKC moiety. Furthermore, the alginate bead encapsulating the FKC can only be released from the bead when the alginate-Ca2+ cross-link is sufficiently broken. In the meantime, we planned to control the encapsulation and release behaviors of the alginate beads by utilizing the clay nanoparticles that can modify the status of FKC suspension and alginate cross-linking. Various clay moieties have been reported to interact with microbes through surface adsorption, flocculation, agglomeration, etc., [37,38]. These physicochemical reactions were expected to affect the encapsulation and release of FKC for alginate beads. Furthermore, clay nanoparticles have often been reported to control the swelling property or drug release property of alginate polymers [39,40]. In this case, we selected two different kinds of clay nanoparticles, KA and LDH, to control the encapsulation and the release of FKC for alginate beads.
In this relation, KA and LDH are different in terms of their surface charge although both clays are similar in their plate-like morphology and layer-by-layer stacking structure. Here, KA is known to have an electric point between 4 and 6 [41], implying negative surface charges at a neutral condition. On the other hand, LDHs have a high isoelectric point of ~11, being positively charged at a neutral pH [42]. Although the surface charges of streptococci are different from each other depending on the strain, it was generally known that streptococcus bacteria have an isoelectric point ~4, indicating negative surface charges in a neutral pH. Therefore, we could expect that the noted dispersion state of Streptococcus parauberis FKC would vary upon the type of clay nanoparticles used. Notably, Streptococci themselves tend to form pairs or chains in these cases. Because of this reason, the pairs and chains would assemble in larger agglomerates under the presence of LDH due to the electrostatic interaction. Individual chains or pairs of FKC would be well dispersed in the presence of KA because of electrostatic repulsion. Even though we could expect some pairs or chains to be broken into single cells in the presence of KA, as it was previously reported, it was expected that the mechanical stress separated chained streptococci into single cells [43]. As a result of the study, this hypothesis was clearly demonstrated in the DLS analysis (Figure 1 and Table 1) and in the SEM observations (Figure 2).
As has been seen, the KA dramatically enhanced the encapsulation efficiency, while LDH slightly hindered the encapsulation of FKC (Table 2). It is not easy to fully comprehend the effect of the nanoclays at this stage. However, we could suggest several explanations. First, the alginate-Ca2+ cross-linking occurs readily and homogeneously in the aqueous system, as soon as the alginate solution meets the Ca2+ cations [44]. During this process, the degree of homogeneous dispersion is considered to affect the encapsulation efficiency whereby the more homogeneously dispersed, the easier alginate-Ca2+ network can capture the FKC moiety. In this instance, Table 1 shows that the PdI of the FKC + KA mixture was 20% lower than that of FKC only, suggesting the possibility of FKC chain separation into single cells. Furthermore, an increase in freely floating FKC facilitated encapsulation into alginate beads. On the other hand, the FKC + LDH mixture was clearly shown to produce large lumps; and therefore the alginate-Ca2+ network could not efficiently encapsulate large lumps that might produce a type of sediment without being encapsulated.
It is important to realize that the release behavior was also strongly affected by the existence of clay nanoparticles. As shown in Table 3, FKC alone-encapsulated alginate beads could release ~2% of FKC, while the FKC + clay mixture containing alginate beads did not significantly show the release amount. In an aqueous system, the dried alginate bead is expected to absorb water. As time goes by, the more water that gets into the alginate beads produced the characteristic resulting in a swelling and release of the encapsulated moiety. In this way, the clay nanoparticles, LDH and KA are known to absorb water easily when they form a composite with the polymer [45,46]. Therefore, clay nanoparticles in the alginate beads are expected to absorb water instead of alginate side chains, consequently prohibiting the swelling of alginate beads. In the gastric condition, all alginate beads regardless of the presence of clay nanoparticles preserved the encapsulated FKC efficiently until 4 h. This can be explained by the chemical nature of alginate. As noted, the guluronate groups coordinate with Ca2+ to form cross-links. However, mannuronic acid groups remained because of their acidic property. Generally speaking, at a low pH, mannuronic acid loses solubility dramatically to repel water moiety [35,47]. In this way, alginate beads can preserve an intact FKC at a gastric pH. In the intestinal condition, because of the high pH and various electrolyte conditions, a significant FKC release was apparent for all alginate beads. Both clay nanoparticles increased their FKC release which was attributed to the alginate network disturbed by the nanoparticles. As shown in Figure 3, the inner wall of the alginate only bead was homogenously patterned. However, the addition of clay nanoparticles (Figure 5) altered the inner surface to be more heterogeneous. As a wedge is driven in a matrix, the clay nanoparticles were expected to make holes in the alginate matrix to release more FKC. More release was found in the FKC + LDH mixture since LDH could agglomerate FKC in large lumps, so a larger amount of FKC could be released out at one time. Taking into account the loading capacity and cumulative release in the intestinal condition, the total FKC delivered into the intestine was 3.2 × 104 cells/g-bead, 7.8 × 104 cells/g-bead, 6.7 × 104 cells/g-bead for FKC only, FKC + LDH, and FKC + KA encapsulated beads, respectively. However, in terms of FKC utilization efficiency during encapsulation, FKC + KA is concluded to be advantageous in preparing an oral vaccine.

5. Conclusions

We have developed alginate beads that can be potentially utilized as oral vaccines for starry flounder. To stimulate the immune system, we selected Streptococcus parauberis as a model pathogen and inactivated the toxicity by using a formalin treatment. Thus, the prepared FKC could be successfully encapsulated by alginate beads by treating alginates with Ca2+ in an aqueous system. To control not only the encapsulation efficiency but also the release at the target organ, which is the intestine, we added clay nanoparticles as colloidal state controllers. Positively charged clay, LDH, induced a strong agglomeration of the FKC while the negative one, KA, enhanced the homogeneity in the dispersion. An effect of the clay nanoparticles was also found in the encapsulation efficiency. In line with this result, the FKC encapsulation efficiency was slightly reduced by LDH but dramatically increased with the KA. All alginate beads with and without clay nanoparticles seemed to safely preserve the encapsulated FKC in the DW and gastric conditions. Under intestinal conditions, the release of FKC was significant. The released amount was as follows: FKC + LDH bead > FKC + KA bead > FKC only bead. The FKC + LDH combination was considered to be advantageous in the total intestinal delivery of FKC, and the FKC + KA combination is economical in terms of the bead formulation process.

Author Contributions

S.-B.L.: writing—original draft; J.-Y.K.: methodology; K.K.: formal analysis; K.-J.A.: resources and project administration; T.-i.K.: writing—review and editing; J.-M.O.: supervision and writing—review and editing. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Dongguk University Research Fund of 2019. This work was funded by the Ministry of SMEs and Startups (Development of assessment method in fish vaccine by using antigen protein originated from low-hemolysing bacteria and low molecular alginate, P0002949).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. FAO. The State of World Fisheries and Aquaculture 2018-Meeting the Sustainable Development Goals; Licence: Cc By-Nc-Sa 3.0 Igo; FAO: Rome, Italy, 2018; p. 147. [Google Scholar]
  2. Holey, M.E.; Elliott, R.F.; Marcquenski, S.V.; Hnath, J.G.; Smith, K.D. Chinook Salmon Epizootics in Lake Michigan: Possible Contributing Factors and Management Implications. J. Aquat. Anim. Health 1998, 10, 202–210. [Google Scholar] [CrossRef]
  3. Kim, D.-H.; Han, H.-J.; Kim, S.-M.; Lee, D.-C.; Park, S.-I. Bacterial enteritis and the development of the larval digestive tract in olive flounder, Paralichthys olivaceus (Temminck & Schlegel). J. Fish. Dis. 2004, 27, 497–505. [Google Scholar] [CrossRef] [PubMed]
  4. Yanong, R.P. Use of Antibiotics in Ornamental Fish Aquaculture; University of Florida Cooperative Extension Service, Institute of Food and Agricultural Sciences Extension: Ruskin, FL, USA, 2003. [Google Scholar]
  5. Ferdous, J.; Bradshaw, A.; Islam, S.K.M.A.; Zamil, S.; Islam, A.; Ahad, A.; Fournie, G.; Anwer, M.S.; Hoque, M.A. Antimicrobial Residues in Chicken and Fish, Chittagong, Bangladesh. EcoHealth 2019, 16, 429–440. [Google Scholar] [CrossRef]
  6. Food and Drug Administration. Fish and Fishery Products Hazards and Controls Guidance; US Department of Health and Human Services Food and Drug Administration: Silver Spring, MD, USA, 2011.
  7. European Commission. Antibiotics in Water and the Risk of Drug-Resistance Bacteria. Available online: https://ec.europa.eu/jrc/en/news/antibiotics-water-and-risk-drug-resistant-bacteria (accessed on 26 March 2019).
  8. Adams, A. Progress, challenges and opportunities in fish vaccine development. Fish. Shellfish Immunol. 2019, 90, 210–214. [Google Scholar] [CrossRef] [PubMed]
  9. Souter, B. Immunization with vaccines. Dep. Fish. Ocean. Winn. Mannitoba 1984, 20, 111–117. [Google Scholar]
  10. Vaccination Strategies and Procedures. In Fish Vaccination; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2014; Chapter 12; pp. 140–152. [CrossRef]
  11. Li, L.; Huang, T.; Liang, W.; Chen, M. Development of an attenuated oral vaccine strain of tilapia Group B Streptococci serotype Ia by gene knockout technology. Fish Shellfish Immunol. 2019, 93, 924–933. [Google Scholar] [CrossRef]
  12. Halimi, M.; Alishahi, M.; Abbaspour, M.R.; Ghorbanpoor, M.; Tabandeh, M.R. Valuable method for production of oral vaccine by using alginate and chitosan against Lactococcus garvieae/Streptococcus iniae in rainbow trout (Oncorhynchus mykiss). Fish Shellfish Immunol. 2019, 90, 431–439. [Google Scholar] [CrossRef]
  13. Embregts, C.; Rigaudeau, D.; Tacchi, L.; Pijlman, G.; Kampers, L.; Veselý, T.; Pokorová, D.; Boudinot, P.; Wiegertjes, G.; Forlenza, M. Vaccination of carp against SVCV with an oral DNA vaccine or an insect cells-based subunit vaccine. Fish Shellfish Immunol. 2019, 85, 66–77. [Google Scholar] [CrossRef]
  14. Mutoloki, S.; Munang’andu, H.M.; Evensen, Ø. Oral Vaccination of Fish—Antigen Preparations, Uptake, and Immune Induction. Front. Immunol. 2015, 6, 519. [Google Scholar] [CrossRef] [Green Version]
  15. Rodriguez, Y.E.; Laitano, M.V.; Pereira, N.A.; López-Zavala, A.A.; Haran, N.S.; Fernández-Gimenez, A.V. Exogenous enzymes in aquaculture: Alginate and alginate-bentonite microcapsules for the intestinal delivery of shrimp proteases to Nile tilapia. Aquaculture 2018, 490, 35–43. [Google Scholar] [CrossRef]
  16. Geethanjali, S.; Subash, A. Optimization and immobilization of purified Labeo rohita visceral protease by entrapment method. Enzym. Res. 2013, 2013, 874050. [Google Scholar] [CrossRef] [Green Version]
  17. Rosas-Ledesma, P.; León-Rubio, J.M.; Alarcón, F.J.; Moriñigo, M.A.; Balebona, M.C. Calcium alginate capsules for oral administration of fish probiotic bacteria: Assessment of optimal conditions for encapsulation. Aquac. Res. 2012, 43, 106–116. [Google Scholar] [CrossRef]
  18. Andrea, T.; Marcela, F.; Lucía, C.; Esther, F.; Elena, M.; Simona, M. Microencapsulation of lipase and savinase enzymes by spray drying using arabic gum as wall material. J. Encapsul. Adsorpt. Sci. 2016, 6, 161–173. [Google Scholar] [CrossRef] [Green Version]
  19. Yoo, S.-H.; Song, Y.-B.; Chang, P.-S.; Lee, H.G. Microencapsulation of α-tocopherol using sodium alginate and its controlled release properties. Int. J. Biol. Macromol. 2006, 38, 25–30. [Google Scholar] [CrossRef] [PubMed]
  20. Rahman, M.B.A.; Tajudin, S.M.; Hussein, M.Z.; Rahman, R.N.Z.R.A.; Salleh, A.B.; Basri, M. Application of natural kaolin as support for the immobilization of lipase from Candida rugosa as biocatalsyt for effective esterification. Appl. Clay Sci. 2005, 29, 111–116. [Google Scholar] [CrossRef]
  21. Tietjen, T.; Wetzel, R.G. Extracellular enzyme-clay mineral complexes: Enzyme adsorption, alteration of enzyme activity, and protection from photodegradation. Aquat. Ecol. 2003, 37, 331–339. [Google Scholar] [CrossRef]
  22. de Fuentes, I.E.; Viseras, C.A.; Ubiali, D.; Terreni, M.; Alcántara, A.R. Different phyllosilicates as supports for lipase immobilisation. J. Mol. Catal. B Enzym. 2001, 11, 657–663. [Google Scholar] [CrossRef]
  23. Manjanna, K.; Kumar, T.P.; Shivakumar, B. Calcium alginate cross-linked polymeric microbeads for oral sustained drug delivery in arthritis. Drug Discov. Ther. 2010, 4, 109–122. [Google Scholar]
  24. Segale, L.; Giovannelli, L.; Mannina, P.; Pattarino, F. Calcium alginate and calcium alginate-chitosan beads containing celecoxib solubilized in a self-emulsifying phase. Scientifica 2016, 2016, 5062706. [Google Scholar] [CrossRef] [Green Version]
  25. Layek, B.; Mandal, S. Natural polysaccharides for controlled delivery of oral therapeutics: A recent update. Carbohydr. Polym. 2020, 230, 115617. [Google Scholar] [CrossRef]
  26. Mahmoudi, Z.; Mohammadnejad, J.; Razavi Bazaz, S.; Abouei Mehrizi, A.; Saidijam, M.; Dinarvand, R.; Ebrahimi Warkiani, M.; Soleimani, M. Promoted chondrogenesis of hMCSs with controlled release of TGF-β3 via microfluidics synthesized alginate nanogels. Carbohydr. Polym. 2020, 229, 115551. [Google Scholar] [CrossRef] [PubMed]
  27. Busatto, C.A.; Taverna, M.E.; Lescano, M.R.; Zalazar, C.; Estenoz, D.A. Preparation and Characterization of Lignin Microparticles-in-Alginate Beads for Atrazine Controlled Release. J. Polym. Environ. 2019, 27, 2831–2841. [Google Scholar] [CrossRef]
  28. Hu, M.; Zheng, G.; Zhao, D.; Yu, W. Characterization of the structure and diffusion behavior of calcium alginate gel beads. J. Appl. Polym. Sci. 2020, 137, 48923. [Google Scholar] [CrossRef]
  29. Machado, J.P.E.; de Freitas, R.A.; Wypych, F. Layered clay minerals, synthetic layered double hydroxides and hydroxide salts applied as pickering emulsifiers. Appl. Clay Sci. 2019, 169, 10–20. [Google Scholar] [CrossRef]
  30. Varajāo, A.F.D.C.; Gilkes, R.J.; Hart, R.D. The Relationships between Kaolinite Crystal Properties and the Origin of Materials for a Brazilian Kaolin Deposit. Clays Clay Miner. 2001, 49, 44–59. [Google Scholar] [CrossRef]
  31. Pasetti, M.F.; Simon, J.K.; Sztein, M.B.; Levine, M.M. Immunology of gut mucosal vaccines. Immunol. Rev. 2011, 239, 125–148. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Rolhion, N.; Chassaing, B. When pathogenic bacteria meet the intestinal microbiota. Philos. Trans. R. Soc. B Biol. Sci. 2016, 371, 20150504. [Google Scholar] [CrossRef] [Green Version]
  33. Baird, A.W.; Campion, D.P.; O’Brien, L.; Brayden, D.J. Oral delivery of pathogens from the intestine to the nervous system. J. Drug Target. 2004, 12, 71–78. [Google Scholar] [CrossRef]
  34. Mikula, K.; Skrzypczak, D.; Ligas, B.; Witek-Krowiak, A. Preparation of hydrogel composites using Ca2+ and Cu2+ ions as crosslinking agents. Sn Appl. Sci. 2019, 1, 643. [Google Scholar] [CrossRef] [Green Version]
  35. Agulhon, P.; Robitzer, M.; David, L.; Quignard, F. Structural Regime Identification in Ionotropic Alginate Gels: Influence of the Cation Nature and Alginate Structure. Biomacromolecules 2012, 13, 215–220. [Google Scholar] [CrossRef]
  36. Muller, M.V.G.; Petry, A.; Vianna, L.P.; Breier, A.C.; Michelin-Tirelli, K.; Pires, R.F.; Trindade, V.M.T.; Coelho, J.C. Quantification of glucosylceramide in plasma of Gaucher disease patients. Braz. J. Pharm. Sci. 2010, 46, 643–649. [Google Scholar] [CrossRef] [Green Version]
  37. Cao, Y.; Wei, X.; Cai, P.; Huang, Q.; Rong, X.; Liang, W. Preferential adsorption of extracellular polymeric substances from bacteria on clay minerals and iron oxide. Colloids Surf. B Biointerfaces 2011, 83, 122–127. [Google Scholar] [CrossRef] [PubMed]
  38. Labille, J.; Thomas, F.; Milas, M.; Vanhaverbeke, C. Flocculation of colloidal clay by bacterial polysaccharides: Effect of macromolecule charge and structure. J. Colloid Interface Sci. 2005, 284, 149–156. [Google Scholar] [CrossRef] [PubMed]
  39. Surya, R.; Mullassery, M.D.; Fernandez, N.B.; Thomas, D. Synthesis and characterization of a clay-alginate nanocomposite for the controlled release of 5-Flurouracil. J. Sci. Adv. Mater. Devices 2019, 4, 432–441. [Google Scholar] [CrossRef]
  40. Zhang, J.-P.; Wang, Q.; Xie, X.-L.; Li, X.; Wang, A.-Q. Preparation and swelling properties of pH-sensitive sodium alginate/layered double hydroxides hybrid beads for controlled release of diclofenac sodium. J. Biomed. Mater. Res. Part. B Appl. Biomater. 2010, 92B, 205–214. [Google Scholar] [CrossRef]
  41. Yuan, J.; Pruett, R.J. Zeta potential and related properties of kaolin clays from Georgia. Min. Metall. Explor. 1998, 15, 50–52. [Google Scholar] [CrossRef]
  42. Li, Y.; Hou, W.-G.; Zhu, W.-Q. Adsorption of cationic starch on aluminum magnesium hydrotalcite-like compound. Colloids Surf. A Physicochem. Eng. Asp. 2007, 303, 166–172. [Google Scholar] [CrossRef]
  43. Braga, P.C.; Bovio, C.; Culici, M.; Dal Sasso, M. Flow cytometric assessment of susceptibilities of Streptococcus pyogenes to erythromycin and rokitamycin. Antimicrob. Agents Chemother. 2003, 47, 408–412. [Google Scholar] [CrossRef] [Green Version]
  44. Bajpai, S.; Sharma, S. Investigation of swelling/degradation behaviour of alginate beads crosslinked with Ca2+ and Ba2+ ions. React. Funct. Polym. 2004, 59, 129–140. [Google Scholar] [CrossRef]
  45. Hussien, R.A.; Donia, A.M.; Atia, A.A.; El-Sedfy, O.F.; El-Hamid, A.R.A.; Rashad, R.T. Studying some hydro-physical properties of two soils amended with kaolinite-modified cross-linked poly-acrylamides. CATENA 2012, 92, 172–178. [Google Scholar] [CrossRef]
  46. Win, P.; Lin, C.-G.; Long, Y.; Chen, W.; Chen, G.; Song, Y.-F. Covalently cross-linked layered double hydroxide nanocomposite hydrogels with ultrahigh water content and excellent mechanical properties. Chem. Eng. J. 2018, 335, 409–415. [Google Scholar] [CrossRef]
  47. Draget, K.; Bræk, G.S.; Smidsrød, O. Alginic acid gels: The effect of alginate chemical composition and molecular weight. Carbohydr. Polym. 1994, 25, 31–38. [Google Scholar] [CrossRef]
Applsci 10 02679 g001 550
Figure 1. Hydrodynamic radius of formalin- killed cells (FKC) (solid line), clays (dashed line), and FKC + clay mixture (circle and line) with respect to (a) layered double hydroxide (LDH) and (b) kaolinite (KA).
Figure 1. Hydrodynamic radius of formalin- killed cells (FKC) (solid line), clays (dashed line), and FKC + clay mixture (circle and line) with respect to (a) layered double hydroxide (LDH) and (b) kaolinite (KA).
Applsci 10 02679 g001
Applsci 10 02679 g002 550
Figure 2. Scanning electron microscope images of FKC, clays, and FKC + clay mixtures. (a) FKC; (b) LDH; (c) FKC + LDH; (d) KA; and (e) FKC + KA.
Figure 2. Scanning electron microscope images of FKC, clays, and FKC + clay mixtures. (a) FKC; (b) LDH; (c) FKC + LDH; (d) KA; and (e) FKC + KA.
Applsci 10 02679 g002
Applsci 10 02679 g003 550
Figure 3. Scanning electron microscopy images of the inside of dried alginate bead encapsulating FKC. The dashed circles indicated the location of the FKCs on the alginate network. (a) half-crushed dried alginate bead; (b) assemblies of FKC; (c) magnified images of FKC.
Figure 3. Scanning electron microscopy images of the inside of dried alginate bead encapsulating FKC. The dashed circles indicated the location of the FKCs on the alginate network. (a) half-crushed dried alginate bead; (b) assemblies of FKC; (c) magnified images of FKC.
Applsci 10 02679 g003
Applsci 10 02679 g004 550
Figure 4. Fluorescence and confocal microscopy of wet alginate beads containing FITC-labelled FKC. The specimens were prepared by simply compressing wet alginate beads between slide glasses. EX: 492 nm, EM: 520 nm.
Figure 4. Fluorescence and confocal microscopy of wet alginate beads containing FITC-labelled FKC. The specimens were prepared by simply compressing wet alginate beads between slide glasses. EX: 492 nm, EM: 520 nm.
Applsci 10 02679 g004
Applsci 10 02679 g005 550
Figure 5. Scanning electron microscopy images for the inside of the dried alginate bead encapsulating (a) FKC + LDH and (b) FKC + KA mixture. The dashed circles indicated the location of the FKC + LDH agglomerates. To trace the FKC inside the alginate bead, the cells were labelled with FITC. In order to check that this FITC-FKC could be utilized in the quantification of FKC, the relationship between the FKC concentration and fluorescence intensity was fitted into a linear equation.
Figure 5. Scanning electron microscopy images for the inside of the dried alginate bead encapsulating (a) FKC + LDH and (b) FKC + KA mixture. The dashed circles indicated the location of the FKC + LDH agglomerates. To trace the FKC inside the alginate bead, the cells were labelled with FITC. In order to check that this FITC-FKC could be utilized in the quantification of FKC, the relationship between the FKC concentration and fluorescence intensity was fitted into a linear equation.
Applsci 10 02679 g005
Applsci 10 02679 g006 550
Figure 6. Linear regression result for FITC- labelled FKC concentration (number/mL) and fluorescence intensity at 520 nm.
Figure 6. Linear regression result for FITC- labelled FKC concentration (number/mL) and fluorescence intensity at 520 nm.
Applsci 10 02679 g006
Table
Table 1. Dynamic light scattering results of FKC, clays, and FKC + clay mixture.
Table 1. Dynamic light scattering results of FKC, clays, and FKC + clay mixture.
ParametersFKCClayFKC + Clay Mixture
LDHKALDHKA
Z-average (nm)1571244.3114628471176
PdI a)0.4950.0690.4030.8990.396
a) PDI: polydispersity index equals to (standard deviation)2/average.
Table
Table 2. Encapsulation efficiency and loading capacity of FKC in alginate beads with and without clays nanoparticles.
Table 2. Encapsulation efficiency and loading capacity of FKC in alginate beads with and without clays nanoparticles.
-Loading ParameterFKC AloneFKC + LDH MixtureFKC + KA Mixture
Encapsulation efficiency (%)37.75 ± 12.1028.06 ± 18.1667.22 ± 11.74
Loading capacity (cells/g)1.89 × 105 ± 6.05 × 1041.28 × 105 ± 8.25 × 1043.06 × 105 ± 5.33 × 104
Table
Table 3. Cumulative FKC release from alginate beads at various media.
Table 3. Cumulative FKC release from alginate beads at various media.
Release ConditionsFKC Alone (%)FKC + LDH Mixture (%)FKC + KA Mixture (%)
DW at 2 h (%)1.96 ± 0.79N.D.N.D.
Gastric simulation at 4 h (%)N.D.N.D.N.D.
Intestinal simulation at 2 h (%)17.06 ± 1.6061.12 ± 12.6022.26 ± 17.27

Share and Cite

MDPI and ACS Style

Lee, S.-B.; Kim, J.-Y.; Kim, K.; Ahn, K.-J.; Kim, T.-i.; Oh, J.-M. Encapsulation and Release Control of Fish Pathogen Utilizing Cross-Linked Alginate Networks and Clay Nanoparticles for Use with a Potential Oral Vaccination. Appl. Sci. 2020, 10, 2679. https://doi.org/10.3390/app10082679

AMA Style

Lee S-B, Kim J-Y, Kim K, Ahn K-J, Kim T-i, Oh J-M. Encapsulation and Release Control of Fish Pathogen Utilizing Cross-Linked Alginate Networks and Clay Nanoparticles for Use with a Potential Oral Vaccination. Applied Sciences. 2020; 10(8):2679. https://doi.org/10.3390/app10082679

Chicago/Turabian Style

Lee, Su-Bin, Ji-Yeon Kim, Kyusik Kim, Kyoung-Jin Ahn, Tae-il Kim, and Jae-Min Oh. 2020. "Encapsulation and Release Control of Fish Pathogen Utilizing Cross-Linked Alginate Networks and Clay Nanoparticles for Use with a Potential Oral Vaccination" Applied Sciences 10, no. 8: 2679. https://doi.org/10.3390/app10082679

APA Style

Lee, S. -B., Kim, J. -Y., Kim, K., Ahn, K. -J., Kim, T. -i., & Oh, J. -M. (2020). Encapsulation and Release Control of Fish Pathogen Utilizing Cross-Linked Alginate Networks and Clay Nanoparticles for Use with a Potential Oral Vaccination. Applied Sciences, 10(8), 2679. https://doi.org/10.3390/app10082679

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop