1. Introduction
In recent times, material jetting processes have gained increasing importance, particularly in the field of three-dimensional bioprinting and bioassembly [
1,
2,
3,
4,
5,
6].
Cellular aggregates, such as spheroids and organoids, can serve as building blocks for organ-printing processes by dispensing distinct amounts of these aggregates into hydrogel structures [
7]. Furthermore, the three-dimensional structure of spheroids and organoids can be used to replicate metabolic tissue properties in a realistic manner, thus enabling the testing of their reaction against drugs [
2,
3,
4,
5,
6]. In all of the previously mentioned cases, distinct amounts of cellular aggregates must be deposited, which is predominantly achieved through material jetting processes [
2,
4,
5,
6].
The process of material jetting describes the deposition of build materials by forming droplets and depositing these droplets on distinct target spots [
8]. The generation of droplets is typically achieved through the use of piezoelectric or thermal ink jetting [
1,
2,
4,
8]. Hereby, thermal expansion or piezoelectric displacement of a plunger generates displacement pressure pulses which eject droplets [
5,
8]. The integration of automatic sorting and selection of individual cellular aggregates, such as spheroids, contained within the build material facilitates the handling of these systems, thereby increasing throughput and significantly reducing processing times [
1,
2,
4]. The aforementioned process is alternatively referred to as the Drop-on-Demand approach, as described in [
2,
4].
For the deposition of distinct amounts of cellular aggregates, reliable detection of dispensed aggregates is required. In most cases, a label-free approach for the detection of theses cellular aggregates is desired, as labels can influence aggregates and sequential experiments carried out on these aggregates [
1,
2,
3,
4,
6]. To ensure a label-free detection approach, the number of dispensed spheroids is usually tracked by monitoring the nozzle region of the dispenser, predicting the amount of aggregates dispensed with the next droplet [
2,
3,
4,
7]. The systems of Zieger et al., as well as the system of Parent et al., demonstrated high detection accuracies within the capillary for spheroids in the size range of 50 μm up to 200 μm [
4,
7]. However, these methods depend on the optical properties of the observed nozzle region or dispense head, which limits the possibilities of applied dispensers [
4,
6]. Additionally, the opaqueness of the meniscus region of the dispensing nozzle can obscure a particle, hence making impossible the detection of whether the particle has been dispensed or is hidden in the meniscus [
3,
4]. Therefore, we propose the detection of cellular aggregates in already ejected, free-falling droplets, enhancing the independence of the detection process from dispense head geometry and materials.
So far, further solutions for characterization of free-falling droplets have mainly focused on the size and velocity of the droplets [
9,
10]. Due to the spherical lens behavior of droplets, which leads to the focusing of incident light towards the center of the droplet, imaging of the contained substances within the droplets becomes an issue [
11,
12]. Resulting images of droplets that are illuminated by only one light source show dark edges due to the focusing of the light [
2,
12].
Approaches to detect single cells as small as 25 μm within droplets by characterizing the velocity or deceleration of droplets have already been demonstrated by Huang et al. [
13]. By applying a machine learning algorithm to the generated data, detection accuracies between 77% and 82% were achieved [
13]. Nevertheless, real-time detection of particles within the droplets is not feasible using this concept due to the high computational effort required. Furthermore, the detection accuracies are considerably inferior to those of the concepts which detect particles within the capillary [
4,
7,
13].
In contrast to direct imaging approaches and the characterization of droplets, Heinisch investigated content-dependent glare point changes on the droplet surface [
14]. With that, they were able to detect the position and size of enclosed air bubbles in trapped droplets. Here, our goal is to use glare point changes on the image of the free-falling droplet to detect enclosed spheroids for a more precise and stable determination of the number of spheroids dispensed.
To optically detect particles inside a spherical object, it is necessary to understand the propagation of light through a spherical object.
Light-scattering properties on droplets or translucent spherical particles are described by the Gustav Mie theory [
15]. For spherical particles without an inclusion, this theory can be easily applied and is already the basis of many droplet- and particle-based measurement techniques [
14,
15]. In contrast, for spherical objects or droplets with an inclusion, such as air bubbles or cellular aggregates, this theory must be extended to a much more complex form, which leads to increased computational effort [
14,
15]. To investigate light propagation for the illumination concepts in this work, we do not follow this approach for these reasons. Instead, the ray optic theory is sufficient for a basic understanding of light propagation. The results of this approach for illuminating a droplet with different wavelengths can be seen in
Figure 1.
From the perspective of geometrical ray optics, the surface of the droplet causes refraction of the incident beam that strikes the droplet surface. The ray enters a medium with a higher refractive index than the ambient medium, causing refraction towards the centre of the droplet. Subsequently, the light is partially reflected at the phase transitions from an optically denser medium to an optically thinner medium (ambient air) on the back of the droplet. Total reflection can be achieved, contingent on the medium of the droplet and the angle of incidence.
The reflected light beam is directed back towards the front of the droplet. This phenomenon allows the observation of characteristic glare points for empty droplets, which are dependent on the aforementioned variables.
Enclosed particles cause changes in the described light propagation due to absorption and scattering effects [
14]. For cellular aggregates like spheroids, organoids, or microtissue, light penetration is limited by the different nonhomogeneous refractive index distributions which result from the different components of the cells. Changes in the speed and angle of light propagation are induced and promote the scattering and absorption of the light [
17].
These alterations in light propagation, which are contingent upon the dimensions, composition, and scattering characteristics of the particles within the droplet, are anticipated to manifest as the emergence or absence of glare points or alterations in the dimensions and brightness of the preexisting glare points on the droplet surface. However, this phenomenon occurs only when the light interacts with the particle during its propagation through the droplet. The use of specially designed droplet illuminations, which either project glare points onto the droplet surface or reduce the optical effects caused by the lens behavior of the droplet due to light entering the droplet in a diffuse way from every direction, will be employed in the following section to investigate the efficacy of different approaches to particle detection in free-falling droplets.
3. Results and Discussion
3.1. Wavelength and Number of Light Sources
The initial investigation focused on the characteristic glare points of empty water droplets, employing anterior and posterior light modalities. The droplets were illuminated by evenly distributed LEDs around the dispense axis, resulting in bright glare points, using either anterior or anterior-posterior illumination for all wavelengths (see
Figure 9). The differences between illumination concepts are primarily reflected in the number of glare points on the droplet surface and the varying brightness levels of these points, which depend on the wavelength and number of light sources used. The greater number of glare points on droplets illuminated by six LEDs suggested that the information gained from these points might be more extensive than that gained from droplets illuminated by four LEDs. Consequently, further experiments were only conducted with the six LED arrangement.
For subsequent experiments, MCF-7 spheroids with diameters ranging from 80 to 130 μm were cultivated, harvested, and resuspended in phosphate-buffered saline (PBS) for dispensing.
Five image series, each containing 200 images, were captured using the spheroid PBS suspension for the tests. This process was repeated for each wavelength. The droplets were dispensed onto a microscope slide, and the number of dispensed spheroids was counted under the microscope after each run. An experienced researcher manually evaluated the images and counted the number of images with potential detections.
Figure 10 shows sample images of droplets. New bright points or large points with lower intensity appear as soon as a spheroid is encapsulated in the droplet. This effect is most pronounced for a wavelength of 405 nm. The reason for this effect can be seen in the dispersion effect caused by the different wavelengths. In the case of 405 nm, the light is refracted to a higher degree than light with a wavelength of 635 nm [
22]. Consequently, the 405 nm light propagates on different path within the droplet compared to light with a higher wavelength. Hereby, the light hits the backside of the droplet at a sharper angle. This causes a reflection at a sharper angle with respect to the center axis of the droplet (see
Figure 1), resulting in a higher intensity on the camera sensor as more light is guided towards the lens setup. This, in turn, produces brighter images of the glare points. As soon as a particle is within the path of light propagation within the droplet, the refraction caused by this particle is, furthermore, stronger due to the different refraction angle, resulting in more pronounced changes in the appearance of the glare point. In accordance with Snell’s law, which describes the change of the direction of the incident light when transferring from one medium to another, depending on the refractive index of the media, the same effect can be achieved by different droplet media, provided that the refractive indices of the droplet media differ to a significant extent.
In the next step, the number of images in which the researcher detected a spheroid within the droplet and the number of spheroids counted using the microscope were compared to calculate the manual detection accuracy according to Equation (1).
The detection accuracy of the version with violet light was more constant and on average higher (93.8%) than with red (59.2%) and blue (84.3%) light. This corroborates the observation that light of a shorter wavelength is deflected more strongly by particles with shorter wavelengths, resulting in greater alterations in the size and intensity of the glare point. The distribution of the accuracies can be seen in
Figure 11.
Based on the manual evaluation, it was decided to proceed with the illumination using six 405 nm LEDs for further experiments towards an automated process. Based on the best accuracies, this setup demonstrated the most promising results in terms of glare point clarity and difference between an empty droplet and a droplet containing a spheroid.
3.2. Code-Assisted Evaluation
Next, we evaluated the effect of different particle types, as mentioned in the
Section 2, on detection accuracy.
For each particle type, we took three series of 200 images each. These images were later analyzed on a separate PC using either the white pixel count, contour count, or droplet similarity algorithm. After each run, we counted the total number of particles ejected using the reference camera pointing at the microscope slide on which the droplets were ejected.
Table 1 displays the detection accuracies achieved by the three considered detection algorithms based on images captured with directed illumination using a six-LED arrangement and violet light.
All three algorithms showed a strong performance on particles larger than 53 μm. For particles smaller than 45 μm, glare point changes became less significant relative to the size of the glare points and were indistinguishable from noise effects. Slight position shifts of droplets within the captured images, for example due to trigger jitter, can cause noise. This leads to a constant, slight change in the glare point position due to the changing angle of incidence.
The droplet image similarity algorithm is highly dependent on identical images, as they are subtracted from one another. Minor variations in droplet appearance and, consequently, glare point shape, can result in increased score values that are comparable to those caused by small particle-induced changes in glare point size. It was not possible to distinguish these errors from actual detections of particles that were in a size range of 45–53 μm. Nevertheless, a comparison of the captured images with a reference image demonstrated significant potential for all other tested particles, particularly spheroids.
The CC algorithm is less prone to errors caused by droplet position changes, as it only counts the number of glare points. However, it may struggle to detect small particles only by the size change of a single glare point, as the glare point only changes in size instead of forming a new one. Additional parameters, such as the area of individual glare points, have shown potential to further increase detection accuracy.
In terms of the time required for image processing to evaluate the presence of particles within droplets, the WPC algorithm can deliver a result within 0.06 s per image. Due to the higher amount of processing steps required for the CC and DIS algorithms, processing times are four to five times higher. In order to achieve a real-time detection, it is necessary to further improve the programming language and the hardware configuration of the PC used to process the image.
Table 2 presents the results achieved by the white pixel count algorithm based on images captured with the diffuse illumination concept. The algorithm demonstrated an accuracy of greater than 95% for blue-colored beads with a size range of 45 μm to 106 μm. However, the algorithm was unable to detect spheroids and transparent beads, as illustrated in
Figure 12. The diffuse illumination resulted in diffuse light reflections indistinguishable from spheroids and transparent particles. Nevertheless, the evaluation algorithms indicated that the accuracy for light-absorbing colored particles was higher than that achieved with the directed light concept.
Figure 13 provides a side-by-side comparison of the achieved accuracies. The processing of glare point images, in particular with the DIS algorithm, is most effective for transparent particles and spheroids. This is due to the fact that pronounced changes in the glare point appearance can be easily distinguished from empty droplet glare point images.
The impact of the aforementioned errors is most pronounced for small opaque particles, as evidenced by the weakness of the algorithm that employs the comparison of images (DIS). The diffuse light setting is the most reliable for the detection of small opaque particles.
A satisfactory compromise is the evaluation of images by the WPC and CC algorithms, which exhibit lower but still high accuracies for the complete range of tested particles. Furthermore, the CC algorithm has the potential to be enhanced by calculating the area of individual contours and comparing them to empty droplet images or previously captured images. It is anticipated that further enhancements in the detection accuracy for small particles will be achieved through a reduction in the size of the glare points. Alternatively, it would be more effective to project more of these smaller, distinct glare points onto the droplets. The alterations compared to the characteristics of individual glare points become more pronounced, thereby facilitating their identification.
The differentiation of the tested particles was not within the scope of the tested concepts. The captured images nevertheless demonstrated a tendency for the glare points to exhibit a different appearance depending on particle size and type. This could be utilized in the future to sort different sizes or types of particles. A larger particle size, for example, results in a greater increase in the glare point surface. Changing refractive indices between particles and spheroids causes the same effects.
4. Conclusions
The concept study presented here demonstrates the possibility of determining the presence of a particle within a free-falling droplet using glare points projected onto its surface or diffuse illumination. This approach offers greater flexibility in the selection of materials for dispensing nozzles, while enabling the detection of particles or cellular aggregates that have left the dispenser without the need for special camera equipment such as high-speed cameras. Moreover, there is no need for fluorescent labels, which can influence the results of biological experiments, to detect these particles.
For glare point projection, six LED lights with a wavelength of 405 nm were arranged anteriorly and posteriorly towards the droplet. Particles within the droplets could be detected by means of characteristic glare point changes with respect to empty droplets. Algorithms were proposed and tested for image analysis. The computer-based detection accuracy achieved was comparable to that of manual image evaluation. For particles equal to or larger than 75 μm, high detection accuracies of >95% were achieved. However, the accuracy decreased for smaller particles. The reason for this can be observed in the comparison of glare points to the particle sizes. The introduction of smaller glare points and their respective changes, in addition to a higher number of glare points projected onto the droplets, represents a potential avenue for enhancement, with the objective of improving the detection of smaller particles. The diffuse white light concept accounts for the lens-like behavior of spherical droplets by uniform illumination from all directions, thus reducing dark drop edge areas. This allowed high detection accuracies of 96%, even for small opaque particles with a size of 45 μm. Nevertheless, with this concept, spheroids and transparent particles cannot be detected.
This study showed that droplet imaging techniques can enhance the quality of the Drop-on-Demand bioprinting process by accurately determining the dispensed quantity of particles. This method is independent of dispense head design and dispense parameters, as it detects particles within the already dispensed droplets. Enabling more freedom in the design of the dispense head allows for the easy exchange of nozzles or capillaries without the need to adapt the system for particle detection. This might be especially useful in terms of automated workflows, where printheads need to be changed according to the different print materials.
The combination of the two illumination concepts and their respective evaluation algorithms is anticipated to enhance particle detection for a wide range of particles, as each concept demonstrates strengths for different particle types and sizes. To achieve this, both illumination concepts could be integrated into a single housing. Filters can be applied to the imaging system to capture two images of the same droplet simultaneously and evaluate them using either WPC or DIS algorithms. It would be advantageous to consider approaches to making the entire detection system more compact in size and to reduce the free-falling path of the droplets in future research. One potential solution is the implementation of optical fibers instead of LEDs, which could help to reduce the size of the module and enhance the glare point appearance due to more focused light propagation towards the droplet. Nevertheless, this technique can assist in enhancing the quality of material jetting printing processes and, especially, the Drop-on-Demand bioprinting process by enabling a more precise determination of the actual dispensed amounts of particles, independent of the printhead material and without the necessity for fluorescent labels. Consequently, the present concept enhances the reliability of drug screening research outcomes on printed models.