1. Introduction
Dental photography serves as an effective way for documentation, patient communication and education, and treatment planning in modern dental practice [
1,
2,
3,
4]. While conventional dental photography captures images in high spatial resolution and in color, it is limited to providing 2D information. In certain cases, without a comprehensive 3D representation, specific dental conditions may necessitate the capture of multiple photos from different angles, thus complicating diagnosis and treatment. In addition to dental photography, 3D dental imaging is also available [
5,
6]. Applications with 3D dental imaging tend to be more specialized areas, such as orthodontics and dental implants. Within orthodontics, 3D scanning has become a standard procedure for assessment, treatment planning, and the fabrication of dental aligners [
7,
8]. In dental implant surgeries, the use of 3D imaging improves surgical outcomes by reducing potential complications and enabling more precise implant placement [
9,
10,
11].
Several imaging techniques have been developed for dental imaging, including structured light and stereo vision [
6,
12]. Structured light uses active illumination to project patterns, such as coded stripes and speckles, onto the surface [
13,
14]. Due to the surface elevation (3D shape), these patterns are deformed, and this deformation is analyzed to reconstruct the 3D shape of the scene. On the other hand, stereo vision utilizes two cameras to image the scene. Due to parallax, identical features are captured at different locations on the two sensors [
6,
15]. This positional difference on the sensor is known as disparity, which is utilized to determine the depth [
16,
17]. While these imaging techniques provide excellent results for orthodontics and implant surgeries, the cost and complexity of the technology probably do not justify adopting such methods for other general imaging needs in dental care.
In recent years, close-range photogrammetry has emerged as an appealing alternative to the existing 3D imaging technologies [
18,
19,
20]. Close-range photogrammetry does not require active illumination and only needs a single camera. It can generate full-color 3D models from a set of images or a video. Photogrammetry estimates the 3D positions of points by triangulating common features found in multiple images [
21]. Due to its straightforward system setup and ease of use, close-range photogrammetry-based dental imaging (CPDI) has shown great potential in dental practice and has undergone extensive exploration [
22,
23,
24]. Notably, the majority of CPDI has been performed on plaster cast dental models typically with a professional digital camera. Data on in vivo CDPI are limited, possibly due to the lack of appropriate intraoral imaging devices. In this study, we focused on (1) demonstrating and evaluating the capability of CDPI to generate 3D dental models in an in vivo setting by (2) developing and testing a compact and cost-effective intraoral camera. An affordable camera would lower the barrier for adopting this technology, potentially improving the quality of patient care.
2. Materials and Methods
2.1. Imaging Device Development
We developed two low-cost imaging setups based on a Raspberry Pi 4 (RPi) single-board computer (1) to conduct parametric studies of the effects of depth-of-field and surface reflection on reconstruction and (2) to evaluate the performance of in vivo teeth imaging. A typical configuration of the imaging system is shown in
Figure 1A. For parametric studies, we adopted an RPi-compatible camera module featuring an M12 mount (UR-261, Arducam, Nanjing, China). This camera is later referred to as the M12 camera. Although this camera type is typically large and not ideal for in vivo imaging, its standard M12 mount allows for the use of a variety of M12 camera lenses. Illumination was provided by two 1 W white LEDs positioned on both sides of the camera. To reduce stray light, two flexible light-blocking sheets were placed between the LED and the camera. A more compact setup was developed for in vivo imaging, which comprised a compact Rpi imaging module (Module V2, Raspberry Pi, Cambridge, UK) with a fixed lens and a single 1 W white LED for illumination. The imaging setup was assembled into a toothbrush-like structure with a compact imaging head (2 × 1 × 1 cm, L × W × D). This camera is later referred to as the intraoral camera. Both imaging setups utilized the same 8 MP imaging sensor (IMX 219, SONY, Tokyo, Japan). Circular polarizers could be added to both imaging modules to reduce specular reflection, given that both illumination and imaging share a common optical path [
25].
Figure 1B illustrates the principle of reflection suppression with a circular polarizer. A circular polarizer consists of a linear polarizer and a quarter-wave plate. The quarter-wave plate introduces a 90 degree phase shift to the polarized light [
26]. In the incident direction, the 90 degree phase shift turns the linear polarized light into circular polarized light. Upon reflection from the teeth surface, the handedness of the circular polarized light is reversed. The 90 degree phase shift from the quarter-wave plate converts the reflected light into linear polarized light with the polarization direction 90 degrees apart from the original one, which is blocked by the linear polarizer. A custom Python script was developed to adjust imaging parameters and capture images. To ensure high-quality and consistent images, we used a fixed white balance setting and an ISO value of 200. Throughout imaging, an auto-exposure setting was used with a fixed exposure compensation value to reduce the risk of overexposure.
2.2. Parametric Studies of the Effects of Depth-of-Field and Surface Reflection on Reconstruction Quality
Intraoral imaging poses several challenges, such as a short working distance and surface reflection, that can potentially impact the quality of the 3D reconstruction. The parametric study was designed to understand these challenges and optimize imaging parameters. The working distance of intraoral imaging needs to be short to accommodate the limited space in the oral cavity. A shorter working distance will result in a shallower depth-of-field (DoF), which means a smaller portion of the object will remain in focus. A lens with a smaller aperture can be used to alleviate this issue at the expense of lower light collection efficiency, potentially affecting the image quality. To understand how DoF affects the reconstruction quality, we imaged a dental model using the M12 camera with two lenses sharing the same focal length but differing in aperture sizes. The F/2.8 lens (89342, Edmund Optics, Barrington, NJ, USA) features a larger aperture, producing a shallower DoF, while the F/5.6 lens (89343, Edmund Optics, Barrington, NJ, USA) has a smaller aperture, resulting in a more moderate DoF. The enamel reflects light strongly under illumination, often leading to image saturation and subsequent loss of information. To study whether the suppression of reflected light can improve the reconstruction quality, we imaged the same dental model both with and without a circular polarizer (88-085, Edmund Optics, Barrington, NJ, USA). A circular polarizer effectively reduces reflection when the illumination and imaging share a common optical path. For each testing condition, the dental model was imaged three times.
2.3. In Vivo Teeth Imaging
The anterior sections of both the upper and lower arches of a volunteer were imaged using an intraoral camera. Prior to imaging, the volunteer brushed the teeth, and excess fluids on the teeth were wiped with gauze. The intraoral camera was positioned approximately 2 cm in front of the teeth and followed the curvature of the arch during image acquisition. Each portion of the arch was imaged from multiple angles with a good focus, which was visually confirmed before taking the picture. It is worth noting that the imaging was performed by the same volunteer without the need for additional assistance, further demonstrating the user-friendly nature of this imaging technique. Due to the specular reflections from the moist teeth surface, a circular polarizer (88-085, Edmund Optics, Barrington, NJ, USA) was used throughout the in vivo teeth imaging to mitigate these reflections. Images were subsequently reconstructed for visualization.
2.4. Photogrammetry-Based 3D Reconstruction
Photogrammetry, as a versatile 3D reconstruction technique, has been widely used in remote sensing and aerial applications [
27,
28,
29], archaeology [
30], and augmented/virtual reality applications [
31]. Detailed descriptions of the working principles of photogrammetry are beyond the scope of this study and have been well documented elsewhere [
20,
30]. Briefly, photogrammetry involves capturing a set of images of the subject from various directions. Common features of the subject within a subset of the images are detected and extracted using certain techniques, such as the scale-invariant feature transformation (SIFT) method [
32]. These detected features are then used to estimate the 3D coordinates of the features based on triangulation [
21]. The point cloud of the subject is generated after the 3D coordinates of all features have been estimated. This point cloud is further processed, meshed, and textured to create a faithful 3D representation of the subject.
In practice, photogrammetry requires that images have sufficient overlap to facilitate feature detection, registration, and reconstruction [
20]. In both parametric studies and in vivo imaging, we maintained approximately 50% image overlap for all acquired images. As illustrated in
Figure 2A, we followed specific imaging paths to ensure comprehensive coverage from various angles. The camera was positioned sequentially at approximately 45, 90, and 135 degrees towards the dental model (indicated by blue arrows in
Figure 2A). At each angle, images were taken while following the curvature of the arch, as depicted by the red arrow.
Figure 2B visualizes the camera positions and angles during a CPDI session. For the parametric study, 30–40 images were acquired to reconstruct the full arch, while in in vivo imaging, 20–30 images were acquired.
2.5. Reconstruction and Trueness Analysis
Following image acquisition, we reconstructed 3D models using PhotoCatch (EOS Innovations LLC) running on a M1 Mac Mini. The reconstructed 3D model was exported in .obj format, which contains both the 3D structure mesh and color texture of the dental model. The 3D model was imported into CloudCompare (V 2.12.4), an open-source point processing software, for visualization and trueness analysis.
To facilitate the trueness analysis, the teeth portion of the reconstructed dental model was manually segmented out in CloudCompare. Due to the lack of a physical dimension of the 3D model generated with the photogrammetry technique, the initial alignment between the 3D teeth model and the ground truth model was performed manually. Specifically, the 3D teeth model was manually scaled to the approximate size of the ground truth mode. Following scaling, the orientation of the 3D teeth model was adjusted to match the orientation of the ground truth model. Once a good manual alignment between these two models was achieved, we utilized the registration function provided in CloudCompare for a more precise registration. During the registration process, we designated the ground truth as the target and allowed the 3D teeth model to be further scaled to minimize the overall error between the two models. Unlike CPDI, the ground truth model was generated on an absolute scale using a commercial intraoral scanner. Local deviations, measured in millimeters, between the ground truth model and the reconstructed model were quantified to indicate the reconstruction errors. As the ground truth for the in vivo dental model was not available, trueness analysis was not conducted for in vivo imaging.
4. Discussion
CPDI aims to fill the gap between conventional 2D dental photography and high-precision 3D dental scanning and to improve the quality of patient care. Similar to 2D dental photography, CPDI provides rich and accurate color information in high spatial resolution, facilitating detailed close-up examinations. Furthermore, this color information is integrated into the 3D dental model, allowing it to be visualized and evaluated from arbitrary angles with the awareness of its 3D geometry. It is worth noting that while CPDI-derived models may not achieve the precision of those obtained with commercial 3D dental scanners, they faithfully reconstruct the most prominent morphological and structural dental features of the dental arch and individual teeth. This compromise is justified as CPDI is not intended for orthodontics and dental surgeries where a highly precise dental scan is critical to ensure optimal treatment outcomes. The low-cost nature of CPDI could lower the barrier for adopting this technology in clinical dental practice, potentially improving the quality of patient care.
The parametric study suggested that while the surface reflection is undesirable in an image, it appears not to significantly affect the reconstruction results. This is potentially because the same region exhibiting high reflection in one image may not do so in other images due to the change in imaging angles. With the information from multiple images, those regions can still be reconstructed. Suppressing reflection seems to deteriorate the reconstruction quality, possibly due to the loss of surface contrast, as discussed in
Section 3.2. In the in vivo imaging, we had to use a polarizer to mitigate excessive reflection. The reconstruction quality, however, was better than the result presented in the parametric study with a polarizer. While the exact cause warrants further investigation, we speculate that sufficient surface contrast of the teeth was retained for this specific volunteer, even with a polarizer. This also suggests that whether or not to use a polarizer for surface reflection suppression needs to be tested and evaluated for each imaging task. A lens with a smaller aperture (offering a larger DoF) indeed improved the overall reconstruction quality, but only by a small margin, suggesting that such a lens is still preferable under adequate illumination. If illumination intensity and potential motion blur become a point of concern, opting for a lens with a larger aperture would still yield satisfactory results.
Through in vivo 3D imaging, we demonstrated accurate reconstructions of both the incisor and canine teeth. However, it proved challenging to reconstruct the pre-molar and molar teeth due to certain limitations within the current system. These limitations, in turn, provide valuable insights for guiding future improvements. Despite the overall compact size of the intraoral camera, the working distance is still too large to comfortably image the teeth positioned deep within the oral cavity. A potential improvement could be achieved by introducing a variable extension tube to the imaging system. This adaptation will allow easy adjustment of the working distance and FoV to facilitate imaging different portions of the arch, such as a larger working distance (larger FoV) for imaging the anterior arch and a shorter working distance (smaller FoV) for imaging the posterior arch. Furthermore, a warm and moist intraoral environment often leads to rapid condensation that complicates imaging. To counteract this issue, an anti-fogging coating on the lens or a heating mechanism to keep the imager warm might be necessary. Finally, the camera needs to move around the teeth in order to capture images from multiple angles. In the case of molar teeth, moving from the anterior to the posterior of the teeth will flip the imaged field vertically. Such a sudden change can disorient the operator and interrupt the imaging process. This issue could be alleviated by incorporating an inertial measurement unit (IMU) within the camera system to estimate the camera orientation and adjust the image accordingly.
The reconstruction algorithm of CPDI relies on subject contrast to triangulate and recover shape. Healthy teeth unfortunately typically lack pronounced visual contrast, which could partly explain the reconstruction errors shown in our studies, particularly on the smooth teeth surface. Surface contrast is determined by the local absorption and scattering properties, which are wavelength dependent. It would be valuable to perform a multispectral analysis of the intrinsic sources of contrast [
34], and identify the most effective wavelengths in red, green, and blue spectral bands to maximize the contrast. Another inherent limitation of photogrammetry lies in its inability to recover the absolute physical scale. If a dimensional measurement is required with CPDI, a calibrator with known dimensions can be placed within the scene to serve as a reference [
24].
The low-cost nature and ease of use of CPDI will potentially benefit patient care both within dental clinics and home settings. 3D full-color dental models have the potential to replace conventional dental images and augment the text-based descriptions in patient charts. CPDI could also reduce the frequency of dental visits and overall costs by enabling at-home applications [
35]. For instance, dental visits after orthodontic procedures could be replaced by providing the 3D dental model generated at home to the dental care providers for progress evaluation. Similarly, a 3D dental model could aid dental examination during remote dental visits.