DRHT: A Hybrid Mathematical Model for Accurate Ultrasound Probe Calibration and Efficient 3D Reconstruction

Abstract

The calibration of ultrasound probes is essential for three-dimensional ultrasound reconstruction and navigation. However, the existing calibration methods are often cumbersome and inadequate in accuracy. In this paper, a hybrid mathematical model, Dimensionality Reduction and Homography Transformation (DRHT), is proposed. The model characterizes the relationship between the image plane of ultrasound and projected calibration lines and homography transformation. The homography transformation, which can be estimated using the singular value decomposition method, reduces the dimensionality of the calibration data and could significantly accelerate the computation of image points in ultrasonic three-dimensional reconstruction. Experiments comparing the DRHT method with the PLUS library demonstrated that DRHT outperformed the PLUS algorithm in terms of accuracy (0.89 mm vs. 0.92 mm) and efficiency (268 ms vs. 761 ms). Furthermore, high-precision calibration can be achieved with only four images, which greatly simplifies the calibration process and enhances the feasibility of the clinical application of this model.

1. Introduction

Currently, there are dozens of commercially available spinal surgery robots that rely on intraoperative imaging equipment, such as CT, for the guidance of robot-assisted spinal surgery [1,2,3]. However, these approaches involve radiation and cannot provide real-time access to the actual subcutaneous instrument location. Ultrasound scanning, with its real-time capabilities and absence of radiation, holds promise when combined with robotic systems to acquire real-time guidance [4,5].

The calibration of the spatial relationship between two-dimensional ultrasound images and three-dimensional space is a prerequisite for ultrasound-guided robotic-assisted surgery [6]. Three mainstream calibration methods exist: the constraint method, the image-based method, and the tracking method. The constraint method involves restricting the ultrasound probe motion using known and accurate motion relationships to stitch together ultrasound sequence frames. The motion constraint method often employs robots [7,8] or high-precision motors [9] to control the probe motion or uses mechanical constraints to ensure the probe captures fixed points [10]. The tracking method does not constrain the motion of the ultrasound probe but measures the position of the probe using tracking devices, such as optical tracking, electromagnetic tracking, and accelerometers [11], completing pose transformation through pre-calibrated transformation matrices.

The above methods are relatively complex or strict in terms of the equipment required. In contrast, the image-based methods, such as deep-learning-based matching methods [12,13,14,15,16] and Hough-transform-based image matching methods [17], do not impose constraints on the motion of the ultrasound probe nor measure the orientation of the probe. The image-based methods employ connections within the ultrasound images for probe motion. Due to the severe distortion in ultrasound images, feature extraction for determining motion relationships is more complex compared to images in natural environments, resulting in insufficient final accuracy, and thus these methods are mostly in the research phase.

The spatial calibration between the ultrasound probe and the localization device (referred to as ultrasound probe calibration) is a prerequisite for the tracking method, which mainly includes hand–eye calibration and multi-point calibration. Hand–eye calibration calculates the ultrasound probe calibration matrix by solving the matrix equation AX=XB, typically requiring movement at multiple positions. In 2020, Cai et al. [18] proposed a three-dimensional freehand ultrasound spatial calibration method based on independent general motion, also solved by the equation AX=XB, with an average angular error of 2.09° and an average distance error of 4.90 mm. However, this method often requires complex motion control and multiple trials, which can be time consuming and may accumulate errors due to the sequential nature of the process. Iommi et al. [19] published an evaluation study on three-dimensional ultrasound guidance, comparing two different 3D ultrasound modes, 3D freehand and 3D sweeping mode, to identify the most suitable mode for clinical guidance applications. Here, 2D ultrasound calibration was performed using pointer-based and image-based methods, with a final registration error of about 1 mm for both, while the 3D hand–eye calibration method had an error of around 2 mm. Although these 2D methods achieve reasonable accuracy, they do not fully address the challenges of 3D spatial calibration required for advanced robotic surgical guidance.

The multi-point calibration method is most widely used, which employs models such as the visual probe model, cross-line model, N-line model, and 3D-printed model. The calibration is achieved by calculating the three-dimensional spatial position of the projected points in the model. Shen et al. [20] proposed a rapid and simple automatic 3D ultrasound probe calibration method based on a 3D-printed phantom and untracked markers, requiring only a single marker without the need for tracking it throughout the calibration process. The method adopts a calibration object with known dimensions, and the coordinates during the ultrasound calibration process are directly from the designed values. Its final calibration and three-dimensional reconstruction error were 0.83 mm and 1.26 mm. While this method simplifies the calibration procedure, it relies on the accuracy of the 3D-printed model and may not account for variations in real-world imaging conditions. In 2020, Wen et al. [21] designed a calibration model combined with a visual probe for the spatial calibration of ultrasound probes. An infrared camera is used for tracking the spatial position of the probe tip, and the pixel positions of the probe in the two-dimensional ultrasound image are simultaneously extracted. The spatial position of the probe tip was modeled via Gaussian distribution, and an iterative nearest neighbor optimization algorithm was used to estimate the spatial transformation of the two sets of points. The calibration can be completed within 5 min, with an error of less than 1 mm. However, this approach depends on the precision of the infrared camera and the accuracy of pixel position extraction, which can be affected by environmental factors and image quality. Meza et al. [22] proposed a low-cost multimodal medical imaging system with stripe projection contouring and three-dimensional freehand ultrasound. They use a binocular camera system and annular marker to track the handheld ultrasound device. In the ultrasound calibration model, they introduced two scale factors in the x and y directions and then calculated homogeneous matrices through the LM algorithm to complete the calibration. However, the use of multiple sensors and the complexity of the LM algorithm may increase the computational load and affect the real-time performance of the system. Rong et al. [23] proposed a geometric calibration method for a freehand ultrasound system based on electromagnetic tracking. In their calibration process, a multi-layer N-line model and a fully automatic image segmentation algorithm are employed, achieving an average calibration error of 1.42 mm. Although this method improves automation, the electromagnetic tracking may be susceptible to interference from metallic objects in the operating environment. In 2021, Meza [24] proposed a three-dimensional multimodal medical imaging system based on freehand ultrasound and structured light. Additionally, the pose of the ultrasound probe is optically estimated based on convolutional neural networks. Finally, a CT-like scanning effect is achieved, and the layer thickness precision reached 0.12 mm. However, the reliance on convolutional neural networks requires extensive training data and computational resources and the generalization ability of the model needs to be further validated across different devices and clinical settings. In 2023, Sahrmann et al. [9] proposed a repeatable three-dimensional ultrasound automatic measurement system for skeletal muscles that utilizes force sensors for position control and ensures good contact between the probe and the skin. The Z-line model was used for calibration, and the three-dimensional coordinate calculation method was the same as above, with a final test three-dimensional reconstruction accuracy error of 0.94 mm (0.23%) and volume error of 0.08 mL (0.23%). While this system achieves high precision, it requires precise force control mechanisms, increasing the complexity and cost of the setup. In 2024, Harindranath et al. [25] introduced a manual 3D ultrasound imaging technique aided by an inertial measurement unit (IMU). This method involves motion-constrained sector scanning, where the ultrasound probe and IMU are mounted on a bracket capable of rotating in a single degree of freedom. The calibration process involves 100 s of random 3D spatial sensor data from the rotating IMU and an accuracy of less than 2 mm is achieved. Although this approach introduces a novel sensing modality, the calibration process is relatively lengthy and the accuracy may be limited by the precision and stability of the IMU over time.

In this paper, a three-tier 24-point calibration fixture based on the N-line model and a novel ultrasound probe calibration method were proposed. This method, which adopts homography transformation to characterize the relationship between the reduced two-dimensional points and the correspondences in the ultrasound image, effectively eliminates the scale factor and thus avoids the accumulation of errors.

2. Proposed Method

2.1. Overview of Ultrasound Probe Calibration

The commonly used ultrasound probe calibration model, illustrated in Figure 1, includes three coordinate systems: the global coordinate system {world}, the coordinate system of the tracking device attached to the probe {probe}, and the image coordinate system {I}. In ultrasound images, the pixel coordinates $q\left(u, v\right)$ correspond to a global point Q(x, y, z), and the relationship between the two coordinates is:

$$\left[\begin{array}{c}x\\ y\\ z\\ 1\end{array}\right]=T_{world}^{probe}\cdot T_{probe}^I\cdot \left[\begin{array}{c}{s}_x\cdot u\\ s_y\cdot v\\ 0\\ 1\end{array}\right]$$

(1)

where $s_x$ and $s_y$ are the scale factors in the x and y directions and $T_{world}^{probe}$ and $T_{probe}^I$ are the homogeneous transformation matrix from {probe} to {world} and {I} to {probe}, respectively.

The above model neglects the local transformations of ultrasound images and only considers scale transformation. The purpose of ultrasound calibration is to determine the transformation matrix $T_{probe}^I$ and the scale factors $s_x$ and $s_y$. The introduction of $s_x$ and $s_y$ increases the computational complexity and thus some researchers [26] have initially calculated $s_x$ and $s_y$ using correspondences of 3D and 2D distances, then followed with the computation of the transformation matrix $T_{probe}^I$. However, the sequential approach may lead to the accumulation of errors.

Considering the characteristics of ultrasound imaging, the calibration points in the {probe} coordinate system form a plane in three-dimensional space. Therefore, the mapping relationship between image coordinates and three-dimensional coordinates can be accurately described by a homography matrix. In this paper, a three-tiered N-line ultrasound calibration model was designed and a novel calibration method for the ultrasound probe based on Dimensionality Reduction and Homography Transformation (referred to as DRHT) was proposed, as illustrated in Figure 2. In the calibration process, the spatial positions of both the ultrasound probe and the calibration model with visual markers can be tracked using a stereo camera. The N-line model is encoded in binary for line recognition. With the acquired ultrasound images and the constraints of the N-line model, the homography matrix can be computed from the projections of the calibration points, thereby completing the calibration process.

By employing the DRHT method, which initially reduces the three-dimensional points to two-dimensional image points, the computational complexity in the reconstruction phase is significantly reduced. It is only necessary to reconstruct the four vertices of the ultrasound image into three-dimensional points, after which the remaining points can be calculated through coordinate interpolation.

2.2. The Three-Layer N-Line Calibration Model

The traditional N-line model [27] consists of only 3 line segments, as depicted in Figure 3 by lines $\overline{P_0{P}_1}$, $\overline{P_1{P}_2}$, and $\overline{P_2{P}_3}$. Assume the intersections of the ultrasound imaging plane and N-line model are $Q_1$ and their projections in the ultrasound image are $q_1$ (i = 0, 1, 2). Based on the principle of similar triangles, the coordinates of point $Q_1$ can be directly calculated by Equation (2).

$${\displaystyle \frac{\left|\overrightarrow{q_1{q}_0}\right|}{\left|\overrightarrow{q_2{q}_0}\right|}}={\displaystyle \frac{\left|\overrightarrow{P_1{Q}_1}\right|}{\left|\overrightarrow{P_2{P}_0}\right|}}$$

(2)

This results in a low utilization rate of the calibration points of only 1/3. To increase the number of usable calibration points and the utilization rate, thereby improving calibration accuracy while reducing the requirement for the calibration images, this paper extends the N-line model to a novel three-tiered 24-line model, as shown in Figure 3. Within each single calibration plane, there are 8 calibration line segments (the solid lines). Thin lines denote “0” and thick lines denote “1”. From left to right, each column is respectively 000, 001, 010, ……, 111, corresponding to decimal numbers 0, 1, 2, ……, 8. This design aims to facilitate identification. As the ultrasound probe may not scan all points each time, distinguishing lines by their thickness and using a specific encoding method can ensure partial points can still be identified and sorted.

The dashed lines represent the intersection of the ultrasound imaging plane with the N-line model, with the intersection points labeled as $q_0~q_{23}$. When all intersection points $q_0~q_{23}$ are identified, a total of 18 points ($Q_1\sim Q_6$, $Q_9\sim Q_{14}$, $Q_{17}\sim Q_{22}$) can be directly calculated using the expanded N-line model, achieving a utilization rate of 3/4 for the calibration points. Excluding the points at the both ends that cannot be calculated, for each of the identified $q_1\sim q_6$, $q_9\sim q_{14}$ and $q_{17}\sim q_{22}$, the following relationship holds:

$${\displaystyle \frac{\left|\overrightarrow{q_i{q}_{i-1}}\right|}{\left|\overrightarrow{q_{i+1}{q}_{i-1}}\right|}}={\displaystyle \frac{\left|\overrightarrow{P_i{Q}_i}\right|}{\left|\overrightarrow{P_{i+1}{P}_{i-1}}\right|}},i=1,2,\dots ,22$$

(3)

In the above equation, ${\displaystyle \frac{\left|\overrightarrow{q_i{q}_{i-1}}\right|}{\left|\overrightarrow{q_{i+1}{q}_{i-1}}\right|}}$ represents the scale factor in three-dimensional space, which can be characterized by the pixel distances on the ultrasound image. This scale factor is crucial for establishing the proportional relationship between the actual physical dimensions and the corresponding pixel measurements within the imaging plane. With the proportional relationship, the three-dimensional coordinate $Q_i$ can be determined as:

$$Q_i=P_i+{\displaystyle \frac{\left|\overrightarrow{q_i{q}_{i-1}}\right|}{\left|\overrightarrow{q_{i+1}{q}_{i-1}}\right|}}\cdot (P_{i+1}-P_i)$$

(4)

As the imaging region of ultrasound is sector-shaped, some points of the N-line model, such as $q_0$ and $q_7$ in Figure 3, may sometimes be lost, which would affect the point identification. Consequently, for the practical fabrication of the model, an encoding scheme based on the thickness of the lines is employed to facilitate the identification of calibration line segments. A thin line is defined as the binary digit “0”, and a thick line as the binary digit “1”. The sequence of lines, from left to right, corresponds to a binary series, which serves to determine the identified points.

2.3. Transformation of Calibration Point

From Equation (4), the intersections $Q_i$ in calibration tool coordinate system have been calculated. However, the ultrasound calibration requires the coordinates in the ultrasound probe coordinate system. Thus, we use a stereo camera to transform the calibration points $Q_i$ from a calibration tool coordinate system to an ultrasound probe coordinate system, as shown in Figure 1. Let the coordinates of the calibration point in the calibration tool and ultrasound probe marker coordinate system be $Q_i={\left[\begin{array}{ccc}{x}_i& y_i& z_i\end{array}\right]}^{\mathrm{T}}$ and $Q_{mi}={\left[\begin{array}{ccc}{x}_{mi}& y_{mi}& z_{mi}\end{array}\right]}^{\mathrm{T}}$, respectively. Then, the coordinate $Q_{mi}$ can be transformed by the following equation:

$$\left[\begin{array}{c}{Q}_{mi}\\ 1\end{array}\right]=T_{\mathrm{camera}}^{\mathrm{probe}}{}^{-1}T_{\mathrm{camera}}^{\mathrm{calib}}\left[\begin{array}{c}{Q}_i\\ 1\end{array}\right]$$

(5)

where $T_{\mathrm{camera}}^{\mathrm{calib}}$ and $T_{\mathrm{camera}}^{\mathrm{probe}}$ are the poses of the calibration tool and ultrasound probe, which can be obtained simultaneously using the stereo camera. With the coordinates $Q_{mi}$ in the ultrasound probe marker coordinate system known, we can establish a new plane coordinate system and calculate the homography matrix of the two planes.

2.4. Establishment of Plane Coordinate System

Since ultrasound imaging is inherently planar, all points are theoretically coplanar within the ultrasound probe marker coordinate system. Therefore, a planar homography can be employed to represent the mapping from the two-dimensional image coordinates to the three-dimensional coordinates in the marker coordinate system. However, this requires an initial transformation from the three-dimensional marker coordinate system to the image coordinates system. Here, we project the three-dimensional points onto a plane within the marker coordinate system to achieve dimension reduction.

A least squares fitting method is utilized to fit a plane to the three-dimensional coordinates in the marker coordinate system. The general form of the plane equation can be given by:

$$z=ax+by+c$$

(6)

where a, b, and c are constants.

As each three-dimensional point in the marker coordinate satisfies the above equation, the problem can be converted to a least square problem:

$$AX=b$$

(7)

where

$$A=\left[\begin{array}{c}\begin{array}{ccc}{x}_{m1}& y_{m1}& 1\end{array}\\ \begin{array}{ccc}{x}_{m2}& y_{m2}& 1\end{array}\\ ⋮\\ \begin{array}{ccc}{x}_{mn}& y_{mn}& 1\end{array}\end{array}\right],X=\left[\begin{array}{c}a\\ b\\ c\end{array}\right],b=\left[\begin{array}{c}{z}_{m1}\\ z_{m2}\\ ⋮\\ z_{mn}\end{array}\right]$$

(8)

and n is the number of identified points. By substituting the three-dimensional point coordinates under the marker coordinate system, a, b, c can be solved, and the normal vector of this plane is $\overrightarrow{z}={\left[\begin{array}{ccc}a& b& -1\end{array}\right]}^{\mathrm{T}}$.

Then, a coordinate system is established within this plane by arbitrarily selecting two points:

$$\begin{array}{l}{l}_1={\left[\begin{array}{ccc}0& 0& c\end{array}\right]}^{\mathrm{T}}\\ l_2={\left[\begin{array}{ccc}10& 0& 10a+c\end{array}\right]}^{\mathrm{T}}\end{array}$$

(9)

Given $\overrightarrow{x}=l_2-l_1$, the $\overrightarrow{y}$ can be computed from the cross product $\overrightarrow{y}=\overrightarrow{z}\times \overrightarrow{x}$. The rotation matrix R for the new coordinate system can be derived as follows:

$$R=\left[\begin{array}{ccc}{\displaystyle \frac{\overrightarrow{x}}{\left|\overrightarrow{x}\right|}}& {\displaystyle \frac{\overrightarrow{y}}{\left|\overrightarrow{y}\right|}}& {\displaystyle \frac{\overrightarrow{z}}{\left|\overrightarrow{z}\right|}}\end{array}\right]$$

(10)

By arbitrarily designating a point as the origin, assumed to be point $I_1$, the translation vector can be defined as:

$$t=-R{I}_1$$

(11)

Finally, the homogeneous transformation matrix $M_P$ from the marker coordinate system to the established plane coordinate system can be represented as:

$$M_{\mathrm{p}}=\left[\begin{array}{cc}R& t\\ 0& 1\end{array}\right]$$

(12)

2.5. Calculation of the Homography Matrix

With matrix $M_{\mathrm{p}}$, all three-dimensional points are projected onto a two-dimensional plane. Let the points within the two-dimensional plane be denoted as $s_i={\left[\begin{array}{cc}{x}_{si}& y_{si}\end{array}\right]}^{\mathrm{T}}$, then

$$\left[\begin{array}{c}{x}_{si}\\ y_{si}\\ k_i\\ 1\end{array}\right]=M_{\mathrm{p}}\cdot \left[\begin{array}{c}{x}_{mi}\\ y_{mi}\\ z_{mi}\\ 1\end{array}\right]$$

(13)

where $k_i$ is the z coordinate after projection onto the plane, which should ideally be 0.

After a projection of three-dimensional coordinates onto a two-dimensional plane, we now proceed to determine the homography matrix $M_h$ that maps the set of image points into the three-dimensional plane of N-line model. The assumptions for this calculation are as follows:

$$\left[\begin{array}{c}{x}_{si}\\ y_{si}\\ 1\end{array}\right]=\alpha M_h\left[\begin{array}{c}{x}_{oi}\\ y_{oi}\\ 1\end{array}\right]=\alpha \left[\begin{array}{ccc}{h}_{11}& h_{12}& h_{13}\\ h_{21}& h_{22}& h_{23}\\ h_{31}& h_{32}& h_{33}\end{array}\right]\left[\begin{array}{c}{x}_{oi}\\ y_{oi}\\ 1\end{array}\right]$$

(14)

Here, α represents the scale transformation factor. Upon expansion and subsequent elimination of α by the third row, we have:

$$\begin{array}{l}-h_{11}{x}_{oi}-h_{12}{y}_{oi}-h_{13}+h_{31}{x}_{si}{x}_{oi}+h_{32}{x}_{si}{y}_{oi}+h_{22}{x}_{si}=0\\ -h_{21}{x}_{oi}-h_{22}{y}_{oi}-h_{23}+h_{31}{x}_{oi}{\mathrm{y}}_{si}+h_{32}{y}_{oi}{\mathrm{y}}_{si}+h_{33}{\mathrm{y}}_{si}=0\end{array}$$

(15)

The problem can be formulated as a least squares minimization, leading to the construction of matrix $Ax=0$, where:

$$A=\left[\begin{array}{ccccccccc}-x_{o1}& -y_{o1}& -1& 0& 0& 0& x_{o1}{x}_{s1}& x_{s1}{y}_{o1}& x_{s1}\\ 0& 0& 0& -x_{o1}& -y_{o1}& -1& x_{o1}{y}_{s1}& y_{o1}{y}_{s1}& y_{s1}\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ -x_{on}& -y_{on}& -1& 0& 0& 0& x_{on}{x}_{sn}& x_{sn}{y}_{on}& x_{sn}\\ 0& 0& 0& -x_{on}& -y_{on}& -1& x_{on}{y}_{sn}& y_{o1}{y}_{sn}& y_{sn}\end{array}\right]$$

(16)

$$x={\left[\begin{array}{ccccccccc}{h}_{11}& h_{12}& h_{13}& h_{21}& h_{22}& h_{23}& h_{31}& h_{32}& h_{33}\end{array}\right]}^{\mathrm{T}}$$

(17)

$Ax=0$ is an overdetermined system of equations, which does not have an exact solution. Therefore, we estimate the solution through Singular Value Decomposition (SVD). Assuming there exists a set of values x that satisfy $Ax=0$, it remains valid when multiplying x by any scalar k. Therefore, we might as well add a constraint $\Vert x\Vert =1$ and formulate a constrained optimization problem:

$$\arg \min \Vert Ax\Vert , \mathrm{s}.\mathrm{t}. \Vert x\Vert =1$$

(18)

Assume the Singular Value Decomposition (SVD) of matrix A is:

$$A=UD{V}^{\mathrm{T}}$$

(19)

Due to the norm-preserving property of orthogonal matrices, we can directly eliminate U from the SVD, which allows us to further simplify the expression to:

$$\arg \min \Vert Ax\Vert =\arg \min \Vert UD{V}^{\mathrm{T}}x\Vert =\arg \min \Vert D{V}^{\mathrm{T}}x\Vert$$

(20)

By defining $y=V^{T}x$, the optimization problem is equivalently transformed into the following form:

$$\arg \min \Vert Ax\Vert =\arg \min \Vert Dy\Vert$$

(21)

Matrix D is a diagonal matrix composed of the singular values of matrix A. When the singular values in the diagonal matrix are arranged in descending order, the condition for $\arg \min \Vert Dy\Vert$ to be minimized is $y={\left[\begin{array}{cccc}0& 0& \cdots & 1\end{array}\right]}^{\mathrm{T}}$.

Given that $x=Vy$ and considering that matrix V is composed of the eigenvectors of matrix A, it can be deduced that x corresponds to the last column of the eigenmatrix V. Finally, all calibration tasks for the ultrasound probe have been completed. The saved matrices $M_{\mathrm{h}}$ and $M_{\mathrm{p}}$ as calibration parameters, can be directly utilized in subsequent three-dimensional ultrasound reconstruction processes.

2.6. Ultrasonic Three-Dimensional Reconstruction

After completing the previous steps for ultrasound probe calibration, it is then possible to convert any two-dimensional point coordinate on the ultrasound image into a three-dimensional coordinate within the ultrasound probe marker coordinate system during subsequent processes, which is a fundamental feature required by all algorithms for three-dimensional ultrasound reconstruction. However, calibration methods such as the one illustrated in Equation (1) involve a computationally intensive process where each point must be individually multiplied by the calibration matrix to derive the three-dimensional coordinates. When handling a large series of ultrasound images, this results in a substantial computational load, which is not conducive to intraoperative use.

In contrast, the DRHT algorithm in this paper is centered around the concept of dimensionality reduction by projecting three-dimensional points onto two-dimensional points, followed by a homography transformation between two-dimensional images. Thus, it can be viewed as a transformation of planar images, which enables the quick computation of the coordinates for all points. As depicted in Figure 4, for an image with dimensions of n by m pixels, the initial step involves calculating the three-dimensional coordinates $I_{11}^{\prime }$, $I_{1n}^{\prime }$, and $I_{m1}^{\prime }$ of the vertices $I_{11}$, $I_{1n}$, and $I_{m1}$ of the ultrasound image.

Let the two-dimensional coordinates be represented as $o_i^{\prime }={[\begin{array}{cc}{x}_{oi}^{\prime }& y_{oi}^{\prime }\end{array}]}^{\mathrm{T}}$ and the correspondences in the plane coordinate system be represented by $s_i^{\prime }={[\begin{array}{ccc}{x}_{si}^{\prime }& y_{si}^{\prime }& 0\end{array}]}^{\mathrm{T}}$. According to the principles of homography transformation, the relationship between these coordinates can be expressed as:

$$\left[\begin{array}{c}k{x}_{si}^{\prime }\\ k{y}_{si}^{\prime }\\ k\end{array}\right]=M_h\left[\begin{array}{c}{x}_{oi}^{\prime }\\ y_{oi}^{\prime }\\ 1\end{array}\right]$$

(22)

Utilizing the transformation matrix $M_p$ from the ultrasound marker coordinate system to the plane coordinate system and given a three-dimensional point $m_i^{\prime }={[\begin{array}{ccc}{x}_{mi}^{\prime }& y_{mi}^{\prime }& z_{mi}^{\prime }\end{array}]}^{\mathrm{T}}$ in the ultrasound probe marker coordinate system, the transformation can be described as follows:

$$\left[\begin{array}{c}{x}_{mi}^{\prime }\\ y_{mi}^{\prime }\\ z_{mi}^{\prime }\\ 1\end{array}\right]=M_p\left[\begin{array}{c}{x}_{si}^{\prime }\\ y_{si}^{\prime }\\ 0\\ 1\end{array}\right]$$

(23)

After calculating the coordinates of points $I_{11}^{\prime }$, $I_{1n}^{\prime }$, and $I_{n1}^{\prime }$, the coordinates of all other points can be obtained through interpolation based on these three points. To proceed, we calculate the unit vectors in the x and y directions of the three-dimensional image as follows:

$$v_x={\displaystyle \frac{I_{1n}^{\prime }-I_{11}^{\prime }}{n-1}}$$

(24)

$$v_y={\displaystyle \frac{I_{m1}^{\prime }-I_{11}^{\prime }}{m-1}}$$

(25)

Thus, the coordinate calculation for point $I_{ij}^{\prime }$ is outlined as follows:

$$I_{ij}^{\prime }=I_{11}^{\prime }+(i-1)\cdot v_y+(j-1)\cdot v_x$$

(26)

Through the above steps, it is possible to rapidly compute the three-dimensional coordinates of all two-dimensional points in the ultrasound image within the camera coordinate system.

3. Experiments

3.1. Ultrasound Probe Calibration Scenario Setup

To validate the method described in this paper, an ultrasound probe calibration scenario was constructed, as shown in Figure 5, which includes a binocular camera DHST-1 (Beijing Dahua Wangda Technology Co., Ltd., Beijing, China) with a resolution of 1600 × 1200 pixels and a frame rate > 40 fps; the positioning error is less than 0.3 mm. The ultrasound equipment is the DW360 (Dawei Medical Co., Ltd., Xuzhou, China), with a linear array probe selected and set to a detection depth of 9 cm. The laptop used for data processing is a DELL G7 7590 (Dell Inc., Round Rock, TX, USA), equipped with an Inter(R) Core(TM) i7-8750H processor (Intel Corporation, Santa Clara, CA, USA), an NVIDIA Geforce RTX2060 graphics card (NVIDIA Corporation, Santa Clara, CA, USA), and 16 GB of RAM. An image acquisition card UB530 (TCHD Digital Video Technology Development (Beijing) Co., Ltd., Beijing, China) is employed to capture images from the ultrasound device and transmit them in real-time to the workstation for immediate processing.

The calibration fixture was fabricated using a photo-curable resin material through 3D printing. The black parts of the visual markers were printed using black nylon and assembled afterward. The thin calibration lines were created with nylon threads of 0.2 mm in diameter, while the thick lines were wrapped by a 0.8 mm thick sleeve over the thin threads. To ensure the installation accuracy of the nylon threads, dumbbell-shaped notches were set on the installation columns. After assembly, the nylon threads were tightened and secured at both ends with screws.

The designed coordinates (x, y, z) of the line segment vertices under the calibration marker are presented in

Table 1. Coordinates of the vertex of the calibrated line segment in the calibrated marker coordinate system.

	Column	0	1	2	3	4	5	6	7
Layers
1		29.5	29.5	29.5	29.5	29.5	29.5	29.5	29.5
		18.5	23.5	28.5	33.5	38.5	43.5	48.5	53.5
		−68.2	0.24	−68.2	0.24	−68.2	0.24	−68.2	0.24
2		44.5	44.5	44.5	44.5	44.5	44.5	44.5	44.5
		18.5	23.5	28.5	33.5	38.5	43.5	48.5	53.5
		−68.2	0.24	−68.2	0.24	−68.2	0.24	−68.2	0.24
3		59.5	59.5	59.5	59.5	59.5	59.5	59.5	59.5
		18.5	23.5	28.5	33.5	38.5	43.5	48.5	53.5
		−68.2	0.24	−68.2	0.24	−68.2	0.24	−68.2	0.24

. From top to bottom are the eight segment vertices of the first, second, and third layers, corresponding to P₀~P₂₃ in Figure 3.

3.2. Image Coordinate Extraction

After acquiring the ultrasound images through image capture, the process of interactive point selection is carried out on a software platform developed using Visual Studio 2019, QT 5.12.9, OpenGL 4.5, OpenCV 3.3, Plus 2.8.0, and Vtk 8.0.1, as depicted in Figure 6. Initially, the user identifies the point to be selected on the interface shown in Figure 6a, then clicks on the center of the projected point on the image to complete the selection, repeating this process until all points are selected. As illustrated in Figure 6b, the first and second rows of markers are complete, with all eight points recognizable, making the selection process relatively straightforward in this case. However, in Figure 6c, the first row contains only seven points and the second row contains only six points. Under these circumstances, it is necessary to rely on the size of the projection to discern the points and prevent any errors in selection.

After the point selection, all identified calibration points from the images are consolidated. Utilizing the method described in this paper, the matrices $M_p$ and $M_h$ are calculated and saved as the calibration results.

3.3. Error Determination

Evaluating the quality of the model presented in this paper requires a comprehensive and objective approach. Merely comparing the results with those from the literature may not yield an unbiased assessment. Factors such as the number of calibration points, the quality of the equipment (ultrasound and navigation devices), the precision of 3D printing, and the errors in point selection can significantly affect the evaluation of the results. Therefore, we use the same dataset for calibration with different methods and then compare their respective calibration errors.

In this paper, we compare the calibration accuracy with the Plus Toolkit [28], an open-source toolkit for data acquisition, preprocessing, and calibration in ultrasound-guided interventions. Continuously updated since 2011, it has become one of the most authoritative open-source libraries in the field of ultrasound-guided imaging. The function used is Compute Image To Probe Transform By Linear Least Squares Method, and its calibration model is shown in Equation (1).

The calculation of error is determined as follows: two methods are used to calculate the spatial coordinates $m_i^{\prime }={[\begin{array}{ccc}{x}_{mi}^{\prime }& y_{mi}^{\prime }& z_{mi}^{\prime }\end{array}]}^{\mathrm{T}}$. The first is by projecting the 2D calibration points to the 3D spatial coordinates, and the second is by directly using the N-line model. The average distance between these two sets of coordinates is taken as the calibration error, denoted as:

$$e={\displaystyle \frac{{\displaystyle \sum _{i=1}^n\sqrt{{(x_{mi}-x_{mi}^{\prime })}^2+{(y_{mi}-y_{mi}^{\prime })}^2+{(z_{mi}-z_{mi}^{\prime })}^2}}}{n}}$$

(27)

3.4. Calibration Experiment

We conducted a total of three calibration experiments, collecting datasets consisting of 3, 9, and 14 groups, respectively. Each dataset included ultrasound images, the pose of the calibration fixture marker $T_{\mathrm{camera}}^{\mathrm{calib}}$ in the camera coordinate system, and the pose of the ultrasound probe marker $T_{\mathrm{camera}}^{\mathrm{probe}}$. The ultimate calibration errors for the three experiments are presented in

Table 2. Calibration errors of the three groups of experiments.

Image Num	3	9	14
Error of PLUS	1.15 ± 0.88 mm	1.12 ± 0.82 mm	1.05 ± 0.63 mm
Error of DRHT	1.09 ± 0.77 mm	0.98 ± 0.76 mm	0.86 ± 0.54 mm

.

Analysis of the three sets of data revealed that the DRHT algorithm demonstrated an error of 0.86 mm, which was superior to the 1.05 mm error of the PLUS algorithm. The DRHT also showed an advantage in terms of variance. Consequently, for the same data, the DRHT outperforms the algorithms in the PLUS library in terms of accuracy. It is worth noting that the DRHT computation did not employ any nonlinear optimization; thus, there is potential for further improvement in accuracy if optimization algorithms such as the Levenberg–Marquardt (LM) method are utilized.

From another perspective, the calibration errors of the three experiments were not significantly different, with the error using three sets of data being nearly equivalent to that using fourteen sets of data. To further explore the robustness of the DRHT and to determine the minimum number of images required for calibration, we designed a second experiment.

In this experiment, we collected 20 sets of data and conducted 17 trials, using 1 to 17 sets of data for calibration and the remaining 3 sets for validation, with the validation data not used for calibration. To ensure the credibility of the experiment, both the calibration and validation data were randomly selected. The specific implementation involved reshuffling the 20 sets of data for each trial, using the first n sets for calibration in the nth trial, and the last 3 sets for validation. Ultimately, four metrics were measured: the calibration error and validation error for DRHT, as well as the calibration error and validation error for the PLUS library.

Figure 7 illustrates the results of the 17 calibration experiments plotted as curves. It can be observed that with only a single calibration image, the calibration error was less than 0.5 mm, yet the validation error approximated 1.2 mm, which means the calibration model was somewhat overfitted at this point. When more than four images were used, the calibration error increased and eventually stabilized. The average calibration error for DRHT was 0.8940 mm; for PLUS, it was 0.9212 mm. The validation error remained stable below 1.3 mm, with DRHT’s validation error at 1.0272 mm and PLUS’s at 1.0629 mm. In terms of both calibration and validation accuracy, DRHT outperformed PLUS. From another perspective, when both calibration methods use over four images, their calibration and validation errors fluctuate between 0.8 mm and 1.2 mm, showing similar trends. This is because manual selection of all calibration points introduces subjectivity, leading to result fluctuations. While automated calibration point selection could address this, it is beyond this paper’s scope, so will not be described here.

Despite the inherent randomness in calibration procedures and data, notable fluctuations in the validation error were observed, including instances where the error dipped below 0.8 mm in the 6th and 12th datasets. Nevertheless, a discernible trend emerged: as the quantity of calibration data increased, the variability in the calibration error diminished and eventually reached a stable state.

From the first experiment, it was evident that the DRHT model proposed in this paper was feasible in terms of validation accuracy. Even without using a nonlinear optimization algorithm, the results were still better than the calibration algorithm of the PLUS library. In the second experiment, a cross-validation method was employed to compute the calibration and validation errors for both algorithms. The results showed that with the three-tier 24-line calibration model proposed in this paper, only four images were needed to provide sufficient calibration information, thus completing the calibration. Compared to methods that required 100 images [25] or thousands of images [29] for calibration, this approach offered a significant advantage.

3.5. Three-Dimensional Reconstruction Experiment

In the three-dimensional reconstruction experiment, five groups of ultrasonic datasets were collected from fingers, spinal model bone, and 3D-printed parts, as shown in the first row of Figure 8. In each dataset, about 300 to 700 ultrasonic images were collected for ultrasonic three-dimensional reconstruction. The original images acquired were 1920 × 1080, and the area of interest was 700 × 508.

In the three-dimensional reconstruction experiment, the proposed reconstruction method and the PLUS method were used for ultrasonic three-dimensional reconstruction to evaluate the reconstruction performance of the algorithms. The reconstruction of the PLUS method is via the multiplication of points and calibration parameters from PLUS. The third row shows the 3D reconstruction results of the PLUS algorithm, and the fourth row shows the results of the algorithm in this paper. As can be seen from the figure, there is no significant difference between the reconstruction results of the two algorithms. Therefore, it can be preliminarily determined that the dimensionality reduction process of the DRHT algorithm in this paper has no significant effect on the accuracy of 3D reconstruction.

The overall reconstruction time is shown in Figure 9. The average 3D reconstruction time for the DRHT method is 268 ms, while that for the PLUS method is 761 ms. The results show that the reconstruction time of the proposed method is about 1/3 of the PLUS method, indicating that the reconstruction efficiency of the proposed method is significantly superior to that of the PLUS method.

4. Conclusions

Three-dimensional ultrasound reconstruction has long been a focal point of research because ultrasound can detect tissues beneath the skin without radiation. However, the information provided to physicians by 2D ultrasound imaging is quite limited and lacks a global perspective, which is not conducive to guiding surgical robots. In contrast, reconstructing a series of two-dimensional ultrasound images into a three-dimensional image can greatly expand the field of view, providing more comprehensive information for surgical procedures. In orthopedic surgery, ultrasound 3D reconstruction offers real-time and radiation-free navigation, boosts surgical precision, supports minimally invasive techniques, reduces risks, and improves efficiency. For tumor resection, it aids in preoperative planning, enables intraoperative navigation and precise excision, and is useful for postoperative evaluation. To achieve this, precise calibration of the ultrasound probe is a prerequisite. The DRHT method proposed in this paper is a calibration technique for ultrasound probes that simplifies computations and enhances accuracy.

The contributions of this paper are as follows: Firstly, we propose a novel ultrasound probe calibration method based on dimensionality reduction and homography transformation. This method effectively maps three-dimensional spatial points to two-dimensional points by fitting the ultrasound imaging plane and then uses homography transformation to characterize the relationship between the reduced two-dimensional points and the ultrasound image. This approach effectively eliminates the scale factor commonly found in traditional calibration models, avoiding the amplification of calibration errors caused by scale factor estimation inaccuracies. The effectiveness of the model is verified by comparing it with the calibration algorithms in the PLUS library. Secondly, we designed a three-tier 24-point calibration fixture based on the N-line model, encoded using binary principles for easy identification. A single image can effectively identify 18 calibration points, and experiments confirmed that only four images are needed to complete the calibration, greatly simplifying the ultrasound calibration process and reducing calibration costs. Ultimately, the average calibration error of the DRHT algorithm is 0.95 mm, and the average validation error is 1.02 mm. Since the calibration data are currently collected manually, further improvements in accuracy are expected if a robotic arm is used for data collection.

In future work, we plan to adopt a nonlinear optimization algorithm to optimize the DRHT algorithm, further enhancing the model’s accuracy. We will also develop an automatic calibration point recognition method to achieve automatic calibration, reducing the impact of manual point selection on calibration errors.