1. Introduction
Ultrasound (US) guided needle procedures have been widely applied in the clinical contexts of tumour biopsy, regional anaesthesia and fetal medicine [
1]. Clear visibility of the needle tip within the US image is essential to reach the procedure target and prevent erroneous needle placement, which may lead to adverse events such as nerve damage, organ injury [
2,
3] or pregnancy loss [
4]. In fetal medicine, 0.5% of women who undergo chorionic villus sampling or amniocentesis—two common diagnostic procedures—will miscarry, while 6% of procedures fail and need to be repeated [
5].
The visibility of a needle within a B-mode, pulse-echo US image depends on its echogenicity relative to the surrounding tissue [
3,
6,
7]. Two acoustic phenomena that contribute to needle echogenicity are diffuse scattering and specular reflection. The small diameter of the needle relative to the US wavelength results in diffuse scattering in planes perpendicular to the needle’s long axis, while its smooth surface and long length cause specular reflection in the plane parallel to its long axis. At steep insertion angles, specular reflection can cause US waves to miss the receiving US aperture, resulting in poor visibility [
3]. Artefacts caused by reverberation and side lobes can also complicate the localisation of the needle tip [
8], and at large depths, acoustic attenuation in tissue can deteriorate visibilty [
9]. Even if the needle tip is well-visualised, the intersection of the needle shaft with the imaging plane may be incorrectly identified as the needle tip, causing misplacement, even during insertions performed by experienced operators [
3].
Over the past three decades, a variety of passive methods have been proposed to improve needle visibility, including: techniques for distinguishing the needle from its surrounding tissues using image processing algorithms [
10,
11,
12]; detection of a needle’s motion relative to the US probe with colour Doppler US [
13,
14,
15,
16]; US beam steering designed to reduce losses due to specular reflection [
17]; and modification of a needle’s surface to enhance diffuse scattering, thereby reducing specular reflection [
18,
19,
20]. Such methods have been reported to be effective to some extent, and surface-modified needles have seen success in the clinic [
19,
20]. However, reliable, clear and accurate localisation of the needle tip still poses a challenge, particularly in high echogenicity tissues [
21,
22], with steep insertion angles (greater than 45
) and in heterogeneous tissues containing structures resembling needle shafts [
23].
Active needle tracking methods have also been recently proposed, typically relying on either electromagnetic (EM) [
24,
25,
26] or US transducers embedded within the needle shaft and tip [
1,
27,
28,
29,
30,
31,
32]. In EM tracking, a magnetic sensor is located at the needle tip and exposed to a known EM field. The magnetic gradient detected by the sensor is then used to derive the spatial position of the sensor within the field. This is registered to the coordinates of the US probe, which is fitted with its own sensor. This method is, however, sensitive to environmental EM field disturbances—for example from surgical instruments [
33]—and its resolution can exceed 3 mm [
34,
35]. The requirement for an external EM field generator and a tracking sensor on the US probe is also disruptive to the clinical workflow. Winsberg et al. introduced US needle tracking in 1991 with a piezoelectric element located at the needle tip; when struck by a transmission from the US probe, the electrical signal from the receiver was delayed and injected into the echo train of the US imaging system, causing a bright echo to appear in the US image at the needle tip position [
36]. Later work by Langberg et al. utilised a piezoelectric transponder that returned acoustic signals to the US probe [
1]. Recent advancements in fibre-optic US devices have enabled the development of similar methods utilising photoacoustic transmitters, receivers and transponders with the potential to provide sub-millimetre accuracy [
32,
34,
37]. We previously presented an ultrasonic tracking technique based on a fibre-optic hydrophone (FOH) integrated into an intraoperative needle [
28,
29,
30]. The system converted a temporal ultrasonic signal received at the needle tip into a spatial position which was clearly visualised as a cross-hair on the US image [
28,
29,
30]. However, the system required a dedicated US transmission sequence for tracking that was interleaved with the imaging sequence. A later system utilised a bespoke 1.5D US imaging probe [
38,
39]. Both techniques required the use of a research US platform, featuring only simple B-mode US image reconstruction and therefore degraded image quality, which impeded their use in the clinic.
This paper presents a next-generation, real-time FOH tracking system capable of locating the needle tip within the imaging plane of a commercial US system (Voluson E10, GE, Zift, Austria), using only limited knowledge of the transducer geometry and transmission sequence. The system has been developed under the ISO 13485 Medical Devices quality standard [
40], and is intended for application in fetal-medicine diagnostic procedures such as chorionic villus sampling and amniocentesis. Three in vitro (homogenous medium) and two ex vivo (heterogeneous medium) experiments assessing aspects of tracking accuracy, directivity and signal-to-noise ratio are presented and their results discussed. Two experiments related to safety are also presented: temperature rise at the FOH tip was measured in vitro in a water phantom, and ex-vivo placenta tissue was histologically examined after exposure to the FOH, demonstrating that no tissue damage was caused by laser-induced heating from the FOH.
3. Experimental Methods
Three in vitro and two ex vivo experiments assessing the performance of the UNT system were carried out. The in vitro experiments took place in a tank of water, aiming to assess the performance of the system in a homogenous medium. The first homogenous-medium experiment assessed tracking accuracy at 356 known locations in the imaging plane; the second assessed the out-of-plane tracking accuracy at a fixed lateral and elevational position; and the third assessed the dependence of FOH signal amplitude and tracking accuracy on needle insertion angle. In tissue, attenuation and heterogeneities in sound-speed and acoustic impedance were likely to aberrate the acoustic field and reduce signal amplitude, degrading tracking accuracy. The first ex vivo experiment therefore assessed tracking accuracy in a bovine tissue and water phantom designed to mimic a transabdominal fetal medicine procedure, by comparison to manual annotation of the apparent location of the needle tip in B-mode US images. The second heterogeneous-medium experiment assessed the dependence of SNR on depth in another phantom comprised solely of bovine tissue.
Two experiments relating to safety were carried out. The first was an in vitro experiment assessing the temperature rise at the tip of the needle in water. The second experiment histologically assessed ex vivo placenta samples for signs of thermal damage after insertion with a needle with integrated FOH. The placentas were from women undergoing elective Caesarean section at term who gave informed written consent (“Fetal Stem Cells” REC reference: 14/LO/0863. IRAS ID: 133888).
3.1. In-Plane Tracking Accuracy in a Homogenous Medium
For the homogenous-medium experiment, the US probe was placed over a water tank with dimensions 70 × 40 × 50 cm. A bespoke probe mount allowed manual adjustment of its pitch, yaw and roll as shown in
Figure 3. The probe was held pointing vertically downwards with its face immersed in the room-temperature water to a minimum depth of approximately 5 mm. The base of the tank was lined with an acoustically absorbing material to prevent reflections. The needle was clamped to an arm attached to a 3-axis, 2.0
m resolution computer controlled Cartesian motion system (1N150 linear stages, Thorlabs, Newton, NJ, USA) and positioned in the tank below the imaging probe. The needle was held with its long axis horizontal to the water surface, maximising the range of accessible positions within the US field of view. The needle was oriented with its bevel facing upwards, exposing the integrated FOH to incident ultrasound. The needle clamp and motion control system were positioned on the right hand side of the imaging probe, with the needle pointing left into the US imaging plane as shown in
Figure 4.
3.1.1. US Imaging System Configuration
The US imaging system was configured with an imaging depth of 300 mm and field of view of 55
, resulting in a frame rate of 18 Hz [
41]. These settings were chosen based on proprietary details of the pulse transmission sequence provided by GE. A single focal zone was set at a depth of 15 mm, ensuring that all measurements were made in the acoustic far-field. The acoustic output was set to the default value of 98%. Settings that did not affect the acoustic output of the probe were not recorded, as they had no effect on tracking performance.
3.1.2. Determining True Needle Positions
Determination of the true needle positions relative to the US probe was achieved by alignment of the US probe to the needle’s motion axes. True, 2D needle positions within the imaging plane were determined in Cartesian coordinates relative to the centre of curvature of the probe, x being the lateral direction and y being the axial (vertical) direction. This comprised first aligning the lateral axis of the motion control system to the imaging plane, and then aligning the central vertical axis of the US image to the axial motion of the control system.
The lateral motion of the needle tip was aligned to the imaging plane as follows: the US probe was oriented pointing vertically downwards using a spirit level; the needle tip was positioned along the central vertical axis of the US image, approximately 75 mm from the probe face; the elevational position of the needle was adjusted such that the brightness of the needle in the US mage was maximised; the needle was translated laterally by 75 mm; the roll of the probe was adjusted such that the brightness of the needle in the US mage was maximised. Success was confirmed by translating the needle tip laterally from one side of the imaging plane to the other and observing that its brightness in the US image did not change.
An axial alignment procedure was designed based on the iterative “near-far” method for aligning the beam axis of a single-element, axisymmetric US source to the axial motion of a hydrophone described in [
49]. The cited axisymmetric alignment procedure comprises repeatedly moving the hydrophone between two distantly-spaced far-field positions nominally along the beam axis, maximising the US signal at each position by making adjustments to either the position of the hydrophone or orientation of the source. After several iterations, the axial motion of the hydrophone and the beam axis of the source will converge. This alignment procedure was adapted to allow alignment of the central vertical axis of the B-mode US image to the axial motion of the FOH. The needle was translated between 40 mm and 140 mm nominally directly below the centre of the probe face, with the distance determined from the time of arrival of the US pulse to the FOH embedded in the needle tip combined with knowledge of the delay between the trigger signal and US transmissions internal to the imaging system, and the radius of curvature of the probe; although no pulse was emitted from the centre of the probe, the scan lines either side of the central axis were very close to the centre and as such their use introduced a negligible error. At the 40 mm depth, the lateral and elevational positions of the needle were adjusted to position its tip on the central axis. At the 140 mm depth, the pitch and yaw of the probe were adjusted to coincide the central axis with the needle tip position.
At each iteration, an approximate alignment of the FOH tip to the central vertical axis could be achieved by adjusting the position of the needle or orientation of the probe until the B-mode image of the needle tip coincided with the central axis of the image, using a rigid vertical marker placed over the screen of the imaging system as a guide. However, although the focal depth of the US imaging system was adjusted at each iteration to optimise the resolution of the US image at the needle depth, US image artefacts around the needle tip made it difficult to precisely locate the needle tip using the B-mode image alone. Following this, a more precise alignment was achieved by examining the FOH signal visualised with an oscilloscope, using the signal amplitude to find the midpoint (i.e., the amplitude minimum) between the two central scan-lines of the US imaging frame.
3.1.3. Data Collection
It was expected that tracking accuracy would depend on both the depth and lateral position of the needle tip within the imaging plane: the probe’s curvilinear design results in the spacing between scan-lines increasing with both depth and lateral distance from the central vertical axis of the scan plane. Tracking accuracy was therefore assessed at 356 locations within the imaging plane, equally spaced across the right-hand side plane between 40 mm and 140 mm below the probe face, forming a 5 mm resolution grid as shown in
Figure 5. The fetal medicine procedure of particular interest—chorionic villus sampling—typically occurs between 11 to 16 weeks of gestation with the target located well within this depth range [
50], and previous US needle tracking systems have been assessed over a similar range [
30,
39]. Due to physical constraints of the motion control system and hardware setup, it was not possible to laterally translate the needle across the whole field of view, but in any case tracking accuracy could be expected to be symmetrical along the central vertical axis of the probe.
At each position, 18 frames (i.e., 1 s) of FOH waveforms were acquired providing 18 measurements of the position of the needle tip. Five repeats of this experiment were carried out, with the needle and probe being removed and remounted and the above alignment procedure being repeated between each.
3.1.4. Data Analysis
Two accuracy metrics were determined at each of the 356 locations: tracking error and tracking repeatability . Tracking error was defined as the vector displacement between the tracked and true positions, averaged over the 18 measurements. Tracking repeatability was defined as the standard deviation of the 18 tracked positions.
It was expected that, due to the use of a convex probe, any systematic tracking error introduced by the US imaging system would likely manifest as a constant error in polar space. This would result in a spatially-varying tracking error in Cartesian space. Tracking error and repeatability were therefore assessed in both polar and Cartesian coordinates. Each true needle tip location was derived from the alignment procedure in Cartesian coordinates
relative to the centre of curvature of the probe. These were directly compared with the Cartesian, scan-converted, tracked positions. For comparison with pre-scan-conversion polar tracked positions, the Cartesian true positions were converted to polar coordinates
:
Tracking error
was then determined in Cartesian coordinates as
and polar coordinates as
where
and
were the average of the 18 tracked position in Cartesian and polar coordinates respectively. Repeatability
was simply the standard deviation of the 18 individual measurements of tracked position,
or
.
At each location, five measurements of and were made—one from each of the five repeat datasets. These five measurements of error and repeatability were averaged for each location. The random uncertainty in the measurement of was estimated by dividing the standard deviation of the repeat measurements of by , where (the number of repeat measurements).
The final values of error and repeatability for each position were then averaged across the whole scanned area, and the standard-deviation taken to provide single values for the spatial variation of tracking error and repeatability respectively. The absolute value of the spatial-average tracking area in each direction was also of interest.
In summary:
Mean and standard deviation at each position provide the spatially-varying tracking error and repeatability of the system
These metrics were measured five times and averaged, with the standard deviation across the five repeats used to estimate the measurement uncertainty
Spatial average and standard deviation of tracking error and repeatability were also taken to provide single values for the systems tracking performance.
3.2. Out-of-Plane Tracking Accuracy in a Homogenous Medium
The experimental setup and alignment for the assessment of out-of-plane tracking accuracy was identical to that of the in-plane tracking accuracy assessment described above. The needle tip was initially positioned 145 mm from the probe face along the central vertical axis of the US image, and then translated elevationally to discrete positions either side of the imaging plane, and tracking accuracy was assessed at each position. The experiment was carried out with two different US imaging focal-depths: 15 mm and 140 mm, expecting that the out-of-plane tracking performance may depend the shape of the ultrasound beam profile at the needle tip depth. An FOH frame was also acquired with the US output disabled (a “noise” dataset), for determination of signal-to-noise ratio.
3.2.1. Data Collection
The needle tip was scanned along a straight line covering elevational positions from −36 mm to 36 mm in 1 mm steps, where the US imaging plane was located at an elevational position of 0 mm. At each position, a set of 18 FOH frames was acquired. This process was repeated three times for each focal depth, with the needle removed and realigned between each repeat.
3.2.2. Data Analysis
At each position, the tracked positions determined from the 18 FOH frames were averaged and compared to the true in-plane position, resulting in one axial tracking displacement and one lateral tracking displacement. The standard deviations of the tracked lateral and axial positions were taken as the tracking repeatabilities in those directions.
Noise was expected to be independent of position, so common values for noise level were used in the calculation of SNR at all positions. The root-mean-square (RMS) voltage in each noise waveform of the acquired frame was determined, and then these RMS values were averaged to provide the common noise level. Signal-to-noise ratio in each measurement frame was the calculated by dividing the peak voltage in then frame by the noise level.
The means and standard deviations of the tracking displacements, repeatabilities and signal-to-noise ratios were then taken across the three repeat measurements, with the standard deviations indicating the measurement uncertainty.
3.3. Assessment of the Effects of Needle Insertion Angle
The angular dependence of tracking accuracy and FOH signal amplitude were assessed using the same experimental setup and imaging system configuration as the homogenous-medium tracking accuracy assessments described above. The needle mount included a miniature rotational stage that allowed the angle of the needle to be set with a precision of approximately ±5. For all angles, the needle shaft remained aligned to the imaging plane.
3.3.1. Data Collection
A set of 18 FOH frames was collected with the needle at each of thirteen angles equally spaced between 0 and 60 relative to horizontal. For each acquisition, the needle tip was positioned directly below the US probe, 150 mm from the probe face, with the depth determined using the time of arrival of the US pulse as described above for the tracking accuracy experiment. This was the minimum depth at which the needle tip could be positioned at all angles of interest, due to physical constraints imposed by the experimental setup. The experiment was repeated three times, with the needle removed and realigned between each repeat.
3.3.2. Data Analysis
The averaged of the amplitude of the 18 waveforms acquired in each measurement was taken, before and after processing with the matched filter and envelope detector. The five repeat measurements of amplitude at each angle were averaged, and the standard deviation taken to assess the measurement uncertainty.
3.4. Tracking Accuracy in Tissue
Ex vivo assessment of tracking accuracy took place in a phantom comprising a layer of room temperature bovine tissue held at the surface of a small tank of room temperature water, designed to simulate transabdominal fetal-medicine diagnostic procedures such as chorionic villus sampling and amniocentesis. A schematic diagram of the phantom is shown in
Figure 6. The tissue layer was approximately 50 mm thick and the water below it approximately 200 mm deep. The base of the tank was lined with an acoustically absorbing material to prevent reflections. While the needle tip was in the water layer and therefore clearly visualised, tracked positions were compared to the apparent position of the needle tip within acquired B-mode images. The configuration of the US imaging system was identical to that used for the in vitro experiment.
3.4.1. Data Collection
The imaging probe was fixed in a clamp with its face pressed onto the top surface of the tissue pointing vertically downwards, coupled with commercially available US gel. The UNT software was configured to record its received data to disk, and a needle insertion was made into the top surface of the tissue at an angle of 55 relative to horizontal, down to a depth of approximately 150 mm, aiming to keep the needle shaft within the imaging plane throughout. The total recording time was 46 s, during which the needle was inserted and withdrawn.
3.4.2. Determining Reference Needle Positions
The UNT software includes a feature that allows recorded datasets to be stepped through frame by frame, and a manually-labelled position to be annotated on each frame by clicking on the correct position in the US image. These annotated positions are then stored to disk alongside their associated frames. This feature was used to generate reference needle positions for the recorded dataset. Only frames for which the needle tip was in the water layer of the phantom, where the tip was clearly visualised, were annotated (793 out of a total of 832 frames). During the homogenous-medium experiment, it was noticed that the visually-apparent location of the needle tip in the US image was often displaced along the needle shaft from the location of the tip of the FOH (as identified from examination of the FOH signal). This apparent extension of the needle length in the US image was likely caused by diffuse reflection at the needle tip and the point spread function of the imaging system. This offset was quantified by manually labelling four datasets from the homogenous-medium experiment (with the needle tip positioned on the central vertical axis) and determining the displacement along the needle shaft between the true positions and the manual labelled positions. For the tissue dataset, manually-labelled positions were corrected for this effect by subtraction of this displacement, after rotating it by 55 (the difference between the angles of the needle in the homogenous-medium and tissue experiments).
3.4.3. Data Analysis
The reference and tracked positions in each frame were compared as described for the in vitro experiments, although no assessment of repeatability was possible as the needle tip moved each frame. The average distance between the tracked and reference positions across the whole dataset was calculated.
3.5. Assessment of Signal-to-Noise Ratio and Repeatability in Tissue
Signal-to-noise ratio (SNR) and tracking repeatability were assessed in a 150 mm deep ex vivo phantom comprising four vertically stacked, approximately 4 cm thick pieces of bovine tissue. SNR was measured with the needle tip manually inserted to 15 locations along the central vertical axis of the US image. The US imaging system configuration was identical to that used for the ex vivo accuracy experiment described above.
3.5.1. Data Collection
For all 15 depths, the needle was manually inserted from the side of the phantom with the shaft horizontal. This was done to ensure the angle of incidence of US at the needle tip did not contribute to any difference in SNR between positions. Once the needle had been inserted, the position of the US imaging probe was adjusted to maximise the brightness of the needle tip in the US image, ensuring it was in imaging plane. At each depth, 18 frames of FOH waveforms were acquired with the US output of the imaging system enabled (“signal” datasets), and another 18 frames were acquired with the US output disabled (“noise” datasets).
3.5.2. Data Analysis
SNR was calculated for both raw and processed (matched-filtered and envelope-detected) FOH waveforms. Noise was independent of position, so common values for raw and processed noise level were used in the calculation of SNR at all positions. The root-mean-square (RMS) voltage in each noise waveform was determined, and then these RMS values were averaged to provide the common noise level. The standard deviation of RMS voltage across the waveforms was also take to ensure no significant variation in noise level.
SNR was only determined for the maximum amplitude FOH waveform in each “signal” frame. The SNR was calculated by dividing the peak waveform voltage by the corresponding (i.e., raw or processed) noise level. The SNR values determined at each position were then averaged, resulting in a raw and processed SNR value for each position.
Manually labelling of the needle tip positions was challenging because US attenuation and sound-speed heterogeneities within the tissue resulted in poor visualisation of the needle tip. As such, tracking error was not determined. Tracking repeatability, however, was determined at each position as described for the in vitro experiment and compared to the signal-to-noise ratio.
The validity of the tracking uncertainties estimated by the UNT system during this experiment was assessed by comparison to the determined repeatabilities. The absolute values of the two metrics were compared, and the Pearson correlation coefficient was calculated.
3.6. Measurement of Laser-Induced Heating
During normal operation, the infrared beam of the interrogation laser remains enclosed within the FOH, reflected by the Fabry-Pérot cavity at the tip of the fibre. The temperature rise of the FOH tip was measured using a thermocouple in water to assess the potential for laser-induced heating of the cavity to cause tissue damage in vivo.
Water was used as a medium, having a similar infrared absorption spectrum to tissue [
51]. If the Fabry-Pérot tip was damaged, exposing the fibre tip, then light would be emitted from the fibre, being quickly absorbed by the surrounding tissue and causing heating. Temperature rise in water was therefore also measured at the tip of a bare optical fibre—identical to that used in the manufacture of the FOH—connected to the interrogation laser. Resulting temperature rises were compared to the limit of 6
C set for invasive diagnostic ultrasound devices by the relevant IEC safety standard [
52].
In each case, the fibre or FOH tip was submersed in a beaker of approximately 200 mL of water, positioned approximately 1 mm from the water surface. A wire thermocouple was placed close to (<1 mm from) the fibre or FOH tip and connected to a temperature logging system (TC-01, National Instruments, Austin, TX, USA) with a resolution of 0.1 C. The interrogation laser was enabled at its maximum possible power of 25 mW; this power setting—several times greater than that used during the tracking performance assessments—was chosen so that higher laser powers could be used to boost the signal-to-noise ratio (and therefore tracking accuracy) in future if required, without needing to repeat the experiment. The temperature rise (relative to the temperature at switch-on) was recorded with a rate of 1 Hz for a period of 30 min. The maximum increase in temperature observed during the 30 min period was then logged. This procedure was repeated three times for both the FOH and bare fibre.
3.7. Heating of Ex Vivo Placenta Tissue
To ensure that laser-induced heating would not cause tissue damage in vivo, 25 mm × 25 mm × 10 mm excised samples of fresh placenta were exposed and then histologically examined for signs of damage to the chorionic villi. Six samples were exposed: three to heating from an FOH and three to the light emitted from a bare optical fibre. Exposures were carried out by inserting an amniocentesis needle with integrated FOH or bare fibre a few millimetres into each sample as shown in
Figure 7. With the needle inserted, the interrogation laser was enabled at its maximum power of 25 mW for 10 min. To assist with histological examination, the location of the insertion was marked with a small spot of white paint. The needle was then extracted, and the exposed samples were placed into tissue cassettes and submersed in 4%
w/
v formaldehyde.
For histological examination, samples were processed, cut (4 m slice thickness) and stained with Haemotoxylin and Eosin. The marked location of the needle insertion was used as a guide for the cutting of each sample. The site of the needle tip was then identified from the artefactual architecture change in the villi caused by the insertion of the needle, and the surrounding tissue was inspected for thermal damage such as coagulative necrosis and heat diathermy.
5. Discussion
We have presented the first performance assessment of a fibre-optic US needle tracking system compatible with a commercial US imaging system and which is therefore directly translatable to the clinic. The technology has been developed under the ISO 13485 Medical Devices quality standard [
40] to assist minimally-invasive interventional procedures in fetal medicine such as chorionic villus sampling and amniocentesis, where automated tracking of the needle tip could reduce the frequency of adverse events such as miscarriage. During operation, the system sits alongside the US imaging device, connected to it with an HDMI cable and BNC trigger cable. The reusable components comprise a PC workstation, signal acquisition electronics and a laser source assembled into a Karl Storz (Tuttlingen, Germany) medical stack trolley. Custom single-use needles with integrated FOHs are provided sterilised and individually packaged. When in use, these are connected to the system with a thin fibre-optic cable.
The FOH embedded in the needle tip provides several advantages over alternative active US tracking technologies such as those based on an integrated piezoelectric element [
31]. It provides a small active area and therefore a broad directivity [
53]—reducing the likelihood that needle angle will affect tracking variance—without compromising sensitivity; when using piezoelectric elements, a smaller element broadens directivity while reducing sensitivity. The FOH also has a broadband frequency response with high sensitivity up to at least 50 MHz [
53], allowing a single fibre to be used for clinical procedures of all typical US centre frequencies; this is in contrast to the narrow response of many piezoelectric elements. Delivery of signals from the sensor tip to the rest of the tracking system is trivial, as the optical signal can be examined at the end of the narrow (150
m outer diameter) fibre which fits readily within the intraoperative needle; piezoelectric systems must integrate electrical connections into a bespoke needle, and utilise an element small enough to fit into or onto the needle tip. While the single-use component of the system—the FOH—is extremely low cost compared to piezoelectric transducers, one disadvantage is the cost of the wavelength-tunable interrogation laser, which is much greater than the electronics required to operate a piezoelectric element.
Previous FOH-based US tracking systems have relied on bespoke US transmission sequences dedicated to tracking [
28,
30] and/or modified US imaging probes [
38,
39]. Our next-generation system is capable of determining the needle position using only the US imaging transmissions, with the only modification to the US imaging system being the provision of a trigger signal. This overcomes a major hurdle in clinical translation: clinicians are able to use a familiar, commercial US imaging system featuring controls and image quality with which they are comfortable, rather than a research US system with unfamiliar controls and substandard image quality. While the UNT system was developed to operate specifically with a GE Voluson E10 C1-5-D convex US probe, the software was designed to allow the system to be used with other systems. The properties of the US imaging system upon which tracking is dependent can be manually changed in the UNT software using a configuration file—these include the imaging depth, scan angle (or width for a linear array probe), the number of scan lines per frame, the pulse transmission sequence, the radius of curvature of the probe, the coordinates of the centre of the probe face within the acquired US images, and the physical size of the pixels of the acquired US images. Many of these properties can be automatically obtained from analysis of a single high-gain US frame, as previously described, while others require proprietary knowledge of the US imaging system. Tracking is derived from the pulse transmissions intended for imaging, and therefore properties that affect US image resolution would also be expected to affect tracking performance, for example lateral scan resolution and focal depth. A limitation of the UNT system is that it is not currently possible to adjust (or automatically detect) the US imaging configuration during a procedure: the system must be restarted in order for changes to the configuration file to take affect, and while it may be possible to determine some US imaging properties in real time from the acquired US images or changes to the trigger signal, this has not yet been explored. US frame rates much higher than the 18 Hz for which the system was developed may also introduce tracking lag, as there would be less time for acquiring and processing each frame of data.
The UNT system’s positioning as an adjunct to existing medical devices—i.e., a US imaging system and needle—simplifies the regulatory processes required for clinical translation. The only change to the clinical procedure is the use of a needle with integrated FOH. A risk assessment carried out as part of the system’s development under the ISO quality management standard [
40] highlighted two areas of risk relating to this: the risk of tissue damage due to heating from the FOH; and the risk of the presence of the optical fibre trailing from the proximal end of the needle impeding the clinicians ability to manoeuvre the needle. The results of the temperature rise and ex vivo placenta experiments demonstrate that the FOH, even without its reflective tip in place, is not capable of causing thermal damage to tissue through laser-inducer heating. Formative usability tests with the system have indicated that the trailing optical fibre does not impede the motion of the needle. A further usability study involving a larger group of clinicians is planned.
In the homogenous medium, the system was found to have a spatial-average tracking error magnitude of 0.67 mm with a spatial-average measurement uncertainty of ±0.58 mm. Ninety-five percent of the scanned spatial positions had a tracking error lower than 1.5 mm, and 95% of measurements had an uncertainty less than ±1 mm. The tracking accuracy of the previous FOH system, which utilised a bespoke 1.5D tracking array and pulse transmission sequence, was only assessed down to a depth (i.e., distance from the US probe face) of 60 mm [
39], and a maximum absolute tracking error component (in any particular Cartesian direction) of 0.38 mm was reported. Restricted to this depth range, the accuracy of the current system is comparable: the lateral direction has the highest spatial-average absolute tracking error of 0.34 mm, with 95% of locations having an absolute error less than 0.9 mm. Spatial-average tracking error magnitude was 0.46 mm, with 95% of locations having an error less than 1 mm. Given the estimated measurement uncertainties, it is likely that the two systems have a similar tracking accuracy.
As expected, tracking performance was best in the radial direction; radial position was derived from time-of-flight and therefore benefited from the high sample rate and short US pulse length. Error and repeatability were worst in the tangential direction (the direction orthogonal to the radial direction), which was derived from the angle of the needle tip around the centre of curvature of the probe face and therefore adversely affected by the relatively poor spatial resolution of the radial A-lines of the US imaging system. This resulted in a poorer lateral accuracy than axial accuracy, on average. The tangential (and therefore lateral) accuracy worsened with depth as the gap between adjacent A-lines increased.
The results of the out-of-plane accuracy experiments demonstrate that, in water, signal-to-noise ratio remains better than 20 dB up to 30 mm either side of the imaging plane, even with the needle tip positioned far from the focal depth. The reduced out-of-plane tracking performance with the needle tip in the far-field is therefore likely due to the broader ultrasound beam profile, rather than loss of signal amplitude. This is evidenced by the worse lateral than axial tracking performance far from the imaging plane: the lateral ultrasound beam profile widens at elevational positions further from the imaging plane, increasing uncertainty in the determination of the centre-of-mass of the profile and therefore the lateral position of the needle tip. Axial tracking, which depends on time-of-flight measurement, is less affected. The results show that at the depth tested the UNT system is able to track the needle position within positions at least 15 mm either side of the imaging plane, and up to 30 mm either side with the needle tip in focus. With the needle tip in focus, out-of-plane signal-to-noise ratios were high enough that even with the additional attenuation introduced in tissue (shown in
Figure 13), out-of-plane tracking performance would likely be similar in vivo to that observed in water.
During fetal medicine procedures, the clinician aims to keep the needle visualised within the imaging plane at all times by adjusting the position and orientation of the US probe. Knowledge of the distance of the needle tip from the imaging plane may assist with this process. In anaesthetics, it is common for the needle to be inserted orthogonally to the imaging plane [
30], and therefore out-of-plane tracking is desirable. The high SNR provided by the UNT system combined with the US slice thickness enables the needle tip to continue to be tracked far outside the imaging plane. The software has the capability to visualise the tracking uncertainty or signal-to-noise ratio by changing the size, colour or opacity of the tracking cross-hair—this change in appearance of the cross-hair can be used to determine when the needle tip is in plane. Feedback from clinicians during usability studies has suggested that this method is useful for insertions into a tissue phantom. It was, however, challenging to configure the visualisation such that it was useful for indicating the distance of the needle tip from the imaging plane, and the resulting configuration was specific to that particular phantom. The method could not provide a quantitative elevational position of the needle tip, or even inform the user at which side of the imaging plane the tip was located. For these reasons, further work is required to improve the out-of-plane tracking provided by the UNT system. It is likely that the relative amplitudes of the signals received by the FOH from each aperture of the US probe encode its elevational position, as has been shown in similar work [
54], and some preliminary work has started looking at using machine learning techniques to decode this, as has been demonstrated in the literature [
55,
56,
57]. In [
54], a photoacoustic beacon, rather than FOH, is embedded in the surgical instrument, and transmissions from it are received by the imaging probe and used to determine its 3D position. The current system could potentially be adapted to track in 3D using this principal. The work reported in [
54] also makes use of a Kalman filter for smoothing tracked positions. The UNT software includes the option to enable a constant-velocity or constant-acceleration Kalman filter for reducing tracking jitter; however, usability tests determined that the apparent jitter in the position of the tracking cursor was preferable to the time-lag introduced by the Kalman filter.
In the heterogeneous tissue-and-water phantom, the average distance between tracked and true positions was 1.06 mm, nominally in the direction of the needle insertion. No such bias was seen in the homogenous-medium experiment. In the tissue-and-water phantom experiment, both the tracking system and the imaging system were operating under the same (incorrect) assumption of a homogenous sound-speed of 1540 m s
, and therefore any displacement due to sound-speed heterogeneity affects the image and tracked positions equally and do not cause an offset between the two. This suggests that the displacement is likely a residual difference between the apparent position of the needle tip in the US image and the true position of the tip of the embedded FOH; in contrast, the homogenous experiment demonstrated better tracking performance because the ground truths were derived from FOH signals rather than the US image of the needle tip. Previous work involving manual labelling of the position of a needle tip in water estimated the accuracy of manual labelling to be approximately ±0.5 mm [
46], so it is likely that human error is also a significant contributing factor. For our experiment, the needle tip was particularly difficult to locate precisely due to the short focal-depth used.
Tracking repeatability was excellent in water (spatial mean 0.28 mm) where there was little acoustic attenuation and therefore SNR greater than 60 dB. The results of measurements of SNR in the bovine tissue phantom suggest that in vitro signal to noise ratios could drop to 30 dB, resulting in worsened tracking repeatability of approximately 0.5 mm. The observed reduction in SNR is equivalent to an attenuation of approximately 1 dB cm
MHz
. This is significantly higher than literature values of acoustic attenuation in soft tissues [
58] and therefore suggests that the shape of the US field provided a significant contribution to the reduction in SNR. In the clinic, a clinician would be expected adjust the US focal depth to optimise the image of the needle, improving the SNR. However, the observed worst-case repeatability of approximately 0.5 mm is already smaller than the diameter of the needle and therefore should not cause issues in the clinic.
The real-time estimates of tracking uncertainty provided by the UNT system software were well correlated with but much higher than the measured tracking repeatability. While true tracking uncertainty of the system will be higher than the repeatability, including systematic as well as random contributions, the accuracy results presented here indicate that the uncertainty likely remains well below the software’s estimates, which reach up to ±5.5 mm. The uncertainty estimates are currently used to create a visual (but qualitative) indicator of tracking uncertainty. The ability to visualise a quantitative estimate of tracking uncertainty during operation would improve clinical user experience, and will be the subject of future work.
The 170 ms latency introduced to the video feed by the frame grabber has a noticeable effect, particularly when the needle changes direction. For a new UNT system currently under development, latency has been reduced to 70 ms through the use of an alternative frame grabber (ProCapture HDMI, Nanjing Magewell Electronics Co., Nanjing, China). This amount of latency is close to the threshold of human detectability [
59]. The software library written to enable the use of the device within a real-time Python application has been released as an open source Python package [
60].
The angle of the needle relative to the probe face was found have a significant effect on the amplitude of the raw FOH signal: signal amplitude was approximately 67% higher with a needle insertion angle of 40
than 0
. The shape of the directional response seen was different to what would be expected from a bare FOH [
53], presumably due to acoustic interaction with the needle tip. The amplitude of processed FOH waveforms did not have a significant dependence on insertion angle. Inspection of the FOH waveforms and frequency spectra at difference stages of processing (
Figure 11b) demonstrated that the additional amplitude at 40
was provided by frequency components outside of the bandwidth of the matched filter, causing the matched filter to equalise the directional response of the embedded FOH. It can be concluded that changing insertion angle does not adversely affect tracking performance.