1. Introduction
Small form factor lidar sensors have become commonplace within numerous research areas and commercial applications, including autonomous vehicle navigation [
1,
2,
3], mapping from unoccupied aerial systems (UAS), as known as remotely piloted aircraft systems (RPAS) [
4,
5,
6], terrestrial-based personal mobile mapping systems [
7,
8], and recently for improving imaging and augmented reality capabilities on mobile devices [
9,
10]. Before integrating a lidar sensor into a remote sensing platform it is fundamental to understand the capabilities of the sensor, and this can only be achieved by a systematic evaluation of the accuracy, repeatability, and stability of the sensor [
11]. Most commercially available, small form factor lidar sensors have already been systematically evaluated by researchers and the results published within literature (e.g., [
11,
12,
13,
14]). However, the Livox Mid-40 lidar sensor (
Figure 1a)—which became commercially available in 2019—has only been partially evaluated to date, as its angular observations have not been rigorously evaluated. The principal topics of this study are the angular observations of the Mid-40 sensor and its suitability for use within UAS-based lidar mapping applications.
Unlike a multi-line lidar sensor (e.g., Velodyne VLP-16) which uses a mechanical rotating array of multiple laser transceivers, the Mid-40 uses only a single laser transceiver that is in a stationary position and orientation relative to the sensor’s own coordinate system, similar to most terrestrial laser scanners (TLS). However, unlike TLS which commonly use mirrors to reflect the laser beam, the beam of the Mid-40 is steered using a Risley prism optical mechanism consisting of two wedge-shaped glass prisms aligned sequentially in the direction of the optical scan axis (
Figure 1b). Each prism can rotate about the optical scan axis resulting in the incident beam being refracted by the glass prisms. The emergent beam is deviated in a direction according to the relative orientation of the prisms with respect to each other [
15].
Using a Risley prism mechanism allows for a wide range of unique scan patterns to be realized simply by adjusting the rotation rates and directions of each prism, see [
15] for complete details. The scan pattern observed by the Mid-40 is commonly described as a rosette pattern and it occurs within a circular field of view (FoV) around the optical scan axis (
Figure 2a). The rotation rates for the glass prisms inside the Mid-40 are set so the lidar observations are not repetitively taken along a single projected path within the FoV, rather the observation density within the FoV increases with time (
Figure 2b). The observation density within the FoV is nonhomogeneous with a peak at the center, similar to the response of the human retina [
16].
In comparison to the laborious manufacturing techniques used for mechanical multi-line lidar sensors, the Mid-40 is mass produced using a straightforward approach which enables the sensor to be sold at a lower cost [
16]. In addition to the lower cost, the reported accuracy specifications and physical properties of the sensor make it an attractive option for UAS lidar mapping applications (
Table 1). However, in order to fully understand the quality of the point clouds produced by the Mid-40, an assessment of the sensor’s accuracy specifications is needed. The primary research of this study is a rigorous examination of the systematic errors that affect the raw angular observations (i.e., azimuth angle and zenith angle) of the Livox Mid-40 lidar sensor for the purposes of accuracy assessment.
To date, two studies have been published that provide an initial assessment of the accuracies of the Mid-40. In Ortiz Arteaga et al. [
18], the range and angular observation accuracies of the sensor were examined by comparing features within the collected point clouds (i.e., signalized targets and planes) to the same features within a more precise reference point cloud collected by a TLS. The study concluded that at longer standoff distances—40 m to 130 m—the Mid-40 achieved range measurements to an accuracy of ±20 mm. The angular accuracy was inferred by a trigonometric approximation of the ratio of the observed length error between known signalized targets over the standoff distance between the Mid-40 and the targets. The study concluded the angular accuracy of the Mid-40 to be 0.1
, but reported that an angular systematic effect may be present based on the appearance of concentric “ripples” of noise artifacts within some of the collected point clouds. The artifacts were believed to be correlated with the deflection angle of the emergent beam with respect to the optical scan axis and appeared within point clouds of planar surfaces that were observed from small standoff distances.
In Glennie and Hartzell [
19], the range and angular observation accuracies were again examined by comparing the collected point cloud datasets to a more precise reference point cloud from a TLS. The analysis included 127 planar comparisons collected from standoff distances between 3 m to 35 m, and from incidence angles between approximately 5
and 85
. The study concluded that the Mid-40 exhibited two classifications of range accuracies, ±8 mm for range observations greater than 20 m and ±21 mm for range observations less than 20 m. The angular accuracy was not explicitly analyzed, but a visual examination to identify any systematic effects correlated to the angular observations was conducted, see [
19] for complete details. An attempt was made to duplicate the systematic “ripple effect” reported in [
18], but no point cloud artifacts were found. The study concluded that no obvious systematic trends in the angular observations were observed.
Refs. [
18,
19] quantified the range accuracy of the Mid-40 to be in agreement with the manufacturer’s specification of ±20 mm and provided some evidence that the angular accuracy agrees with the manufacturer’s specification of <0.1
. However, both studies stated that the lack of information regarding the internal hardware configuration and operating principles of the Mid-40 restricted the ability to perform further analysis and uncover additional systematic errors.
This study presents the development of an angular observation model for the Livox Mid-40 lidar sensor. The observation model captures the physical prism parameters and systematic errors that affect the incident beam as it travels and refracts through the Risley prism steering mechanism and exits the sensor as the observable emergent beam. The observation model is used within an extended Kalman filter (EKF) to suitably represent the physical dynamics of the sensor and to statistically estimate the model parameters and the rotation angles for the individual prisms that correspond to the sequential azimuth and zenith observations. The filter’s inputs are the azimuth and zenith observations output directly by the Mid-40 sensor.
An initial analysis is performed using actual observations from a Mid-40 sensor to quantify the accuracy of the angular observation model and provide realistic estimates for the systematic errors. A secondary analysis is performed using simulated input observations—with a known amount of noise—to illustrate the filter’s ability to estimate the systematic errors and the prism rotation angles accurately and precisely. A final analysis is performed using simulated observations from an incorrectly calibrated sensor to a planar surface, to illustrate that the angular observations from an incorrectly calibrated sensor can be improved by using the range observations and applying a planar constraint within a least-squares adjustment. Lastly, a discussion on possible systematic errors within the range observation and the Cartesian coordinates representation of the observations is presented.
2. Materials and Methods
When discussing a conventional Risley prism steering mechanism there are four possible configurations for the internal arrangement of the two wedge-shaped glass prisms. Each prism has a perpendicular face (P) and an angled face (A) with respect to the optical scan axis. Therefore, the four configurations are PA-PA, PA-AP, AP-AP and AP-PA (
Figure 3). Though each configuration follows the same generalized mathematical form for the direction of the emergent beam, selecting the correct configuration for the specific Risley prism mechanism being studied is important. Based on the information presented in [
16], the Mid-40 Risley prism configuration is PA-AP.
2.1. Ideal Angular Observation Model
The angular observation model for a Risley prism steering mechanism starts with the unit vector of the incident beam and computes four successive refractions—using the 3D vector form of Snell’s law (Equation (10))—at the perpendicular and angled faces of each prism. Under ideal conditions, the incident beam, the planar normal on the perpendicular face of prism A, the axis of rotation for both prisms, and the planar normal on the perpendicular face of prism B are all collinear with the optical scan axis. The deflection angle between the perpendicular planar normal and the angled planar normal is equal to the wedge angle of the prism. At the zero position of the prisms, all four planar normals for the faces of the prisms lie in a common vertical plane (
Figure 4). The prisms are geometrically identical and made of the same homogeneous medium with a precise refractive index. The prisms rotate at constant angular velocities and the rotation angles for each prism—with respect to the zero position—at the time of observation are expressed by:
where
and
represent the rotation angles for prism A and prism B at the time of observation
, respectively, and
and
represent the constant angular velocities for prism A and prism B, respectively.
The reference coordinate system for the ideal observation model is defined as a right-handed system with its origin at the intersection of the axis of rotation of prism B and the perpendicular face of prism B. The optical scan axis defines the X coordinate axis and the common vertical plane contains the orthogonal Z coordinate axis (
Figure 4). The unit vectors for the direction of the incident beam (
), the planar normals for all four prism faces (
,
,
,
), and the axes of rotation for the prisms (
,
) are expressed by:
where
is the wedge angle of the prisms.
The 3D vector form of Snell’s law, modified from [
20], is expressed by:
where
and
(for
i = 0, 1, 2, 3) represent the unit vectors for the direction of the initial beam and the refracted beam, respectively.
represents the unit vector for the direction of the planar normal on the refracting prism face, as expressed in Equations (4)–(7).
and
represent the refractive indices for the mediums (i.e., air or glass) on the initial and refracted sides of the prism face, respectively.
Once the unit vector for the direction of the emergent beam (
) has been determined, the equivalent azimuth and zenith angular observations—as defined with respect to the reference coordinate system of the Mid-40 (
Figure 5)—are expressed by:
where
and
are the azimuth and zenith angular observations, respectively, and
,
,
are the components of the unit vector for the emergent beam.
2.2. Practical Angular Observation Model
Several studies [
20,
21,
22] have been published that discuss the error sources and their effects on the angular accuracy of Risley prism-based observations. Li et al. [
20] state that the systematic error sources of a Risley prism mechanism include component errors and assembly errors. The component errors include the errors in the wedge angle and refractive index for each prism. The assembly errors include the tilt errors, axis of rotation alignment errors, and rotational position (i.e., encoder) errors for each prism, and also the incident beam direction error.
In reality, some of the assumptions made within the ideal angular observation model do not hold true. Within the practical angular observation model it is still assumed that the prisms are geometrically identical and made of the same homogeneous medium. However, the assumption that the planar normals for the faces of each prism all lie in a common vertical plane is no longer valid. Rather, for each prism a unique plane exists that contains the prism’s two planar normals and defines the so-called
principal section of the prism (
Figure 6). The zero position for each prism (i.e.,
) is defined when its principal section is orientated vertically. Only by determining the principal section of each prism can the initial orientation of each prism be accurately calibrated and defined as the reference for calculating the prism rotation angle [
20].
The reference coordinate system for the practical observation model is defined as a right-handed system with its origin at the intersection of the axis of rotation of prism B and perpendicular face of prism B. The axis of rotation of prism B (
) defines the X coordinate axis and the vertically oriented principal section of prism B contains the orthogonal Z coordinate axis (
Figure 6). Equation (9) from the ideal observation model still correctly defines the unit vector for
. However, the axis of rotation for prism A (
) is no longer considered collinear with
, and is affected by the so-called
bearing tilt error. As a result, the unit vector for
is now expressed by:
where
and
represent the bearing tilt error for the axis of rotation of prism A within the azimuth (XY) plane and corresponding zenith (⊥ XY) plane, respectively.
The unit vector for the direction of the incident beam (
) is also no longer considered collinear with
, and is affected by the so-called
incident beam direction error. As a result, the unit vector for
is now expressed by:
where
and
represent the incident beam direction error within the azimuth (XY) plane and corresponding zenith (⊥ XY) plane, respectively.
The unit vectors for the direction of the planar normals for all four prism faces are affected by the so-called
prism tilt errors for each respective prism. As a result, the unit vectors at the zero position of each prism (
,
,
,
) are expressed by:
where
and
represent the tilt error within the azimuth (XY) plane and corresponding zenith (⊥ XY) plane for prism A, respectively, and
and
represent the tilt error within the azimuth (XY) plane and corresponding zenith (⊥ XY) plane for prism B, respectively.
The unit vectors for the direction of the planar normals for all four prism faces at the time of observation (
,
,
,
) can be determined by rotating the unit vectors at the zero position about the axis of rotation for each prism by the respective rotation angles
and
. The rotation angles at the time of observation are defined by Equations (1) and (2). The unit vector for the axis of rotation for prism A (
) is non-trivial, as expressed in Equation (13), because
suffers from bearing tilt error. As a result, a quaternion representation is used to perform the rotation of
and
about
by the rotation angle
. The
and
unit vectors are now expressed by Equations (23) and (24).
where
represents a unit quaternion and the terms
i,
j,
k are the fundamental quaternion units, the terms
,
,
are the components of the unit vector
, the terms
,
,
are the components of the unit vector
, the terms
,
,
are the components of the unit vector
, and
represents the Hamilton product.
The axis of rotation for prism B (
) defines the X axis of the reference coordinate system, therefore the
and
unit vectors are simply expressed by:
Once the unit vectors for the directions of the planar normals at the time of observation have been defined, the unit vector for the direction of the emergent beam () can be determined. Equations (14) and (23) are substituted into Equation (10) which determines the unit vector for the direction of the first refracted beam (). Equation (10) is applied three more times by substituting in Equations (24)–(26) to determine the unit vectors , , and , respectively. Lastly, Equations (11) and (12) are used to convert the unit vector into the equivalent azimuth and zenith angular observations.
Within the practical angular observation model a total of 13 model parameters are needed to determine the azimuth and zenith observations for the emergent beam at the time of observation. A time of observation equal to zero corresponds to the prisms being in their zero positions, as previously discussed. The model parameters are summarized in
Table 2.
2.3. Observability of the Model Parameters
Liu et al. [
16] state that the Mid-40 is composed of two identical prisms with a refractive index of 1.51, wedge angles of 18
, and angular velocities of
4664 revolutions per minute (RPM) and 7294 RPM for prisms A and B, respectively, following a right-hand rule for positive rotation around the X coordinate axis. However, the refractive index of air, the refractive index of the prisms, and the wedge angle of the prisms model parameters are perfectly correlated due to the use of Snell’s law within the observation model. As a result, the refractive index of air is set to a constant of 1.00 and the wedge angle of the prisms is set to a constant of 18
within the observation model. The
refractive index of the prisms model parameter is not fixed to a constant value to allow it to be optimized based on the sensor’s observations.
The azimuth and zenith angular observations—which are reported to a precision of 0.01—are the raw outputs available from the Mid-40 sensor. It is important to analyze the sensitivity of the output observations with respect to changes in the model parameters, as well as the correlations between the model parameters, in order to understand the observability of the model. Using an extended Kalman filter—as described in the next section—analyses were performed to study the nature of these sensitivities and correlations. It was discovered that changes in all of the model parameters, except for the horizontal tilt error for prism A (), were observable with respect to changes in the azimuth and zenith observations. The horizontal tilt error for prism A demonstrated to be relatively insensitive to changes in the azimuth and zenith observations and indicated a strong negative correlation (p = 0.99) with the computed estimates for the rotation angle for prism A () and the vertical tilt error for prism A model parameter (). As a result, the horizontal tilt error for prism A model parameter was assumed to be zero, and therefore had no effect on the practical observation model and was removed.
2.4. Estimation of the Model Parameters and Prism Rotation Angles Using a Kalman Filter
The practical angular observation model is utilized within an EKF to estimate the model parameters and the rotation angles of each prism at the observation times. The EKF is processed in both the forward and backward time directions, and the respective results are combined to produce an optimal smoothed estimate [
23]. The EKF dynamics model for the system is expressed by:
where
and represents the state transition noise.
The EKF measurement model for the system is highly non-linear and requires the use of Equations (13)–(26) and four successive applications of Equation (10) before the following measurement equations can be utilized:
where
and represents the measurement noise.
The EKF is initialized by examining the output azimuth and zenith observations and finding a time of observation that indicates the prisms are approximately in their zero positions. This occurs when the azimuth observation is approximately
and the zenith observation is approximately at its maximum value (
). This time of observation is set equal to zero and all observations preceding this instance are disregarded and all proceeding observation times are adjusted accordingly. The EKF state variables are initialized as:
It was discovered that a Mid-40 sensor—installed with v03.08.0000 firmware—operates using two different combinations of angular velocities for the prisms. The first combination uses nominal values of (—4664 RPM) and (7294 RPM) for and , respectively, while the second combination uses nominal values of and for and , respectively. The combinations alternate in occurrence when the sensor is power cycled, and it is speculated that this increases the lifespan of the brushless direct current (BLDC) motors that rotate the prisms. Therefore, the initial values for the 2nd and 3rd vector elements within Equation (30) must change when the second combination of angular velocities is experienced during data collection.
2.5. Estimation of the Model Parameters Using a Planar Constrained Least-Squares Adjustment
The practical angular observation model is also utilized within a non-linear least-squares adjustment to estimate the model parameters—excluding the angular velocities and the
refractive index of the prisms—based on constraining the resulting Cartesian coordinates of the observed points to best-fit a planar surface. The point coordinates are calculated within the reference coordinate system of the Mid-40 using Equation (31):
where
represent the Cartesian coordinates of point
, and
represent the range, azimuth, and zenith observations from a Mid-40 sensor to point
, respectively.
Before the adjustment begins, the input azimuth and zenith observations are processed using the EKF—as previously discussed—to estimate the current model parameters and the instantaneous rotation angles of the prisms that correspond to the input observations. Each iteration of the least-squares adjustment begins with computing new estimates for the azimuth and zenith angular observations for all points, using the practical angular observation model and the current estimates for the model parameters and the prism rotation angles. The point coordinates are then calculated and used within a singular value decomposition (SVD) problem to estimate the current best-fitting plane based on all the point coordinates. The perpendicular distances between the points and the best-fit plane are then calculated and used to define the minimization criteria for the least-squares adjustment. Equation (32) illustrates the unweighted least-squares solution:
where
represents the corrections to the current estimates for the model parameters,
represents the Jacobian matrix containing the partial derivatives of the point-to-plane distances with respect to the model parameters, and
represents the vector containing the negated values of the point-to-plane distances. The solution is repeated until convergence of the standard deviation of unit weight is achieved.
4. Discussion and Conclusions
Within the analyses of this study, real and simulated angular observations for a Mid-40 sensor were used to demonstrate that the practical angular observation model is accurate and precise, and enables rigorous uncertainty estimation of the angular observations from a Risley prism beam steering mechanism to be performed. The residuals between the model’s angular observations and those from an actual Mid-40 sensor were shown to be normally distributed with a mean of
and a standard deviation of
. This suggests that the angular observations for the Mid-40 sensor are more accurate than the manufacturer’s specification of <
. However, this finding does not take into consideration the angular uncertainty associated with the direction of the emergent beam due to beam divergence. The previous studies [
18,
19] provided inferred and qualitative evidence that the angular accuracy of the Mid-40 sensor was in agreement with the manufacturer’s specification, but also stated that the lack of information regarding the internal hardware configuration and operating principles of the sensor restricted further analysis. This study was performed to overcome these restrictions, by introducing an angular observation model and providing a rigorous accuracy assessment of the Mid-40’s angular observations.
Using the angular observation model within an extended Kalman filter allows for the azimuth and zenith observations to be used to estimate the model parameters as well as a set of lower-level observations, namely the rotation angles for each prism. The filter can accurately estimate the model parameters and the rotation angles for the prisms even when the prisms’ angular velocities are not constant. The inherent benefit of the prism rotation angles is that they are not affected by the systematic component errors and assembly errors of the sensor. By using the prism rotation angles and the corresponding range observations, an independent estimation of the sensor’s calibration values can be performed, and provides a method for improving the angular observations of an incorrectly calibrated Mid-40 sensor. However, the uncertainty in the range observations does set a limit to the achievable accuracy of the independent estimation.
In addition to the systematic errors that affected the angular observations, earlier research conducted as part of this study included analyzing errors within the range observation and the Cartesian coordinate representation of the azimuth, zenith, and range observations. No obvious systematic errors within the range observation could be identified; if any errors remain, they are likely non-geometric in nature and small in magnitude. Within the Cartesian coordinate representation of the observations a systematic error was discovered, and was a result of the emergent beam not coinciding with the origin of the sensor’s reference coordinate system. The error caused the Y and Z coordinates to be incorrect by upwards of 4.5 mm, see the
Appendix A for details. Future work will include (1) conducting experiments using real Mid-40 angular and range observations to planar surfaces and estimating an improved set of calibration values for a sensor and quantifying that improved angular observations can be realized; (2) analyzing the angular observations from a triple Risley prism lidar sensor (e.g., Livox Horizon and Livox); and (3) studying the spatial distribution properties of a point cloud generated from a UAS lidar mapping system that utilizes a Mid-40 sensor.