2.1.1. The Principle of Binocular Stereo Imaging
In response to the specific requirements of night-time vehicle driving and rapidly changing work environments, we designed a wide-field-binocular stereo perception system with dual-band-electro-optical imaging. The visiblelight-imaging component incorporates a large-array CMOS device from Rockchip Electronics Co., Ltd. (Jiangsu, China) that has high sensitivity and definition. This system boasts a pixel resolution of 1920 (H) × 1080 (V), with each pixel measuring 13 μm (H) × 13 μm (V), and it operates at a frame rate of 50 Hz. This component can function effectively under low-light conditions down to 10−3 lx. Furthermore, the infrared-imaging component employs a non-cooled infrared focal plane detector from Yantai IRay Technology Co., Ltd. (Yantai, China) that offers a pixel resolution of 1024 (H) × 768 (V), with each pixel measuring 14 μm. The frame rate of the infrared component is also 50 Hz.
Binocular stereo sensing systems are mainly divided into two types according to their placement [
27]: a parallel model and a convergent model. The parallel model has two camera optical axes parallel to each other. Moreover, the structure is simple and easy to calculate. The advantage of this structure is the presence of only a negative horizontal parallax, with no vertical parallax. Disadvantages include a small common area and a lack of stereoscopic information in the left and right sides of the single viewing area, which will cause a waste of information. The convergence model can adjust the angle between the two optical axes to obtain a larger effective field-of-view, with positive-, negative-, or zero-horizontal parallax; however, the camera body will produce vertical parallax, which causes a certain gradient distortion. In this study, considering the advantages and disadvantages of the parallel model and the convergence model, the visible and infrared components were fused. For this purpose, we selected an optical design based on the parallel model, as shown in
Figure 3. To realize binocular stereo vision, the binocular stereo sensing system consisted of two visible light-objective lens groups and two infrared-objective lens groups.
Binocular-imaging-distance-measurement technology relies on the binocular disparity to establish an ideal model for binocular ranging. In this model, both cameras have identical specifications and parameters, including matching camera models, consistent focal lengths, and the parallel alignment of optical axes. The model is shown in
Figure 4.
Here,
represents the camera’s inter-image plane spacing (baseline width),
and
are the distance from the image point to the image plane center,
is the camera’s focal length, and
is distance to the object. Using the principles of similar triangles, the following relationships can be derived:
where
represents the difference in the imaging positions of the object point in the two fields of view, which is commonly referred to as the parallax value.
Under the conditions of parameter determination in a binocular-stereo-perception system (such as focal length and baseline), ranging accuracy is determined by the parameter
. Presently, binocular stereo-matching algorithms can achieve sub-pixel-level matching accuracy, resulting in superior disparity precision. Results can be obtained by differentiating Equation (1):
Hence, as long as the parameters of the binocular-stereo-perception system are determined, the distance to the target can be calculated by measuring the disparity. System parameters typically consist of intrinsic and extrinsic parameters. Intrinsic parameters include the focal length, principal point coordinates, and distortion coefficients of the left and right cameras, among others. Extrinsic parameters encompass the relative transformation between the left and right cameras, involving rotation and translation matrices. Due to potential errors during camera installation, such as non-parallel alignment of the lens and imaging plane, it is necessary to recalibrate the camera to obtain updated focal lengths, intrinsic parameters, and extrinsic parameters.
2.1.2. The Simulation and Design of Visible- and Infrared Objective Lenses
- A.
Simulation design for the visible light objective lens
In the wide-field binocular stereo perception-optical-imaging system, the visible light component employs a high-resolution and low-light CMOS-imaging module from Rockchip Electronics Co., Ltd., which enables imaging in both day and night scenes. This component features large pixels and sensor-imaging areas, imposing stringent requirements on the optical system. We utilized the CODE V(10.2) software for optical system design, implementing a “telephoto-type” optical path structure with 12 lenses. By introducing appropriate non-spherical elements while keeping the total number of lenses, glass thickness, and imaging quality constant, we enhanced the light-gathering capabilities of the visible light objective lens without compromising its transmittance. The system design is depicted in
Figure 5.
Table 1 presents the optical-design specifications for the visible light objective lens.
The maximum effective range
of the visible light-optical-imaging system is as follows:
where
represents human height,
represents shoulder width, and
represents pixel size.
At a distance of
, there is a depth-calculation deviation of
. Under the condition of a matching algorithm precision at the 0.1 pixel level, the baseline
can be determined as follows:
where
represents the distance from the target to the system, which is referred to as the test distance.
is set at a 0.1 pixel level, and
stands for the resolvable distance.
At a distance of
, the depth-calculation deviation of
. When these values are incorporated into Equation (4), the following result is obtained:
It can be seen clearly that
. The formula for visible light stereo acuity
is
where
is a constant of 206,265 when converting from radians to arcseconds:
Stereopsis is the ability to resolve the smallest horizontal disparity between retinal images from both eyes. The normal value for stereopsis should be less than . A smaller value of stereopsis indicates better stereo vision.
Utilizing CONE V software, we simulated environmental temperature variations and obtained the transfer functions for the visible light-optical system at 20 °C, 50 °C, and −40 °C, as well as the diffuse spots, as shown in
Figure 6,
Figure 7, and
Figure 8, respectively.
This paper addresses the design of a wide-field, relative aperture long-wave infrared optical system for the 1024 (H) × 768 (V) long-wave infrared detector from Yantai IRay Technology Co., Ltd. The infrared objective lens system, as depicted in
Figure 9, operates without active cooling over a wide temperature range. The front surface of the first lens is non-spherical, enabling passive temperature compensation at different temperatures by adjusting the system’s back focal length.
Table 2 provides the optical-design specifications for the infrared objective lens.
The maximum effective range
of the infrared objective lens-optical system is
At a distance of
, with a depth-calculation deviation
, the baseline
can be calculated as follows:
At a distance of
, with a depth-calculation deviation
, the baseline
can be calculated as follows:
According to Formulas (9) and (10),
The formula for the stereo acuity
is as follows:
where
is the baseline length,
is the resolvable distance,
is the distance from the target to the system, and
is the binocular ranging accuracy. The formula is as follows:
With a decrease in temperature, the infrared optical system experiences changes in inter-lens spacing, lens thickness, refractive index, and curvature radius. Therefore, temperature variations inevitably lead to defocusing of the system’s focal plane, resulting in a degradation of image quality. The transfer functions and diffuse spots of the infrared optical system based on system simulation analysis are shown in
Figure 10,
Figure 11, and
Figure 12, respectively, at temperatures of 20 °C, 50 °C, and −40 °C.
In summary, without moving any optical elements, we achieved alignment of the image plane with the detector target surface during changes in environmental temperature. With temperature variations, the lens assembly, employing optical passive thermal compensation, can maintain consistent magnification and requires no active optical components. The relative positions of the optical axes remain relatively unchanged, resulting in high image registration accuracy. Large-field optical systems often exhibit significant distortion to improve the field-of-view. Image distortion is corrected to obtain high-quality fused images. Based on the imaging quality of the fused lens system and image registration effectiveness, further improvements in the system’s ranging accuracy were achieved.
Baseline distance
; if baseline distance
, then the ranging accuracy is as follows:
Table 3 presents the calculated accuracy of visible and infrared ranging at test distances of 15 m and 30 m.