Environment Perception with Chameleon-Inspired Active Vision Based on Shifty Behavior for WMRs

Featured Application

Inspired by the visual behavior mechanism of the chameleon predation process, this paper proposes a chameleon-inspired active-vision-based environment perception strategy based on a shifty-behavior mode for mobile robots. It achieved the reproduction of the visual behavior of chameleons in the vision system of mobile robots with satisfactory results, which is of great significance for improving the environmental perception ability and adaptability of mobile robots.

Abstract

To improve the environment perception ability of wheeled mobile robots (WMRs), the visual behavior mechanism of the negative-correlation motion of chameleons is introduced into the binocular vision system of WMRs, and a shifty-behavior-based environment perception model with chameleon-inspired active vision for WMRs is established, where vision–motor coordination is achieved. First, a target search sub-model with chameleon-inspired binocular negative-correlation motion is built. The relationship between the rotation angles of two cameras and the neck and the camera’s field of view (FOV), overlapping angle, region of interest, etc., is analyzed to highlight the binocular negative-correlation motion compared with binocular synchronous motion. The search efficiency of the negative-correlation motion is doubled compared with binocular synchronous motion, and the search range is also greatly improved. Second, the FOV model of chameleon-inspired vision perception based on a shifty-behavior mode is set up. According to the different functional requirements of target searching and tracking stages, the shift of the robot visual behavior is analyzed from two aspects, measuring range and accuracy. Finally, a chameleon-inspired active-vision-based environment perception strategy for mobile robots is constructed based on the shifty-behavior mode, and experimental verification is deployed, which achieves the reproduction of the visual behavior of chameleons in the vision system of mobile robots with satisfactory results.

1. Introduction

To enable robots to perceive and acquire environmental information with their eyes, such as with humans or animals, and to quickly and accurately identify and track targets, people have attempted various methods to simulate human or animal eyes or reproduce intelligent behaviors related to biological vision, leading to the emergence of machine vision technology [1,2,3]. Active vision, as opposed to traditional passive vision, emphasizes the activity and selectivity of the visual information acquisition and processing process, i.e., the interaction ability between the visual system and its environment [4,5,6]. A chameleon is called a “lion on the ground” because its visual system plays an important role in its defense and predation. A chameleon’s two eyes can move independently without constraint for panoramic observation, and they can also move synchronously for binocular stereo vision, accurately measuring the depth information of prey, thus allowing its tongue to precisely capture the prey [7,8,9,10]. The independent eye movements of a chameleon are shown in Figure 1.

Many scholars have studied the specificity of the behavioral mechanism of the vision system of chameleons. Ott and Schaeffel noted that, unlike other vertebrates, the eyes of chameleons have a negative focal mechanism that allows them to calculate target information more precisely [11,12,13]. In addition, Ott pointed out through further studies that during the sweeping of the environment by a chameleon’s two eyes with the high-speed motion of the prey, the two eyes move with different amplitudes and alternate between fixed-focus and non-focus states. Therefore, when a chameleon rapidly searches for targets in the environment, the motions of its two eyes are independent of each other, and this is known as the negative-correlation saccade pattern, which greatly improves the efficiency of scanning the environment. In addition, since a chameleon has relative motion between its eyes and head, the environmental search range of a chameleon can be further improved by the dual motion of the head and eyes. Avni used an experimental design to study the gaze direction of the eyes of chameleons when scanning the environment for prey targets, and the experimental results confirmed that the eyes of chameleons obeyed a “negative-correlation” motion pattern when scanning the environment for targets [14,15]. In addition, Prasad applied the idea of a chameleon’s eyes to a missile propulsion controller for obstacle avoidance [16]. Xu studied the coordinated motion of a mobile robot equipped with a chameleon-inspired vision system during target tracking [17]. Zhao from Yanshan University applied it to a compass parallel robot [18,19]. Zhou and Tsai applied the binocular negative-correlation-motion mechanism of chameleons to a surveillance system to improve the flexibility of surveillance [20,21]. Chen constructed a vision-based pose alignment platform inspired by a chameleon for multi-position prey capture and a concept verification demonstration [22].

In previous research, most robotic vision systems have adopted the binocular cooperative motion search mode, such as the Mars rover and lunar rover. At present, the mainstream technologies of binocular vision include stereo vision, structured light, and time of flight (TOF). Based on the principle of the human oculomotor nervous system, Liu proposed an algorithm to simulate the human oculomotor system using a binocular motion system [23]. Zhang designed the device and algorithm of bionic eye anthropomorphic bionic vision and realized the function of six-degrees-of-freedom vision control and binocular stereo vision control [24,25]. To overcome the shortcomings of narrow binocular vision and low monocular depth perception accuracy, Wang and Zou proposed a method to simulate the pursuit and saccade actions of the human visual system, which improves the accuracy of 3D coordinates obtained at the best observation position [26,27]. In response to the depth measurement errors generated by bionic eyes, Wang proposed some effective bionic eye design guidelines [28]. Fan and Liu proposed a 3D triangulation method based on eye gaze to improve the measurement error of bionic eyes [29]. Chen proposed an integrated dual attitude calibration method to obtain accurate and stable head–eye parameters and kinematic models [30]. In addition, many scholars have designed different forms of humanoid vision systems. Chen developed a bionic compound eye based on the neural network of a fiber bundle stereo, which can quickly and accurately acquire the 3D spatial information of a target [31]. To improve the lack of rotational freedom in most bionic eyes, Chen and Wang designed a three-degrees-of-freedom tandem-parallel vision mechanism that enables the eccentric-free spherical motion of a bionic eye [32]. In addition, machine learning algorithms are widely used in the field of image recognition and trajectory tracking for robots, which can significantly improve the accuracy and stability of control [33,34,35,36].

When autonomous mobile robots work in the field, one of their most essential tasks is to use their binocular vision system to quickly scan the surroundings over a wide area to discover and target an object of interest. Improving the search efficiency and expanding the search range is a tricky and urgent problem. In previous studies, most of the robot binocular vision systems have used the following two search modes: (1) the cameras are fixed, and the mobile platform rotates to search; (2) the two cameras synchronously rotate to search, which is inspired by the human visual system. These two search modes have a large binocular correlation, symmetry, and a large overlapping field of view, so the search efficiency is low, and the search range is limited. In contrast, the binocular independent-motion vision system of a chameleon can realize both independent motion for panoramic rapid scanning and cooperative motion for stereo vision, which has absolute advantages in rapid target searching. However, the current research on the visual behavior of chameleons mainly focuses on the experimental study of the negative-correlation-motion mechanism and its application in monitoring systems. There are still few applications for the visual behavior mechanism of chameleons in the vision systems of wheeled mobile robots (WMRs).

Therefore, in this paper, according to the visual function requirements of the robot at different stages, we propose a bionic binocular active-vision-based environment perception model for WMRs equipped with a binocular vision system based on shifty behavior and inspired by the predation process of chameleons, which can improve the search efficiency and expand the search range. The shifty-behavior model emphasizes the shift from one behavior to another to achieve different functional goals, and in this paper, it refers to the behavior shift of the vision system from binocular negative-correlation motion to binocular synchronous motion in order to achieve a wide range of rapid environmental scanning and the accurate localization and tracking of targets. In this process, the coordinated motion of the robot body and the vision system with the neck is achieved, which is referred to as vision–motor coordination, which is an important functional requirement in active-vision environment perception.

2. Visual Perception Modelling in Chameleon-Inspired Shifty-Behavior Mode for WMRs

The construction process of the chameleon-inspired vision system for WMRs is shown in Figure 2. First, considering that a chameleon’s eyes have zoom and rotation characteristics, which can achieve approximately 180° rotation in the horizontal direction and 90° rotation in the vertical direction, and they have negative-correlation motion characteristics, dual PTZ cameras were used to imitate the visual behavior mechanism of chameleons. Furthermore, the rotation of the neck can further improve the search range and search flexibility. Second, once a chameleon finds a target, it will first lock on to the target with one eye, and then quickly adjust the other eye to focus on the target, thus forming binocular stereo vision to accurately determine the location of the target. Finally, through real-time feedback from the vision system, the robot achieved an accurate tracking of the target.

In the field of mobile robots, the optical zoom type has obvious advantages. Therefore, an optical zoom lens was used in this paper. To adapt to the robot’s working requirements, an avenir-SSL06036M 6× zoom lens was selected as the robot lens in this paper. Zoom, focus, and iris were controlled by a 12v voltage signal. To realize a more convenient connection with industrial computers and reduce the number of intermediate transformation steps, a CMOS camera with a USB interface was chosen. To match the lens, a camera with a target surface size of 1/3″ was chosen. To adapt to the mobile robot tracking target, the update frequency of CMOS was maintained above 30 fps to ensure a resolution of 320 × 240, thus ensuring the success rate of target searching and tracking. Therefore, a Suntime-200 industrial camera was selected in this paper. The specific parameters of the lens and CMOS are shown in

Table 1. Parameters of lens and CMOS.

Sensors	Parameters	Values
Lens	Model	SSL06036M
	Size of target surface	1/3″
	Focal length	6.0–36.0 mm
	Minimum focus distance	1.3 m
	Back focus distance	12.50 mm
CMOS	Model	Suntime-200
	Pixel size	3.2 μm × 3.2 μm
	Scanning type	Line-by-line scanning
	Resolution	320 × 240, 640 × 480, 1280 × 1024
	Transmission rate	30 fps: 320 × 240, 640 × 480

.

When a mobile robot with a chameleon-inspired vision system searches for targets, it involves changes in the position and the orientation of the camera imaging and the PTZ servo mechanism. Therefore, according to the constructed vision platform, a coordinate system was established as shown in Figure 3, and all coordinate systems conformed to the right-hand rule [37].

In the above figure, $W,R$ denotes the world coordinate system and the robot body coordinate system, both located at the geometric midpoint of the body differential mechanism; $G_b$ denotes the neck coordinate system, located at the center of the PTZ beam; $G_l,G_r$ denote the left and right PTZ coordinate system, located at the rotation centers of the left and right sides of the PTZ; $C_l,C_r$ denote the left and right camera coordinate system, located at the optical centers of the left and right cameras; $X_l,X_r$ denote the left and right camera imaging coordinate system; and $I_l,I_r$ denote the left and right camera image coordinate system; $P_{cl}\left(x_{cl},y_{cl},z_{cl}\right)$ denotes the coordinates of the target point in the left camera coordinate system; $P_{xl}\left(x_{xl},y_{xl}\right)$ denotes the coordinates of the target point under the left camera imaging coordinate system; $P_{il}\left(u,v\right)$ denotes the coordinates of the target point under the left camera image coordinate system.

During the process of environmental perception using their eyes, chameleons adopt a shifty-behavior mode according to the functional requirements at different stages. In the target search stage, the binocular negative-correlation motion mechanism with the wide-angle camera mode is used to perform a rapid wide-range scanning of the environment. In the target tracking stage, once the target is detected, the two eyes are coordinated and synchronized to achieve stereo vision, which precisely measures the depth information of the target and makes the outstretched tongue catch the prey just right, and the focal length is appropriately increased to improve the measurement accuracy. Based on the visual behavioral mechanism of chameleons, a visual perception model in the chameleon-inspired shifty-behavior mode for WMRs was established, as shown in Equation (1).

$$\begin{array}{l}{P}_v\left\{\begin{array}{l}{S}_{ng}\left[\left({\varphi }_{ngb},{\phi }_{ngl},{\phi }_{ngr},{\theta }_{ngl},{\theta }_{ngr},f_{ng}\right)\to \left(F_{ngv},C_{ng}\right)\right],\\ B_{co}\left[\left({\varphi }_{cob},{\phi }_{col},{\phi }_{cor},{\theta }_{col},{\theta }_{cor},f_{co}\right)\to \left(F_{cov},D_{co}\right)\right]\end{array}\right\}\\ \mathrm{s}.\mathrm{t}.\left\{\begin{array}{l}{\left({\varphi }_b\right)}_{\min }\le {\varphi }_b\le {\left({\varphi }_b\right)}_{\max }\\ {\left({\phi }_l\right)}_{\min }\le {\phi }_l\le {\left({\phi }_l\right)}_{\max }\\ {\left({\phi }_r\right)}_{\min }\le {\phi }_r\le {\left({\phi }_r\right)}_{\max }\\ {\left({\theta }_l\right)}_{\min }\le {\theta }_l\le {\left({\theta }_l\right)}_{\max }\\ {\left({\theta }_r\right)}_{\min }\le {\theta }_r\le {\left({\theta }_r\right)}_{\max }\\ f_{\min }\le f\le f_{\max }\end{array}\right.\end{array}$$

(1)

where $P_v\left(·\right)$ denotes the visual perception model in the chameleon-inspired binocular shifty-behavior mode for WMRs, $S_{ng}\left(·\right)$ denotes the target search model with the chameleon-inspired binocular negative-correlation motion, $\left({\varphi }_{ngb},{\phi }_{ngl},{\phi }_{ngr},{\theta }_{ngl},{\theta }_{ngr},f_{ng}\right)$ denotes the action set of the dual PTZ vision system in the target search stage, and $\left(F_{ngv},C_{ng}\right)$ denotes the output set of the vision system in the target search stage, and the action set determines the output set. $F_{ngv}$ denotes the field of view (FOV) acquired by the chameleon-inspired binocular negative-correlation motion, $C_{ng}$ denotes the target features detected by the selective attention algorithm, $B_{co}\left(·\right)$ denotes the target tracking model with the binocular synchronous motion, $\left({\varphi }_{cob},{\phi }_{col},{\phi }_{cor},{\theta }_{col},{\theta }_{cor},f_{co}\right)$ denotes the action set of the dual PTZ vision system in the target tracking stage, and $\left(F_{\mathit{cov}},D_{co}\right)$ denotes the output set of the vision system in the target tracking stage, and the action set determines the output set. $F_{cov}$ denotes the FOV acquired by the binocular synchronous motion, and $D_{co}$ denotes the depth information of the target acquired by the binocular stereo vision.

3. Target Search Modelling with Chameleon-Inspired Binocular Negative-Correlation Motion for WMRs

For the chameleon-inspired vision system of WMRs, a binocular negative-correlation motion search model was established to analyze the negative-correlation-motion behavior of the two eyes and the synchronous-motion behavior of the two eyes and the neck when the vision system is searching for a target. The relationship between the rotation angles of the two cameras and the neck and the FOV, the overlapping angle, and the region of interest of the cameras was quantified. To compare with the chameleon-inspired binocular negative-correlation-motion search model, a binocular synchronous-motion search model was established. The direction of the camera rotating to the right and upward was defined as the positive direction.

$$\begin{array}{l}S\left[\left({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,f,Mod(j=0,1)\right)\to \left(F_v,C_o\right)\right]\\ \mathrm{s}.\mathrm{t}.\left\{\begin{array}{l}{\left({\varphi }_b\right)}_{\min }\le {\varphi }_b\le {\left({\varphi }_b\right)}_{\max }\\ {\left({\phi }_l\right)}_{\min }\le {\phi }_l\le {\left({\phi }_l\right)}_{\max }\\ {\left({\phi }_r\right)}_{\min }\le {\phi }_r\le {\left({\phi }_r\right)}_{\max }\\ {\left({\theta }_l\right)}_{\min }\le {\theta }_l\le {\left({\theta }_l\right)}_{\max }\\ {\left({\theta }_r\right)}_{\min }\le {\theta }_r\le {\left({\theta }_r\right)}_{\max }\\ f_{\min }\le f\le f_{\max }\end{array}\right.\end{array}$$

(2)

where $S\left(·\right)$ denotes the target search model in different search modes, $\left({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,f,Mod(j=0,1)\right)$ denotes the action set of the dual PTZ vision system in the target search phase, and $\left(F_v,C_o\right)$ denotes the output set of the vision system in the target search phase, and the action set determines the output set. ${\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r$ denotes the rotation angle of the neck in the horizontal plane and the rotation angles of the left and right cameras in the horizontal plane and the vertical plane, respectively. $f$ denotes the focal length of the camera. $Mod(j=0,1)$ denotes the chameleon-inspired binocular negative-correlation-motion and binocular synchronous-motion search modes, respectively.

The relationship between the rotation angles of the two cameras and the neck and the FOV, overlapping angle, and region of interest of the cameras is shown in Equation (3).

$$\begin{array}{l}R=\left\{{\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,Mod\left(j=0,1\right)\right\}\\ =R\left(\alpha ,\beta ,\delta ,r,s,\mathsf{\Psi }\times \mathsf{\Phi },Mod\left(j=0,1\right)\right),\\ \mathrm{s}.\mathrm{t}.\left\{\begin{array}{l}{\left({\varphi }_b\right)}_{\min }\le {\varphi }_b\le {\left({\varphi }_b\right)}_{\max }\\ {\left({\phi }_l\right)}_{\min }\le {\phi }_l\le {\left({\phi }_l\right)}_{\max }\\ {\left({\phi }_r\right)}_{\min }\le {\phi }_r\le {\left({\phi }_r\right)}_{\max }\\ {\left({\theta }_l\right)}_{\min }\le {\theta }_l\le {\left({\theta }_l\right)}_{\max }\\ {\left({\theta }_r\right)}_{\min }\le {\theta }_r\le {\left({\theta }_r\right)}_{\max }\end{array}\right.\end{array}$$

(3)

Therefore, the number of rotations of the vision system in the horizontal and vertical planes can be deduced from the FOV, the overlapping angle, the region of interest, and the search mode used by the two cameras [38], as shown in Equation (4).

$$\begin{array}{l}\left\{\begin{array}{l}2\alpha r_{ng}-2\delta \left(r_{ng}-1\right)={\mathsf{\Psi }}_0^{+\alpha }\\ \beta s_{ng}-\delta \left(s_{ng}-1\right)={\mathsf{\Phi }}_0^{+\beta }\end{array}\right.,(j=0)\\ \left\{\begin{array}{l}\alpha r_{co}-\delta \left(r_{co}-1\right)={\mathsf{\Psi }}_0^{+\alpha }\\ \beta s_{co}-\delta \left(s_{co}-1\right)={\mathsf{\Phi }}_0^{+\beta }\end{array}\right.,(j=1)\end{array}$$

(4)

Subsequently, the rotation angles of the two cameras and the neck in the two search modes to achieve scanning of the horizontal region of interest can be obtained as follows:

$$\left\{\begin{array}{ll}{\varphi }_{ngb}+{\phi }_{ngi}={(-1)}^c& \left(0.5\mathsf{\Psi }-0.5\left(2p_{ng}-1\right)\alpha +\left(p_{ng}-1\right)\delta \right),\\ & \left(p_{ng}=r_{ng},r_{ng}-1,\dots ,1\right),(j=0)\\ {\phi }_{coi}=0.5\mathsf{\Psi }-0.5& \left(2p_{co}-1\right)\alpha +\left(p_{co}-1\right)\delta ,\\ & \left(p_{co}=r_{co},r_{co}-1,\dots ,1\right),(j=1)\end{array}\right.$$

(5)

where $i\in \left\{l,r\right\}$ denotes the camera label, and $c=0,1$ denotes the left and right camera labels. For the binocular negative-correlation-motion search mode with neck rotation, the best combination of rotation angles ${\varphi }_{ngb}$ with ${\phi }_{ngl},{\phi }_{ngr}$ are obtained by the rotation optimization model based on a genetic algorithm [39].

Similarly, the rotation angles of the two cameras in the vertical plane in both search modes can be obtained as follows:

$$\begin{array}{l}{\theta }_i=0.5\mathsf{\Phi }-0.5\left(2q-1\right)\beta +\left(q-1\right)\delta ,\\ \left(q=s,s-1,\dots ,1\right),(j=0,1)\end{array}$$

(6)

where $\mathsf{\Psi }\times \mathsf{\Phi }$ denotes the region of interest of the robot, $\alpha ,\beta$ denotes the horizontal and vertical field-of-view angles of the cameras, and they are the same, and $\delta$ denotes the overlapping angle of the FOV, and they are equal for the horizontal and vertical FOV.

Through the modeling analysis of the above two search modes, it can be found that, on the one hand, for the camera’s rotation angles in the horizontal plane, in the case of the same size of the horizontal region of interest, first, the chameleon-inspired binocular negative-correlation-motion search mode reduces the camera’s rotation times by twice as much as the binocular synchronous-motion search mode, which means that not only can the power consumption of the camera servos be reduced twice but also the search efficiency is increased twice. Second, the number of images processed in the binocular negative-correlation-motion search mode is half that of the binocular synchronous-motion search mode, which can significantly reduce the burden of image processing for the robot computer and improve the processing efficiency. Finally, the neck increases the degrees of freedom of the vision system, making the same search range be accomplished by the rotation of three axes, thus improving the search efficiency. This rotation combination of the neck and cameras makes the search more flexible and the camera’s rotation angles smaller when scanning the same range, i.e., the vision system has a larger search range when the camera reaches its rotation limit. On the other hand, for the camera’s rotation angles in the vertical plane, in the case of the same size of the vertical region of interest, although there is no difference between the two modes in the case of normal camera operation, if the camera is once contaminated then the chameleon-inspired binocular negative-correlation-motion mode can give full play to its advantage of negative-correlation motion to achieve the complementation of visual contamination.

4. Modelling of FOV in Chameleon-Inspired Active Vision Based on Shifty-Behavior Mode for WMRs

The FOV in the chameleon-inspired active-vision shifty-behavior mode in different stages is modeled for mobile robots, including the FOV model of binocular negative-correlation motion for target searching and the FOV model of binocular synchronous motion for target tracking. Under these two FOV models with different visual perception behaviors, the spatial variations in the monocular FOV, the binocular overlapping FOV, and the binocular merging FOV of the vision system are analyzed. In addition, the quantitative relationships between the vision system parameters and the FOV and the resolution of the vision system are analyzed to show the behavioral shift in the vision system based on different functional requirements in the target search and target tracking stages. The binocular vision system based on the shifty behavior has a better extension of the measurement range and a better adaptivity in terms of measurement accuracy than the traditional binocular vision system.

4.1. FOV Model with Binocular Synchronous Motion

The FOV model is shown in Figure 4 when the cameras are in synchronous motion, and the PTZ beam carrying the two cameras is rotated $\alpha$ to the left around its central axis, resulting in the two cameras rotating $\alpha$ to the left simultaneously. The direction of the beam rotating to the right around its central axis is defined as positive. In this paper, the FOV of the binocular vision system was analyzed by taking the horizontal FOV as an example. The parameters of the binocular visual system of WMRs are shown in

Table 2. Parameters of the binocular visual system of WMRs.

Parameters	Descriptions	Range (mm)	Parameters	Descriptions	Range (mm)
$a$	Length of CMOS target surface	$\left[3.6,12.7\right]$	${\theta }_1,{\theta }_2$	Monocular horizontal FOV	$\left[9.3°,50.9°\right]$
$f$	Focal length	$\left[6,36\right]$	${\theta }_{12}$	Binocular overlapping horizontal FOV	N/A
$l$	Distance between two eyes	$\left[90,360\right]$	${\theta }_0$	Binocular merging horizontal FOV	N/A
$d$	Object distance	N/A	$r_1,r_2$	Radius of the inner tangent circle of the monocular horizontal FOV	N/A
$d_{12}$	Distance from the object to the vertex of the binocular horizontal overlapping FOV	N/A	$r_{12}$	Radius of the inner tangent circle of the binocular horizontal overlapping FOV	N/A
$d_0$	Distance from the object to the vertex of the binocular horizontal merging FOV	N/A	$r_0$	Radius of the inner tangent circle of the binocular horizontal merging FOV	N/A
$A$	Length of target object	N/A	$\mathrm{\Delta }w$	Size of CMOS image element	$\left[0.002,0.01\right]$
$P\left(x,z\right)$	Position of the target object in the effective horizontal FOV	N/A

.

To reflect the size change relationship of the FOV, the inner tangent circle of the FOV defined to represent the size of the FOV [40], as shown in Figure 4a. In this binocular synchronous motion mode, the radii of the inner tangent circles of the monocular horizontal FOV, the binocular horizontal overlapping FOV, and the binocular horizontal merging FOV are calculated by the following equations:

$$\left\{\begin{array}{l}{r}_1=r_2=ad/\left(2\sqrt{a^2/4+f^2}+a\right)\\ r_{12}=\left(ad-lf\right)/\left(2\sqrt{a^2/4+f^2}+a\right)\\ r_0=\left(ad+lf\right)/\left(2\sqrt{a^2/4+f^2}+a\right)\end{array}\right.$$

(7)

To analyze the measurement accuracy of the binocular vision system, the resolution is analyzed. Figure 4b shows the imaging model of the binocular vision system, with $OXZ$ representing the vision system’s coordinate system and $O_1{X}_1{Z}_1,O_2{X}_2{Z}_2$ representing the left and right camera coordinate systems, respectively. Assuming that the image element size of CMOS is $\mathrm{\Delta }w$, then the resolution $\mathrm{\Delta }X,\mathrm{\Delta }Z$ of the vision system in the $X,Z$ direction in the binocular synchronous motion mode is given by the following equations:

$$\left\{\begin{array}{l}\mathrm{\Delta }X=\min \left(\left|\frac{z}{f}\right|\mathrm{\Delta }w,\left|\frac{z}{f}\right|\mathrm{\Delta }w\right)\\ \mathrm{\Delta }Z=\min \left(\left|\frac{2z^2}{f\left(l+2x\right)}\right|\mathrm{\Delta }w,\left|\frac{2z^2}{f\left(l-2x\right)}\right|\mathrm{\Delta }w\right)\end{array}\right.$$

(8)

4.2. FOV Model with Binocular Negative-Correlation Motion

In the chameleon-inspired binocular negative-correlation-motion mode, the two cameras are rotated ${\alpha }_1,{\alpha }_2$ around their optical axes to the left and right, respectively, where ${\alpha }_1=-{\alpha }_2=-\alpha$. The FOV model is shown in Figure 5.

In the binocular negative-correlation-motion mode, the equations for the radius of the inner tangent circle of the binocular horizontal overlapping FOV and the binocular horizontal merging FOV are shown below:

$$\left\{\begin{array}{l}{r}_{12}=\left\{d\sin \left(\arctan \frac{a}{2f}-\alpha \right)-\frac{l}{2}\cos \left(\arctan \frac{a}{2f}\right)\right\}/\left\{\cos \alpha +\sin \left(\arctan \frac{a}{2f}-\alpha \right)\right\}\\ r_0=\left\{d\sin \left(\arctan \frac{a}{2f}+\alpha \right)+\frac{l}{2}\cos \left(\arctan \frac{a}{2f}\right)\right\}/\left\{\cos \alpha +\sin \left(\arctan \frac{a}{2f}+\alpha \right)\right\}\end{array}\right.$$

(9)

In addition, the resolutions $\mathrm{\Delta }X,\mathrm{\Delta }Z$ of the vision system in the $X,Z$ directions are shown in Equation (10):

$$\left\{\begin{array}{l}\mathrm{\Delta }X=\min \left(\left|\frac{{\left(z-\left(x+0.5l+z\tan \alpha \right)\sin \alpha \cos \alpha \right)}^2}{zf{\cos }^2\alpha }\right|\mathrm{\Delta }w,\left|\frac{{\left(z-\left(0.5l-x+z\tan \alpha \right)\sin \alpha \cos \alpha \right)}^2}{zf{\cos }^2\alpha }\right|\mathrm{\Delta }w\right)\\ \mathrm{\Delta }Z=\min \left(\left|\frac{{\left(z\cos \alpha -\left(x+0.5l\right)\sin \alpha \right)}^2}{f\left(x+0.5l\right)}\right|\mathrm{\Delta }w,\left|\frac{{\left(z-\left(0.5l-x+z\tan \alpha \right)\sin \alpha \cos \alpha \right)}^2}{f\left(0.5l-x\right){\cos }^2\alpha }\right|\mathrm{\Delta }w\right)\end{array}\right.$$

(10)

4.3. Simulation Analysis of FOV Model

Through the above analysis, it can be found that the FOV model in the binocular negative-correlation-motion target search stage introduces a binocular negative-correlation rotation angle compared to that in the binocular synchronous motion target tracking stage, so the simulation analysis from the measurement range and accuracy for the binocular negative-correlation-motion FOV model is carried out without any loss of generality. According to Equation (9), the relationship surfaces of the binocular horizontal overlapping FOV and binocular horizontal merging FOV with the parameters of the vision system as well as the main effect diagram and Pareto diagram [41,42] obtained by orthogonal experimentation can be obtained, as shown in Figure 6 and Figure 7.

As shown in Figure 6, the rotation angles of the two cameras with negative-correlation motion have a very significant negative effect on the change in the size of the binocular horizontal overlapping FOV, with an effect rate of about 68%. Therefore, if we want to increase the binocular horizontal overlapping FOV, we can reduce the rotation angles of the negative-correlation motion of the two cameras. Therefore, in the target tracking stage, the binocular synchronous-motion mode is used, where the rotation angles of the two cameras with negative-correlation motion is 0 in order to ensure an effective FOV of sufficient size.

As shown in Figure 7, the rotation angles of the two cameras with the negative-correlation motion have a very significant positive effect on the change in the size of the binocular horizontal merging FOV, with an effect rate of about 73%. Therefore, if we want to increase the binocular horizontal merging FOV, we can increase the rotation angles of the two cameras with negative-correlation motion. However, to ensure the continuity of the search area, the rotation angles of the two cameras with negative-correlation motion should not be too large.

According to Equation (10), the relationship surfaces of the resolution in the X and Z directions with the parameters of the vision system can be obtained, as can the main effect diagram and Pareto diagram obtained by orthogonal experimentation, as shown in Figure 8 and Figure 9.

According to the importance of these factors, the corresponding parameters can be adjusted according to the functional requirements. To increase the $X$ direction resolution, we can reduce the $Z$ coordinate of the target object in the effective FOV, reduce the CMOS pixel size, and increase the camera focal length. Therefore, in the target tracking stage, the camera focal length is appropriately increased to improve the measurement accuracy.

From Figure 9, it can be found that in order to increase the $Z$ resolution, the absolute value of the $X$ coordinate of the target object in the effective FOV can be increased, the $Z$ coordinate of the target object in the effective FOV can be decreased, the camera focal length can be increased, or the CMOS pixel size can be decreased.

Based on the above analysis, in the target search stage, the goal to be achieved by the mobile robot is the rapid perception of a large region of interest, and there is no need to achieve stereo vision for the accurate observation of the target, so the chameleon-inspired binocular negative-correlation-motion mode (with a large binocular merging FOV) and wide-angle camera mode (with a large FOV) are used for environment perception. However, to ensure the continuity of the search area, the rotation angles of the two cameras with the negative-correlation motion should not be too large. In addition, the focal length of the camera should not be too small, otherwise it will affect the initial detection effect of the target. For the functional requirements of the mobile robot in the target tracking stage, on the one hand, to realize stereo vision, the effective binocular FOV (i.e., the binocular overlapping FOV) must be increased; on the other hand, in order to improve the measurement accuracy of the vision system to accurately locate the target, the resolution of the vision system must be improved, so in this stage, the binocular synchronous-motion mode (with a large binocular overlapping FOV) and the long-focus mode of the camera (with a high measurement accuracy) are adopted. However, the focal length of the camera should not be too large, otherwise it will lead to a too-small effective FOV, affecting the detection effect of the target.

5. Environment Perception Strategy in Chameleon-Inspired Active Vision Based on Shifty-Behavior Mode for WMRs

Chameleons are called “lions that crawl on the ground” because their vision systems play an important role in defending and hunting. Chameleons have peculiar eyes, which can move independently without interlocking to achieve panoramic observation, and they can move simultaneously to achieve binocular stereo vision. Inspired by this visual behavior of chameleons, this paper proposes an active-vision environment-perception strategy for WMRs based on a shifty-behavior mode, as shown in Figure 10. This method can reproduce the predation process of chameleons by executing a series of rhythmic movements to achieve the rapid searching, accurate positioning, and tracking of targets, and it can realize the coordinated motion of the robot’s body, neck, and eye in that process, i.e., vision–motor coordination [8].

(1) Target search stage with chameleon-inspired binocular negative-correlation motion. The functional goal of this stage is to achieve a panoramic perception of the environment; therefore, to speed up the target search and reduce the burden of panoramic information processing, the search mode of the chameleon-inspired binocular negative-correlation motion is combined with the low-resolution mode of short focus. Given that the WMR proceeds in forward motion and the target behind the robot has been searched and tracked, the search range is set to the area of 160° in front of the robot, and the FOV of the two cameras in the wide-angle mode is 48°. In addition, to ensure the continuity and integrity of the search area, the overlapping FOV of the two cameras is 8°. According to the chameleon-inspired binocular negative-correlation-motion search model established above, the specific implementation process is shown in Figure 11. The FOV relationship of the robot is shown in Equation (11).

(2) Target recognition stage based on selective attention algorithm. In this paper, we used the selective attention algorithm proposed by Itti based on visual physiology and psychology [17], which uses a bottom-up data-driven model to obtain image saliency maps, including the extraction of multi-scale low-level features of images, the generation of multi-scale saliency maps, and attention focus acquisition and transferring.

$$\begin{gathered} \left\{\begin{array}{ll}{Z}_1=& \{Z({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,\alpha ,\beta ,\delta ,r,s,\psi ,\mathsf{\Phi },i)|\hfill \\ & \begin{array}{cc}& \end{array}{\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,\alpha ,\beta ,\delta ,r,s,\psi ,\mathsf{\Phi },i=\\ & \begin{array}{cc}& \end{array}20°,36°,-36°,30°,30°,48°,36,8,1,160,1\}\\ Z_2=& \{Z({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,\alpha ,\beta ,\delta ,r,s,\psi ,\mathsf{\Phi },i)|\\ & \begin{array}{cc}& \end{array}{\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,\alpha ,\beta ,\delta ,r,s,\psi ,\mathsf{\Phi },i=\\ & \begin{array}{cc}& \end{array}20°,36°,-36°,30°,30°,48°,36,8,1,160,2\}\\ Z_4=& \{Z({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,\alpha ,\beta ,\delta ,r,s,\psi ,\mathsf{\Phi },i)|\\ & \begin{array}{cc}& \end{array}{\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,\alpha ,\beta ,\delta ,r,s,\psi ,\mathsf{\Phi },i=\\ & \begin{array}{cc}& \end{array}-20°,36°,-36°,30°,30°,48°,36,8,r,160,1\}\\ Z_5=& \{Z({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,\alpha ,\beta ,\delta ,r,s,\psi ,\mathsf{\Phi },i)|\\ & \begin{array}{cc}& \end{array}{\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r,\alpha ,\beta ,\delta ,r,s,\psi ,\mathsf{\Phi },i=\\ & \begin{array}{cc}& \end{array}-20°,36°,-36°,30°,30°,48°,36,8,r,160,2\}\end{array}\right. \\ \left\{\begin{array}{l}{Z}_{3,4}=Z_3\cap Z_4\\ Z_{2,6}=Z_2\cap Z_6\\ Z_1\cup Z_{3,4}\cup Z_{2,6}\cup Z_5=160°\end{array}\right. \end{gathered}$$

(11)

(3) Target alignment stage. To achieve the accurate alignment of the robot to the target, a combination of the robot body and neck steering-alignment strategy and the zoom-accurate-alignment strategy is used. The rotation angles $\omega ,{\varphi }_b$ of the robot body and neck are $\gamma ,-{\varphi }_b^I$, where $\gamma$ denotes the angle between the target and the longitudinal direction of the main robot body, and ${\varphi }_b^I$ denotes the initial deviation angle of the neck (i.e., the angle between the neck and the longitudinal direction of the main robot body in the initial state), where $-270°\le \omega \le 270°,-180°\le \gamma \le 180°$. Unlike the requirements of the target search stage, the target tracking stage is carried out to achieve precise positioning and tracking of the target, so binocular synchronous motion is used to achieve stereo vision combined with a long-focus high-resolution mode.

(4) Target tracking stage based on particle filtering algorithm. The particle filtering algorithm is used to approximate the posterior [43] probability distribution of random variables by using an importance sampling technique using the Monte Carlo simulation method [44]. Since the selective attention algorithm pays a lot of attention to the edge regions of specific objects and cannot select out the object centers, the target selection process is supervised in this paper by adopting a pre-learning approach for the objects to be tracked [45]. In addition, since the traditional particle filtering algorithm uses a fixed template, it is not very adaptable to changes in the shape, pose, and light of the tracked object. In order to improve the robustness of tracking, this paper improves the tracking accuracy by periodically updating the target model.

6. Experiments

To verify the effectiveness of the proposed chameleon-inspired active-vision environment-perception strategy based on a shifty-behavior mode for mobile robots, experiments were conducted based on a 4WD4WS mobile robot on rigid, flat ground and soft–rough sand, respectively.

6.1. Target Tracking Experiment of WMRs on Rigid, Flat Ground

In this experiment, the robot was located indoors on rigid, flat ground to search and track a red target in front of it, as shown in Figure 12.

Target search stage: first, the robot was initialized with $\left({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r\right)=\left(0,0,0,30,30\right)$, and the camera entered the short-focus mode. Then, at the 11th second and 15th second, respectively, the robot performed a rapid scan of the environment using the chameleon-inspired binocular negative-correlation-motion target search model.

$$\left({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r\right)=\left\{\begin{array}{l}\left(20,36,-36,30,30\right),q=1\\ \left(-20,36,-36,30,30\right),q=2\end{array}\right.$$

(12)

Subsequently, the four acquired images were compared and analyzed using the selective attention algorithm, and it was determined that when the neck and right camera were rotated $\left({\varphi }_b,{\phi }_r,{\theta }_r\right)=\left(20,-36,30\right)$, respectively, the area located at the coordinate $\left(248,77\right)$ of the image had the highest saliency. The horizontal and vertical orientation of the target was derived by the PTZ rotation equation as $\left(-27.1°,28.0°\right)$. As shown in Figure 13a, we first used a histogram-based selective attention model to identify the targets [46], which uses histograms for image color statistics, first quantizing color images in RGB space and then performing a color-space smoothing operation, which is more suitable for mobile robot environment scenes. As shown in Figure 13b, we used superpixel segmentation [47] to improve the accuracy, which is more desirable in terms of execution speed, the compactness of the generated superpixels, and contour preservation.

Target alignment stage: first, in the 24th second, the robot body and vision system rotated by $\left({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r\right)=\left(0,0,0,28.0,28.0\right),\omega =-27.1°$, respectively, aligning with the target, and the cameras moved together to realize stereo vision. Then, in the 41st second, the camera zoomed and entered the long-focus mode to obtain high-resolution pictures of the target to achieve precise target alignment.

Target tracking stage: starting from the 41st second, the robot started to track the target. The experimentally collected rotation angles of each axis of the vision system and the target tracking error are shown in Figure 14. The tracking error curve showed that although the tracking error fluctuated due to the vibration problem of the robot itself, it always fluctuated around the 0 axis and was basically between plus and minus 15 pixels, indicating that the target tracking was achieved.

6.2. Target Tracking Experiment of WMRs on Soft–Rough Sand

In this experiment, the robot was located on soft–rough sand to search and track a red target in front of it, as shown in Figure 15.

Target search stage: first, the robot was initialized with $\left({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r\right)=\left(0,0,0,40,40\right)$, and the camera entered the short-focus mode. Then, at the 11th second and 15th second, the robot performed a rapid scan of the environment using the chameleon-inspired binocular negative-correlation-motion target search model.

$$\left({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r\right)=\left\{\begin{array}{l}\left(20,36,-36,40,40\right),q=1\\ \left(-20,36,-36,40,40\right),q=2\end{array}\right.$$

(13)

Then, the four acquired images were compared and analyzed using the selective attention algorithm, and it was determined that when the neck and right camera were rotated by $\left({\varphi }_b,{\phi }_r,{\theta }_r\right)=\left(20,-36,40\right)$, respectively, the area located at the coordinate $\left(248,49\right)$ of the image had the highest saliency. The horizontal and vertical orientation of the target was derived by the PTZ rotation equation as $\left(-28.6°,29.7\right)$. The image processing during the target search phase is shown in Figure 16.

Target alignment stage: first, in the 23rd second, the robot body and vision system rotated by $\left({\varphi }_b,{\phi }_l,{\phi }_r,{\theta }_l,{\theta }_r\right)=\left(0,0,0,29.7,29.7\right),\omega =-28.6°$, respectively, aligning with the target, and the cameras moved together to realize stereo vision. Then, in the 40th second, the camera zoomed into the long-focus mode to obtain high-resolution pictures of the target to achieve precise target alignment.

Target tracking stage: starting from the 40th second, the robot started to track the target. The experimentally collected rotation angles of each axis of the vision system and the target tracking error are shown in Figure 17. The tracking error curve showed that although the tracking error fluctuated due to vibration problems with the robot itself, it always fluctuated around the 0 axis and was basically between plus and minus 15 pixels, indicating that it could achieve the tracking of the target.

The experiments showed that this chameleon-inspired active-vision environment-perception strategy based on a shifty-behavior mode for mobile robots could achieve the rapid search and tracking of the target, and in this process, the coordinated movement of various parts could realize a sequence of actions being executed rhythmically, which achieved the reproduction of the visual behavior of chameleons in the vision system of the mobile robot with satisfactory results.

7. Conclusions

In this paper, the reproduction of the visual behavior of chameleons in the vision system of mobile robots was investigated with the following main results:

(1) The chameleon-inspired binocular negative-correlation-motion search mode significantly improved the search efficiency, range, flexibility, and image processing volume compared with the traditional binocular synchronous-motion search mode.

(2) The shifty-behavior mode binocular vision system had a better extension of the measurement range and a better adaptivity of the measurement accuracy compared with the traditional binocular vision system. In the target search stage, the wide-angle camera mode (with a large FOV) and the binocular negative-correlation-motion mode (with a large binocular merging FOV) were used for the rapid and large-range perception of the environment. In the target tracking stage, the long-focus camera mode (with a high measurement accuracy) and the binocular synchronous-motion mode (with a large binocular overlapping FOV) were used to achieve stereo vision for accurate target measurement and tracking.

(3) The effectiveness of the proposed chameleon-inspired active-vision environment perception model based on a shifty-behavior mode for mobile robots was verified through experiments with satisfactory results, which could realize its vision–motor coordination.