Mallard Detection Using Microphone Arrays Combined with Delay-and-Sum Beamforming for Smart and Remote Rice–Duck Farming

Abstract

This paper presents an estimation method for a sound source of pre-recorded mallard calls from acoustic information using two microphone arrays combined with delay-and-sum beamforming. Rice farming using mallards saves labor because mallards work instead of farmers. Nevertheless, the number of mallards declines when they are preyed upon by natural enemies such as crows, kites, and weasels. We consider that efficient management can be achieved by locating and identifying the locations of mallards and their natural enemies using acoustic information that can be widely sensed in a paddy field. For this study, we developed a prototype system that comprises two sets of microphone arrays. We used 64 microphones in all installed on our originally designed and assembled sensor mounts. We obtained three acoustic datasets in an outdoor environment for our benchmark evaluation. The experimentally obtained results demonstrated that the proposed system provides adequate accuracy for application to rice–duck farming.

1. Introduction

Rice–duck farming, a traditional organic farming method, uses hybrid ducks released in a paddy field for weed and pest control [1]. Farmers in northern Japan use mallards because of their utility value as a livestock product [2]. Moreover, mallard farming saves labor because mallards work instead of farmers. Mallard farming is a particularly attractive approach for farmers, especially in a regional society that faces severe difficulties posed by the population decrease and rapid progress of aging [3]. As mallard farming has attracted attention, it has become popular not only in Japan, but also in many Asian countries.

One difficulty posed by mallard farming is that a sord of mallards must be gathered to a specific area in a paddy field. They trample down rice, which produces stepping ponds in which it is difficult to grow rice. Another shortcoming is that weed control effects are not obtained in areas outside the range of mallard activities. For this case, accurate position estimation of mallards in a paddy field is necessary. However, as depicted in the right photograph of Figure 1, detecting mallards among grown rice plants involves difficulties because the targets are not visible. Moreover, the number of mallards decreases because of predation by their natural enemies such as crows, kites, and weasels. to protect mallards from their natural enemies, specifying and managing mallard positions in real time are crucially important tasks.

To mitigate or resolve these difficulties, we evaluated an efficient management system created by locating and identifying mallards and their natural enemies using acoustic information that can be sensed widely in a paddy field. Furthermore, one can provide effective control to elucidate exactly when, where, and what kinds of natural enemies are approaching. This study was conducted to develop a position estimation system for mallards in a paddy field. Developing stable production technology is strongly demanded because rice produced by mallard farming trades at a high price on the market. For this system, we expect technological development and its transfer to remote farming [2], which is our conceptual model for actualizing smart farming.

This paper presents a direction and position estimation method for a sound source of pre-recorded mallard calls using acoustic information with arrayed microphones combined with delay-and-sum (DAS) beamforming. Using acoustic information, the approach can detect mallards that are occluded by stalks or grass. Based on the results, we inferred that an efficient management system can be actualized from locating and identifying mallards and their natural enemies using acoustic information that can widely sense a paddy field. We developed a prototype system with a 64-microphone array installed on our originally designed and assembled mount. To reproduce the sounds obtained using a microphone in advance, we conducted a simulated experiment in an actual environment to evaluate the estimation accuracy of the method.

This paper is structured as follows. Section 2 briefly reviews state-of-the-art acoustic and multimodal methods of automatic bird detection from vision and audio modalities. Subsequently, Section 3 and Section 4 present the proposed localization method using an originally developed microphone system based on DAS beamforming. The experimental results obtained using our original acoustic benchmark datasets obtained using two drones are presented in Section 5. Finally, Section 6 presents the conclusions and highlights future work.

2. Related Studies

Detecting birds that fly across the sky in three-dimensional space is a challenging task for researchers and developers. Over a half-century ago, bird detection using radar [4] was studied to prevent bird strikes by aircraft at airports. By virtue of advances in software technology and improvements in the performance and affordability of sensors and computers, bird detection methods have diversified, especially in terms of offering improved accuracy and cost-effective approaches. Recent bird-detection methods can be categorized into two major modalities based on survey articles: visual [5,6,7,8] and audio [9,10,11]. Numerous outstanding methods [12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49] have been proposed from both modalities.

Benchmark datasets and challenges for the verification and comparison of detection performance have been expanded in both vision [50,51,52,53] and acoustic [54,55,56,57,58,59,60,61,62,63,64,65] modalities. During the first half of the 2010s, probability models and conventional machine-learning (ML) models combined with part-based features obtained using local descriptors were used widely and popularly. These methods comprise k-means [66], morphological filtering (MF) [67], principal component analysis (PCA), Gaussian mixture models (GMMs), expectation–maximization (EM) algorithms [68], boosting [69], scale-invariant feature transform (SIFT) [70], random forests (RFs) [71], bag-of-features (BoF) [72], histograms of oriented gradients (HOGs) [73], support vector machines (SVMs) [74], local binary patterns (LBPs) [75], background subtraction (BS) [76], and multi-instance, multi-label (MIML) [77]. These methods require preprocessing of input signals for the enhancement of the features. Moreover, algorithm selection and parameter optimizations conducted in advance for preprocessing have often relied on the subjectivity and experience of the developers. If data characteristics differ even slightly, then performance and accuracy drop drastically. Therefore, parameter calibration is necessary and represents a great deal of work. In the latter half of the 2010s, deep-learning (DL) algorithms have been predominant, especially after the implementation of the error back-propagation learning algorithm [78] in convolutional neural networks (CNNs) [79].

Table 1. Representative studies of vision-based bird detection methods reported during the last decade.

Year	Authors	Method	Dataset
2011	Qing et al. [12]	Boosted HOG-LBP + SVM	original
2011	Descamps et al. [13]	Image energy	original
2011	Farrell et al. [14]	HOG + SVM + PNAD	original
2012	Mihreteab et al. [15]	HOG + CS-LBP + linear SVM	original
2013	Liu et al. [16]	HOG + linear SVM	[52]
2014	Xu et al. [17]	linear SVM classifier	original
2015	Yoshihashi et al. [18]	BS + CNN	[18]
2016	T’Jampens et al. [19]	SBGS + SIFT + BoW + SVM	original
2016	Takeki et al. [20]	CNN (ResNet) + FCNs + DeepLab	original
2016	Takeki et al. [21]	CNN + FCN + SP	original
2017	Yoshihashi et al. [22]	CNN (ResNet)	[53]
2017	Tian et al. [23]	Faster RCNN	original
2018	Wu et al. [24]	Skeleton-based MPSC	[50,52]
2019	Lee et al. [25]	CNN	original
2019	Vishnuvardhan et al. [26]	Faster RCNN	original
2019	Hong et al. [27]	Faster RCNN	original
2019	Boudaoud et al. [28]	CNN	original
2019	Jo et al. [29]	CNN (Inception-v3)	original
2020	Fan et al. [30]	RCNN	original
2020	Akcay et al. [31]	CNN + RPN + Fast-RCNN	original
2021	Mao et al. [32]	Faster RCNN (ResNet-50)	original
2021	Marcoň et al. [33]	RNN + CNN	original

and

Table 2. Representative studies of sound-based bird detection methods reported during the last decade.

Year	Authors	Method	Dataset
2011	Jančovič et al. [34]	GMM	original
2012	Briggs et al. [35]	MIML	original
2014	Stowell et al. [36]	PCA + k-means + RF	[55]
2015	Papadopoulos et al. [37]	GMM	[60]
2015	Oliveira et al. [38]	DFT + MF	original
2017	Adavanne et al. [39]	CBRNN	[62]
2017	Pellegrini et al. [40]	DenseNet	[62]
2017	Cakir et al. [41]	CRNN	[62]
2017	Kong et al. [42]	JDC-CNN (VGG)	[56,62]
2017	Grill et al. [43]	CNN	[56]
2018	Lassecket al. [44]	CNN	[65]
2020	Liang et al. [45]	BNN (XNOR-Net)	[58,61]
2020	Solomes et al. [46]	CNN	[65]
2021	Hong et al. [47]	2D-CNN	[58,59]
2021	Kahl et al. [48]	CNN (ResNet-157)	original
2021	Zhong et al. [49]	CNN (ResNet-50) + GAN	original

respectively present representative studies of vision-based and sound-based bird detection methods reported during the last decade. As presented in the fourth columns, the representative networks and backbones used in bird detection methods are the following: regions with CNN (RCNN) [80], VGGNet [81], Inception [82], ResNet [83], XNOR-Net [84], densely connected convolutional neural networks (DC-CNNs) [85], fast RCNN [86], faster RCNN [87], you only look once (YOLO) [88], and weakly supervised data augmentation network (WS-DAN) [89]. End-to-end DL models require no pre-processing for feature extraction of the input signals [90]. Moreover, one-dimensional acoustic signals can be input to the DL model as two-dimensional images [91].

Recently, vision-based detection methods have targeted not only birds, but also drones [92]. In 2019, the Drone-vs-Bird Detection Challenge (DBDC) was held at the 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) [93]. The goal of this challenge was the detection of drones that appear at several image coordinates in videos without detecting birds that are present in the same frames. The developed algorithms must alert detected drones only, without alerting birds. Moreover, it is necessary to provide estimates of drone locations on the image coordinates. As a representative challenge for acoustic bird detection, bird acoustic detection [65] was conducted at the Detection and Classification of Acoustic Scenes and Events (DCASE) in 2018. Among the 33 submissions evaluated for this challenge, the team of Lasseck et al. [44] was awarded the champion title. For this detection task, the organizer provided 10 h of sound clips, including negative samples with human imitations of bird sounds. The algorithms produced a binary output of whether or not the sounds were birds, based on randomly extracted sound clips of 10 s. There are differences between our study and the extraction challenge of the coordinates of birds inferred from sound clips of the same period. To improve the accuracy for wide applications, the algorithms developed for the challenge incorporated domain adaptation, transfer learning [94], and generative adversarial networks (GANs) [95]. By contrast, our study is specialized for application to mallard farming in remote farming [2].

In comparison to a large number of previous studies mentioned above, the novelty and contributions of this study are as follows:

• The proposed method can detect the position of a sound source of pre-recorded mallard calls output from a speaker using two parameters obtained from our originally developed microphone arrays;
• Compared with existing sound-based methods, our study results provide a detailed evaluation that comprises 57 positions in total through three evaluation experiments;
• To the best of our knowledge, this is the first study to demonstrate and evaluate mallard detection based on DAS beamforming in the wild.

Based on the underlying method used in drone detection [96] and our originally developed single-sensor platform [97], this paper presents a novel sensing system and shows experimentally obtained results obtained from outdoor experiments.

3. Proposed Method

3.1. Position Estimation Principle

Figure 2 depicts the assignment of two microphone arrays for position estimation. The direction perpendicular to the array was set to 0° of the beam. Let $(x,y)$ be the coordinate of an estimation target position based on origin O. The two microphone arrays, designated as M1 and M2, are installed at distance $p_x$ horizontally and at distance $p_y$ vertically.

Assuming ${\theta }_1$ and ${\theta }_2$ as the angles between the lines perpendicular to $p_x$ and $p_y$ and the straight line to $(x,y)$, and letting $L_1$ and $L_2$ represent the straight distances between the respective microphone array systems and $(x,y)$, then using trigonometric functions, $p_x$ and $p_y$ are given as presented below [98].

$$\begin{array}{ccc}\hfill p_x& =& L_{1}cos{\theta }_1+L_{2}sin{\theta }_2,\hfill \end{array}$$

(1)

$$\begin{array}{ccc}\hfill L_1& =& \frac{p_x-L_{2}sin{\theta }_2}{cos{\theta }_1},\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill p_y& =& L_{1}sin{\theta }_1+L_{2}cos{\theta }_2\hfill \end{array}$$

(3)

$$\begin{array}{ccc}& =& \frac{p_x-L_{2}sin{\theta }_2}{cos{\theta }_1}sin{\theta }_1+L_2\frac{cos{\theta }_{1}cos{\theta }_2}{cos{\theta }_1}\hfill \end{array}$$

(4)

$$\begin{array}{ccc}& =& L_2\frac{cos{\theta }_{1}cos{\theta }_2}{cos{\theta }_1}-L_2\frac{sin{\theta }_{1}sin{\theta }_2}{cos{\theta }_1}+p_x\frac{sin{\theta }_1}{cos{\theta }_1}.\hfill \end{array}$$

(5)

The following expanded equation is obtained by solving for $L_2$.

$$\begin{array}{ccc}\hfill L_2\frac{cos{\theta }_{1}cos{\theta }_2-sin{\theta }_{1}sin{\theta }_2}{cos{\theta }_1}& =& p_y-p_x\frac{sin{\theta }_1}{cos{\theta }_1}\hfill \end{array}$$

(6)

$$\begin{array}{ccc}\hfill L_2\frac{-cos({\theta }_1+{\theta }_2)}{cos{\theta }_1}& =& \frac{p_{y}cos{\theta }_1-p_{x}sin{\theta }_1}{cos{\theta }_1}\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill L_2& =& \frac{p_{x}sin{\theta }_1-p_{y}cos{\theta }_1}{cos({\theta }_1+{\theta }_2)}.\hfill \end{array}$$

(8)

One can obtain $(x,y)$ as follows:

$$\left[\begin{array}{c}x\\ y\end{array}\right]=\left[\begin{array}{c}{p}_x-L_{2}cos{\theta }_2\\ p_y-L_{2}sin{\theta }_2\end{array}\right].$$

(9)

Alternatively, using $L_2$, one can obtain $p_y$ as follows:

$$\begin{array}{ccc}\hfill L_2& =& \frac{p_x-L_{1}cos{\theta }_1}{sin{\theta }_2},\hfill \end{array}$$

(10)

$$\begin{array}{ccc}\hfill p_y& =& L_{1}sin{\theta }_1+\frac{p_x-L_{1}cos{\theta }_1}{sin{\theta }_2}cos{\theta }_2\hfill \end{array}$$

(11)

$$\begin{array}{ccc}& =& L_1\frac{sin{\theta }_{1}sin{\theta }_2}{sin{\theta }_2}+p_x\frac{cos{\theta }_2}{sin{\theta }_2}-L_1\frac{cos{\theta }_{1}cos{\theta }_2}{sin{\theta }_2}.\hfill \end{array}$$

(12)

The following expanded equation is obtained by solving for $L_1$.

$$\begin{array}{ccc}\hfill L_1\frac{sin{\theta }_{1}sin{\theta }_2-cos{\theta }_{1}cos{\theta }_2}{sin{\theta }_2}& =& p_y-p_x\frac{cos{\theta }_2}{sin{\theta }_2}\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill L_1\frac{-cos({\theta }_1+{\theta }_2)}{sin{\theta }_2}& =& \frac{p_{y}sin{\theta }_2-p_{x}cos{\theta }_2}{sin{\theta }_2}\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill L_1& =& \frac{p_{x}sin{\theta }_2-p_{y}sin{\theta }_2}{cos({\theta }_1+{\theta }_2)}.\hfill \end{array}$$

(15)

Using $L_1$ and ${\theta }_1$, one can obtain $(x,y)$ as follows:

$$\left[\begin{array}{c}x\\ y\end{array}\right]=\left[\begin{array}{c}{p}_x-L_{1}cos{\theta }_1\\ p_y-L_{1}sin{\theta }_1\end{array}\right].$$

(16)

Angles ${\theta }_1$ and ${\theta }_2$ are calculated using the DAS beamforming method, as shown in Figure 3.

3.2. DAS Beamforming Algorithm

Beamforming is a versatile technology used for directional signal enhancement with sensor arrays [99]. Letting $y\left(t\right)$ be a beamformer output signal at time t and letting M be the number of microphones, then for $z_m\left(t\right)$ and $w_m\left(t\right)$, which respectively denote a measured signal and a filter of the m-th sensor, $y\left(t\right)$ is calculated as:

$$y\left(t\right)=\sum _{m=1}^M{w}_m\left(t\right)\otimes z_m\left(t\right),$$

(17)

where symbol ⊗ represents convolution.

Based on our previous study [96], we considered DAS beamforming in the temporal domain. Assuming that single plane waves exist and letting $s_m\left(t\right)$ be a set of acoustic signals, then delay ${\tau }_m$ as expressed for the formula below occurs for incident waves observed for the m-th sensor.

$$z_m\left(t\right)=s_m(t-{\tau }_m),$$

(18)

where M represents the total number of microphones.

The delayed $-{\tau }_m$ of incident waves was offset by the advanced +${\tau }_m$ of incident waves for a filter. Signals from the direction of $\theta$ are enhanced because of the gathered phases of signal $s_m\left(t\right)$ in all channels. The temporal compensated filter $w_m\left(t\right)$ is defined as:

$$w_m\left(t\right)=\frac{1}{M}\delta (t+{\tau }_m),$$

(19)

where $\delta$ is Dirac’s delta function.

Letting $\theta$ be an angle obtained using beamforming, then, for the comparison of acoustic directions, the relative mean power level $G\left(\theta \right)$ of $y\left(t\right)$ is defined as:

$$\begin{array}{c}\hfill G\left(\theta \right)=\frac{1}{T}\sum _{t=0}^T{y}^2\left(t\right),\end{array}$$

(20)

where T represents the interval time length.

We changed $\theta$ from −90° to 90° with 1° intervals. Let $P_1\left(\theta \right)$ and $P_2\left(\theta \right)$ respectively denote $G\left(\theta \right)$ obtained from M1 and M2. Using $P_1\left(\theta \right)$ and $P_2\left(\theta \right)$, one can obtain ${\theta }_1$ and ${\theta }_2$ as:

$${\theta }_n=\underset{-90\le \theta \le 90}{\arg \max }{P}_n\left(\theta \right),n=\{1,2\}$$

(21)

4. Measurement System

4.1. Mount Design

For this study, we designed an original sensor mount with installed microphones and electric devices: an amplifier, an analog–digital converter, a battery, an inverter, and a laptop computer. Figure 4 depicts the design and assembled mount. We used 20 mm square aluminum pipes for the main frame. The terminals were joined with L-shaped connectors, bolts, and nuts. To set the electrical devices, acrylic boards were used for the plate located at a 575 mm height. Microphones were installed on a bracket arm as an array for a straight 1200 mm long line with a 30 mm gap.

As for the installation of microphones, a spatial foldback occurs in high-frequency signals if the gap is wide. In general [98], the distance d between microphones and the upper frequency f where no foldback occurs is defined as:

$$d\le \frac{c}{2f}$$

(22)

where the constant c is the speed of sound. We confirmed from our preliminary experimental results that the power of mallard calls was concentrated in the band below 5 kHz. Therefore, we installed the microphones in 30 mm intervals because the upper limit is approximately 5.6 kHz for $d=0.030$. In addition, the directivity is reduced for low frequencies, if d is narrow.

The height from the ground to the microphones was 1150 mm. We used cable ties to tighten the microphones to the bracket arm. We also labeled the respective microphones with numbers that made it easy to check the connector cables to the converter terminals.

4.2. Microphone Array

Table 3. System components and quantities for the respective microphone arrays: M1 and M2.

Item	M1	Amount	M2	Amount
Microphone	DBX RTA-M	×32	Behringer ECM8000	×32
Amplifier	MP32	×1	ADA8200	×4
AD converter	Orion32	×1	(included amplifier)
Battery	Jackery 240	×1	Anker 200	×1

presents the components and their quantities for the respective microphone arrays. We introduced different devices for M1 and M2 because of the various circumstances such as an introduced period, research budget preparation, and coordination with our related research projects. The microphones used for M1 and M2 were, respectively, DBX RTA-M (Harman International Inds., Inc., Stamford, CT, USA) and ECM-8000 (Behringer Music Tribe Makati, Metro Manila, Philippines).

Table 4. Detailed specifications of microphones of two types.

Parameter	DBX RTA-M	Behringer ECM8000
Polar pattern	Omnidirectional	Omnidirectional
Frequency range	20–20,000 Hz	20–20,000 Hz
Impedance	259 $\mathsf{\Omega }$ ± 30% (@1 kHz)	600 $\mathsf{\Omega }$
Sensitivity	$-63\pm 3$ dB	$-60$ dB
Mic head diameter	10 mm	(no data)
Length	145 mm	193 mm
Weight	(no data)	120 g

presents detailed specifications for both microphones. The common specification parameters are an omnidirectional polar pattern and the 20–20,000 Hz frequency range. For the amplifier and analog-to-digital (AD) converter, we used separate devices for M1 and an integrated device for M2. Here, both converters include a built-in anti-aliasing filter. Under the condition of the arrayed microphone interval, $d=0.030$ m, grating lobes may appear for the upper limit of 5.6 kHz and above. However, we assumed that the effect is relatively small because the target power of mallard calls is concentrated in the band below 5 kHz. For outdoor data acquisition experiments, we used portable lithium-ion batteries of two types. Moreover, to supply measurement devices with 100 V electrical power, the commercial power supply voltage in Japan, we used an inverter to convert direct current into alternating current.

Figure 5 portrays photographs of all the components in Table 3 installed on the mount depicted in Figure 4. In all, the 64 microphones were connected to their respective amplifiers with 64 cables in parallel.

5. Evaluation Experiment

5.1. Experiment Setup

Acoustic data acquisition experiments were conducted at the Honjo Campus of Akita Prefectural University (Yurihonjo City, Akita Prefecture, Japan). This campus, located at 39°23′37″ north latitude and 140°4′23″ east longitude, has a size of 204,379 m². The left panel of Figure 6 portrays an aerial photograph of the campus. The surroundings comprise an expressway to the east and a public road to the south. The experiment was conducted at the athletic field on the west side of the campus, as depicted in the right panel of Figure 6.

We used a pre-recorded sound of a mallard as the sound source. We played this sound on a Bluetooth wireless speaker that had been loaded on a small unmanned ground vehicle (UGV) to move to the respective measurement positions. In our earlier study, we developed this UGV as a prototype for mallard navigation, particularly for remote farming [2]. We moved it remotely via a wireless network. Our evaluation target was the detection of stationary sound sources. While the UGV was stopped at an arbitrary position, we played a mallard call sound file for 10 s. The sound pressure level produced from the speaker was $75\pm 10$ dB.

We conducted experiments separately on 19, 26, and 28 August 2020.

Table 5. Meteorological conditions on the respective experiment days.

Parameter	Experiment A	Experiment B	Experiment C
Date	19 August 2020	26 August 2020	28 August 2020
Time (JST)	14:00–15:00	14:00–15:00	14:00–15:00
Weather	Sunny	Sunny	Sunny
Air pressure (hPa)	1008.1	1008.8	1007.2
Temperature (°C)	28.6	31.7	33.4
Humidity (%)	65	53	58
Wind speed (m/s)	4.3	3.8	4.9
Wind direction	WNW	WNW	W

shows the meteorological conditions on the respective experimental days. All days were clear and sunny under a migratory anticyclone. Experiments were conducted during 14:00 to 15:00 Japan Standard Time (JST), which is 9 h ahead of the Coordinated Universal Time (UTC). The temperature was approximately 30 °C. The humidity was in the typical mean range for Japan. The wind speed was less than 5 m/s.

We conducted three evaluation experiments, designated as Experiments A–C. Experiment A comprised a sound source orientation estimation experiment using a single microphone array. The objective of this experiment was to verify the angular resolutions. Subsequently, Experiments B and C provided position estimation experimental results obtained using two microphone arrays. The difference between the two experimental setups was the field size and the orientations of the respective microphone arrays.

5.2. Experiment A

Figure 7 depicts the setup of the sound source positions and the microphone array for Experiment A. We used M1 alone to evaluate the angular resolution as a preliminary experiment. The sound source comprises 26 positions: P1–P26. We divided these positions into two groups as P1–P13 and P14–P26 which were located, respectively, on the circumferences of half circles with radii of 10 m and 20 m. The angle interval of the respective positions from the origin was 15°. Here, P1–P13 existed on the straight lines of P14–P26 from the origin.

Let ${\theta }_g$ denote the ground-truth (GT) angle of $\theta$. Figure 8 depicts the angle estimation results for 16 positions at θ_g = ±15°, ±30°, ±60°, and ±90°. The experiment results demonstrated a unimodal output distribution of $G\left(\theta \right)$. Moreover, $G\left(\theta \right)$ at positions with the radius of 10 m were found to be greater than those of 20 m. The purple-filled circles denote the vertices of the respective output waves. We let ${\theta }_m$ represent the maximum angle of $G\left(\theta \right)$ in $-{90}^{°}\le \theta \le {90}^{°}$ calculated from the Formula (21).

Table 6. Angle estimation results and error values found from Experiment A (°).

Position	${\mathit{\theta }}_{\mathit{g}}$	${\mathit{\theta }}_{\mathit{m}}$	E	Position	${\mathit{\theta }}_{\mathit{g}}$	${\mathit{\theta }}_{\mathit{m}}$	E
P1	−90	−90	0	P14	−90	−83	7
P2	−75	−72	3	P15	−75	−68	7
P3	−60	−54	6	P16	−60	−56	4
P4	−45	−44	1	P17	−45	−42	3
P5	−30	−29	1	P18	−30	−29	1
P6	−15	−15	0	P19	−15	−15	0
P7	0	0	0	P20	0	0	0
P8	15	15	0	P21	15	15	0
P9	30	29	1	P22	30	29	1
P10	45	43	2	P23	45	43	2
P11	60	54	6	P24	60	54	6
P12	75	67	8	P25	75	63	12
P13	90	74	16	P26	90	72	18

presents ${\theta }_m$ at the respective positions. The error E between ${\theta }_g$ and ${\theta }_m$ is calculated as:

$$E=\sqrt{{({\theta }_g-{\theta }_m)}^2}.$$

(23)

The experimentally obtained results demonstrated that $E=0$ in $-{15}^{°}\le {\theta }_m\le {15}^{°}$. As an overall tendency, the increase of the angles and E demonstrated a positive correlation apart from the negative angles of the 10 m radius.

Figure 9 presents the two-dimensional distributions of $(x_m,y_m)$ for comparison with $(x_g,y_g)$, which represents the GT coordinates of the respective positions. The theoretical distance between M1 and the respective sound source positions was unobtainable for this experimental setup. Therefore, provisional positions were calculated by substituting $L_1$ as 10 m or 20 m.

Figure 10 presents scatter plots of ${\theta }_g$ and ${\theta }_m$. The distribution results for $L_1$ = 10 m and $L_1$ = 20 m are presented, respectively, in the left and right panels. On the plus side, ${\theta }_m$ for ${\theta }_g$ exhibited smaller values as the angle increased. By contrast, on the minus side, it exhibited greater values as the angle increased. As the angle increased, the absolute ${\theta }_m$ for absolute ${\theta }_g$ tended to decrease.

5.3. Experiment B

Experiment B was performed to evaluate the position estimation using two microphone arrays: M1 and M2. Figure 11 depicts the setup of the sound source positions and the microphone arrays for Experiment B. The experiment field was a square area of 30 m in length and width. The coordinates of M1 and M2 were, respectively, (0, 30) in the upper left corner and (30, 0) in the lower right corner. The microphone frontal orientations of M1 and M2 were, respectively, horizontal and vertical. For this arrangement, the positions on the diagonal between M1 and M2 were undetermined because the intersection of $L1$ and $L2$ was indefinite in the condition of ${\theta }_1=-{\theta }_2$. Therefore, the estimation target comprised 12 positions at 10 m intervals along the vertical and horizontal axes, excluding coordinates on the diagonal.

Figure 12 depicts the angle estimation results for 8 of 12 positions. The respective output waves exhibited a distinct peak, especially for shallow angles.

Table 7. Angle estimation results and error values found from Experiment B (°).

Position	${\mathit{\theta }}_{\mathit{g}1}$	${\mathit{\theta }}_{\mathit{g}2}$	${\mathit{\theta }}_{\mathit{m}1}$	${\mathit{\theta }}_{\mathit{m}2}$	${\mathit{E}}_1$	${\mathit{E}}_2$
P1	−90	90	−86	87	4	3
P2	−72	90	−71	87	1	3
P3	−56	90	−56	87	0	3
P4	−90	72	−87	72	3	0
P5	−63	63	−63	62	0	1
P6	−34	0	−34	0	0	0
P7	−90	56	−83	55	7	1
P8	−27	27	−27	27	0	0
P9	−18	0	−18	0	0	0
P10	0	34	0	34	0	0
P11	0	18	0	18	0	0
P12	0	0	0	0	0	0

presents the experiment results: ${\theta }_{g1}$, ${\theta }_{g2}$, ${\theta }_{m1}$, ${\theta }_{m2}$, $E_1$, and $E_2$. The mean error values of M1 and M2 were, respectively, 1.25° and 0.92°. The highest error of M1 was 7° at P7. This error increased the mean M1 error.

Table 8. Position estimation results obtained from Experiment B (°).

Position	${\mathit{x}}_{\mathit{g}}$	${\mathit{y}}_{\mathit{g}}$	${\mathit{x}}_{\mathit{m}}$	${\mathit{y}}_{\mathit{m}}$	${\mathit{E}}_{\mathit{x}}$	${\mathit{E}}_{\mathit{y}}$
P1	0	0	1.47	2.00	1.47	2.00
P2	0	10	1.05	9.97	1.05	0.03
P3	0	20	0.53	19.88	0.53	0.12
P4	10	0	9.40	1.08	0.60	1.08
P5	10	10	10.73	9.82	0.73	0.18
P6	10	30	9.76	30.00	0.24	0.00
P7	20	0	20.16	1.21	0.16	1.21
P8	20	20	19.87	19.87	0.13	0.13
P9	20	30	20.25	30.00	0.25	0.00
P10	30	10	30.00	9.76	0.00	0.24
P11	30	20	30.00	20.25	0.00	0.00
P12	30	30	30.00	30.00	0.00	0.00

presents the position estimation results. Coordinates ($x_m$,$y_m$) were calculated from ${\theta }_{m1}$ and ${\theta }_{m2}$ based on the proposed method (21). The error values $E_x$ and $E_y$ were calculated from the following.

$$\left[\begin{array}{c}{E}_x\\ E_y\end{array}\right]=\left[\begin{array}{c}\sqrt{{(x_g-x_m)}^2}\\ \sqrt{{(y_g-y_m)}^2}\end{array}\right].$$

(24)

The mean error values of $E_x$ and $E_y$ are, respectively, 0.43 m and 0.42 m. The details showed that P11 and P12 had no error. The error at P1 was the highest. Figure 13 presents the distributions of the GT coordinates $(x_g,y_g)$ and estimated coordinates $(x_m,y_m)$. The six positions on the upper right indicate that the error values were small. This resulting tendency was attributed to the measurement angles of M1 and M2 being within 45°. The six positions on the bottom left indicate that the error values were large. This resultant tendency was attributed to the measurement angles of M1 and M2, which were greater than 45°.

Figure 14 presents scatter plots of the GT coordinates and estimated coordinates. The distribution results for x and y are presented, respectively, in the left and right panels. Compared to Experiment A, the arrangement of the microphone array for Experiment B used 90°, which is half of the effective measurement range. However, the estimated angle error increased as it approached 90°. The error distribution trend demonstrated that the estimated position coordinates were calculated as a larger value than the GT position coordinates.

5.4. Experiment C

Figure 15 shows the experimental setup used for Experiment C. Position estimation was conducted in a rectangular area of 30 m in width and 20 m in length. The two microphone arrays, M1 and M2, were installed, respectively, at coordinates (0, 20) and (30, 0). The microphone frontal orientation of M1 and M2 was set to the equivalent of their diagonals to use the small error range around 0° effectively. The position detection interval was 5 m horizontally and vertically. For Experiment C, the position that could not be calculated on the diagonal was merely (15, 10). This field consists of a rectangular shape, unlike the square field for Experiment A. The estimation target for Experiment C had 32 positions: P1–P32.

Figure 16 depicts the angle estimation results obtained for each group with similar vertical positions to those shown in Figure 15. The respective results demonstrated that distinct peaks shown as filled purple circles from the output waves were obtained at the source direction angles.

Table 9. Angle estimation results and errors obtained from Experiment C (°).

Position	${\mathit{\theta }}_{\mathit{g}1}$	${\mathit{\theta }}_{\mathit{g}2}$	${\mathit{\theta }}_{\mathit{m}1}$	${\mathit{\theta }}_{\mathit{m}2}$	${\mathit{E}}_1$	${\mathit{E}}_2$
P1	45	−45	44	−45	1	0
P2	31	−45	31	−45	0	0
P3	18	−45	18	−44	0	1
P4	8	−45	7	−43	1	2
P5	0	−45	0	−44	0	1
P6	−6	−45	−5	−44	1	1
P7	45	−36	44	−34	1	2
P8	27	−34	27	−33	0	1
P9	11	−31	11	−31	0	0
P10	0	−27	0	−27	0	0
P11	−8	−18	−7	−17	1	1
P12	−14	0	−14	0	0	0
P13	−18	45	−18	45	0	0
P14	45	−27	45	−25	0	2
P15	18	−23	18	−23	0	0
P16	0	−18	0	−18	0	0
P17	−18	0	−18	0	0	0
P18	−23	18	−23	18	0	0
P19	−27	45	−27	45	0	0
P20	45	−18	44	−18	1	0
P21	0	−14	0	−13	0	1
P22	−18	−8	−18	−7	0	1
P23	−27	0	−27	0	0	0
P24	−31	11	−31	10	0	1
P25	−34	27	−34	27	0	0
P26	−36	45	−35	44	0	1
P27	−45	−6	−45	−6	0	0
P28	−45	0	−44	0	1	0
P29	−45	8	−44	8	1	0
P30	−45	18	−45	17	0	1
P31	−45	31	−44	30	1	1
P32	−45	45	−45	45	0	0

presents the estimated angles and error values. The mean error values of M1 and M2 were, respectively, 0.28° and 0.53°. The error values were smaller than 2° because an effective angular range smaller than ±45° was used.

Table 10. Position estimation results obtained from Experiment C (m).

Position	${\mathit{x}}_{\mathit{g}}$	${\mathit{y}}_{\mathit{g}}$	${\mathit{x}}_{\mathit{m}}$	${\mathit{y}}_{\mathit{m}}$	${\mathit{E}}_{\mathit{x}}$	${\mathit{E}}_{\mathit{y}}$
P1	0	0	0.35	0.00	0.35	0.00
P2	5	0	4.99	0.00	0.01	0.00
P3	10	0	10.01	0.35	0.01	0.35
P4	15	0	15.22	0.52	0.22	0.52
P5	20	0	19.82	0.18	0.18	0.18
P6	25	0	23.70	0.11	1.30	0.11
P7	0	5	0.25	5.78	0.25	0.78
P8	5	5	4.75	5.37	0.25	0.37
P9	10	5	10.15	4.95	0.15	0.05
P10	15	5	15.19	4.81	0.19	0.19
P11	20	5	16.22	7.33	3.78	2.33
P12	25	5	25.05	4.95	0.05	0.05
P13	30	5	30.00	4.71	0.00	0.29
P14	0	10	0.00	10.92	0.00	0.92
P15	5	10	5.06	10.08	0.06	0.08
P16	10	10	9.61	10.39	0.39	0.39
P17	20	10	20.39	9.61	0.39	0.39
P18	25	10	24.94	9.92	0.06	0.08
P19	30	10	30.00	10.25	0.00	0.25
P20	0	15	0.08	15.24	0.08	0.24
P21	5	15	3.34	16.66	1.66	1.66
P22	10	15	12.65	13.55	2.65	1.45
P23	15	15	14.81	15.19	0.19	0.19
P24	20	15	19.38	15.17	0.62	0.17
P25	25	15	25.09	15.12	0.09	0.12
P26	30	15	29.74	14.76	0.26	0.24
P27	5	20	5.30	20.00	0.30	0.00
P28	10	20	10.18	19.82	0.18	0.18
P29	15	20	15.13	19.74	0.13	0.26
P30	20	20	19.37	20.00	0.63	0.00
P31	25	20	24.76	19.57	0.24	0.43
P32	30	20	30.00	20.00	0.00	0.00

presents the position estimation results. The mean error values of $E_x$ and $E_y$ were, respectively, 0.46 m and 0.38 m. The error value at P11 was the highest. Moreover, the error values at P6, P21, and P22 were high. These positions were present in the shallow angles. For this setup of the microphone frontal directions, slight angular errors induced greater coordinate errors.

Figure 17 portrays the distributions for the GT positions and estimated positions. This tendency indicated that the error values were significant at positions near the diagonal between M1 and M2.

Figure 18 portrays the scatter plots between the GT coordinates and estimated coordinates for the respective axes. The distribution variation shown for the results of Experiment C was higher than that of the scatter plots for Experiment B. By contrast, the distribution variation was low in the lower left and upper right regions, which were more distant from both microphone arrays.

5.5. Discussion

In Japanese rice cultivation, paddy rice seedlings are planted with approximately a 0.3 m separation. Therefore, mallards move in a grid up to 0.3 m.

Table 11. Estimation accuracies within grids up to the third neighborhood.

Experiment	Axis	0.3 m	0.6 m	0.9 m	1.2 m
B	x	58.3%	75.0%	83.3%	91.7%
B	y	75.0%	75.0%	75.0%	83.3%
C	x	71.9%	81.3%	87.5%	87.5%
C	y	65.6%	84.4%	87.5%	90.6%
Mean		67.7%	78.9%	83.3%	88.3%

presents the simulated evaluation results of the estimation accuracy for mallard detection within grids up to the third neighborhood in each coordinate for Experiments B and C. The experimentally obtained results revealed that the proposed method can detect mallards with 67.7% accuracy in a similar grid. Moreover, accuracies of 78.9%, 83.3%, and 88.3% were obtained, respectively, in the first, second, and third neighborhood regions. We considered that the allowable errors can be set widely for various mallard group sizes. Although this study was conducted to assess the application of the method in a rice paddy field, the accuracy can be expected to vary greatly depending on other applications.

6. Conclusions

This study was undertaken to detect mallards based on acoustic information. We developed a prototype system comprising two sets of microphone arrays. In all, we used 64 microphones installed on our originally designed and assembled sensor mounts. For the benchmark evaluation, we obtained three acoustic datasets in an outdoor environment. For the first experiment, the angular resolutions were evaluated using a single microphone array. An error accumulation tendency was demonstrated as the angle increased from the front side of the microphones. For the second experiment, the sound sources were estimated at 12 positions with 10 m grids in a square area. The accumulated position errors were affected by the frontal orientation settings of the two microphone arrays. For the third experiment, sound sources were at 23 positions with 5 m grids in a rectangular area. To minimize blind positions, the two microphone arrays were installed with their front sides facing diagonally. Although the error increased in the diagonal direction because of the effect of the angular resolution of shallow angles, the positional errors were reduced compared to the second experiment. These experimentally obtained results revealed that the proposed system demonstrated adequate accuracy for application to rice–duck farming.

As future work aimed at practical use, we expect to improve our microphone array system in terms of miniaturization and waterproofing to facilitate its practical use for remote farming. For the problem of missing sound sources on the diagonal line between the two microphone arrays, we must consider superposition approaches using probability maps. We would like to distinguish mallard calls from noises such as the calls of larks, herons, and frogs in paddy fields. Moreover, for the protection of the ducks, we expect to combine acoustic information and visual information to detect and recognize the natural enemies of mallards. For this task, we plan on developing a robot that imitates natural enemies such as crows, kites, and weasels. We expect to improve the resolution accuracy of multiple mallard sound sources. Furthermore, we would like to discriminate mallards from crows and other birds and predict the time series mobility paths of mallards using a DL-based approach combine with state-of-the-art backbones. Finally, we would like to actualize the efficient protection of mallards by developing a system that notifies farmers of predator intrusion based on escape behavior patterns. This is because physical protection of mallards, such as electric fences and nylon lines around over paddy fields, requires a significant effort from farmers.