Vision-Based Parking-Slot Detection: A Benchmark and A Learning-Based Approach

Abstract

Recent years have witnessed a growing interest in developing automatic parking systems in the field of intelligent vehicles. However, how to effectively and efficiently locating parking-slots using a vision-based system is still an unresolved issue. Even more seriously, there is no publicly available labeled benchmark dataset for tuning and testing parking-slot detection algorithms. In this paper, we attempt to fill the above-mentioned research gaps to some extent and our contributions are twofold. Firstly, to facilitate the study of vision-based parking-slot detection, a large-scale parking-slot image database is established. This database comprises 8600 surround-view images collected from typical indoor and outdoor parking sites. For each image in this database, the marking-points and parking-slots are carefully labeled. Such a database can serve as a benchmark to design and validate parking-slot detection algorithms. Secondly, a learning-based parking-slot detection approach, namely $PS{D}_L$, is proposed. Using $PS{D}_L$, given a surround-view image, the marking-points will be detected first and then the valid parking-slots can be inferred. The efficacy and efficiency of $PS{D}_L$ have been corroborated on our database. It is expected that $PS{D}_L$ can serve as a baseline when the other researchers develop more sophisticated methods.

1. Introduction

Owing to the increased demand for autonomous driving [1,2], the development of automatic parking assistant systems (PAS) has become a topic of intense study. A typical automatic parking system starts by target position designation, i.e., locating a vacant parking space for the vehicle. To resolve the problem of target position designation, various solutions have been proposed in the literature and they roughly fall into two categories, infrastructure-based ones and in-vehicle sensor based ones.

A typical infrastructure-based method usually resorts to a pre-built map and infrastructure-level sensors. Desirable target positions are designated based on the infrastructure and the vehicle receives the parking information through vehicle-infrastructure communication [3,4,5,6,7,8]. Obviously, the infrastructure-based methods have an advantage of managing all parking-slots; however, they may not be applicable in a short time due to the requirement of additional hardware installation on current park-sites and vehicles.

When the infrastructure is not available, the PAS may need to depend on an in-vehicle sensor-based method to identify an appropriate parking space. Such methods can perceive the available parking spaces during the movement of the vehicle by making only use of sensors it carries. These methods can be categorized into two groups, the free-space-based ones and the parking-slot-marking-based ones.

A free-space-based approach designates a target position by recognizing a vacant space between two adjacent vehicles. This is the most widely used approach as it can be implemented using various range-finding sensors, such as ultrasonic sensors [9,10,11,12,13,14,15], laser scanners [16,17,18], short-range radars [19,20,21], structured light [22], depth cameras [23], stereo cameras [24,25,26,27,28,29], etc. However, the free-space-based approach has an inherent drawback in that it cannot find free spaces when there is no adjacent vehicles and its accuracy depends on the positions and poses of adjacent vehicles.

Parking-slot-markings refer to the regular line segments painted on the ground, indicating the valid areas for parking. Unlike the free-space-based approaches, a parking-slot-marking-based approach finds parking spaces by recognizing visual slot-markings from image sequences acquired by cameras mounted on the vehicle and thus its performance does not depend on the existence or poses of adjacent vehicles. Moreover, in most cases, slot-markings can provide more accurate parking information than “free-space”. Meanwhile, most car manufacturers have started to produce vehicles equipped with wide FOV (field of view) imaging sensors, usually used in an AVM (around view monitoring) system. For these reasons, the parking-slot-marking-based approach has began to draw a lot of attention recently in the field of parking space detection, which is also our focus of this paper.

1.1. Related Work

In this paper, we focus on studying the problem of in-vehicle vision-based parking-slot detection. Usually to fulfill this task, a bird’s-eye-view image or a surround-view image (obtained by synthesizing multiple bird’s-eye-view images) is generated in real-time first and then the parking-slots are detected from it. Representative work in this area will be reviewed here.

The research in this area started from Xu et al.’s pioneer work [30]. In [30], Xu et al. claimed that the colors of parking-slot’s markings are quite uniform and different from the background and thus they trained a neural network to segment parking-slot-markings. Then, they estimated two perpendicular lines as the parking-slot contour. The drawback of this simple parking-slot model is that the type (perpendicular or parallel) of the parking-slot cannot be obtained. In [31], Jung et al. presented a one-touch method that recognizes the line segments of a parking-slot by checking the directional gradient based on a manually designated point inside the target parking-slot. Since this method can handle only a single type of parking-slot, Jung et al. [32] extended it to a two-touch method. This method can recognize various types of parking-slot-markings based on two points of a parking-slot entrance provided by the driver. In [33], Du and Tan developed a reverse parking system. To detect the parking-slot, they first applied a ridge detector on the image and then medial axes of the slot markings can be obtained after the steps of noise filtering, connected components labeling, and removal of components with a small number of pixels. However, their system relies on human drivers to identify an empty parking-slot first before initiating the parking process. The apparent drawback of the methods in [31,32,33] is that they are not fully automatic.

Fully automatic parking-slot detection methods are developed along two main streams, the line-based ones and the corner-based ones, according to the primitive visual features they extract. In [34], Jung et al. assumed that marking-lines consist of lines with a fixed width and recognized them by applying peak-pair detection and clustering in Hough space [35]. Separating marking-line segments were at last recognized by a T-shaped template matching. Following the similar idea as Jung et al.’s work [34], Wang et al. [36] proposed to detect marking-line segments in Radon space [37] as they considered that Radon transform has a better noise-tolerance ability and is more robust than Hough transform. One potential drawback of the methods in [34,36] is their sensitiveness to the marking-line width. In [38], after obtaining the edge map of the surround-view image, Hamada et al. made use of the probabilistic Hough transform [39] to extract all line segments and then they inferred the valid parking-slots based on some geometric constraints. In [40], Suhr and Jung designed a parking-slot detection approach specially for underground and indoor environments. In their approach, the guide line is detected first and then the separating lines are detected. To detect the guide line, they utilized the RANSAC (RANdom SAmpling Consensus) [35] algorithm for robust line fitting from edge pixels and to detect the separating lines, they used a distance transform-based chamfer matching [41]. The limitations of Suhr and Jung’s method mainly lie in two points: (1) it can only detect perpendicular parking-slots but cannot detect parallel ones; (2) it requires that the guide line of the parking-slots should be visible. In [42], Lee and Seo proposed the so-called “cone-hat” filter for line-marking extraction and then the extracted line features were assigned to parking-line segments via entropy-based clustering. After that, the sequential RANSAC algorithm [43] was utilized to fit parking-lines from clusters. At last, parking-slot candidates were generated and then validated by a Bayesian network model [44].

Different from line-based ones, a few other parking-slot detection methods are based on corners and among them Suhr and Jung’s work is a representative one [45,46]. The method proposed in [45,46] detects corners via the Harris corner detector [47] first and then generates junctions by combining these corners; finally, parking-slots are inferred from junction pairs. Thus, the success rate of this method highly depends on the robustness of the Harris corner detector.

1.2. Our Motivations and Contributions

Having investigated the literature, we find that in the field of vision-based parking-slot detection, there is still large room for further improvement in at least two aspects.

Firstly, vision-based parking-slot detection is not an easy problem. The backgrounds can vary significantly in different parking sites as the ground surfaces can have various textures. More seriously, the illumination condition can change greatly, especially at the outdoor parking sites. Though a plethora of solutions have been proposed, nearly all the existing state-of-the-art ones are based on low-level visual features, such as lines and corners, detected by some low-level vision algorithms (such as Canny edge detector, Hough transform, Radon transform, Harris corner detector, etc.). These features actually are not quite distinguishable and even worse, they are unstable and unrepeatable to changes in environment aroused by noise, clutter, illumination variation, etc. These serious drawbacks inevitably limit the performance of methods depending on low-level features. Actually, how to efficiently and accurately detect parking-slots using vision-based methods in uncontrolled environment is still a great challenge. In addition, none of the researchers have ever published their source codes, which undoubtedly hinders the development of this field.

Secondly, vision-based parking-slot detection is actually a “pattern classification” problem. To cope with it, a publicly-available large-scale benchmark dataset is highly desired, which in fact is indispensable for researchers to design and compare the detection algorithms. Unfortunately, previous researchers have never published their datasets and thus such a dataset is still lacking in this area. Without a common benchmark dataset, it is impossible to make fair comparisons among different parking-slot detection algorithms.

In this work, we make an attempt to fill the aforementioned research gaps to some extent. Our contributions in this paper are summarized as follows:

(1) A data-driven learning-based approach, namely $PS{D}_L$ (Parking-Slot Detection based on Learning), is proposed for parking-slot detection. Given a surround-view image, $PS{D}_L$ detects the marking-points using a pre-trained detector first and then infers the valid parking-slots from them. Marking-points are defined as the cross-points of parking-lines. In Figure 1, examples of marking-points on two surround-view images are marked by yellow circles. The advantages of $PS{D}_L$ over the existing parking-slot detection methods are twofold. Firstly, $PS{D}_L$ is built upon marking-point patterns, which are more distinguishable and stable than primitive visual features, such as lines or corners. Secondly, for detecting marking-points, $PS{D}_L$ adopts a data-driven learning-based strategy, which is much more robust to changes of imaging conditions than low-level vision algorithms. To our knowledge, our work is the first to use learning-based techniques to detect visual patterns in the field of parking-slot detection. $PS{D}_L$ can detect both the perpendicular and the parallel parking-slots. Besides, it can work equally well in indoor and outdoor environments. Its efficacy and efficiency have been thoroughly evaluated in experiments. As we have already published its source code, $PS{D}_L$ can serve as a baseline when the other researchers develop more sophisticated algorithms.

(2) To facilitate the study of vision-based parking-slot detection, we have established a large-scale benchmark dataset and have made it publicly available to the research community. This dataset comprises 8600 surround-view images collected from typical indoor and outdoor parking sites with our experimental car and all the images were manually labeled with care. Such a dataset can be employed for training and testing new parking-slot detection algorithms. To our knowledge, this dataset is the first of its kind in the field of vision-based parking-slot detection. Please refer to Section 4.1 for more details about this dataset.

To make the results reported in this paper fully reproducible, the collected benchmark dataset and all the relevant source code have been made publicly available at https://cslinzhang.github.io/ps/.

The remainder of this paper is organized as follows. Section 2 presents the steps to generate the surround-view image. Section 3 introduces our novel approach $PS{D}_L$ for parking-slot detection. Experimental results are presented in Section 4. Finally, Section 5 concludes the paper.

2. Surround-View Generation

The high-level structure of a typical vision-based PAS system is shown in Figure 2. In such a system, the surround-view image sequence is synthesized in real-time from outputs of multiple wide-angle cameras mounted on the vehicle. Taking the surround-view image as input, the parking-slot detection module detects the valid parking-slot(s) and then send the coordinates of parking-slot(s) with respect to the vehicle coordinate system to the decision module for further process (For example, usually the decision module will use an ultrasonic radar to test whether the parking-slot returned by the parking-slot detection module is vacant or not). In this section, we briefly introduce the steps for surround-view generation.

The automotive surround-view camera system normally consists of 4 to 6 wide-angle cameras mounted around the vehicle, each facing a different direction. On our experimental car, 4 low-cost fish-eye cameras are mounted. From these camera inputs, a ${360}^{\circ }$ surround-view image around the vehicle can be synthesized.

Actually, the surround-view is the composite view of four bird’s-eye-views, the front-view, the left view, the rear view, and the right view. Two adjacent bird’s-eye-views overlap with each other. The key step for generating the bird’s-eye-view image is to build an off-line lookup table $T_{B\to F}$, mapping a point ${\mathit{x}}_B$ on the bird’s-eye-view image to a position ${\mathit{x}}_F$ on the input fish-eye image, by conducting a set of calibration operations. To determine $T_{B\to F}$, we need to determine $P_{B\to W}$ the transformation matrix from the bird’s-eye-view coordinate system to the world coordinate system, $P_{W\to U}$ the transformation matrix from the world coordinate system to the undistorted input image coordinate system, and $T_{U\to F}$ the lookup table mapping a point on the undistorted input image to a position on the original input fish-eye image. Figure 3 illustrates the relationships among the coordinate systems involved in bird’s-eye-view generation.

The distortion coefficients of the fish-eye camera can be estimated by Zhang’s calibration method [48,49] and accordingly, the mapping table $T_{U\to F}$ can be obtained.

$P_{B\to W}$ is a similarity transformation matrix, which can be determined straightforwardly if the size (measured by pixels) of the bird’s-eye-view image and the corresponding physical visible range (measured by millimeters) are determined beforehand. As illustrated in Figure 4, suppose that the size of the bird’s-eye-view image is $M\times N$ and the height of the corresponding physical area is H mm, it is easy to verify that $P_{B\to W}$ can be expressed as,

$$P_{B\to W}=\left[\begin{array}{c}\frac{H}{M}0-\frac{HN}{2M}\hfill \\ 0-\frac{H}{M}\frac{H}{2}\hfill \\ 001\hfill \end{array}\right]$$

(1)

To estimate the homography matrix $P_{W\to U}$, a calibration field is used, in which the world coordinates of the feature points are known beforehand. An image of the calibration field is captured and then undistorted using $T_{U\to F}$. Then, on the undistorted image, $N_F$ feature points ($N_F$ cannot be smaller than 4 and based on our experience it is usually set between 10 and 20.) can be manually selected as illustrated in Figure 5. Of course, for each selected feature point i we know its coordinates ${\mathit{x}}_W^i$ in the world coordinate system and its coordinates ${\mathit{x}}_U^i$ in the undistorted image coordinate system. The relationship between ${\mathit{x}}_W^i$ and ${\mathit{x}}_U^i$ is ${\mathit{x}}_U^i=P_{W\to U}{\mathit{x}}_W^i$. Therefore, $P_{W\to U}$ can be estimated by solving a least-square problem based on correspondence pairs ${\left\{{\mathit{x}}_W^i,{\mathit{x}}_U^i\right\}}_{i=1}^{N_F}$.

When the matrices $P_{B\to W}$ and $P_{W\to U}$, and the mapping table $T_{U\to F}$ are ready, the mapping table $T_{B\to F}$ can be finally determined. In our case, we use four fish-eye cameras and thus we can have four mapping tables ${\left\{T_{B\to F}^i\right\}}_{i=1}^4$, each being responsible for generating one bird’s-eye-view. To synthesize the surround-view, at last we need to determine the stitching line between each pair of adjacent bird’s-eye-views. With the four stitching lines, for a point on the surround-view, we can know which bird’s-eye-view it should come from and accordingly, which mapping table and the associated input fish-eye image should be used to determine its pixel value.

In Figure 6, we show an example for surround-view generation. Figure 6a–d is four fish-eye images and Figure 6e is the surround-view image synthesized from Figure 6a–d.

3. ${\mathit{PSD}}_{\mathit{L}}$: A Learning Based Approach for Detecting Parking-Slots

In this section, our proposed parking-slot detection approach $PS{D}_L$ will be presented in detail. It is designed to detect typical perpendicular and parallel parking-slots as illustrated in Figure 7. Based on Figure 7, it can be seen that the entrance-lines of parking-slots that can be correctly detected by our $PS{D}_L$ approach are composed of “T-shaped” or “L-shaped” marking-point patterns. $PS{D}_L$ actually comprises two phases, detecting marking-points and then inferring valid parking-slots from detected marking-point patterns.

3.1. Marking-Point Detection

A marking-point pattern refers to a local image patch centered at a cross-point of parking-line segments, as indicated by the yellow circular marks in Figure 7. To detect marking-point patterns, a binary classifier is designed, which takes a local image patch as input and outputs a binary value indicating whether the input is a marking-point pattern or not. Based on the labeled benchmark dataset (see Section 4.1 for details), the positive training image patches can be simply extracted from labeled surround-view images while the negative training patches are extracted using a bootstrapping process during training. It needs to be stressed that based on our experience, chrominance information is unstable for parking-slot detection since the lighting conditions can vary significantly in different parking occasions. Hence, the colorful training surround-view images are first converted to their gray-scale versions. Similarly, at the testing stage, the input surround-view image will also be converted to gray-scale.

To train the marking-point detector, features and the classifier model need to be determined. With respect to features, three types of features are used. The first feature is the normalized intensity. The second feature is the gradient magnitude. Given an image $I\left(\mathit{x}\right)$, its partial derivatives $G_x\left(\mathit{x}\right)$ and $G_y\left(\mathit{x}\right)$ can be computed by filtering $I\left(\mathit{x}\right)$ with Sobel gradient operators [35]. Then, the gradient magnitude of $I\left(\mathit{x}\right)$ is computed as $GM\left(\mathit{x}\right)=\sqrt{G_x^2\left(\mathit{x}\right)+G_y^2\left(\mathit{x}\right)}$. The last type of feature is the oriented gradient magnitudes [50]. The ith oriented gradient magnitude map $Q_i\left(\mathit{x}\right)$ of $I\left(\mathit{x}\right)$ is defined as,

$$Q_i\left(\mathit{x}\right)=GM\left(\mathit{x}\right)·\mathbf{1}\left(\mathrm{\Theta }\left(\mathit{x}\right)==i\right),i=1,2\dots ,N_O$$

(2)

where $\mathbf{1}\left(·\right)$ is the indicator function, $\mathrm{\Theta }\left(\mathit{x}\right)$ is the quantized gradient angle of $I\left(\mathit{x}\right)$ and the range of $\mathrm{\Theta }\left(\mathit{x}\right)$ values is $[1,N_O]$. Given an image patch, its normalized intensity map, its gradient magnitude map, and its oriented gradient maps are vectorized and then concatenated together as the final feature vector.

With respect to the classifier, we adopt the popular AdaBoost framework [51]. A boosted classifier H consisting of M weak classifiers can be represented as,

$$H\left(\mathit{x}\right)=\sum _{t=1}^M{\alpha }_t{h}_t\left(\mathit{x}\right)$$

(3)

where each $h_t$ is a weak classifier and ${\alpha }_t$ is its associated weight. $\mathit{x}$ is classified as positive if $H\left(\mathit{x}\right)>0$ and $H\left(\mathit{x}\right)$ serves as the confidence. In terms of the weak classifier, we use the shallow decision tree. The training is conducted in several stages and after each stage a bootstrapping process is performed to extract negative samples for the next stage.

At the detection stage, if the base AdaBoost classifier is utilized, it would be quite slow. A cascade structure is a common way to reduce the computational burden of evaluating a complex classifier over an entire image [52]. To simplify training, we use the “constant soft-cascade” strategy [53] instead of a real cascade structure. During training, for the node i of the tree $h_t$, we record its weighted log-ratio defined as,

$$l_t^i=\frac{1}{2}{\alpha }_{t}ln\frac{p_i}{1-p_i}$$

(4)

where $p_i$ is the ratio of positive samples to all the samples reaching this node. $l_t^i$ can measure the “positiveness” of samples reaching the node i of the tree $h_t$. At the testing stage, when a testing sample $\mathit{t}$ reaches a node whose associated weighted log-ratio is smaller than a pre-defined constant threshold $\theta$, the testing stops since the probability of $\mathit{t}$ being positive will be quite low. Major steps for training the marking-point detector and applying it on a given surround-view image to detect marking-points are illustrated in Figure 8.

Another issue needs to be considered is that since marking-point patterns can be of any directions, a single detector would not be accurate enough. Thus, we train multiple detectors, each of them being responsible for detecting marking-point patterns whose directions are within a specific range. To keep the balance between the detection accuracy and the speed, we train four detectors ${\left\{de{t}_j\right\}}_{j=1}^4$ and $de{t}_j$ is responsible for detecting marking-point patterns whose directions are within the range $\left[-\frac{3\pi }{4}+\frac{\pi }{2}j,-\frac{\pi }{4}+\frac{\pi }{2}j\right]$. In order to train multiple detectors, when we label positive samples of marking-point patterns, their directions are also labeled besides their positions. The directions of marking-slot patterns are labeled in a way as illustrated in Figure 9. When positive image patches are extracted, they are first rotated to make their directions equal to 0. To train $de{t}_j$, we rotate each positive image patch with a set of angles ${\left\{\frac{j-1}{2}\pi +\frac{\pi }{4}{r}_k\right\}}_{k=1}^K$, where $r_k$ is a random number uniformly distributed over $[-1,1]$ and K defines the number of possible rotations. It can be known that if $N_P$ positive samples are labeled, there will be $K{N}_P$ positive samples for training $de{t}_j$.

In Figure 10, marking-point detection results on two typical surround-view images are shown.

3.2. Parking-Slot Inference

Having detected the marking-points, we then can infer valid parking-slots from them based on some rules.

Given two marking-points $P_1$ and $P_2$, we need to check whether the line $\overrightarrow{P_1{P}_2}$ can be a valid entrance-line of a parking-slot. At first, the length of $\overrightarrow{P_1{P}_2}$ should satisfy some length constraints obtained from the prior knowledge. Then, we check whether the image patterns around $P_1$ and $P_2$ conform to the parking-slot model. By examining the ideal parking-slot models shown in Figure 7, it can be seen that to be a valid parking-slot entrance-line, image patterns around $P_1$ and $P_2$ should conform to one of the pattern models as shown in Figure 11. Inspired by this analysis, we propose to use Gaussian line templates to check the image patterns around $P_1$ and $P_2$. Six Gaussian line templates are used and their positions and orientations relative to $P_1$ and $P_2$ are illustrated in Figure 12. By checking the six responses $r_1$∼$r_6$ with a set of simple rules, whether $\overrightarrow{P_1{P}_2}$ can be a valid entrance-line can be determined. Besides, whether the parking-slot is at the clockwise side or the anti-clockwise side of $\overrightarrow{P_1{P}_2}$ can also be determined. The checking-rules are summarized in Algorithm 1.

After completing the abovementioned steps, we can get a set of “entrance-line” candidates. However, there is one case that they may contradict with each other as illustrated in Figure 13. In Figure 13, $\overrightarrow{P_1{P}_2}$ and $\overrightarrow{P_2{P}_3}$ are two entrance-line candidates for two perpendicular parking-slots while $\overrightarrow{P_1{P}_3}$ is an entrance-line candidate for a parallel parking-slot. However, in fact, $\overrightarrow{P_1{P}_3}$ is an invalid entrance-line. Hence, when the entrance-line candidates are ready, we need to remove the ones passing any valid marking-points to annihilate contradictions.

Finally, valid entrance-lines are remained, each of which represents a valid parking-slot. Their information is then sent to the decision-making module. It needs to be noted that the “depth” of the parking-slot is determined by prior knowledge. Moreover, if multiple parking-slots are detected on the current surround-view image, it is the responsibility of the decision-marking module to choose the most appropriate one. In Figure 14, six typical surround-view images with marked parking-slots detected by $PS{D}_L$ are shown. From these samples, it can be seen that the proposed algorithm $PS{D}_L$ has a strong capability for accurately detecting different types of parking-slots under various conditions.

Algorithm 1: Checking-Rules for Determining the Validity of $P_1{P}_2$ for Being an Entrance-Line and the Parking-Slot Orientation

4. Experimental Results

4.1. Benchmark Dataset

In order to train and test our proposed parking-slot detection approach $PS{D}_L$, we have established a large-scale benchmark dataset, which is publicly available at https://cslinzhang.github.io/ps/. Surround-view images contained in this dataset were collected from typical indoor and outdoor parking sites using our self-developed AVM system equipped on a SAIC Roewe E50 (SAIC MOTOR, Shanghai, China) electric car [54]. Two types of parking-slots, the perpendicular ones and the parallel ones, are included. The spatial resolution of each surround-view image is $600\times 600$, corresponding to a 9600 mm × 9600 mm square region on the physical ground. This dataset comprises two subsets, the training set and the testing set.

In the training set, we labeled 5100 surround-view images for extracting positive marking-point patterns. For each marking-point pattern, we recorded its center and its local orientation (as illustrated in Figure 9). Altogether, we have 13,364 positive marking-point pattern samples. In addition, we labeled 2400 images for extracting negative samples used for training the marking-point detector. Specifically, on each image, all the possible marking-points were marked by bounding-boxes. During training, patches sampled from these images can be used as negative samples if they do not overlap with any bounding boxes.

The testing set comprises 500 labeled surround-view images. It can be used for testing the accuracy of a marking-point detection algorithm and also can be used for testing the final accuracy of a parking-slot detection algorithm.

4.2. Evaluating the Performance of Marking-Point Detection

In our proposed parking-slot detection approach $PS{D}_L$, marking-point detection is a crucial step. In this experiment, we evaluated the performance of our marking-point detection algorithm and also compared it with several other classical methods in the field of object detection, VJ [52], HoG + SVM [55], HoG + LBP [56], PLS [57], HIKSVM [58], MultiFtr [59] and Roerei [60].

For a ground-truth marking-point ${\mathit{g}}_i$, if there is a detected marking point ${\mathit{d}}_i$ satisfying $∥{\mathit{g}}_i-{\mathit{d}}_i∥<\delta$, where $\delta$ is a predefined threshold, we deem that ${\mathit{g}}_i$ is correctly detected and ${\mathit{d}}_i$ is a true positive. In our experiments, $\delta$ was set as 10. To compare various detectors, we plot miss rate against false positives per image (FPPI) using log-log plots by varying the threshold on detection confidence. The plots are shown in Figure 15. As recommended in [61], we use the log-average miss rate (LAMR) to summarize detector performance, computed by averaging miss rate at nine FPPI rates evenly spaced in log-space in the range ${10}^{-2}$ to ${10}^0$. Log-average miss rates achieved by different methods are also shown in Figure 15.

From the results shown in Figure 15, it can be seen that for the task of marking-point detection, our proposed method can achieve much higher accuracy than the other widely used methods in the field of object detection. Specifically, the LAMR of our approach is 18.82% while Roerei [60] is the runner-up whose LAMR is 23.77%.

4.3. Evaluating the Performance of Parking-Slot Detection

In this experiment, the detection accuracy of our proposed parking-slot detection algorithm $PS{D}_L$ was evaluated on the test set. Besides, the performance of several state-of-the-art methods in this field, including Jung et al.’s method [34], Wang et al.’s method [36], Hamda et al.’s method [38], and Suhr&Jung’s method [45], was also evaluated for comparison. We use the precision-recall rates as the performance measure, which are defined as,

$$\begin{array}{c}precision=\frac{truepositives}{truepositives+falsepositives}\hfill \\ recall=\frac{truepositives}{truepositives+falsenegatives}\hfill \end{array}$$

(5)

Each labeled parking-slot is represented as $P{S}_i=\left\{P_1^i,P_2^i,o^i\right\}$, where $P_1^i$ and $P_2^i$ are the coordinates of the two marking-points forming the entrance-line and $o^i$ represents the parking-slot’s orientation. If $o^i$ is 1, it means that the parking-slot $P{S}_i$ is at the anticlockwise side of $\overrightarrow{P_1{P}_2}$; if $o^i$ is −1, it means that $P{S}_i$ is at the clockwise side of $\overrightarrow{P_1{P}_2}$. Suppose that $P{S}_d=\left\{P_1^d,P_2^d,o^d\right\}$ is a detected parking-slot and $P{S}_l=\left\{P_1^l,P_2^l,o^l\right\}$ is a labeled ground-truth parking-slot. If $P_1^d$ matches with $P_1^l$, $P_2^d$ matches with $P_2^l$, and $o^d$ is equal to $o^l$, $P{S}_d$ is regarded as a true positive; if $P_1^d$ matches with $P_2^l$, $P_2^d$ matches with $P_1^l$, and $o^d$ is equal to $-o^l$, $P{S}_d$ is also regarded as a true positive; otherwise, $P{S}_d$ is a false positive.

In order to make the results achieved by different approaches comparable, we carefully adjusted the parameters of all the competing methods to make them achieve nearly the same high precision rates. Then, we could compare their recall rates. In this case, the approach that can achieve the highest recall rate is the best. The results are summarized in

Table 1. Performance evaluation of vision-based parking-slot detection methods.

Method	Precision Rate	Recall Rate
Jung et al. [34]	98.70%	59.13%
Wang et al. [36]	98.59%	61.76%
Hamada et al. [38]	98.61%	63.82%
Suhr&Jung [45]	98.70%	76.22%
PSD_L	98.87%	92.38%

. From Table 1, it can be observed that when operating at a high precision rate, $PS{D}_L$ can achieve a much higher recall rate than all the other competitors. The superiority of $PS{D}_L$ over the other state-of-the-art competitors corroborates that: (1) marking-point patterns are more stable and distinguishable than primitive visual features (lines or corners); and (2) data-driven learning-based detection strategy is more robust to imaging condition variations than low-level vision algorithms.

$PS{D}_L$ was implemented in C++ and tested on an in-vehicle industrial computer with a 2.4 GHZ Intel Core i5 CPU and 4G RAM. It can process 20∼25 surround-view image frames per second and thus it can satisfy the requirements of most automatic PASs.

4.4. Discussion about the Usability of Our Parking-Slot Detection System

Parking-slot detection is a key component in a self-parking system. Most of the commercial parking-slot detection systems are based on ultrasonic radars. These systems share an inherent drawback that they rely on vehicles which have already been properly parked as references. In addition, they cannot detect parking-slots defined by parking line segments painted on the ground either. The vision-based technology is a useful complement to ultrasonic radar based ones. In our work, a vision-based parking-slot detection system using low-cost imaging sensors was developed. With respect to the parking-slot detection algorithm, a learning-based approach PSD_L was proposed. Our system has been internally tested by SAIC MOTOR and it works quite well in practice.

5. Conclusions

Vision-based parking-slot detection is still an unresolved challenging problem. In this paper, we made two contributions to this field. Firstly, we collected and labeled a large-scale surround-view image dataset and have made it publicly available to the research community. Such a dataset will for sure benefit the study of parking-slot detection. Secondly, we proposed a novel learning-based parking-slot detection approach $PS{D}_L$. Its high efficacy and efficiency have been corroborated by comprehensive experiments and it has already been deployed in practice on our experimental car. $PS{D}_L$ can serve as a baseline when the other researchers develop more advanced approaches.