**1. Introduction**

Road traffic accidents are a significant social problem and it is estimated that, depending on the country, their costs amount to 1% up to 3% of gross domestic product [1]. Although road safety is improving in most European countries, the progress remains slow and misaligned with the established targets. This slow progress is partially due to the dynamic and complex nature of road traffic, and safety performance depends on a number of interconnected factors related to roadway environment, vehicles and humans.

In the past decade, with new technological breakthroughs, a significant effort has been devoted to improving vehicle safety systems. These safety systems can be divided into two categories: passive safety systems and active safety systems. A passive safety system reduces injuries sustained by passengers when an accident occurs, while active ones try to keep a vehicle under control and avoid accidents [2,3].

In general, Advanced Driver Assistance System (ADAS) is a collection of numerous intelligent units integrated into the vehicle itself that perform different tasks and assist the human driver in driving. Common ADAS functions include adaptive speed control, lane departure warning, forward collision warning, automatic high beam assist, traffic sign recognition, pedestrian and object detection, automatic emergency braking, etc. All these functions base their operations on different cameras, RADARs, LIDARs and other sensors which "scan" the environment around the vehicle in order to gather the needed information. Since the efficiency of such systems majorly depends on the data collected from the surrounding environment, it is clear that different road infrastructure elements, such as traffic signs or road markings, provide necessary cues not only to human drivers but also to built-in vehicle technologies.

Traffic Sign Recognition System (TSRS) is designed to detect and interpret roadside information in the form of signage. Its basic infrastructure can be generalized into three

**Citation:** Babi´c, D.; Babi´c, D.; Fioli´c, M.; Šari´c, Ž. Analysis of Market-Ready Traffic Sign Recognition Systems in Cars: A Test Field Study. *Energies* **2021**, *14*, 3697. https://doi.org/10.3390/en14123697

Academic Editors: Arno Eichberger, Zsolt Szalay, Martin Fellendorf and Henry Liu

Received: 12 May 2021 Accepted: 17 June 2021 Published: 21 June 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

17

specific components: visual sensor, image processor and vehicle display [4]. Acquiring information from traffic signs involves traffic sign detection (TSD), which consists of finding the location, orientation and size of traffic signs in natural scene images, and traffic sign recognition (TSR), or classifying the detected traffic signs into types and categories in order to extract the information they are providing to drivers [5]. In order to detect and recognize traffic signs, as mentioned before, vehicles are equipped with different technologies. Cameras are the most common sensors and can be used for TSR, TSD or both at the same time. LIDAR sensors have been used for TSD. Their 3D perceptive capabilities are useful to determine the position of a sign and its shape, and can also use the intensity of reflected light to improve detection accuracy based on the high reflectivity of traffic signs [5]. Potentially, the best solution presents the combination of LIDAR and cameras. Such fusion enables the collection of information from different sources, their comparison and analysis, and thus better detection and recognition of signs. Besides detecting and recognizing road signs, TSRS also use digital maps with already implemented signs (mainly related to the speed limit signs).

After sensors and front-facing cameras collect data, algorithms are used to segmen<sup>t</sup> and analyze the stimuli. This process includes shape, color and symbol detection as well as classification of signs based on the aforementioned characteristics [4]. A vast body of literature has analyzed the efficiency of different algorithms for segmentation and classification of signs [6–9]. Several review papers on this subject have also been published in the past few years [10–13]. Besides the overall review of the working procedures, studies identified main issues and challenges regarding the accuracy of TSRS. They are generally related to fading and blurring of traffic signs, visibility levels of signs in comparison to the environment, differences between existing traffic sign systems, multiple appearances of signs, damaged or partially obscured signs, correct location of signs, unavailability of public databases, electronic signs, etc. An on-road study conducted in Australia and New Zealand confirmed some of the aforementioned issues [4]. For the purpose of this study, authors used five cars with TSRS and drove a number of trials designed and conducted in order to identify key issues existing in the current TSRS and to investigate potential causes of the found issues. The study highlighted several applicable changes that could improve traffic sign readability, including electronic signs, installation and maintenance, sign positioning and location, sign face design, vehicle-mounted signs and other advisory and information signs.

From the literature review, one can conclude that general problems regarding TSRS are well known. However, different car manufacturers use different hardware and software solutions that may differ in traffic sign detection and readability accuracy. In addition, there are some differences between the signs being detected and presented to the driver. Namely, some cars can only read "speed limit" signs, while others in addition to "speed limit" signs can read other signs such as "prohibition of overtaking", "end of all restrictions", "start and end of highway", etc.

For this reason, the main aim of this study is to determine whether differences between detection and readability accuracy of market-ready TSRS exist, and to what extent. Moreover, since the problem of partially obscured signs was identified in literature as one of the main challenges of TSRS, the objective of the study was to test how simulated "graphical changes" affect their detection accuracy.

#### **2. Materials and Methods**

In this section, the research methodology is presented. The section consists of four subsections each describing a part of the methodology, from the testing track, vehicles, scenarios and test procedures to data analysis.

#### *2.1. Testing Track*

The experiment was conducted on a road inside the campus of the University of Zagreb. The road is a typical two-way road with almost no traffic since it is only used for connecting buildings on the campus. The total length of the road is 1.2 km and it consists of straight sections, four curves and five intersections with the right of way in the direction in which tests were conducted. Nine traffic signs were placed on straight sections of the testing track, as shown in Figure 1. Seven of them were used, i.e., their graphical image was changed according to scenario designs, while the remaining two (8 and 9) were placed for control purposes only and their "readings" were not recorded and analyzed. Only speed limit and prohibition of overtaking signs were used since the majority of current TSRS read only these signs. In total, four "speed limit" signs (50 km/h, 60 km/h, 90 km/h and 100 km/h) and three "prohibition of overtaking" signs were used. The speed limits (50 km/h, 60 km/h, 90 km/h) were chosen based on the fact that they are the most common and that their meaning could easily be altered. For example, with a very small modification, 60 km/h could easily be altered to 80 km/h. The 100 km/h speed limit was chosen since it contains three digits. All signs were newly made and fulfilled all technical properties (visibility, chromaticity, etc.) defined by the Croatian standard [14]. The design of the signs was also according to the Croatian standard, which is based on the Vienna convention. All of them are 60 cm in diameter and were made using class II sheeting with prismatic retroreflection. Furthermore, all signs were placed according to the Croatian standard [14], which implies a 30–75 cm distance from the edge of the road and a height between 1.2 to 1.5 m. The distance between each sign was set to a minimum of 100 m and the locations of the signs did not have any environmental "disturbances" which could affect the TSRS. Although vegetation next to the road is present, all signs were clearly visible, i.e., they were not covered with vegetation. The layout of the signs at each location was as follows: Location 1—50 km/h speed limit; Location 2—prohibition of overtaking; Location 3—60 km/h speed limit; Location 4—prohibition of overtaking; Location 5—90 km/h speed limit; Location 6—prohibition of overtaking; Location 7—100 km/h speed limit; Location 8—prohibition of overtaking (control sign); Location 9—60 km/h speed limit (control sign).

**Figure 1.** Layout of the test route and placement of the signs.
