Design and Implementation of a Car’s Black Box System Using a Raspberry Pi and a 4G Module

Abstract

The design and implementation of a car’s black box system using a Raspberry Pi microcomputer and an Internet of things module is presented in this research. This system was built using a Raspberry Pi microcomputer and different sensors, including a GPS, camera module, audio module, alcohol sensor module, and signals from the electronic control unit. The data were stored in both a secure digital card and in the cloud using the Waveshare SIM7600G-H 4G module. The results show that this embedded system can acquire and process video, audio, GPS data, alcohol concentration, speed, temperature, etc. Finally, a graphics user interface was developed to process the stored files. This system is similar to a black box in airplanes, which records all flight information into the black box using a specific algorithm. This makes the recorded flight data accessible to authorities when needed.

1. Introduction

In the last two decades, there have been increases in the population around the world, consequently generating an increase in vehicle users over the world [1]. This has resulted in an increase in the number of accidents on land roads [2]. It should be noted that sometimes it is difficult to retain the position of vehicles after an accident. This is problematic because some authorities and institutions need this information to make decisions. Also, if there is no information about the speed of the vehicle, videos, or GPS signals, it is difficult to reach the best decision for both drivers regarding the accident. A solution to determine the causes of an accident is to collect an accurate record of the actions carried out by the drivers, which are reflected by different parameters of the vehicle. Some parameters could be collected by means of a car’s black box system located in the unit [3,4]. In general, a car’s black box is different from an aircraft’s black box [5,6,7]. In a commercial aircraft, the black box is comprised of two separate devices—a flight data recorder (FDR) and a cockpit voice recorder (CVR)—which are often combined to form a single unit [8,9,10]. The FDR monitors a minimum of 88 parameters, including time, altitude, airspeed, heading, pilot input, pitch, roll and yaw angles, etc. The CVR is required to record four channels of cockpit audio data for a minimum duration of two hours. The aircraft’s black box does not perform any safety functions; it is simply there to record flight data and serve as a crash-survival memory unit. In the same way, a car′s black box or event data recorder (EDR) stores data to a flash memory, typically in the vehicle′s airbag control module (ACM) [11,12,13,14]. The primary function of the ACM is to sense a developing collision, determine if any safety devices need to be deployed, and deploy any necessary devices accordingly. Crash data recording is a secondary function of the ACM. The crash data recording of a car is similar to an aircraft’s FDR in that it records valuable information about the vehicle in the moments before a collision; however, the amount of data that are recorded is not as extensive [15]. Furthermore, a car′s black box does not record any audio as in the case of an aircraft′s CVR [5]. While the automotive black box does not record as much data as an aircraft’s black box, it is still capable of recording an abundance of data. In 2006, regulations were introduced in the Part 563 rule which specified 15 minimum parameters that must be recorded, should a manufacturer decide to install an EDR in a vehicle. These parameters include latitudinal and longitudinal delta-v (speed change), vehicle indicated speed, engine throttle percentage, brake status (on/off), ignition cycle–crash, ignition cycle–download, and safety belt status–driver [16]. In addition to the required data, the Part 563 rule also set standards for 30 other types of data if manufacturers voluntarily configured their EDRs to record them. For example, if a manufacturer configured their EDR to record steering input or anti-lock braking system (ABS) activity, it would have to record 5 s of data sampled twice per second. Other optional parameters include latitudinal and longitudinal acceleration, engine RPM, and vehicle roll angle, among others. The research on developing a car’s black box system continues, and some efforts have been carried out. For example, the authors of [4,11,17,18,19] proposed systems using GSM modules that focused on notifying the authorities about accidents. The authors of [13,14] proposed systems that collected data about the vehicle as well as the position of the brake, intensity of lights, and speed of the car. This information was stored in an EEPROM memory. A black box system was proposed in [20], which obtained visual information on the route, the vehicle speed, the location, the brake status, and the current time. This system used a processor that manipulated the information after it was acquired. The authors of [21] presented a black box system that was an advanced in-vehicle data recorder on a massively-sensorized vehicle for car driver behavior experimentation. In [17,22], an intelligent vehicle black box using the IoT was presented that received information from various sensors, such as a breath analyzer, as well as information about acceleration, the distance of surrounding vehicles, and push and panic buttons. However, these systems had no communication with the ECU.

In this research, the design and implementation of a car’s black box system using a Raspberry Pi microcomputer and a 4G module to connect the system to the Internet was carried out. This black box system was based on an embedded system and a graphical user interface (GUI) for the analysis of the acquired data. The system acquired video and audio signals using a camera and a microphone, respectively, connected to the Raspberry Pi microcomputer. This system also detected the alcohol concentrations inside the vehicle using an MQ3 module. The car’s black box uses a plug-and-play method, meaning that it can detect the addition of a new input or output device and automatically activate the appropriate control software.

2. Materials and Methods

The main function of a car’s black box system is to acquire, process, and store information during the vehicle’s journey. It is used frequently in airplanes; however, this system can also be useful in cars. Figure 1 shows the proposed embedded system with the input signals. The system utilized sensors placed in different parts of the vehicle to collect information such as audio, video, alcohol concentration, velocity, GPS, and all variables from the electronic central unit. The audio signals were collected with a voice recorder, which allowed the creation of an audio recording directly from a browser by using a microphone. For video, we connected the Raspberry Pi camera module to the microcomputer. For the alcohol concentration detection inside the vehicle, we used an MQ3 module. This module is a low-cost semiconductor sensor which can detect the presence of alcohol gases at concentrations from 0.05 mg/L to 10 mg/L. The material used for this sensor was SnO₂, which has a lower conductivity in clean air. Its conductivity increases as the concentration of alcohol gases increases. It has high sensitivity to alcohol and a good resistance to disturbances due to smoke, vapor, and gasoline. This alcohol sensor provides both digital and analog outputs and is suitable for detecting the alcohol concentration on your breath, similarly to a common breathalyzer. It has a high sensitivity and a fast response time. We also used a GPS for localization. Finally, this microcomputer was also connected with the central electronic unit to obtain other signals from the vehicle. Figure 2 shows a scheme of the proposed system, where it can be observed that a microcomputer is used as the main component to govern the system and interface with the other devices that comprise it through different communication protocols. The connection between the Raspberry Pi and the ECU can also be seen.

Figure 3 shows the processing and analyzing of the data from the stored files. The information can be viewed graphically and through video and audio files.

This system was designed in a modular way, meaning that different hardware elements were incorporated to reduce costs and obtain a high-performance system. Also, this architecture allows faults to be found more quickly, is efficient, and allows for the interchange of parts easily. This system continues its operation if any hardware element breaks down or is disconnected, since when such a situation occurs the system tries to reincorporate the device iteratively to resume the connection. If the connection is not established again definitively, an error code is stored in a log file that mentions which element failed and the time it lost the connection. With this, it is easier to take action and repair the car’s black box system.

The methodology used to carry out this system is shown in in Figure 4. In the first step, the initialization process is performed where the communication protocols corresponding to each hardware element are configured and thus make the links correctly. Then, certain initial parameters are provided to each device before the device starts its task. Subsequently, the connection with the car’s ECU is established through the CAN communication protocol. Once the above is ready, the system uses the “threading” method to carry out the data acquisition stage. This method allows the execution of different processes simultaneously, allowing all devices to work at the same time without the task of one element interfering with that of another. The data are stored continuously within a dynamic buffer. After 10 min, the data in the buffer are updated by eliminating the oldest information and adding the new data in a new buffer space. This algorithm is shown in Figure 5. Finally, the information acquisition and storage stages are carried out iteratively. At each stage, a review is made of the connection state of the microcomputer with the sensors in order to detect possible failures and errors in the system. In case of any problems, the system automatically generates a record that provides information about which element is failing.

The GUI was developed through the Python programming language and some packages that allowed the creation of interfaces, the handling of different file formats, and the building of an executable that can be used on different computers without the need to install external programs for its operation. This GUI allowed us to load the information and display it graphically and numerically in the case of text files. Audio and video files can be seen through players corresponding to each format, which are integrated into the GUI as shown in Figure 6. The purpose of the GUI is to facilitate the analysis of the collected information to deduce the causes of accidents and consequently to know if they happened due to human error or due to problems with the vehicle’s condition. In this research, the GUI was carried out in Spanish.

3. Results

The designed car’s black box system can be seen in Figure 7. It was tested in a Kia Soul 2018 vehicle, where the components were adapted and the connection with the ECU was made to extract information about the vehicle. The tests were carried out with the car’s black box consisting of the acquisition of data from the sensors and the vehicle’s ECU. The system recognized when one or more components were connected or disconnected and, at the same time, checked that the information was being stored in the corresponding formats. The tests carried out in the GUI relied on the correct reading of the different file formats, the visual display of information through graphs, the numerical values and video, and a reproduction of the audio.

Table 1. The number of files and data for each parameter measured over a 10 min journey.

Parameters	Files	Format	Data	Enabled
Video	60	.mp4	*	Yes
Audio	60	.wav	*	Yes
Alcohol	1	.txt	732	Yes
GPS	1	.txt	203	Yes
Accelerometer	1	.txt	259	Yes
Distance	1	.txt	259	Yes
Time	1	.txt	259	Yes
RPM	1	.txt	259	Yes
Speed	1	.txt	259	Yes
% Combustible	1	.txt	259	Yes

* No data.

shows the number of files and data for each parameter measured over a 10 min journey.

Table 2. Results of some of the tests carried out; E is enabled and D is disabled.

Parameters	Test 1	Test 2	Test 3	Test 4
Video	E	D	E	E
Audio	E	E	E	D
Alcohol	E	E	D	E
GPS	E	E	E	D
Accelerometer	E	E	E	E
Distance	E	E	E	E
Time	E	E	E	E
RPM	E	E	E	E
Speed	E	E	E	E
% Combustible	E	E	E	E

shows the results of some of the tests carried out for the connection and disconnection of the devices with the microcomputer. None of the processes were affected at all if any of the sensors did not work correctly. In addition, when the device was connected again, it was successfully reincorporated into its task without generating interruptions to the execution of the system.

Figure 8 shows some of the records made when a sensor was connected and disconnected. When the sensor was available, the storage of information was observed; otherwise, a warning message was saved. This makes it possible to collect accurate and reliable information in addition to knowing if there are faults in the system, so that corrective maintenance can be carried out to eliminate said faults.

In this research, we also developed a GUI as shown in Figure 6; however, these files can be read with any commercial software. Figure 9a shows the distance traveled by car. Figure 9b shows the RPM of the vehicle’s motor.

Figure 10a shows the vehicle’s velocity and Figure 10b shows a screenshot of the audio player. These results are very important because they give us the information from the last 10 min of the journey.

In this research, we can record the journey of the vehicle using the Raspberry Pi camera module. Figure 11 shows the video player, and you can see the video.

These results are very important because they allow us to review the last 10 min of the vehicle’s journey. If there is an accident, the authorities can review the data and files, and determine the causes.

4. Discussion

A crucial factor in traffic safety is driver behavior. Therefore, a better understanding of driver actions will help in determining the most common reasons for car accidents. In the last decade, there have been many efforts by groups over the world to report research in this field in order to help reduce accidents due to driver distraction. This paper presents a car’s black box system using a simple Raspberry Pi microcomputer, which is a powerful computer that can help researchers study car driver behavior. When comparing this embedded system with others, for example with the Argos system [21], our system is an improved in-vehicle data recorder that allows the recording of many kinds of alphanumerical data, such as the speed, the point of gaze, and the current distance to lateral road marks. Our system is simple and can interact with the ECU, and it also incorporates additional sensors such as an alcohol concentration sensor and an audio recorder. Also, our system is connected to the cloud using the ThingSpeak platform, where we can use MATLAB to read data from a particular time range.

5. Conclusions

The design and implementation of a car’s black box system using a Raspberry Pi microcomputer and an Internet of things module was presented in this research. This system was built using a microcomputer Raspberry Pi and different sensors, such as a GPS, camera module, audio module, alcohol sensor module, and signals from the electronic control unit. The data were stored in both a secure digital card, and in the cloud using the Waveshare SIM7600G-H 4G module and the ThingSpeak platform. The results show that this embedded system can acquire and process video, audio, GPS, alcohol concentration, speed, temperature, etc. This system can be useful when any type of accident occurs due to any reason, as the car’s black box would provide the necessary information and data to generate a report of the accident and its causes.