ABSTRACT

Driving a vehicle is one of the most common daily yet hazardous tasks. One of the great interests in recent research is to characterize a driver’s behaviors through the use of a driving simulation. Virtual reality technology is now a promising alternative to the conventional driving simulations since it provides a more simple, secure and user-friendly environment for data collection. The driving simulator was used to assist novice drivers in learning how to drive in a very calm environment since the driving is not taking place on an actual road. This paper provides new insights regarding a driver’s behavior, techniques and adaptability within a driving simulation using virtual reality technology. The theoretical framework of this driving simulation has been designed using the Unity3D game engine (5.4.0f3 version) and programmed by the C# programming language. To make the driving simulation environment more realistic, the HTC Vive Virtual reality headset, powered by Steamvr, was used. 10 volunteers ranging from ages 19 - 37 participated in the virtual reality driving experiment. Matlab R2016b was used to analyze the data obtained from experiment. This research results are crucial for training drivers and obtaining insight on a driver’s behavior and characteristics. We have gathered diverse results for 10 drivers with different characteristics to be discussed in this study. Driving simulations are not easy to use for some users due to motion sickness, difficulties in adopting to a virtual environment. Furthermore, results of this study clearly show the performance of drivers is closely associated with individual’s behavior and adaptability to the driving simulator. Based on our findings, it can be said that with a VR-HMD (Virtual Reality-Head Mounted Display) Driving Simulator enables us to evaluate a driver’s “performance error”, “recognition errors” and “decision error”. All of which will allow researchers and further studies to potentially establish a method to increase driver safety or alleviate “driving errors”.

Keywords:

Driving Simulator, Virtual Reality, Head Mounted Display, Driver Behavior, Safety, Driving Errors, Unity3D

1. Introduction

Driving simulation has become an important tool in the field of car industry. Researchers are providing an environment which is safer for drivers because there are no physical obstacles or potential for harm in the form of driving simulation using VR [1] . The way that people look around while they’re driving, and control of the vehicle (speed control, braking and lane changing) can manifest as driver behavior errors [2] . Recognition errors, decision making errors and performance errors, in particular, are 3 of the most common errors. Driver error has been identified as an important factor in road traffic accidents. For instance, if a driver accelerates instead of braking, it is considered an error. Also, it is considered an error if a driver performs an incorrect action intentionally or unintentionally such as passing a yellow light when they can stop, or braking hard on an icy or slippery road. Most errors occur when someone drives at a high speed in a situation that requires quick decision making within a small window of time. According to McLean, there are four types of collisions occurring at intersections: Right-Angel, Right-turn-Against, Rear-End and crashes involving pedestrians [3] .

Driving a vehicle is one of the most common daily and hazardous tasks. Studies have also shown the accident rate among senior drivers in Japan has been increasing. Because Japan is a rapidly aging society [4] . There are many factors that cause driver errors, such as weak eyesight, visual observation problem, physical conditions and fatigue, etc. Each driver, on the road has different characteristics and behaviors; the advancement of analyzing these factors by use of driving simulation has become a great interest in recent research. Novel virtual reality technology is now capable of helping researchers gather data more easily and precisely. This technology provides an easy, secure and user-friendly environment for data collection for researchers [1] . Driving simulators are meant to be used to assist novice drivers to learn how to drive in a very safe environment. In our study, we provide new insights regarding driver behavior, techniques and adaptability in driving simulation using virtual reality technology. We also measured driver tendencies (head movements), speed control (accelerating and braking), position of the vehicle, control of the vehicle (steering wheel rotation), Gaze information (The eye movement of user is tracked and calculated to allocate where the user is looking at) and so on. The specific tasks examined, during the simulation included: adjusting speed when approaching the bridge, turning and passing intersections and keeping left and driving between right lanes, controlling the car while accident occurs and when other cars on the road are suddenly stopping.

In summary, the theoretical framework of this driving simulation has been designed using the Unity3D game engine and programmed by C# programming language. Previously, research has been done with the use of traditional monitors and projectors. In past research, 3 monitors were used to simulate the inside of a frontal view within the inside of a car. One, main screen, in the middle represents the view of out of the windshield while 2 other monitors to the left and right of the center monitor represent the left and right window views. This method has been used for a decade. For the sake of creating a more realistic and immersive environment during a simulation, researchers have tried to provide users with a 3D environment which is technically still in 2D. There are difficulties in adopting this past system due to the system being unable to completely mimic a true 3D environment, which makes data gathering often imprecise. Conventional research usually concentrates on analyzing driver behavior based on acceleration, breaking and steering wheel data, and visual data. But, through the traditional method of using flat displays which gives researchers various restrictions which prevents them gathering precise data.

In our study, to make a driving environment more realistic, the HTC Vive headset was used, which was given to our volunteer drivers providing them with a wide 3D view of the virtual world around him/her. By using HTC (Head Mount Display) HMD we could gather and analyze behavior data for each driver. The Fove HMD was used to gather behavior and gaze data for each driver while they were driving in driving simulation. Not to mention, the Unity engine gives us the freedom to design new scenarios, we can modify and develop codes anytime during the experiment and create. Unity can also update itself through the support of its creators to be compatible with new hardware such as HMDs and so on.

Volunteers ranging from the ages of 19 - 37 participated in this virtual reality driving experiment. 3 different driving courses (routes) on a single city map were assigned to each driver. The routes were determined by us, which included both a start line and a finish line which were highlighted and explained to each volunteer. In other words, despite the map being open world, volunteers were aware of exactly of what the boundaries were during each of the courses within this large driving space. For this experiment, we used only 1 of the 3 courses to obtain our data. In course 1 (refer to Figure 3) driver fatigue, patience and adaptively to the system was examined. The results show, some of the drivers grew tired of driving with a static speed on the same course. So, sometimes volunteers accelerated more to gain speed, moved sporadically around the course and stopped paying attention to what they were doing. Some drivers maintained their speed, which did not require them to brake so much. Some drivers had a rough start because they had never tried VR in the past, but as time passed they improved their techniques and got used to the driving simulator and the course, causing them to gradually improve.

The results of this study can suggest that the performance of the drivers is closely associated with the individual’s behavior and adaptability to the VR system. In addition, the analysis results show that an individual’s behavior and characteristics such as maintaining and controlling speed, braking when needed, remaining calm, paying attention to the environment changes around them, while driving, can affect their performance in a simulated environment.

2. Advantages and Disadvantages of Driving Simulations

Driving simulators have been utilized to help novice drivers, for racing training, road safety scientific research, and as entertainment at home. Driving simulators offer various advantages compared to real vehicles, including [5] .

2.1. Advantages of Driving Simulators

1) They are easy to control and easy to make, reproduce and normalization. Changing the traffic mode, weather conditions, and the road design can be easily adjusted based on the needs of the researcher or designer. It will also give the developer more room to design different scenarios and enables the user to try and practice freely for any amount for time. By using simulators, drivers can experience driving under the exact same conditions and the same scenarios decided by researchers. This is an important factor for producing research results [5] .

2) Gathering data will be easier and more precise. With a driving simulator, measuring performance will be more accurate and efficient. On the other hand, gathering data with a real vehicle is problematic when collecting corresponding, accurate data. It is difficult to determine the distance between two vehicles in real time because the vehicles can’t easily be manipulated by researchers. While in the simulation, data can be recorded more easily and precisely because everything about the simulation can be manipulated at any time with coding from a VR program [6] . It helps a researcher evaluate object detection and hazard awareness by using eye-tracking [7] .

3) Experiencing and encountering accidents without being physically harmed. It is possible to study dangerous driving situations by putting the driver in dangerous driving situations, which is an ethically challenging when using a real vehicle [7] .

4) Ease in feedback and instruction. It is possible to pause, reset, or replay the same, established scenario many times. Feedback and instruction can be delivered in different ways such as speech and visual overlays used to highlight dangerous features in the environment [5] .

2.2. Disadvantages of Driving Simulators

Simulators have some disadvantages and challenges which are:

1) Limited physical, perceptual, and behavioral fidelity. Low-fidelity simulators may evoke unrealistic driving behavior and therefore produce invalid research outcomes. Simulator fidelity is known to affect user opinion. Participants may become demotivated by a limited-fidelity simulator and prefer a real vehicle instead (or a costlier high-fidelity simulator for that matter) [5] .

2) Shortage of research demonstrating validity of simulation. A growing body of evidence indicates that driving-simulator measures are predictive for on-the-road driving performance [2] [8] . However, only a few studies have investigated whether skills learned in a driving simulator transfer to the road (see [9] [10] for many such studies).

3) Simulator sickness, especially in older people or under demanding driving conditions. Simulator sickness symptoms may undermine training effectiveness and negatively affect the usability of simulators. This is a serious concern, but fortunately, useful technological and procedural guidelines are available to alleviate it [11] . Research shows that simulator sickness is less of a problem for young drivers [12] . Experience shows that limiting the horizontal field of view, avoiding sharp curves or stops during driving, and using short sessions (≤10 min) with sufficient rest breaks significantly reduces simulator sickness.

3. Research Hypothesis

The first research hypothesis is as follows:

The first question is that in this research what would happen if the users could adapt to the new VR technology and feel comfortable driving in a completely virtualized 3D environment? As we mentioned earlier, there are pros and cons of using this kind of environment, dependent on people and their human condition such as age, wellness, etc. The second reason we are conducting these experiments is to see whether or not it’s beneficial to use a virtual reality system to observe driver behavior. We are trying to develop a new way to gather data base of driver behavior so that other researchers can apply our new method to their own research. This new VR technology is low cost and can use multi-sensors located on a VR headset and is portable. It can be set up in a working environment such as an office or even at home. We hope that other researchers will be able to adapt this technology to their own research and continue to expand the potential uses for this VR system.

4. Experimental Setup

There are limitations regarding external validity in this study, however, a driving simulator study includes some advantages compared to on-road studies: Different flexible driving scenarios can be designed and can be changed anytime based on needs of the researchers and participant status. The experiment was carried out in Sasaki & Matsushita Laboratory based on a game engine and virtual reality environment. The simulator displays the driving scene on a monitor in the form of an open city with real world traffic, AI and realistic road elements. Because of the monitor display, instructors can also view what’s happening within the simulation. Participants can of course view everything through the VR headset. The simulator recorded the data at about 30 frames per second on a CSV file. Furthermore, everything from start to finish can be recorded via a video capturing device. Two external speakers are positioned in front of the driver. Participants controlled the vehicle within the game using a Thrust master T150 steering wheel controller supplied with gas and brake pedals and gear shifting was set to automatic. The steering wheel has built in sensors that allow Unity to translate steering wheel rotation by a driver into raw data.

4.1. Experimental Design

Figure 1 graphs the sensors, the hardware used in the experiment (displays, PC) and the game engine/programming language.

Figure 2 represents a difference between previous experiment setups and the setup we used in our experiment.

4.2. Equipment

Conventional research almost always relied on having to design a Virtual reality program by using flat displays or projecting the visual images on monitors to project the view for the user. In this study, we have the advantage of using recent HMDs technologies to design and develop an immersive environment for users.

This experiment is driving around a circular course 10 times with all the sensors attached to our system allowed us to evaluate the behavior and characteristics of a driver in a long and repeated course. There are some factors such as performance and learning capabilities, concentration, fatigue which effect driver behavior. These results have been analyzed and studied in this research.

4.3. Scenario and Roadway Design

As Figure 3 describes, the driver starts the course from the parking lot represented by the purple bar. The blue line illustrates the driving path for the course which drivers need to navigate fully 10 times. The bridge acts as an obstacle because it forces drivers to be more considerate of their actions while driving. Drivers are not allowed to leave the designated course boundaries marked by the blue line. This means that they cannot go off road. Also, they are not allowed to make a full stop. If the driver exhibits any of these actions during any of the 10 laps during the simulation, the entire experiment must be restarted from lap 1.

4.4. Participants

A total of 10 participants (9 males, 1 female) with an average age of 28 participated in the experiment.

5. Experimental Results and Discussion

By implementing an integrated driving simulator with VR-HMD with a multi-modal sensor attachment to it we were able to gather data for:

Multi Sensing:

i) Steering wheel and handling;

ii) Accelerator and Brake;

iii) Vision Image;

iv) Head Movement;

v) Eye Movement (This data has been captured and recorded and is to be analyzed for the future work).

5.1. Time-Distance Comparison

Comparison involving the top 5 drivers who showed the most signs of improvement and adjustment to the simulation. Figure 4 illustrates Time-Distance comparison of individual performance based on group data. In this graph the distance and time of 5 drivers presented, which notably shows the 5^th driver spent more time and was driving slower than other drivers, and 1^st driver has a faster performance and has different data in each laps.

5.2. Accelerator & Brake

As Figures 5(a)-(c) show, drivers slowly and smoothly increased their speed and applied pressure on the acceleration pedal. The 1^st and 3^rd drivers have braking data in their results which represents how they controlled their speed at curves and other situations which required them to slow down. The 2^nd driver however shows a different result in his performance as he kept his speed controlled without braking. The 2^nd driver smoothly used only the accelerator to maintain his speed. The 1^st and 3^rd drivers controlled the vehicle and speed by pushing the brake pedal smoothly as Figure 5(a) and Figure 5(c) illustrate. Figure 5(e) presents the 5^th driver’s performance. This individual took a long time to finish the 10 laps around the course, which mean his speed was overall slower than other drivers through the whole simulation experiment. The 5^th driver continued to control his speed by alternating between the accelerator and the break pedal at a steady, slower rate when compared to the other drivers. Be-

cause he was driving more slowly he did not need to brake as hard as other drivers. As a result of that he has fewer brake pressure data in his results. Figure 5(d) shows the 4^th driver’s performance results. This driver suddenly started increasing their speed as well as suddenly braking hard. This result shows that participants who drive at lower speed have an easier time maintaining a more consistent relationship between speed and braking pressure. In this case, consistent can mean staying near the 0.5 on any of the graphs.

5.3. Time Lap

Figures 6(a)-(e) are representing the time which each driver spent to finish each lap, the laps are selected from 2^nd to 9^th to be more precise in the results and avoid any unnecessary braking and accelerating data in the first and the last lap. As Figures 6(a)-(c) show, drivers improved their skills and they adopted to the system and they got better in driving. The 5^th driver drove constantly slow and Figure 6(e) presents that the driver was constantly driving slow and did not improve his driving style and skill. In the other hand, Figure 6(d) shows that this driver increased and decreased his speed sometimes, few laps slow and again faster in other laps. This result can show that this driver was bored of driving and did not improve his performance, but it does not necessarily means this dirver is not a good driver according to other data from other figures. The result of these graphs and the time record indicates drivers’ proficiency and motivation in driving.

5.4. Steering Angle/Car Angle (Yaw)

Following graphs are representing the steering wheel angle and car angle (Yaw), the results of these graphs will indicate that adaptively to the course, individual recognition ability and it also shows how stable the drivers were driving in the driving course.

Figures 7(a)-(e) show that drivers have a Stable driving performance and show that driver are memorized the course and getting better each lap in recognizing the map. Figure 7(d) unlike other figures shows that the driver has an unstable driving performance and it is not an Intuitive driving performance. As

we discussed about the 5^th driver in other sections and figures that he drove slowly and had a poor performance, but in terms of controlling the car he had shown a good result as he was able to control the vehicle easily due to his low speed. This figures and overlapping lines indicate how well drivers drove the same way in each lap or it shows the correlation between speed and controlling a vehicle.

5.5. Driver’s Head Position (Yaw)

The following graphs shows that the faster a person drives, the wider the variation of a driver’s head movements become. Figure 8(a) illustrates the head

movements of the first diver, which is Approximately ± 30 degree the highest degree. Both 4^th and 5^th drivers have a low rotation rate in their performance approximately +15 deg./−5 deg. and approximately ± 15 deg. respectively. As other figures presented the performance of these two drivers, speed, controlling the vehicle and head movements data are somehow connected. Figure 8(e) and Figure 8(d) show the lower head movements data, which indicates that head movements data in driver helps the individual recognition ability.

The result of this head movements data represents the individual recognition ability. It shows how relative the speed and head movements are in terms of controlling the car at corners and curves. Looking back at the data, it is clear that a driver who has better performance has completely different data based on his/her head-movements. As [13] represents as well, the elder drivers have less tendency to look around while they are overtaking other cars on the road. This shows how visual abilities in making a quick decision and head movements are closely related.

5.6. Discussion

The results of experiment show the relationship between speed, individual driving ability, performance and individual recognition. The results indicate that learning the course and controlling the speed will help a driver control the vehicle easier. This study shows that head movements and decision making are different in different individuals, and it shows how that effects driving performance. In this study, we used a modal multi sensor head-mounted display and used it in an immersive environment for the driver. Except for the head- mounted display there were not any other sensors attached to the driver.

Table 1 shows the summary of the results of the experiment which divided into two parts, analysis of the data and the results of the analysis. Table 1 listed the relationship between the analyses and a driver’s characteristics: (1) the driver’s eye and head movement information can be extracted with the VR-HMD vision image; (2) individual performance on driving adaptability can be evaluated through acceleration and braking; (3) the graph on the time/car distance traveled (Figure 4) demonstrates individual performance between all the drivers; (4) the time record transition indicates individual performance on driving adaptability; (5) the individual performance on turning can be evaluated with the combination of steering, car speed and driver’s horizontal head angle; (6) the individual performance of each driver can be evaluated with a driver’s vertical head angle. Overall, the sensors in the multi-model system contribute to the evaluation of each individual driver’s performance and recognition.

As for the gaze analysis, different driving scenarios were designed: 2. The driver must tailgate a car which unexpectedly stops, challenging drivers to adjust their speed and keep their distance while controlling the brake and accelerator. In addition, drivers needed to be careful by continuing to monitor the car they were tailgating as well as the surrounding area. 3. The driver had to follow a car and that car would crash with another car, so the driver behavior, reaction and

gaze data would help us to analysis the behavior of each driver when a sudden event would occur.

We are going to analyze and study driver behavior and errors. Specifically, we are going to study the eyes movements, gaze and glances of the driver. In order to do that, we are going to use two different eye tracking technology Fove head- mounted display. We will gather the data of the driver and analyze and compare two different eye tracking technologies results as well as compare and analyze the driver behavior. There are some signs of dizziness and nausea in female drivers who could not complete their tasks. By gathering electroencephalogram (EEG) signals we can measure muscular and nerve responses for each participant.

6. Conclusions

Driving a vehicle is one of the most common daily activities, and requires stable driving, concentration and recognition of one’s surroundings. As Japan is an aging country, recently driving violations caused by elderly drivers have been increasing [14] . It has become an important social issue to evaluate their ability, recognize and evaluate their environmental awareness while driving. The major, general evaluation method has been based on the driving simulator with 3 monitor displays (i.e., frontal view, left side window view with side view mirror included and right side window view the right-side view mirror included). However, this is not a true 360-degree view which can be provided with a VR headset. Driving simulators provide various driving scenes in a safe, stationary environment such as an office room. They enable researchers to measure and analyze how a driver uses the accelerator, the brake, and the steering wheel.

Therefore, the research goal of this paper is to develop a driving simulator based on a virtual reality (VR) head-mount display (HMD), and to demonstrate the driver’s performance and behavior analysis. The results of this study show a driver’s behavior and its relationship between human factors (recognition, visual inspections, decision making, and driving ability) and each individual performance (No two performances are the same). The results also indicate that drivers with better performance tend to look around more and be more attentive, and control their speed constantly. On the other hand, based on the relationship between human factors which mentioned in this study, drivers who were inattentive while driving, show less attention to head movements and maintaining their speed.

Based on our findings, it can be said that with a VR-HMD Driving Simulator, which is inexpensive and highly portable can provide us with 3 scenarios that analyze a driver’s characteristics: the 10-lap course enables us to evaluate a driver’s “performance error”. The head and eye movements captured by the VR headset enable us to evaluate “recognition errors”. And finally, the multi-model sensing device enables us to evaluate “decision error”. All of which will allow researchers and further studies to potentially establish a method to increase driver safety or alleviate “driving errors”.

Cite this paper

Taheri, S.M., Matsushita, K. and Sasaki, M. (2017) Development of a Driving Simulator with Analyzing Driver’s Characteristics Based on a Virtual Reality Head Mounted Display. Journal of Transportation Technologies, 7, 351- 366. https://doi.org/10.4236/jtts.2017.73023

References