Let your eyes speak to autonomous vehicles

In everyday actions, we often already – literally – look ahead to objects that are needed for the next step. These anticipatory eye movements are currently the focus of a research project carried out by junior professor Dr. Christina Pfeuffer, who is investigating the reaction of passengers to the driving style of autonomous vehicles. The psychologist is junior professor of Human-Technology Interaction at the Catholic University of Eichstätt-Ingolstadt.

In Pfeuffer’s project, study participants complete several trips in a simulated driving situation. On a screen, they look out on the road through the windshield. The system plays alternating phases with active and passive tours: At one point, the study participants have to steer into a bend themselves using a steering wheel; in other simulations, the steering wheel moves as if by magic and independently corrects the direction of travel, just as an autonomous vehicle would. A special eye-tracking system records participants’ eye movements, and they also have to assess how safe they felt in each driving situation. Junior professor Pfeuffer explains the background of the study: "When we approach a bend with the car, we can observe a typical eye movement pattern. We look at an area around the tangent point of the curve already about one to two seconds before even entering the bend. It is assumed that such anticipatory eye movements serve to prepare and execute appropriate steering movements." In the current study, Pfeuffer is now interested in the extent to which such eye movements in passengers who are drivers themselves or passengers in an autonomous vehicle reflect their current perception of safety.

Although there have already been initial studies comparing eye movements during active and autonomous driving, they are not yet widespread. “Furthermore, the question of how eye movements can be used to provide a feedback to the system and adjust the driving style to the passenger’s current perception of safety has to date not been at the center of attention in research.”, says Pfeuffer. Especially during the transitional phase leading towards autonomous mobility, she continues to say, it was of great importance to ensure that persons who usually drive cars themselves still felt safe.

The experiments alternate between phases with active and passive tours: At one point, the study participants have to steer into a bend themselves using a steering wheel; in other simulations, the steering wheel moves as if by magic and independently corrects the direction of travel, just as an autonomous vehicle would.
The experiments alternate between phases with active and passive tours: At one point, the study participants have to steer into a bend themselves using a steering wheel; in other simulations, the steering wheel moves as if by magic and independently corrects the direction of travel, just as an autonomous vehicle would.

In this study, Pfeuffer therefore investigates how early or late and for how long drivers direct their gaze to the area around the tangent point of a bend during active or autonomous driving before actually entering the bend and whether these eye movements reflect the study participant’s feeling of safety. “In the experiments, we manipulate the speed in which the bend is approached, for example. We expect that the timing and duration of eye movements not only during active but also during passive driving will mirror the participants' perception of safety." If this is confirmed, future autonomous vehicles could use eye movements in appropriate driving situations to assess the safety experience of the passengers and adjust their driving style accordingly so that passengers feel as comfortable as possible during autonomous driving.

While the planned number of participants in the ongoing study is not yet suitable for drawing general conclusions, it will provide starting points for further research projects in the sense of basic research. The psychologist says: “The basic challenge for research and implementation in practice is that eye movement patterns of humans can be very similar in specific situations but are still highly individual.”

A special eye-tracking system records participants’ eye movements, and they also have to assess how safe they felt in each driving situation.
A special eye-tracking system records participants’ eye movements, and they also have to assess how safe they felt in each driving situation.

Eye tracking is generally regarded as a relatively direct measure of attention or shifts in attention. In this context, it is important to clarify exactly which processing steps take place in each case. "The mere fact that a person is looking at a certain point does not allow very clear conclusions to be drawn about the processing that takes place in the brain. To infer processing steps, we would have to design experiments in an appropriately clever way so that they allow different possible processes to be distinguished from one another." For universal statements, it is also always essential that a correspondingly large minimum number of people participate in the study and that samples are as diverse as possible and representative of the general population. Often, this cannot be achieved in single experiments, which is why a series of experiments is necessary to gain a comprehensive picture.

In her previous research, junior professor Pfeuffer has generally dealt with anticipatory eye movements with regard to the anticipation of consequences of one's own actions. For example, when test subjects generate a light signal by pressing a button, she investigated whether the subjects look at the expected location of the signal in advance. With the action of driving a car, such couplings of eye movements and action control are particularly pronounced, since steering actions and the current position of the vehicle, which we detect through our eye movements, must be continuously coordinated with each other. Here, too, expectations about how the vehicle will behave according to steering actions play a central role.

However, Ms. Pfeuffer also sees possible fields of application for transferring findings from her basic research into practice in the field of education – for example, with regard to determining and providing feedback on implicit assessments of one's own performance to support learning success. In addition, there could be further perspectives in the future, for example, to be able to offer people with motor impairments support in everyday life by having corresponding technical systems respond to their eye movements and derive their intentions from them.