Tracking facial features to make driving safer and more comfortable

Today’s “intelligent” cars, equipped with multiple sensors and algorithms, can react to emergency situations, regulate speed, assist with parking and respond to voice commands. But they don’t know who’s at the wheel or how that driver is feeling. The face is a valuable source of this information, and a project initiated by EPFL’s Transportation Center involving EPFL’s Signal Processing 5 Laboratory (LTS5) and PSA Peugeot Citroën aims to mine it for use in the automobiles of the future.

Face-related technologies have progressed rapidly in recent years, and are used in many areas. Facial recognition is used in real time to relay the exact position of an individual, and eye-tracking devices indicate the direction in which he or she is looking. Analyzing the movements of the head and eyes can also indicate a state of fatigue or sleepiness. EPFL’s LTS5 and its start-up, nViso, are conducting research into capturing facial expressions and exploring the wide range of possible scientific, behavioral and commercial applications that the technology offers.

Technological challenges
The scientists had to tackle the challenge of adapting the technology for use in cars. “Our goal is to build the technological base to detect and situate a driver’s face at any moment in time,” explains LTS5 director Jean-Philippe Thiran. “Using this tool, it will then be possible to build and test various driver assistance applications such as eye tracking, fatigue detection, lip reading, and so on.”

“With this study we are trying to make the interface between the car and the driver more intuitive; reading intentions from facial features is a very natural interactive mode,” adds Olivier Pajot, PSA Peugeot Citroën’s representative on the EPFL campus.

The instrument, developed using a prototyping platform provided by the French car manufacturer, is currently operational, although the research is still far from finished. Vehicles impose a number of specific limitations, beginning with the question of where to place the camera. It cannot obstruct the driver’s field of vision. “One of the possible options is to place it behind the steering wheel, which would require a system that’s quick enough to recapture the face detection after the arms of the steering wheel interrupt it,” explains Thiran. Another issue is adapting to changing lighting conditions, in order to track the driver’s face as the car goes into a tunnel, for example, or when driving into a setting sun. And, finally, the tool must perform as well when the driver is facing it as when she turns her head to the side.

Improving performance
“We are currently working on improving the system’s performance, particularly by increasing the number of images processed,” continues Thiran. “The next step is to test it in realistic conditions. PSA Peugeot Citroën will install the platform on a car and our lab will test the system under various application scenarios.”

PSA Peugeot Citroën opened an office in EPFL’s Innovation Square in summer 2011, with the goal of collaborating on research and innovation projects. Spearheaded by EPFL’s Transportation Center, other projects with EPFL laboratories have been launched, including energy systems, driver assistance systems, human-machine interfaces and predictions for the structure of the automotive market.