A multimodal drowsiness dataset using video, biometric, and behavioral data

Why Staying Awake at the Wheel Matters

Drowsy driving is a hidden threat on the road, contributing to thousands of crashes every year. We often imagine drifting off suddenly, but in reality, sleepiness creeps in gradually: our heartbeat slows, our grip loosens, our posture sags, and our eyes grow heavy long before we actually nod off. This article describes a new public dataset designed to capture those subtle, early warning signs in great detail, giving researchers the raw material they need to build smarter systems that could one day warn drivers before a disaster occurs.

A New Look at Sleepy Drivers

Most existing collections of driver drowsiness data focus on just one type of signal, such as video of the face or recordings of brain waves. The new University of Louisiana Drowsiness Dataset (UL-DD) takes a broader view. Nineteen volunteers drove a truck simulator in a lab while a dense web of cameras and wearable sensors recorded their behavior and body signals. Each person completed one session while fully alert and another when they already felt sleepy, producing continuous recordings lasting about 40 minutes per session and totaling roughly 1,400 minutes of driving data. Instead of tagging drivers as simply "alert" or "drowsy," the study asked them every four minutes to rate their sleepiness on a nine-step scale, capturing the slow slide from wide awake to fighting sleep.

What the Sensors Saw and Felt

The setup watched drivers from multiple angles while also “listening” to their bodies. One infrared camera tracked facial details, performing well even in dim light; a 3D depth camera captured the shape and distance of the head and upper body; and a color camera focused on posture. From these videos, the team extracted facial landmarks—precise points around the eyes, mouth, and brows—and facial “action units” that describe expressions such as eye closing, brow movement, and yawning. Another algorithm traced body pose landmarks to reveal slouching, head tilt, and other posture changes linked to fatigue.

Reading the Body’s Quiet Signals

At the same time, wrist-worn devices and a pulse oximeter measured the inner signs of tiredness. Heart activity and pulse rate, blood oxygen levels, skin temperature, sweat-based electrical activity, motion, and wrist acceleration were all recorded at their own sampling speeds. Pressure sensors on both sides of the steering wheel tracked how firmly each hand gripped, while software tapped into the simulator to log speed, steering behavior, and gear changes dozens of times per second. All these streams were carefully synchronized so that, at any instant in the drive, researchers can see the exact combination of facial expression, posture, heart rhythm, breathing hints, hand force, and driving style.

Checking the Quality of the Signals

Collecting data is only useful if it is reliable, so the authors spent considerable effort validating the recordings. They examined how the body signals changed with low, medium, and high sleepiness, using statistical tests suited to data that differ from person to person. Measures such as skin temperature, sweat activity, blood oxygen saturation, and pulse rate all shifted in meaningful ways as drivers grew more tired, while some others, like one blood-flow measure, proved less informative. The team also checked that the self-reported sleepiness scores were consistent by comparing them with an expert’s ratings of a subset of the videos, finding strong agreement. Signal-to-noise analysis showed that most sensors, especially heart-related ones, produced clean data, and video checks confirmed that frames were not being dropped.

How Well Can Combined Signals Spot Sleepiness?

To see whether this rich mix of information could actually help detect drowsiness, the authors trained several machine-learning models on different combinations of signals. When they used only one type of data—say, biometrics, grip pressure, or facial features—the results were modest. But when they combined biometric, behavioral, and facial information, performance rose sharply. A modern fusion approach that learns how different data types interact reached an accuracy of about 88 percent in classifying drivers as alert, moderately sleepy, or highly sleepy. This suggests that no single cue is enough; instead, fatigue reveals itself most clearly when many small hints are woven together.

What This Means for Safer Roads

The UL-DD dataset does not deliver an off-the-shelf drowsiness detector, and it does have limits: the volunteers were relatively few, mostly male, and drove in a simulator rather than on real highways, and the study did not include direct brain recordings. Still, by making a detailed, well-labeled, and fully synchronized collection of facial, bodily, and driving signals freely available, the authors give scientists and engineers a powerful test bed. With it, they can explore how early our bodies begin to signal trouble, refine algorithms that combine multiple clues, and move closer to in-car systems that gently intervene before a sleepy moment turns into a tragedy.

Citation: Bodaghi, M., Hosseini, M., Gottumukkala, R. et al. A multimodal drowsiness dataset using video, biometric, and behavioral data. Sci Data 13, 506 (2026). https://doi.org/10.1038/s41597-025-06540-1

Keywords: driver drowsiness, multimodal sensing, wearable biometrics, driving simulator, road safety