Latent Body-Pose guided DenseNet for Recognizing Driver’s Fine-grained Secondary Activities

Over the past two decades, there has been an increasing research in developing self-driving vehicles, with many industries pushing the bounds alongside academia. Automatic recognition of in-vehicle activities plays a key role in developing such vehicles. In this work, we propose a novel human-pose driven approach for video-based monitoring of driver’s state/activity and is inspired by the recent success of deep Convolutional Neural Network (CNN) in visual recognition tasks. The approach infers the driver’s state/activity from a single frame and thus, could operate in real-time. We also bring together ideas from recent works on human pose detection and transfer learning for visual recognition. The adapted DenseNet integrates these ideas under one framework, where one stream is focused on the latent body pose and the other stream is on appearance information. The proposed method is extensively evaluated on two challenging datasets consisting various secondary nondriving activities. Our experimental results demonstrate that the driver activity recognition performance improves significantly when the latent body-pose is integrated into the existing deep networks.