CHARM: Context-aware Human Activity Recognition and Monitoring for intelligent vehicles

Smart and connected Autonomous Vehicle relies on cyber-physical infrastructure. This will play a significant role in intelligent mobility for future societies. It offers our desire for a better world in which accident injuries and fatalities are rare, congestion is lesser, and many societal and environmental benefits are far greater. This desire is unlikely to become a reality unless mindful attention is paid to human behaviour since human error is overwhelmingly to blame for as many as 94% automobile accidents.

CHARM aims to address this by developing novel algorithms for context-aware real-time monitoring of human activities. This is a challenging problem since human activity is complex and highly diverse, with applications to many real-life cyber-physical systems in which the system’s perception of contexts (e.g. dynamic environment) and human actions/activities/behaviour, are required for appropriate support and interventions.

CHARM aims to recognise in-vehicle activity (attentiveness, mobile phones, drinking, etc.), possible next activity and be able to anticipate the context (congestion, cyclist, roadworks, etc.) to enable the vehicle to be context-aware. CHARM’s output will be used by the ADAS for developing strategies for automatic steering and braking, collision avoidance plans, etc. This output can also be used to warn drivers using audiovisual signals through dashboard display and speaker for their attention.