A Novel Vision-Based Tracking Algorithm for a Human-Following Mobile Robot

Meenakshi Gupta, Swagat Kumar, Laxmidhar Behera, Venkatesh K. Subramanian

Research output: Contribution to journalArticle (journal)peer-review

119 Citations (Scopus)


The ability to follow a human is an important requirement for a service robot designed to work along side humans in homes or in work places. This paper describes the development and implementation of a novel robust visual controller for the human-following robot. This visual controller consists of two parts: 1) a robust algorithm that tracks a human visible in its camera view and 2) a servo controller that generates necessary motion commands so that the robot can follow the target human. The tracking algorithm uses point-based features, like speeded up robust feature, to detect human under challenging conditions, such as, variation in illumination, pose change, full or partial occlusion, and abrupt camera motion. The novel contributions in the tracking algorithm include the following: 1) a dynamic object model that evolves over time to deal with short-term changes, while maintaining stability over long run; 2) an online K-D tree-based classifier along with a Kalman filter is used to differentiate a case of pose change from a case of partial or full occlusion; and 3) a method is proposed to detect pose change due to out-of-plane rotations, which is a difficult problem that leads to frequent tracking failures in a human following robot. An improved version of a visual servo controller is proposed that uses feedback linearization to overcome the chattering phenomenon present in sliding mode-based controllers used previously. The efficacy of the proposed approach is demonstrated through various simulations and real-life experiments with an actual mobile robot platform.

Original languageEnglish
Article number7637024
Pages (from-to)1415-1427
Number of pages13
JournalIEEE Transactions on Systems, Man, and Cybernetics: Systems
Issue number7
Early online date24 Oct 2016
Publication statusPublished - 1 Jul 2017


  • Feedback linearization
  • K-D tree
  • Kalman filter
  • out-of-plane rotations
  • speeded up robust feature (SURF)-based human tracking
  • visual servo controller


Dive into the research topics of 'A Novel Vision-Based Tracking Algorithm for a Human-Following Mobile Robot'. Together they form a unique fingerprint.

Cite this