HiFI: A Hierarchical Framework for Incremental Learning using Deep Feature Representation

The presented work focuses on automatic recognition of object classes while ensuring near real-time training required for recognizing a new object not seen previously. This is achieved by proposing a two-stage hierarchical deep learning framework which first learns object categories using a Nearest Class Mean (NCM) classifier applied directly to CNN features and then, uses a two-layer artificial neural network to learn the object labels within each category. In order to recognize a new object not seen earlier, the category is identified first and then the second stage neural network is incrementally trained with the features of the new object without forgetting previously learnt labels. The proposed hierarchical framework is shown to provide comparable recognition accuracy with significant reduction in overall computational time in recognizing new objects compared to methods that use end-to-end re-training. The efficacy of the approach is demonstrated through comparison with existing state-of-the-art methods on the publicly available CORe50 dataset.