Attend and Guide (AG-Net): A Keypoints-driven Attention-based Deep Network for Image Recognition

ASISH BERA, Zachary Wharton, YONGHUAI LIU, NIKOLAOS BESSIS, ARDHENDU BEHERA

Research output: Contribution to journalArticle (journal)peer-review

36 Citations (Scopus)
582 Downloads (Pure)

Abstract

This paper presents a novel keypoints-based attention mechanism for visual recognition in still images. Deep Convolutional Neural Networks (CNNs) for recognizing images with distinctive classes have shown great success, but their performance in discriminating fine-grained changes is not at the same level. We address this by proposing an end-to-end CNN model, which learns meaningful features linking fine-grained changes using our novel attention mechanism. It captures the spatial structures in images by identifying semantic regions (SRs) and their spatial distributions, and is proved to be the key to modelling subtle changes in images. We automatically identify these SRs by grouping the detected keypoints in a given image. The ``usefulness'' of these SRs for image recognition is measured using our innovative attentional mechanism focusing on parts of the image that are most relevant to a given task. This framework applies to traditional and fine-grained image recognition tasks and does not require manually annotated regions (e.g. bounding-box of body parts, objects, etc.) for learning and prediction. Moreover, the proposed keypoints-driven attention mechanism can be easily integrated into the existing CNN models. The framework is evaluated on six diverse benchmark datasets. The model outperforms the state-of-the-art approaches by a considerable margin using Distracted Driver V1 (Acc: 3.39%), Distracted Driver V2 (Acc: 6.58%), Stanford-40 Actions (mAP: 2.15%), People Playing Musical Instruments (mAP: 16.05%), Food-101 (Acc: 6.30%) and Caltech-256 (Acc: 2.59%) datasets.
Original languageEnglish
Pages (from-to)3691-3704
Number of pages14
JournalIEEE Transactions on Image Processing
Volume30
Early online date11 Mar 2021
DOIs
Publication statusPublished - 11 Mar 2021

Keywords

  • Computer Vision
  • Deep Learning
  • Attention Mechanism
  • Self-Attention
  • Semantic Regions
  • Scale Invariant Feature Transform (SIFT)
  • Fine-grained visual recognition
  • Artificial intelligence
  • Convolutional Neural Network

Fingerprint

Dive into the research topics of 'Attend and Guide (AG-Net): A Keypoints-driven Attention-based Deep Network for Image Recognition'. Together they form a unique fingerprint.

Cite this