Deep Network based Automatic Annotation for Warehouse Automation

Chandan Kumar Singh, Anima Majumder, Swagat Kumar, Laxmidhar Behera

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

The paper presents a deep learning based fully automatic object annotation technique for warehouse application usecase. One of the main challenges that is addressed in this paper is the large amount of manual labour involved in generating datasets for training a deep network. The proposed annotation model is developed by fine-tuning a deep network based object detection framework with ImageNet pre-trained models. We have used Faster RCNN network with pre-trained model VGG-16 and RFCN with ResNet-101. A small set of manually annotated images of single objects are used to automatically generate a dataset of significantly large size within a very short time duration (in real-time). The model also has the competence of precisely localizing the region of any new object that comes into the familiar background. Incorporation of techniques like color augmentation and affine transformation enables the network invariant to rotation, scale and brightness. Augmentation also enables the model to performs well even if the background is different. A clutter generation technique is introduced in the framework which makes the system capable of annotating objects even in a densely populated real-world environment. This work has another significant contribution in detection of objects those are used in Amazon Robotic Challenge (ARC) 2017 where our team was among the four finalist in both picking and stowing task. The automatically generated big dataset is further used to train multi-class detectors using Faster RCNN and RFCN networks to validate the performance of the proposed annotation model. The efficacy of the proposed model is hence demonstrated through various experimental results. The dataset is shared online for the convenience of the reader.
Original languageEnglish
Title of host publicationProceedings of the International Joint Conference on Neural Networks
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Print)9781509060146
DOIs
Publication statusPublished - 10 Oct 2018
Event2018 International Joint Conference on Neural Networks - Rio de Janeiro, Brazil
Duration: 8 Jul 201813 Jul 2018

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2018-July

Conference

Conference2018 International Joint Conference on Neural Networks
CountryBrazil
CityRio de Janeiro
Period8/07/1813/07/18

Fingerprint

Warehouses
Automation
Luminance
Robotics
Tuning
Personnel
Color
Detectors

Keywords

  • Annotation
  • Background subtraction
  • Convolutional Neural Network
  • Faster RCNN
  • RFCN

Cite this

Singh, C. K., Majumder, A., Kumar, S., & Behera, L. (2018). Deep Network based Automatic Annotation for Warehouse Automation. In Proceedings of the International Joint Conference on Neural Networks (Proceedings of the International Joint Conference on Neural Networks; Vol. 2018-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IJCNN.2018.8489424
Singh, Chandan Kumar ; Majumder, Anima ; Kumar, Swagat ; Behera, Laxmidhar. / Deep Network based Automatic Annotation for Warehouse Automation. Proceedings of the International Joint Conference on Neural Networks. Institute of Electrical and Electronics Engineers Inc., 2018. (Proceedings of the International Joint Conference on Neural Networks).
@inbook{da41dae4d761462f9633c54b8e5de3ea,
title = "Deep Network based Automatic Annotation for Warehouse Automation",
abstract = "The paper presents a deep learning based fully automatic object annotation technique for warehouse application usecase. One of the main challenges that is addressed in this paper is the large amount of manual labour involved in generating datasets for training a deep network. The proposed annotation model is developed by fine-tuning a deep network based object detection framework with ImageNet pre-trained models. We have used Faster RCNN network with pre-trained model VGG-16 and RFCN with ResNet-101. A small set of manually annotated images of single objects are used to automatically generate a dataset of significantly large size within a very short time duration (in real-time). The model also has the competence of precisely localizing the region of any new object that comes into the familiar background. Incorporation of techniques like color augmentation and affine transformation enables the network invariant to rotation, scale and brightness. Augmentation also enables the model to performs well even if the background is different. A clutter generation technique is introduced in the framework which makes the system capable of annotating objects even in a densely populated real-world environment. This work has another significant contribution in detection of objects those are used in Amazon Robotic Challenge (ARC) 2017 where our team was among the four finalist in both picking and stowing task. The automatically generated big dataset is further used to train multi-class detectors using Faster RCNN and RFCN networks to validate the performance of the proposed annotation model. The efficacy of the proposed model is hence demonstrated through various experimental results. The dataset is shared online for the convenience of the reader.",
keywords = "Annotation, Background subtraction, Convolutional Neural Network, Faster RCNN, RFCN",
author = "Singh, {Chandan Kumar} and Anima Majumder and Swagat Kumar and Laxmidhar Behera",
year = "2018",
month = "10",
day = "10",
doi = "10.1109/IJCNN.2018.8489424",
language = "English",
isbn = "9781509060146",
series = "Proceedings of the International Joint Conference on Neural Networks",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "Proceedings of the International Joint Conference on Neural Networks",
address = "United States",

}

Singh, CK, Majumder, A, Kumar, S & Behera, L 2018, Deep Network based Automatic Annotation for Warehouse Automation. in Proceedings of the International Joint Conference on Neural Networks. Proceedings of the International Joint Conference on Neural Networks, vol. 2018-July, Institute of Electrical and Electronics Engineers Inc., 2018 International Joint Conference on Neural Networks, Rio de Janeiro, Brazil, 8/07/18. https://doi.org/10.1109/IJCNN.2018.8489424

Deep Network based Automatic Annotation for Warehouse Automation. / Singh, Chandan Kumar; Majumder, Anima; Kumar, Swagat; Behera, Laxmidhar.

Proceedings of the International Joint Conference on Neural Networks. Institute of Electrical and Electronics Engineers Inc., 2018. (Proceedings of the International Joint Conference on Neural Networks; Vol. 2018-July).

Research output: Chapter in Book/Report/Conference proceedingChapter

TY - CHAP

T1 - Deep Network based Automatic Annotation for Warehouse Automation

AU - Singh, Chandan Kumar

AU - Majumder, Anima

AU - Kumar, Swagat

AU - Behera, Laxmidhar

PY - 2018/10/10

Y1 - 2018/10/10

N2 - The paper presents a deep learning based fully automatic object annotation technique for warehouse application usecase. One of the main challenges that is addressed in this paper is the large amount of manual labour involved in generating datasets for training a deep network. The proposed annotation model is developed by fine-tuning a deep network based object detection framework with ImageNet pre-trained models. We have used Faster RCNN network with pre-trained model VGG-16 and RFCN with ResNet-101. A small set of manually annotated images of single objects are used to automatically generate a dataset of significantly large size within a very short time duration (in real-time). The model also has the competence of precisely localizing the region of any new object that comes into the familiar background. Incorporation of techniques like color augmentation and affine transformation enables the network invariant to rotation, scale and brightness. Augmentation also enables the model to performs well even if the background is different. A clutter generation technique is introduced in the framework which makes the system capable of annotating objects even in a densely populated real-world environment. This work has another significant contribution in detection of objects those are used in Amazon Robotic Challenge (ARC) 2017 where our team was among the four finalist in both picking and stowing task. The automatically generated big dataset is further used to train multi-class detectors using Faster RCNN and RFCN networks to validate the performance of the proposed annotation model. The efficacy of the proposed model is hence demonstrated through various experimental results. The dataset is shared online for the convenience of the reader.

AB - The paper presents a deep learning based fully automatic object annotation technique for warehouse application usecase. One of the main challenges that is addressed in this paper is the large amount of manual labour involved in generating datasets for training a deep network. The proposed annotation model is developed by fine-tuning a deep network based object detection framework with ImageNet pre-trained models. We have used Faster RCNN network with pre-trained model VGG-16 and RFCN with ResNet-101. A small set of manually annotated images of single objects are used to automatically generate a dataset of significantly large size within a very short time duration (in real-time). The model also has the competence of precisely localizing the region of any new object that comes into the familiar background. Incorporation of techniques like color augmentation and affine transformation enables the network invariant to rotation, scale and brightness. Augmentation also enables the model to performs well even if the background is different. A clutter generation technique is introduced in the framework which makes the system capable of annotating objects even in a densely populated real-world environment. This work has another significant contribution in detection of objects those are used in Amazon Robotic Challenge (ARC) 2017 where our team was among the four finalist in both picking and stowing task. The automatically generated big dataset is further used to train multi-class detectors using Faster RCNN and RFCN networks to validate the performance of the proposed annotation model. The efficacy of the proposed model is hence demonstrated through various experimental results. The dataset is shared online for the convenience of the reader.

KW - Annotation

KW - Background subtraction

KW - Convolutional Neural Network

KW - Faster RCNN

KW - RFCN

UR - http://www.mendeley.com/research/deep-network-based-automatic-annotation-warehouse-automation

U2 - 10.1109/IJCNN.2018.8489424

DO - 10.1109/IJCNN.2018.8489424

M3 - Chapter

SN - 9781509060146

T3 - Proceedings of the International Joint Conference on Neural Networks

BT - Proceedings of the International Joint Conference on Neural Networks

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Singh CK, Majumder A, Kumar S, Behera L. Deep Network based Automatic Annotation for Warehouse Automation. In Proceedings of the International Joint Conference on Neural Networks. Institute of Electrical and Electronics Engineers Inc. 2018. (Proceedings of the International Joint Conference on Neural Networks). https://doi.org/10.1109/IJCNN.2018.8489424