Unsupervised Learning of Monocular Depth and Ego-Motion using Conditional PatchGANs

Madhu Babu Vankadari, SWAGAT KUMAR, Anima Majumder, Kaushik Das

Research output: Contribution to conferencePaper

Abstract

This paper presents a new GAN-based deep learning framework for estimating absolute scale aware depth and ego motion from monocular images using a completely unsupervised mode of learning. The proposed architecture uses two separate generators to learn the distribution of depth and pose
data for a given input image sequence. The depth and pose data, thus generated, are then evaluated by
a patch-based discriminator using the reconstructed
image and its corresponding actual image. The
patch-based GAN (or PatchGAN) is shown to detect high frequency local structural defects in the reconstructed image, thereby improving the accuracy
of overall depth and pose estimation. Unlike conventional GANs, the proposed architecture uses a
conditioned version of input and output of the generator for training the whole network. The resulting
framework is shown to outperform all existing deep
networks in this field, beating the current state-of-the-art method by 8.7% in absolute error and 5.2% in RMSE metric. To the best of our knowledge,
this is first deep network based model to estimate
both depth and pose simultaneously using a conditional patch-based GAN paradigm. The efficacy of
the proposed approach is demonstrated through rigorous ablation studies and exhaustive performance
comparison on the popular KITTI outdoor driving
dataset.
Original languageEnglish
Pages5677
Number of pages5684
Publication statusPublished - 16 Aug 2019
EventInternational Joint Conference on Artificial Intelligence - Macao, Macao, China
Duration: 10 Aug 201916 Aug 2019
https://www.ijcai.org/proceedings/2019/0787.pdf

Conference

ConferenceInternational Joint Conference on Artificial Intelligence
Abbreviated titleIJCAI
CountryChina
CityMacao
Period10/08/1916/08/19
Internet address

Fingerprint

Unsupervised learning
Discriminators
Ablation
Defects
Deep learning

Keywords

  • Deep learning, Depth Estimation from Images, GANs

Cite this

Vankadari, M. B., KUMAR, SWAGAT., Majumder, A., & Das, K. (2019). Unsupervised Learning of Monocular Depth and Ego-Motion using Conditional PatchGANs. 5677. Paper presented at International Joint Conference on Artificial Intelligence, Macao, China.
Vankadari, Madhu Babu ; KUMAR, SWAGAT ; Majumder, Anima ; Das, Kaushik. / Unsupervised Learning of Monocular Depth and Ego-Motion using Conditional PatchGANs. Paper presented at International Joint Conference on Artificial Intelligence, Macao, China.5684 p.
@conference{dddb1959fd934695bd190f6009ba3597,
title = "Unsupervised Learning of Monocular Depth and Ego-Motion using Conditional PatchGANs",
abstract = "This paper presents a new GAN-based deep learning framework for estimating absolute scale aware depth and ego motion from monocular images using a completely unsupervised mode of learning. The proposed architecture uses two separate generators to learn the distribution of depth and posedata for a given input image sequence. The depth and pose data, thus generated, are then evaluated bya patch-based discriminator using the reconstructedimage and its corresponding actual image. Thepatch-based GAN (or PatchGAN) is shown to detect high frequency local structural defects in the reconstructed image, thereby improving the accuracyof overall depth and pose estimation. Unlike conventional GANs, the proposed architecture uses aconditioned version of input and output of the generator for training the whole network. The resultingframework is shown to outperform all existing deepnetworks in this field, beating the current state-of-the-art method by 8.7{\%} in absolute error and 5.2{\%} in RMSE metric. To the best of our knowledge,this is first deep network based model to estimateboth depth and pose simultaneously using a conditional patch-based GAN paradigm. The efficacy ofthe proposed approach is demonstrated through rigorous ablation studies and exhaustive performancecomparison on the popular KITTI outdoor drivingdataset.",
keywords = "Deep learning, Depth Estimation from Images, GANs",
author = "Vankadari, {Madhu Babu} and SWAGAT KUMAR and Anima Majumder and Kaushik Das",
year = "2019",
month = "8",
day = "16",
language = "English",
pages = "5677",
note = "International Joint Conference on Artificial Intelligence, IJCAI ; Conference date: 10-08-2019 Through 16-08-2019",
url = "https://www.ijcai.org/proceedings/2019/0787.pdf",

}

Vankadari, MB, KUMAR, SWAGAT, Majumder, A & Das, K 2019, 'Unsupervised Learning of Monocular Depth and Ego-Motion using Conditional PatchGANs' Paper presented at International Joint Conference on Artificial Intelligence, Macao, China, 10/08/19 - 16/08/19, pp. 5677.

Unsupervised Learning of Monocular Depth and Ego-Motion using Conditional PatchGANs. / Vankadari, Madhu Babu; KUMAR, SWAGAT; Majumder, Anima; Das, Kaushik.

2019. 5677 Paper presented at International Joint Conference on Artificial Intelligence, Macao, China.

Research output: Contribution to conferencePaper

TY - CONF

T1 - Unsupervised Learning of Monocular Depth and Ego-Motion using Conditional PatchGANs

AU - Vankadari, Madhu Babu

AU - KUMAR, SWAGAT

AU - Majumder, Anima

AU - Das, Kaushik

PY - 2019/8/16

Y1 - 2019/8/16

N2 - This paper presents a new GAN-based deep learning framework for estimating absolute scale aware depth and ego motion from monocular images using a completely unsupervised mode of learning. The proposed architecture uses two separate generators to learn the distribution of depth and posedata for a given input image sequence. The depth and pose data, thus generated, are then evaluated bya patch-based discriminator using the reconstructedimage and its corresponding actual image. Thepatch-based GAN (or PatchGAN) is shown to detect high frequency local structural defects in the reconstructed image, thereby improving the accuracyof overall depth and pose estimation. Unlike conventional GANs, the proposed architecture uses aconditioned version of input and output of the generator for training the whole network. The resultingframework is shown to outperform all existing deepnetworks in this field, beating the current state-of-the-art method by 8.7% in absolute error and 5.2% in RMSE metric. To the best of our knowledge,this is first deep network based model to estimateboth depth and pose simultaneously using a conditional patch-based GAN paradigm. The efficacy ofthe proposed approach is demonstrated through rigorous ablation studies and exhaustive performancecomparison on the popular KITTI outdoor drivingdataset.

AB - This paper presents a new GAN-based deep learning framework for estimating absolute scale aware depth and ego motion from monocular images using a completely unsupervised mode of learning. The proposed architecture uses two separate generators to learn the distribution of depth and posedata for a given input image sequence. The depth and pose data, thus generated, are then evaluated bya patch-based discriminator using the reconstructedimage and its corresponding actual image. Thepatch-based GAN (or PatchGAN) is shown to detect high frequency local structural defects in the reconstructed image, thereby improving the accuracyof overall depth and pose estimation. Unlike conventional GANs, the proposed architecture uses aconditioned version of input and output of the generator for training the whole network. The resultingframework is shown to outperform all existing deepnetworks in this field, beating the current state-of-the-art method by 8.7% in absolute error and 5.2% in RMSE metric. To the best of our knowledge,this is first deep network based model to estimateboth depth and pose simultaneously using a conditional patch-based GAN paradigm. The efficacy ofthe proposed approach is demonstrated through rigorous ablation studies and exhaustive performancecomparison on the popular KITTI outdoor drivingdataset.

KW - Deep learning, Depth Estimation from Images, GANs

M3 - Paper

SP - 5677

ER -

Vankadari MB, KUMAR SWAGAT, Majumder A, Das K. Unsupervised Learning of Monocular Depth and Ego-Motion using Conditional PatchGANs. 2019. Paper presented at International Joint Conference on Artificial Intelligence, Macao, China.