Cross-modality paired-images generation and augmentation for RGB-infrared person re-identification

Guan'an Wang, Yang Yang, Tianzhu Zhang, Jian Cheng, Zengguang Hou, Prayag Tiwari, Hari Mohan Pandey*

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

RGB-Infrared (IR) person re-identification is very challenging due to the large cross-modality variations between RGB and IR images. Considering no correspondence labels between every pair of RGB and IR images, most methods try to alleviate the variations with set-level alignment by reducing marginal distribution divergence between the entire RGB and IR sets. However, this set-level alignment strategy may lead to misalignment of some instances, which limit the performance for RGB–IR Re-ID. Different from existing methods, in this paper, we propose to generate cross-modality paired-images and perform both global set-level and fine-grained instance-level alignments. Our proposed method enjoys several merits. First, our method can perform set-level alignment by disentangling modality-specific and modality-invariant features. Compared with conventional methods, ours can explicitly remove the modality-specific features and the modality variation can be better reduced. Second, given cross-modality unpaired-images of a person, our method can generate cross-modality paired images from exchanged features. With them, we can directly perform instance-level alignment by minimizing distances of every pair of images. Third, our method learns a latent manifold space. In the space, we can random sample and generate lots of images of unseen classes. Training with those images, the learned identity feature space is more smooth can generalize better when test. Finally, extensive experimental results on two standard benchmarks demonstrate that the proposed model favorably against state-of-the-art methods.

Original languageEnglish
Pages (from-to)294-304
Number of pages11
JournalNeural Networks
Volume128
Early online date19 May 2020
DOIs
Publication statusE-pub ahead of print - 19 May 2020

Keywords

  • Adversarial learning
  • Cross-modality
  • Feature disentanglement
  • Image generation
  • Person re-identification

Fingerprint Dive into the research topics of 'Cross-modality paired-images generation and augmentation for RGB-infrared person re-identification'. Together they form a unique fingerprint.

  • Cite this