Retinal Structure Detection in OCTA Image via Voting-based Multi-task Learning

Jinkui Hao, Ting Shen, Xueli Zhu, YONGHUAI LIU, ARDHENDU BEHERA, Dan Zhang, Bang Chen, Jiang Liu, Jiong Zhang, Yitian Zhao*

*Corresponding author for this work

Research output: Contribution to journalArticle (journal)peer-review

8 Downloads (Pure)


Automated detection of retinal structures,
such as retinal vessels (RV), the foveal avascular zone
(FAZ), and retinal vascular junctions (RVJ), are of great importance
for understanding diseases of the eye and clinical
decision-making. In this paper, we propose a novel Votingbased
Adaptive Feature Fusion multi-task network (VAFFNet)
for joint segmentation, detection, and classification
of RV, FAZ, and RVJ in optical coherence tomography
angiography (OCTA). A task-specific voting gate module
is proposed to adaptively extract and fuse different features
for specific tasks at two levels: features at different
spatial positions from a single encoder, and features from
multiple encoders. In particular, since the complexity of
the microvasculature in OCTA images makes simultaneous
precise localization and classification of retinal vascular
junctions into bifurcation/crossing a challenging task, we
specifically design a task head by combining the heatmap
regression and grid classification. We take advantage of
three different en face angiograms from various retinal
layers, rather than following existing methods that use only
a single en face. To validate the superiority of our VAFFNet,
we carry out extensive experiments on three OCTA
datasets acquired using different imaging devices, and the
results demonstrate that the proposed method performs
on the whole better than either the state-of-the-art singlepurpose
methods or existing multi-task learning solutions.
We also demonstrate that our multi-task learning method
generalizes across other imaging modalities, such as color
fundus photography, and may potentially be used as a
general multi-task learning tool. For the first time in the
field of retinal OCTA image analysis, we construct three
datasets for multiple structure detection. To facilitate further
research, part of these datasets with the source code
and evaluation benchmark have been released for public
Original languageEnglish
JournalIEEE Transactions on Medical Imaging
Publication statusAccepted/In press - 23 Aug 2022


  • OCTA
  • multi-task learning
  • retinal structures
  • detection
  • segmentation
  • classification


Dive into the research topics of 'Retinal Structure Detection in OCTA Image via Voting-based Multi-task Learning'. Together they form a unique fingerprint.

Cite this