Enhancing Trustworthiness of Semantic Segmentation in Cataract Surgery Videos via Intra-Phase Label Propagation

Accurate segmentation of semantic features
is a pivotal procedure for cataract surgery assistance, surgical
skill assessment and related applications. However,
previous studies have failed to consider the instance-level
feature similarity of instruments across different surgical
phases in cataract surgery videos, leading to unreliable
decision-making regarding instrument categories. In this
study, we propose a label propagation framework to effectively
leverage the consistency of phase-specific instruments,
which utilizes the initial frame labels from each
surgical phase to predict masks for the remaining frames,
achieving precise and trustworthy semantic segmentation
of cataract surgery videos. Specifically, we design a
pseudo-label generation and filtering strategy to automatically
obtain highly reliable initial frame labels for each
surgical phase. In addition, we establish a fixed-size memory
bank with an adaptive update module to ensure longterm
applicability in real surgical environments. To address
the common problem of blurred edges in cataract surgery
scenes, we develop a semantic edge perception module to
allow the model to focus on and distinguish the edges of
different objects. The proposed method achieved an mIoU
of 80.7% and 88.8% on a publicly available dataset (14 categories)
and a private dataset (12 categories) with a total of
9,723 frames, respectively, significantly outperforming the
state-of-the-art methods and other label propagation-based
approaches. Furthermore, our method minimizes memory
consumption and maintains about 30 FPS while processing
long video sequences.