A semi-supervised approach using label propagation to support citation screening

Georgios Kontonatsios, Austin J Brockmeier, Piotr Przybta, John MvNaught, Mu Tingting, John Y Goulermass, Sophia Ananaidou

Research output: Contribution to journalArticle (journal)peer-review

34 Citations (Scopus)
111 Downloads (Pure)


Citation screening, an integral process within systematic reviews that identifies citations relevant to the underlying research question, is a time-consuming and resource-intensive task. During the screening task, analysts manually assign a label to each citation, to designate whether a citation is eligible for inclusion in the review. Recently, several studies have explored the use of active learning in text classification to reduce the human workload involved in the screening task. However, existing approaches require a significant amount of manually labelled citations for the text classification to achieve a robust performance. In this paper, we propose a semi-supervised method that identifies relevant citations as early as possible in the screening process by exploiting the pairwise similarities between labelled and unlabelled citations to improve the classification performance without additional manual labelling effort. Our approach is based on the hypothesis that similar citations share the same label (e.g., if one citation should be included, then other similar citations should be included also). To calculate the similarity between labelled and unlabelled citations we investigate two different feature spaces, namely a bag-of-words and a spectral embedding based on the bag-of-words. The semi-supervised method propagates the classification codes of manually labelled citations to neighbouring unlabelled citations in the feature space. The automatically labelled citations are combined with the manually labelled citations to form an augmented training set. For evaluation purposes, we apply our method to reviews from clinical and public health. The results show that our semi-supervised method with label propagation achieves statistically significant improvements over two state-of-the-art active learning approaches across both clinical and public health reviews.
Original languageEnglish
Pages (from-to)67-76
Number of pages10
JournalJournal of Biomedical Informatics
Early online date23 Jun 2017
Publication statusPublished - 1 Aug 2017


  • active learning
  • label propagation
  • citation screening
  • semi-supervised learning
  • text classification
  • Active learning
  • Semi-supervised learning
  • Citation screening
  • Label propagation
  • Text classification
  • Automation
  • Humans
  • Natural Language Processing
  • Review Literature as Topic
  • Data Curation


Dive into the research topics of 'A semi-supervised approach using label propagation to support citation screening'. Together they form a unique fingerprint.

Cite this