A Distant Supervision Method based on Paradigmatic Relations for Learning Word Embeddings

Jianquan Li, Renfen Hu, Xiaokang Liu, Prayag Tiwari, Hari Pandey, Wei Chen, Yaohong Jing, Kaicheng Yang, Benyou Wang

Research output: Contribution to journalArticle (journal)peer-review

5 Citations (Scopus)
236 Downloads (Pure)

Abstract

Word embeddings learned on external resources have succeeded in improving many NLP tasks. However, existing embedding models still face challenges in situations where fine-gained semantic information is required, e.g., distinguishing antonyms from synonyms. In this paper, a distant supervision method is proposed to guide the training process by introducing semantic knowledge in a thesaurus. Specifically, the proposed model shortens the distance between target word and its synonyms by controlling the movements of them in both unidirectional and bidirectional, yielding three different models, namely Unidirectional Movement of Target Model (UMT), Unidirectional Movement of Synonyms Model (UMS) and Bidirectional Movement of Target and Synonyms Model (BMTS). Extensive computational experiments have been conducted, and results are collected for analysis purpose. The results show that the proposed models not only efficiently capture semantic information of antonyms but also achieve significant improvements in both intrinsic and extrinsic evaluation tasks. To validate the performance of the proposed models (UMT, UMS and BMTS), results are compared against well-known models, namely Skip-gram, JointRCM, WE-TD and dict2vec. The performances of the proposed models are evaluated on four tasks (benchmarks): word analogy (intrinsic), synonym-antonym detection (intrinsic), sentence matching (extrinsic) and text classification (extrinsic). A case study is provided to illustrate the working of the proposed models in an effective manner. Overall, a distant supervision method based on paradigmatic relations is proposed for learning word embeddings and it outperformed when compared against other existing models.

Original languageEnglish
Pages (from-to)7759-7768
Number of pages10
JournalNeural Computing and Applications
Volume32
Issue number12
Early online date21 Feb 2019
DOIs
Publication statusPublished - 1 Jun 2020

Keywords

  • Neural network
  • Sentence matching
  • Text classification
  • Word embedding

Fingerprint

Dive into the research topics of 'A Distant Supervision Method based on Paradigmatic Relations for Learning Word Embeddings'. Together they form a unique fingerprint.

Cite this