aalto1 untyped-item.component.html
A deep learning approach to real-world entity linking: Extracting and matching organisation mentions from unstructured text
Loading...
URL
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu |
Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
Department
Major/Subject
Mcode
SCI3115
Degree programme
Language
en
Pages
57
Series
Abstract
Entity Linking is a Natural Language Processing task that aims to link entities found in a piece of text to their corresponding representation in a specific Knowledge Base. In an industrial setting, this task naturally arises when attempting to link mentions of organizations found in unstructured text (such as news articles or job advertisements) to their corresponding entry in the Knowledge Base. The value extracted by solving this task is clear: it enables a much more comprehensive understanding of a business through data sources that provide non-standardized information (for example, that a company is looking to hire new people or is in talks about a potential merger). We define a custom Entity Linking task with a training dataset composed of pairs of job advertisements and businesses mentioned in that ad. Following recent literature, we develop two separate approaches. As a first solution, we split the task into two subtasks, Mention Detection (also called Named Entity Recognition or NER in NLP literature) and Entity Disambiguation.
We abbreviate it as MD+ED. A multilingual BERT-based model (XLM-RoBERTa) is finetuned on public datasets available in NER literature to extract mentions of organizations from the input text, and a rule-based approach links the extracted mentions to entities in the Knowledge Base. In the second approach, we attempt to solve the Entity Linking task end-to-end (abbreviated as E2E). We experiment with two common architectures in the Entity Linking literature, the Bi-encoder and the Poly-encoder.
These architectures are trained to compute a compatibility score between the embeddings of the text of a job ad and the corresponding organization's description. These embeddings are derived from separate instances of BERT-like Transformer encoders (LaBSE) that are finetuned through a contrastive loss function. During inference, the organization that is most compatible with an input job ad is predicted as the correct one. Evaluating the two solutions on a held-out test set, the best E2E model is able to achieve 0.85 recall against MD+ED's 0.70. E2E achieves better performance than MD+ED, but it also requires a textual description of each organization that is present in the Knowledge Base, therefore a trade-off must be struck when choosing which approach to implement.