Date of Award
8-17-2020
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Information Science
First Advisor
John Talburt
Abstract
As a fundamental task in data integration and data quality, Entity Resolution (ER) has been investigated for decades in various domains. The emerging volume of heterogeneously structured data, and even unstructured data, poses a challenge to traditional ER methods. This research is to explore machine learning and deep learning approach to address the challenge from unstructured references data. This research starts with pairwise matching, the core function of all ER tasks. Based on the similarity score vector derived from our designed similarity measurement tool, scoring matrix, machine leaning enhances the performance significantly compared to the manually threshold method. Without similarity measure, deep learning enhances pairwise matching F-measures from 0.80 to 0.99. Reference2Vec is proposed to do the reference representation. To capture both the syntactic and semantic similarity, the model is designed at character level rather than the general word level. Convolutional neural network (CNN), Long short-term memory (LSTM), Attention mechanism and the combinations of them are used as feature extractors. Our experiments show that CNN combined with Attention mechanism gets the best performance. The dense “character embedding” is better than one-hot vector for encoding reference. Through controlling quality of training data, we find that, for the same test data, model trained on low quality data performs better than model trained on high quality data, this shows the potential for transfer learning of the Reference2Vec.
Recommended Citation
Li, Xinming, "Machine Learning and Deep Learning Based Entity Resolution Approaches for Unstructured References" (2020). Theses and Dissertations. 953.
https://research.ualr.edu/etd/953
