Date of Award

5-13-2022

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Information Science

First Advisor

John Talburt

Abstract

Entity resolution means finding duplicate records within the same table, across various tables, or in multiple databases. Traditional and rule-based approaches in entity resolution rely on handcrafting rules for matching records. On the other hand, machine learning and deep learning methods are data-intensive and require labeled training datasets. Thus the move toward automating entity resolution for data cleaning, curation, and integration has become the goal for many organizations. Accordingly, unsupervised entity resolution methods have proliferated, relying on an automated pipeline of preprocessing, blocking, feature extraction, matching, clustering, profiling, and canonicalization. Unsupervised entity resolution methods face many challenges due to the need to automate the process entirely. The main challenge is overcoming the quadratic complexity of pairwise matching, relying less on schemas and metadata, and assuming that datasets, documents, or records do not necessarily come with accurate descriptions. Through a comprehensive survey of the literature on the adoption of graph-based methods in unsupervised entity resolution, I observed that the literature is sparse and mainly relies on metadata, schemas, and arbitrary thresholds to match entities that hinder true automation. I also observed that formulating the unsupervised entity resolution problem as a graph-based problem unlocks the possibility of experimenting with a set of mature algorithms and techniques developed in graph theory, network science, and complex networks. This dissertation discusses the design, implementation, and evaluation of two graph-based unsupervised entity resolution frameworks based on the state-of-the-art probabilistic entity resolution method, namely the Data Washing Machine (DWM). The two frameworks were designed to address the DWM challenges in particular and graph-based and unsupervised probabilistic entity resolution in general. The two frameworks leverage multiple graph clustering algorithms to achieve more precise entity profiles in a hierarchical breakdown. The two frameworks, Graph-based Data Washing Machine (GDWM) and Modularity Composite Optimization for Entity Resolution (ModER) have been evaluated extensively and demonstrated their effectiveness on multiple benchmark datasets. The GDWM framework enhanced the average F1-Score across 63% to 72% across synthetic samples. The ModER framework enhanced the F1-Score of a real-world benchmark dataset from 11% to 51%. The research discussed in this dissertation helps advance the state-of-the-art in unsupervised entity resolution and automated data curation.

Download

Share

COinS