Date of Award

5-26-2020

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Information Science

First Advisor

John Talburt

Abstract

Using a weight-based match score has been studied as a way to improve the accuracy of record linking (entity resolution) since Fellegi and Sunter first described the idea of probabilistic agreement and disagreement weights in their seminal work “A Theory of Record Linking.” However, the original work only described weight associated with an entity attribute. Later researchers such as Herzog et al suggested the weighting scheme could be extended to apply to frequently-occurring attribute values (frequency-based weights) instead of just the attribute as in the Fellegi-Sunter scheme. However, there has been little definitive research as to how frequency-based weights should be applied, and when applied, how effective they are at improving the accuracy of record linking. The research described here provides an answer for three of these questions. In particular, it demonstrates that frequency-based agreement weight can improve the accuracy of record linking, beyond simple attribute-level weights, but not for all entity attributes. The research results indicate only attributes whose values have a wide range of frequencies will produce an improvement in linking when using frequency-based weights. In fact, the results show using frequency-based weight for some types of attributes that actually degrade linking accuracy. The results also indicate that using frequency-based disagreement weights can provide an additional improvement to record linking accuracy than using frequency-based agreements only. Finally, this research establishes the best way to implement the logic for weight selection in the case where there are two weights available for an agreement or disagreement.

Download

Share

COinS