The challenges of, and why you should reconsider using, truth sets for optimizing entity resolution: A case study

Pei Wang; Daniel L. Pullen; Maryam Y. Garza; Meredith N. Zozus

The challenges of, and why you should reconsider using, truth sets for optimizing entity resolution: A case study

Pei Wang, Daniel L. Pullen, Maryam Y. Garza, Meredith N. Zozus

Research output: Contribution to conference › Paper › peer-review

Abstract

The creation of high quality Entity Resolution (ER) processes depends on the ability to quickly and effectively identify erroneous outcomes (false positives and false negatives) in ER results. In past and current research, truth sets have been used to provide this ability. Unfortunately, managing the quantity of data provided to reviewers for manual annotation during the generation process often forces researchers to generate sampled data that is not entirely representative of the total amount of variation contained within the original dataset. This often causes an over-fitting of the match logic to the truth set. This case study shows the challenges and issues that can arise when using truth sets for creating and analyzing ER matching logic.

Original language	English (US)
State	Published - 2017
Externally published	Yes
Event	22nd MIT International Conference on Information Quality, ICIQ 2017 - Little Rock, United States Duration: Oct 6 2017 → Oct 7 2017

Conference

Conference	22nd MIT International Conference on Information Quality, ICIQ 2017
Country/Territory	United States
City	Little Rock
Period	10/6/17 → 10/7/17

Keywords

Boolean Match Rule
EHR Data
Entity Resolution
Truth Set

ASJC Scopus subject areas

Safety, Risk, Reliability and Quality
Information Systems

Cite this

@conference{cee7f74f2a384e6a9329c31cdbfa5b23,

title = "The challenges of, and why you should reconsider using, truth sets for optimizing entity resolution: A case study",

abstract = "The creation of high quality Entity Resolution (ER) processes depends on the ability to quickly and effectively identify erroneous outcomes (false positives and false negatives) in ER results. In past and current research, truth sets have been used to provide this ability. Unfortunately, managing the quantity of data provided to reviewers for manual annotation during the generation process often forces researchers to generate sampled data that is not entirely representative of the total amount of variation contained within the original dataset. This often causes an over-fitting of the match logic to the truth set. This case study shows the challenges and issues that can arise when using truth sets for creating and analyzing ER matching logic.",

keywords = "Boolean Match Rule, EHR Data, Entity Resolution, Truth Set",

author = "Pei Wang and Pullen, {Daniel L.} and Garza, {Maryam Y.} and Zozus, {Meredith N.}",

note = "Publisher Copyright: {\textcopyright} 2017 MIT Information Quality Program. All rights reserved.; 22nd MIT International Conference on Information Quality, ICIQ 2017 ; Conference date: 06-10-2017 Through 07-10-2017",

year = "2017",

language = "English (US)",

}

TY - CONF

T1 - The challenges of, and why you should reconsider using, truth sets for optimizing entity resolution

T2 - 22nd MIT International Conference on Information Quality, ICIQ 2017

AU - Wang, Pei

AU - Pullen, Daniel L.

AU - Garza, Maryam Y.

AU - Zozus, Meredith N.

PY - 2017

Y1 - 2017

N2 - The creation of high quality Entity Resolution (ER) processes depends on the ability to quickly and effectively identify erroneous outcomes (false positives and false negatives) in ER results. In past and current research, truth sets have been used to provide this ability. Unfortunately, managing the quantity of data provided to reviewers for manual annotation during the generation process often forces researchers to generate sampled data that is not entirely representative of the total amount of variation contained within the original dataset. This often causes an over-fitting of the match logic to the truth set. This case study shows the challenges and issues that can arise when using truth sets for creating and analyzing ER matching logic.

AB - The creation of high quality Entity Resolution (ER) processes depends on the ability to quickly and effectively identify erroneous outcomes (false positives and false negatives) in ER results. In past and current research, truth sets have been used to provide this ability. Unfortunately, managing the quantity of data provided to reviewers for manual annotation during the generation process often forces researchers to generate sampled data that is not entirely representative of the total amount of variation contained within the original dataset. This often causes an over-fitting of the match logic to the truth set. This case study shows the challenges and issues that can arise when using truth sets for creating and analyzing ER matching logic.

KW - Boolean Match Rule

KW - EHR Data

KW - Entity Resolution

KW - Truth Set

UR - http://www.scopus.com/inward/record.url?scp=85084162056&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85084162056&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85084162056

Y2 - 6 October 2017 through 7 October 2017

ER -

The challenges of, and why you should reconsider using, truth sets for optimizing entity resolution: A case study

Abstract

Conference

Keywords

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this