The Four Generations of Entity Resolution

Entity Resolution (ER) lies at the core of data integration and cleaning and, thus, a bulk of the research examines ways for improving its effectiveness and time efficiency. The initial ER methods primarily target Veracity in the context of structured (relational) data that are described by a schema of well-known quality and meaning. To achieve high effectiveness, they leverage schema, expert, and/or external knowledge. Part of these methods are extended to address Volume, processing large datasets through multi-core or massive parallelization approaches, such as the MapReduce paradigm. However, these early schema-based approaches are inapplicable to Web Data, which abound in voluminous, noisy, semi-structured, and highly heterogeneous information. To address the additional challenge of Variety, recent works on ER adopt a novel, loosely schema-aware functionality that emphasizes scalability and robustness to noise. Another line of present research focuses on the additional challenge ofVelocity, aiming to process data collections of a continuously increasing volume. The latest works, though, take advantage of the significant breakthroughs in Deep Learning and Crowdsourcing, incorporating external knowledge to enhance the existing words to a significant extent. This synthesis lecture organizes ER methods into four generations based on the challenges posed by these four Vs. For each generation, we outline the corresponding ER workflow, discuss the state-of-the-art methods per workflow step, and present current research directions. The discussion of these methods takes into account a historical perspective, explaining the evolution of the methods over time along with their similarities and differences. The lecture also discusses the available ER tools and benchmark datasets that allow expert as well as novice users to make use of the available solutions.

George Papadakis is a research fellow at the National and Kapodistrian University of Athens, Greece. He also worked at the NCSR Demokritos, National Technical University of Athens (NTUA), L3S Research Center, and ＂Athena＂ Research Center. He holds a Ph.D. in Computer Science from the University of Hanover and a Diploma in Electrical Computer Engineering from NTUA. His research interest focuses on web data mining.Ekaterini Ioannou is an Assistant Professor at Tilburg University, the Netherlands. Prior, she worked as an Assistant Professor at Eindhoven University of Technology, as a Lecturer at the Open University of Cyprus, an adjunct faculty at EPFL in Switzerland, a research collaborator at the Technical University of Crete, and as an Independent Expert for the European Commission. Her research focuses on information integration with an emphasis on the challenges of man aging data with uncertainties, heterogeneity or correlations, and, more recently, on achieving a deeper integration of information extraction tasks within databases, and on efficiently retrieving analytics over graphs/hypergraphs with evolving data.Emanouil Thanos is a Ph.D. candidate at CODeS research group of KU Leuven, under the supervision of Prof. Greet Vanden Berghe. He holds a Diploma in Electrical and Computer Engineering from the National Technical University of Athens and a joint Master in Com putational Logic from TU Dresden, FU Bolzano, and UN Lisbon. He has also worked as a research associate at National ICT Australia and the University of Athens. His research inter ests focus on combinatorial optimization and operations research.Themis Palpanas is Senior Member of the French University Institute (IUF), and Professor of Computer Science at the University of Paris (France) where he is director of the Data Intel ligence Institute of Paris (diiP), and of the Data Intensive and Knowledge Oriented Systems (diNo) group. He is the author of two French patents and nine U.S. patents, three of which have been implemented in world-leading commercial data management products. He is the recipient of three Best Paper awards and the IBM Shared University Research (SUR) Award. He is currently serving in the Board of Trustees for the Very Large Data Bases (VLDB) Endowment, as Editor in Chief for BDR Journal, Editorial Advisory Board member for IS Journal, and in the Senior Program Committee of SIGMOD 2021.