The Knowledge Graph Track at OAEI: Gold Standards, Baselines, and the Golden Hammer Bias

The Ontology Alignment Evaluation Initiative (OAEI) is an annual evaluation of ontology matching tools. In 2018, we have started the Knowledge Graph track, whose goal is to evaluate the simultaneous matching of entities and schemas of large-scale knowledge graphs. In this paper, we discuss the desig...

Descripción completa

Detalles Bibliográficos
Autores principales: Hertling, Sven, Paulheim, Heiko
Formato: Online Artículo Texto
Lenguaje:English
Publicado: 2020
Materias:
Article
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7250608/
http://dx.doi.org/10.1007/978-3-030-49461-2_20
_version_ 1783538795369463808
author Hertling, Sven
Paulheim, Heiko
author_facet Hertling, Sven
Paulheim, Heiko
author_sort Hertling, Sven
collection PubMed
description The Ontology Alignment Evaluation Initiative (OAEI) is an annual evaluation of ontology matching tools. In 2018, we have started the Knowledge Graph track, whose goal is to evaluate the simultaneous matching of entities and schemas of large-scale knowledge graphs. In this paper, we discuss the design of the track and two different strategies of gold standard creation. We analyze results and experiences obtained in first editions of the track, and, by revealing a hidden task, we show that all tools submitted to the track (and probably also to other tracks) suffer from a bias which we name the golden hammer bias.
format Online
Article
Text
id pubmed-7250608
institution National Center for Biotechnology Information
language English
publishDate 2020
record_format MEDLINE/PubMed
spelling pubmed-72506082020-05-27 The Knowledge Graph Track at OAEI: Gold Standards, Baselines, and the Golden Hammer Bias Hertling, Sven Paulheim, Heiko The Semantic Web Article The Ontology Alignment Evaluation Initiative (OAEI) is an annual evaluation of ontology matching tools. In 2018, we have started the Knowledge Graph track, whose goal is to evaluate the simultaneous matching of entities and schemas of large-scale knowledge graphs. In this paper, we discuss the design of the track and two different strategies of gold standard creation. We analyze results and experiences obtained in first editions of the track, and, by revealing a hidden task, we show that all tools submitted to the track (and probably also to other tracks) suffer from a bias which we name the golden hammer bias. 2020-05-07 /pmc/articles/PMC7250608/ http://dx.doi.org/10.1007/978-3-030-49461-2_20 Text en © Springer Nature Switzerland AG 2020 This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic.
spellingShingle Article
Hertling, Sven
Paulheim, Heiko
The Knowledge Graph Track at OAEI: Gold Standards, Baselines, and the Golden Hammer Bias
title The Knowledge Graph Track at OAEI: Gold Standards, Baselines, and the Golden Hammer Bias
title_full The Knowledge Graph Track at OAEI: Gold Standards, Baselines, and the Golden Hammer Bias
title_fullStr The Knowledge Graph Track at OAEI: Gold Standards, Baselines, and the Golden Hammer Bias
title_full_unstemmed The Knowledge Graph Track at OAEI: Gold Standards, Baselines, and the Golden Hammer Bias
title_short The Knowledge Graph Track at OAEI: Gold Standards, Baselines, and the Golden Hammer Bias
title_sort knowledge graph track at oaei: gold standards, baselines, and the golden hammer bias
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7250608/
http://dx.doi.org/10.1007/978-3-030-49461-2_20
work_keys_str_mv AT hertlingsven theknowledgegraphtrackatoaeigoldstandardsbaselinesandthegoldenhammerbias
AT paulheimheiko theknowledgegraphtrackatoaeigoldstandardsbaselinesandthegoldenhammerbias
AT hertlingsven knowledgegraphtrackatoaeigoldstandardsbaselinesandthegoldenhammerbias
AT paulheimheiko knowledgegraphtrackatoaeigoldstandardsbaselinesandthegoldenhammerbias

Ejemplares similares