Tutorial: Hands-on Automatic Quality Assessment of Knowledge Graphs

Organized by:

• Panos Alexopoulos

Theme & goals

Knowledge graphs are increasingly becoming important in the AI world as an enabling technology for data integration and analytics, semantic search and question answering, and other cognitive applications. However, as they become larger in size and scope, and are used by bigger and more diverse audiences, their ability to represent semantic information that is accurate and consensual is stressed. Typical semantic modeling mistakes that in small-scale taxonomies and ontologies are controllable and perhaps not so harmful, in large-scale knowledge graphs can become really problematic and hard to contain. 

In this tutorial we will tackle the challenge of detecting and uncovering (potentially systematic) quality issues in knowledge graphs, in an as much as possible automatic way. We will cover quality dimensions like accuracy, completeness, conciseness and understandability, and we will apply problem detection methods inspired from linguistics, statistical modeling, and ontological analysis, in publicly available knowledge graphs. By the end of this tutorial, the attendees will be able to: 

• Identify key quality dimensions that are important to measure in a knowledge graph, along with the typical reasons why a knowledge graph might have low scores in each of them. 
• Develop computational methods for automatically detecting and uncovering quality issues in each of these dimensions. 

Audience 
The expected audience for this tutorial includes researchers and practitioners who develop or use knowledge graphs and other types of semantic models.

Prerequisite knowledge 

• Basic coding skills in Python and Jupyter notebooks 
• Familiarity with RDF/OWL and SPARQL. 

Delivery methodology 
The tutorial is suggested to be delivered in 5 sessions of a total duration of 180 minutes, with one 30 min break in between. Each session will comprise a slide-based presentation of key topics and 
techniques, a hands-on application of these techniques on real-world knowledge graphs with relevant software, and a Q&A. 

Tentative schedule

• Introduction (15 mins) 
• Session 1: Knowledge Graphs and their quality (45min) 
  • Quality dimensions 
  • Typical quality issues and their causes 
  • Quality dilemmas and trade-offs 
• Session 2: Detecting wrong synonyms (30min) 
  • Break (30 minutes) 
• Session 3: Detecting missing synonyms (20min) 
• Session 4: Detecting vague, ambiguous and inaccurate element names (30min) 
• Session 4: Detecting problematic classes and subclasses (30min) 
• Wrapping up (10min) 

Tutor 
Panos Alexopoulos has been working since 2006 at the intersection of data, semantics, and software, contributing to building intelligent systems that deliver value to business and society. Born and raised in Athens, Greece, he currently works as Head of Ontology at Textkernel, in Amsterdam, Netherlands, where he leads a team of Data Professionals in developing and delivering a large cross-lingual Knowledge Graph in the HR and Recruitment domain. Panos holds a PhD in Knowledge Engineering and Management from National Technical University of Athens, and has published more than 60 papers at international conferences, journals and books. He is the author of the book "Semantic Modeling for Data - Avoiding Pitfalls and Breaking Dilemmas" (O'Reilly, 2020), and a regular speaker and trainer in both academic and industry venues.

Relevant recent past tutorials and masterclasses delivered by Panos include: 

• Semantic Modeling for Interoperable FAIR Data, 13th International SWAT4HCLS Conference, January 10th, 2022. 
• Developing NLP Solutions for Automatic Knowledge Graph Construction, Masterclass at Connected Data World 2021, December 2nd 2021. 
• Semantic Modeling for Knowledge Graphs, Tutorial at Knowledge Graph Conference 2021, May 5th 2021. 
• Flawless Semantic Modeling in OWL and SKOS – The Theory and the Practice, Masterclass at Knowledge Connexions Conference, December 1st, 2020. 
• From Taxonomies and Schemas to Knowledge Graphs – Recognizing and tackling the pitfalls of large-scale semantic modeling}, Tutorial at Connected Data London 2019, London, UK, October 3rd, 2019 
• Executing Knowledge Graph Initiatives in Organizations – A Field Guide}, Tutorial at the 15th European Semantic Web Conference, Heraklion, Greece, June 3rd, 2018.