Knowledge-aware Zero-shot Learning (K-ZSL):
Concepts, Methods and Resources

Description

Zero-shot Learning (ZSL), which enables machine learning models to predict new targets without seeing their training samples, has attracted wide research interests in many communities such as computer vision (CV) and natural language processing (NLP). An effective solution is to use external knowledge such as text, attribute descriptions and Knowledge Graphs (KGs) to bridge the gap between the targets with training samples and the targets without training samples. This tutorial aims to introduce ZSL from the perspective of KG, by presenting different kinds of paradigms, studies and resources for addressing ZSL in different tasks including image classification, visual question answering and KG completion. This tutorial is a new edit of the K-ZSL tutorial in ISWC'22, with many new contents such as the recent transformer-based methods.

This is a half (½) day tutorial. The program, as shown below, needs three and half hours.

Length	Content	Speaker
5 mins	Welcome	Jiaoyan Chen
15 mins 20 mins	Part I - Introduction and Background T1: ZSL Definitions and Concepts T2: An Introduction to KG and KG-aware ZSL	Jiaoyan Chen Jiaoyan Chen
20 mins 30 mins 30 mins 20 mins 20 mins 20 mins	Part II - Knowledge-aware ZSL Methods T3: OntoZSL: Ontology-based Sample Generation and ZSL Enhancement T4: Feature Propagation-based Methods for KG-aware ZSL break T5: KG Augmented Zero-shot Visual Question Answering T6: DUET: Cross-modal Semantic Grounding for Contrastive ZSL T7: KG Structure Pretraining for KG-aware ZSL	Yuxia Geng Yuxia Geng Zhuo Chen Zhuo Chen Wen Zhang
20 mins 10 mins	Part III - Resources, Benchmarking and Lessons T8: Hands-on with Resource, Benchmarking, and Demo T9: Conclusion, Discussion, and Future Directions	Yuxia Geng Jeff Z. Pan

Material

The materials presented in the tutorial are as follows:

• Presentation slides (Part I, Part II, Part III)

• Resources on KG-aware zero-shot image classification, relation extraction and KG completion (repository)

• Resource demonstration (Video, Codes, Google Driver)

Presenters

• Yuxia Geng
• Zhuo Chen
• Jiaoyan Chen
• Wen Zhang
• Jeff Z. Pan

Yuxia Geng is an Assitant Professor in School of Computer Science, Hangzhou Dianzi University. She got his PhD degree at Zhejiang University in 2023, advised by Prof. Huajun Chen. Her research interests include Knowledge Graph, Ontology, Zero-shot and Few-shot Learning, XAI, and Neuro-symbolic Integration. In the past few years, she has been contributing to Knowledge-driven Zero-shot Learning, inductive KG representation and reasoning, and Explainable ZSL, with over 10 papers published in leading CS conferences and journals, including Journal of Web Semantics, Semantic Web Journal, ICDE, KDD, WWW, ACL, IJCAI, ISWC and KR, as a main author. She has good presentation skills, with tutorial experience as a main organizer of the K-ZSL tutorial held in ISWC’22 [11], and invited talk experience as e.g., an invited speaker at Chongqing University, China in 2020. E-mail: yuxia.geng@hdu.edu.cn. Website: https://genggengcss.github.io/.

Zhuo Chen is a Ph.D. candidate in the College of Computer Science and Technology at Zhejiang University. He has been advised by Prof. Huajun Chen since 2020. His research interests include Knowledge Graph, Multi-modal Learning, and Low-Resource Learning. He has more than 3 years' research experience in Knowledge-driven Zero-shot and Multi-modal learning, and has published several papers as a main author in various academic conferences and journals, including ICDE, AAAI, ISWC, IJCAI, WWW, Nature Communications and Nature Machine Intelligence. And as the first author, he received the Best Application Paper Award at IJCKG conference (2022). E-mail: zhuo.chen@zju.edu.cn. Website: https://github.com/hackerchenzhuo.

Jiaoyan Chen is a Lecturer (Assistant Professor) in Department of Computer Science, The University of Manchester. Before he joined The University of Manchester in November 2022, he was a Senior Researcher in Department of Computer Science, University of Oxford, and got his PhD in Computer Science in Zhejiang University in 2016. He has over 10 years research experience on Knowledge Graph and Machine Learning, and over 6 years experience on knowledge-aware zero-shot learning with over 10 publications on this topic in leading conferences and journals such as IJCAI, AAAI, WWW, KDD, ICDE, ACL, KR, ISWC and Journal of Web Semantics. He is one of the main organizers of the K-ZSL tutorial in ISWC’22 [11] and an Editor of the Journal of Web Semantics, and has much presentation experience for a number of (invited) tasks, as well as teaching experience for university courses of AI, the Web, and Knowledge Representation and Reasoning. E-mail: jiaoyan.chen@manchester. Website: https://chenjiaoyan.github.io/.

Wen Zhang is an Assistant Professor in School of Software Technology, Zhejiang University. Wen got her PhD in Computer Science and Technology in Zhejiang University in 2020. Her research interests include Knowledge Graph and Knowledge Reasoning. In recent 3 years, she widely explored how to make knowledge graph reasoning methods transferable which is closely related to zero-shot learning and has 5 publications on this topic in the leading conferences and journals such as IJCAI, AAAI, WWW, ICDE and Knowledge-Based Systems. She has much presentation experience related to knowledge graphs in conferences and universities. She also has teaching experience for university courses about Graph Representation Learning, Knowledge Representation and Reasoning, and Knowledge Graph. Email: zhang.wen@zju.edu.cn. Website: https://person.zju.edu.cn/en/wenzhang.

Jeff Z. Pan is a chair of the Knowledge Graph Group at the Alan Turing Institute and is a member of the School of Informatics at The University of Edinburgh. He received his Ph.D. in Computer Science from The University of Manchester in 2004. His research focuses primarily on knowledge representation and artificial intelligence, in particular on knowledge based learning and reasoning, and knowledge based natural language understanding and generations, as well as their applications, such as those in information retrieval, healthcare, software engineering, and open science. He is the first author of the first book on Knowledge Graph and the Chief Scientist and Coordinator of the first EU Marie-Curie project on Knowledge Graph (i.e., the K-Drive project). He has 7 publications on this topic in leading conferences and journals, such as IJCAI, AAAI, WWW, KDD, KR, ISWC and the Journal of Web Semantics. He is an Editor of the Journal of Web Semantics (JoWS) and a Program Chair of the 19th International Semantic Web Conference (ISWC 2020), the premier international forum for the Semantic Web / Knowledge Graph / Linked Data communities. Email: j.z.pan@ed.ac.uk. Website: http://knowledge-representation.org/j.z.pan/.

Reference

[1]. Chen, J., Geng, Y., Chen, Z., Horrocks, I., Pan, J.Z., et al.: Knowledge-aware zero-shot learning: Survey and perspective. In: IJCAI (2021)
[2]. Chen, J., Geng, Y., Chen, Z., Pan, J. Z., He, Y., Zhang, W., ... & Chen, H. Zero-shot and Few-shot Learning with Knowledge Graphs: A Comprehensive Survey. Proceedings of the IEEE (2023).
[3]. Geng, Y., Chen, J., Zhuang, X., Chen, Z., Pan, J. Z., Li, J., Yuan, Z., & Chen, H. Benchmarking Knowledge-driven Zero-shot Learning. Journal of Web Semantics (2022).
[4]. Geng,Y.,Chen,J.,Chen,Z.,Pan,J.Z.,Ye,Z.,Yuan,Z.,Jia,Y.,Chen,H.:OntoZSL: Ontology-enhanced zero-shot learning. In: WWW. pp. 3325-3336 (2021).
[5] Geng, Y., Chen, J., Ye, Z., Zhang, W., Chen, H.: Explainable zero-shot learning via attentive graph convolutional network and knowledge graphs. Semantic Web (2021)
[6]. Chen, Z., Chen, J., Geng, Y., Pan, J. Z., Yuan, Z., & Chen, H. Zero-shot visual question answering using knowledge graph. In International Semantic Web Conference (pp. 146-162) (2021).
[7]. Geng, Y., Chen, J., Zhang W., Xu Y., Chen Z., Pan, J. Z., Huang, Y., Xiong F. & Chen, H. Disentangled Ontology Embedding for Zero-shot Learning. In: ACM SIGKDD (2022).
[8] Geng, Y., Chen, J., Zhang, W., Pan, J. Z., Yang, M., Chen, H. & Jiang, S. Relational Message Passing for Fully Inductive Knowledge Graph Completion.In: ICDE (2023).
[9] Chen, Z., Huang, Y., Chen, J., Geng, Y., Zhang, W., Fang, Y., ... & Chen, H. DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning. In: AAAI (2023).
[10] Zhang, W., Zhu, Y., Chen, M., Geng, Y., Huang, Y., Xu, Y., Song, W. and Chen, H., 2023. Structure Pretraining and Prompt Tuning for Knowledge Graph Transfer. In: WWW (2023).

Acknowledgement

We would like to thank many of colleagues, especially Prof. Huajun Chen from Zhajiang University and Prof. Ian Horrocks from University of Oxford, for their discussion, suggestions and all the other support.

This work is funded by the EPSRC projects ConCur (EP/V050869/1), Samsung Research UK, NSFCU19B2027/91846204, Zhejiang Provincial Natural Science Foundation of China (No. Q23F020051), the joint project DH-2022ZY0012 from Donghai Lab and the Chang Jiang Scholars Program (J2019032).