A Tutorial on Hypergraph Neural Networks:

An In-Depth and Step-by-Step Guide

Part 1. Introduction

Sunwoo Kim* Soo Yong Lee* Yue Gao Alessia Antelmi Mirko Polato Kijung Shin
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Part 1. Introduction

Part 1. Part 2. Part 3. Part 4. Part 5. Part 6.

Introduction Inputs Message Training Applications Discussions
Passing Strategies

The slides are available at [Link]

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 2
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Presenter

Kijung Shin.
Associate Professor @
KAIST

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 3

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Higher-Order Interactions are Everywhere

• [Example 1] Group chats on social messaging apps

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 4

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Higher-Order Interactions are Everywhere (cont.)

• [Example 2] Co-authorship of researchers

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 5

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Higher-Order Interactions are Everywhere (cont.)

• [Example 3] Co-purchase of items

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 6

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Higher-Order Interactions are Everywhere (cont.)

• [Example 4] Interactions of proteins

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 7

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Hypergraphs Model Higher-Order Interactions

• Higher-order interactions are commonly modeled as a hypergraph.
• A hypergraph consists of a node set and a hyperedge set.
• A hyperedge (i.e., a subset of nodes) models a higher-order interaction.

written by
A B

written by
A B C

written by
A B C D
Co-Authorship
A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 8
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Hypergraphs Model Higher-Order Interactions

• Higher-order interactions are commonly modeled as a hypergraph.
• A hypergraph consists of a node set and a hyperedge set.
• A hyperedge (i.e., a subset of nodes) models a higher-order interaction.

written by
A B A B C D
Node set
written by
A B C

written by
A B C D
Co-Authorship Hyperedge set
A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 9
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Hypergraphs Model Higher-Order Interactions

• Higher-order interactions are commonly modeled as a hypergraph.
• A hypergraph consists of a node set and a hyperedge set.
• A hyperedge (i.e., a subset of nodes) models a higher-order interaction.

written by
A B A B C D
A B
Node set
written by D
A B C
C
written by
A B C D Co-authorship
Co-Authorship Hyperedge set hypergraph
A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 10
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Machine Learning Tasks on Hypergraphs

• There are various machine learning tasks on hypergraphs.
• The first example is a node classification task.

[Nodes] Messaging app users

[Hyperedge] A group of users

involving in the same group chat

Messaging app hypergraphs

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 11
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Machine Learning Tasks on Hypergraphs (cont.)

• There are various machine learning tasks on hypergraphs.
• The first example is node classification.

Supporting
Football Club B
Supporting
Football Club A
Unknown

Which club does each user support?

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 12
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Machine Learning Tasks on Hypergraphs (cont.)

• There are various machine learning tasks on hypergraphs.
• The first example is node classification.

The node classification task formalizes this user profiling.

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 13
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Machine Learning Tasks on Hypergraphs (cont.)

• There are various machine learning tasks on hypergraphs.
• The second example is hyperedge prediction.

[Nodes] Metabolites

[Hyperedge] A set of
metabolites involved in
the same chemical reaction.

Metabolic reaction hypergraph

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 14
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Machine Learning Tasks on Hypergraphs (cont.)

• There are various machine learning tasks on hypergraphs.
• The second example is hyperedge prediction.

Would these three metabolites

react together?

Metabolic reaction hypergraph

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 15
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Machine Learning Tasks on Hypergraphs (cont.)

• There are various machine learning tasks on hypergraphs.
• The second example is hyperedge prediction.

This hyperedge
would exist.

This hyperedge
would not exist.

The hyperedge prediction task formalizes this reaction prediction.

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 16
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Machine Learning Tasks on Hypergraphs (cont.)

• There are various machine learning tasks on hypergraphs.
• More examples include anomaly detection, ranking, and alignment.

E.g., fraud in
social messaging apps

Messaging app hypergraphs

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 17
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Machine Learning Tasks on Hypergraphs (cont.)

• There are various machine learning tasks on hypergraphs.
• More examples include anomaly detection, ranking, and alignment.

E.g., most influential user

in social messaging apps

Messaging app hypergraphs

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 18
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Machine Learning Tasks on Hypergraphs (cont.)

• There are various machine learning tasks on hypergraphs.
• More examples include anomaly detection, ranking, and alignment.

Facebook messenger hypergraph WhatsApp hypergraph

Do et al., Unsupervised Alignment of Hypergraphs with Different Scales, KDD 2024 19
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Hypergraph Neural Network (HNN)

• Hypergraph neural networks (HNNs) are family of neural networks
specialized for machine learning on hypergraphs.
B 1 3
1 4
2 4
3
2 A B
A
Hypergraph neural
Hypergraph and node network (HNN) Node (and hyperedge)
(and hyperedge) features embeddings

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 20

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Hypergraph Neural Network (cont.)

• Hypergraph neural networks (HNNs) have achieved state-of-the-art
performance in many machine learning tasks on hypergraphs.

Hypergraph Node Classification Leader Board

Dataset: DBLP Dataset: Trivago Dataset: House

1. HNN 1. HNN 1. HNN

2. GNN 2. SVM 2. GNN
… … …

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 21

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Why Hypergraph Neural Networks?

• For hypergraph learning, one might consider using graph neural networks:
• [Step 1] Represent higher-order interactions as a pairwise graph.
• [Step 2] Apply graph neural networks to the pairwise graph.

B
1 4
3
2
A

Hypergraph
A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 22
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Why Hypergraph Neural Networks? (cont.)

• For hypergraph learning, one might consider using graph neural networks:
• [Step 1] Represent higher-order interactions as a pairwise graph.
• [Step 2] Apply graph neural networks to the pairwise graph.

B
1 4 1 3
3
4
2 2
A

Hypergraph Pairwise graph

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 23
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Why Hypergraph Neural Networks? (cont.)

• For hypergraph learning, one might consider using graph neural networks:
• [Step 1] Represent higher-order interactions as a pairwise graph.
• [Step 2] Apply graph neural networks to the pairwise graph.

B
1 4 1 3
3 GNN
4
2 2
A

Hypergraph Pairwise graph

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 24
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Why Hypergraph Neural Networks? (cont.)

• However, expressing higher-order interactions with a pairwise graph may
cause an information loss [Zhou et al., 2006].

1 2 1 2

3 Resulting in the
3
Resulting
same in the
structure.
same structure.
1 2 1 2

3 3

Zhou et al., Learning with hypergraphs: Clustering, Classification, and Embedding, NeurIPS 2006 25
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Why Hypergraph Neural Networks? (cont.)

• Such information loss can result in a significant performance degradation.
→ Hypergraph neural networks are crucial for hypergraph learning.

Cora-CA dataset Walmart dataset

GNN models
HNN models

Node classification performance on hypergraph benchmark datasets

Chien et al., A Multiset Function Framework for Hypergraph Neural Networks, ICLR 2022 26
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Tutorial Overview
• In this tutorial, we provide an overview of how hypergraph neural networks
are designed, trained, and utilized for various applications.

Design choice Training strategy Application

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 27
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Tutorial Overview (cont.)

• The remainder of our tutorial is divided into the five parts.

We are now
here!

Part 1. Part 2. Part 3. Part 4. Part 5. Part 6.

Introduction Inputs Message Training Applications Discussion
Passing Strategies

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 28

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Tutorial Overview (cont.)

• [Part 2] We cover inputs of hypergraph neural networks.
• 2.1. How can hypergraph structures be represented?
• 2.2. What input features can be used for nodes and hyperedges?
1
A
4 2
1
3 B 3
2 B
A 4

Hypergraph Node and hyperedge

structure features
A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 29
 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Tutorial Overview (cont.)

• [Part 3] We cover message passing in hypergraph neural networks.
• 3.1. Whose messages to aggregate
• 3.2. What messages to aggregate
• 3.3. How to aggregate messages

? ? ? ?

? ? ? ? ?

? ? ? ?

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 30

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Tutorial Overview (cont.)

• [Part 4] We cover training strategies for hypergraph neural networks.
• 4.1. Task-agnostic training
• 4.2. Task-targeted training

Task 1
Task 1
Task 2
Task 3 100

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 31

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Tutorial Overview (cont.)

• [Part 5] We cover practical applications of hypergraph neural networks.
• 5.1. Recommender systems
• 5.2. Bioinformatics and medical science
• 5.3. Time series analysis
• 5.4. Computer vision

A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 32

 S. Kim*, S. Lee*, Y. Gao, A. Antelmi, M. Polato, K. Shin

Tutorial Overview (cont.)

• [Part 6] We discuss open questions related to hypergraph neural networks.
• 6.1. What are theoretical foundation of HNNs in hypergraph learning?
• 6.2. When is using HNNs especially advantageous?
• 6.3. How should HNNs encode more complex hypergraphs?
• 6.4. How to utilize large language models to empower HNNs.
When?_ 5
GNNs 4 4 5
4
Transformers HNNs 1 3 1 3
2 1 3
MLPs 2

Advantages of Time
HNN Theory HNNs Complex Hypergraphs HNNs
A Tutorial on Hypergraph Neural Networks: An In-Depth and Step-by-Step Guide 33