Model aggregation techniques in federated learning 2024-25

CHAPTER 1
INTRODUCTION
[Link]
Federated learning (FL) is a distributed machine learning (ML) approach that enables models to be
Trained on client devices while ensuring the privacy of user data. Model aggregation, also known as
Model fusion, plays a vital role in FL. It involves combining locally generated models from client
devices into a single global model while maintaining user data privacy. ML’s capacity to uncover
concealed information and identify non-linear patterns in data has provided significant
benefits to scientific research and real- life applications. Consequently, it is increasingly being
utilized in industries such as finance, healthcare, and business. However, while ML brings
substantial advantages, its data-driven nature has raised concerns regarding privacy and
security. This is primarily because the potential for data leakage or illicit data collection poses
security risks within the ML space. In traditional artificial intelligence (AI) technologies,
such as ML itself, the data used for training models are usually concentrated in the data
center. Large datasets have enabled incredible advances in ML. Once these data centers storing
a large amount of data are attacked, the consequences are immeasurable. Furthermore, data
are frequently personal or proprietary and are not intended to be shared, making privacy a
critical concern and roadblock to centralized data gathering and model training. Based on the
above-made premise, with the purpose of protecting privacy, a distributed training approach,
federated learning (FL) came into being.
Federated learning represents a novel paradigm in ML that aims to facilitate the training
of high-quality models by coordinating multiple clients or devices, all while preserving the
privacy of their respective local datasets. The foundational framework for FL was initially
proposed by the Google team and since then, it has gained increasing popularity among
researchers. This is primarily due to its inherent ability to provide enhanced privacy
protection compared to traditional ML approaches.
As research interest in FL has grown, several variations of the initial server–client
model have emerged. These include fully decentralized FL, where only clients are
involved as well as FL built on blockchain technology, among others. In the most
common setup, FL consists of a central server and multiple clients. The server’s role is to
coordinate the collaborative training process, while the clients act as individual
participants. These clients can range from small devices with computational capabilities

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 1

Model aggregation techniques in federated learning 2024-25

(such as IoT devices, mobile phones, or computers) to large organizations or institutions.

Importantly, each client retains its own local dataset and does not share it with other
participants, which forms the fundamental premise of FL.
Fig. 1 shows an overview of the general FL framework. In each communication round,
the following steps occur: first, each client

Fig. 1. Overview of the general FL framework. During the tth communication round, each
client downloads the latest global model from the server for initialization ① and uses its
own local dataset for iterative training ②. Then, the updated weights are uploaded to
the server ③ which performs model aggregation ④ to generate a new global model.
These steps are repeated for each communication round.

downloads the latest model from the server to initialize their local model. Second, clients
employ their own local datasets to perform training and update their respective local
models. Subsequently, the clients send their updated models back to the server. Finally,
the server performs model aggregation on the received local models, resulting in a new
global model. These steps are repeated until communication ceases.
Compared to traditional ML approaches where all data is collected centrally, FL ensures
that local data remains on the client devices. This characteristic provides enhanced privacy

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 2

Model aggregation techniques in federated learning 2024-25

and security guarantees. While FL can be considered a form of distributed ML, it also
presents unique challenges and issues to address, such as the statistical heterogeneity of
data and device heterogeneity. Statistical heterogeneity refers to the fact that the data
distributed among clients often have different distributions, deviating from the assumption
of independent and identically distributed (IID) data. This heterogeneity can lead to
challenges such as reduced model accuracy and slower convergence speed. Device
heterogeneity, on the other hand, pertains to variations in computing capabilities and
transmission speeds among different clients, which can impact the operational efficiency
of FL to varying degrees.
Undoubtedly, FL has gained significant traction in various practical applications,
particularly in the realm of edge computing. The proliferation of edge computing has been
enabled by the enhanced computational capabilities of modern devices like mobile phones,
wearables, drones, and autonomous vehicles. By integrating FL technology into the edge
network, local data can be stored on terminal devices, local training can be conducted,
and the data can subsequently be aggregated and updated by a central server. This
approach addresses the imperative need for data privacy on devices while effectively
coordinating a large number of remote devices.
A review of relevant literature on FL reveals that a substantial portion of the research
has focused on applying FL to diverse do- mains such as healthcare , smart transportation
wireless networks , and the Internet of Things (IoT) Several systematic federated
frameworks have been implemented, including the mobile device federated system .
Researchers have dedicated considerable efforts to studying FL, encompassing a broad
range of topics, including data distribution patterns advancements in model aggregation
and practical applications of FL. Since the introduction of the FedAvg algorithm,
which is based on weighted averaging, by McMahanet al an increasing number of studies
have focused on exploring aggregation strategies in FL. Thus, the motivation behind this
paper is to conduct a systematic and com- prehensive survey of existing model
aggregation methods in the field of FL, aiming to provide valuable references for
researchers and practitioners. Currently, only a few surveys are available, such as, which
discuss the crucial issue of model aggregation in FL. However, these surveys cover only
a limited number of research works and techniques. Recognizing the significance and
effectiveness of model aggregation techniques, we conducted a systematic literature
review of model fusion methodologies employed in FL from 2017 to 2023. The objective

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 3

Model aggregation techniques in federated learning 2024-25

of this study is to identify the scope, trends, and methods used in the field of model fusion,
ultimately enhancing our understanding of this domain.
1.1 The contributions of the survey

In recent years, the remarkable growth of FL and its applications has led to an upsurge
in related surveys and reviews. provides a summary of recent surveys that have garnered
significant citations in the field of FL. Notably, the papers [6,19] offer introductions to the
fundamental concepts of FL and discuss the challenges it faces. Meanwhile, [20,21] focus
on security and privacy concerns in FL, summarizing potential risks and vulnerabilities in
the context of FL. And, conducts a survey on personalization in FL, highlighting the
impact of non-IID data on federated models. Furthermore, delves into the exploration of
non-IID data in FL. The authors of present a summary of relevant consensus protocols,
platforms, and real-world cases in FL. Moreover, provide reviews on FL research
applications, covering areas such as healthcare, edge networks, 5G, and IoT.
However, to the best of our knowledge, while there are numerous reviews available that
focus on various aspects of FL, such as privacy and security issues, non-IID data, and
applications in different domains, the topic of model aggregation has not received the
attention it deserves. Despite the claim by to be a review of model aggregation, it merely
provides a brief introduction to several classic aggregation methods without conducting a
comprehensive taxonomy or summary.
This highlights the gap in the existing literature, further emphasizing the need for a
systematic and comprehensive review specifically focusing on model aggregation
techniques in FL. The present study aims to fill this gap and provide a thorough
examination of the various methodologies employed in model aggregation within the
context of FL.
1.2The structure of the survey
This survey is organized as follows: firstly, in Section 2, we outline the systematic
approach employed to retrieve research work on model aggregation in FL, while Section
3 provides a com- prehensive introduction to FL, covering its fundamental concepts and
principles. Section 4 presents a detailed survey and taxonomy of model aggregation
methods in FL. In Section 5, we explore application scenarios related to FL from the
perspective of model aggregation. Section 6 delves into the challenges currently faced by
FL, including statistical heterogeneity of data, communication bottlenecks, and security t

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 4

Model aggregation techniques in federated learning 2024-25

CHAPTER 2
Research Methodology
To evaluate the applications and impact of aggregation methods in FL, a systematic
literature review was conducted. This involved following a systematic approach to
identify and analyze pertinent studies in a specific area of interest, ensuring that the review
process was scientific, transparent, and reproducible. Expert-defined guidelines were
utilized to ensure a comprehensive and effective systematic literature review. Specifically,
the guidelines proposed by Siddaway et [Link] the PRISMA statement by Moher et al.
were studied and adhered to. The Siddaway et al. proposal outlines an eight-step process
for conducting a systematic literature review, including formulating research questions,
defining inclusion/exclusion criteria, searching the literature, screening studies,
extracting data, assessing study quality, analyzing and synthesizing results, and reporting
findings. On the other hand, the PRISMA statement provides a checklist for reporting
systematic reviews and meta-analyses, covering items such as the title, abstract,
introduction, methods, results, discussion, and funding.
In the subsequent sections, we present the search strategy and selection criteria
employed to identify relevant studies for this systematic literature review. The research
questions, as previously formulated in the introduction section, served as a guide
throughout the review process.

2.1 Search strategy

At the beginning of the literature search, we decide to utilize the Scopus and Web of
Science databases, given their comprehensive coverage of papers. As mentioned earlier,
the concept of FL was first introduced by the Google team in October 2016 and the
FedAvg algorithm was defined by McMahan et al. in 2017 .These works established
the foundation for the concept of FL as intended in this survey. Therefore, our literature
search was limited to publications between 2017 and June 2023, encompassing formal
publications and preprints, while excluding non-English articles.
To ensure comprehensive coverage of relevant literature, the primary search term
‘‘federated learning’’ was used to search the titles, abstracts, and keywords of the
literature. Furthermore, in order to account for the possibility that some researchers may
use the terms ‘‘fusion’’ and ‘‘aggregation’’ interchangeably, the sec- ondary search term

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 5

Model aggregation techniques in federated learning 2024-25

‘‘model aggregation’’ OR ‘‘model fusion’’ was employed to search all sections of the
literature. It is important to note that despite using these search keywords, it is possible
that not every relevant publication was captured without omission. This is because, in
the early stages of FL development, some researchers preferred to name their proposed
frameworks instead of explicitly using the term "model aggregation’’. Examples of
such frameworks include FedProx, FedNova, Scaffold, among others. To mitigate the
risk of missing articles that did not explicitly mention the keywords, we also searched
the references of the retrieved papers for further relevant publications.

Fig. 2. The structure of the survey. The survey is organized into several sections, starting
with an overview of the methodology employed for conducting the study. Following
that, it introduces the concept of aggregation methods in FL, providing readers with
essential background information.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 6

Model aggregation techniques in federated learning 2024-25

Fig. 3. The systematic literature review conducted in this study follows a research
article retrieval methodology that consists of four stages: identification, screening and
eligibility, and inclusion. In the identification stage, the appro- priate keywords were
determined to retrieve research articles relevant to the research topic under investigation.

In the screening and eligibility stages, specific criteria were established to screen and
exclude literature that did not meet the research needs and objectives. Finally, in the
inclusion step, the literature that met the criteria and requirements of the study was
determined and included in this paper.

2.2 Selection criteria

The literature search began by conducting a comprehensive search using key terms,
which resulted in finding 495 articles on Scopus and 597 articles on Web of Science;
duplicates were excluded. Additionally, it was acknowledged that some of the initially
retrieved literature may not be relevant to the research topic. To address this, a set of
criteria was developed for screening the literature. Review articles and surveys were

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 7

Model aggregation techniques in federated learning 2024-25

excluded from the analysis, as the focus was specifically on experimental articles. Next,
the titles and abstracts of the articles were scanned to exclude those that did not mention
model aggregation design or research. Finally, the full text of the remaining articles was
reviewed, and articles that only used existing methods in the model aggregation stage
without proposing any new ones were excluded. Following this screening process, a
total of 933 arti- cles were eliminated, resulting in a final set of 201 articles that were
deemed suitable for inclusion in the survey. The literature selection process is depicted in
Fig. 3, comprising four stages: identification, screening, eligibility, and inclusion.
Initial statistics have been compiled for the 201 selected arti- cles. Fig. 4 presents the
distribution of research articles focusing on model aggregation techniques in FL from
2017 to June 2023. It indicates that the majority of published articles have been released
in the past two years (2021 to the present), indicating the active and current nature of
research in model aggregation in FL. Out of the screened papers, 113 are from IEEE
Computer Society, encompassing various field journals such as IEEE Conference on

Computer Vision and Pattern Recognition

Fig. 4. Year-wise distribution of papers on the subject area of model aggregation in FL.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 8

Model aggregation techniques in federated learning 2024-25

CHAPTER 3
FEDERATED LEARNING
3.1. Principles of FL
FL generally refers to a distributed ML process deployed on multiple clients. The process involves N
clients {C1,C2,...,CN} indexed by k, each with its own local dataset {D1,D2,...,DN}, which is kept
locally, and data cannot be shared between clients or collected by a third party. Typically, a server
coordinates different clients and their training. FL involves three key steps:

(1) Initialization: at communication round, the clients download the latest model wt from the server for
initialization;
(2) Local training: each client Ck performs iterative training based on its own local dataset Dk and hyper
parameter η. The local model wt is updated to wt+1 k after certain training epochs according to wt+1 ←
wt+1 wt k (η,Dk), and the n send to the server.
(3) Model aggregation: the server performs model aggregation on received local models, and updates the
global model wt+1 glob ←−Agg (wt+1 k ; k∈[1,...,N]). In this way, FL enables multiple clients to
collaborate on training a model without sharing their data, which is especially useful for privacy-sensitive
applications.
3.2 Features of FL
This section covers unique features that set FL apart from traditional centralized ML
approaches. These distinguishing features include aspects such as data and device
heterogeneity, as well as the specific topology of the federated scenario.

3.2.1 Data distribution

In FL, one of the major challenges is the presence of statistical heterogeneity in the data,
also referred to as non-IID-ness. Unlike traditional centralized ML approaches that assume
independent and identically distributed (IID) data structures, FL data often exhibits
statistical heterogeneity due to variations in client distribution, data generators, or other
factors. This can result in local models that are biased towards the distribution of the
local dataset, which can cause a decline in the performance of the aggregated model.
❖ Three primary sources of non-IID-ness can be identified:
[Link] distribution skew: Label distribution skew refers to the variation in the

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 9

Model aggregation techniques in federated learning 2024-25

distribution of labels, denoted as P (y), across different clients. In typical FL

experimental setups, datasets are sorted by labels and assigned to clients in a round-robin
fashion. The MNIST dataset is assigned to clients, where each client is allocated only
two out of the total ten labels. This means that client i may have data labeled as ‘1’
and ‘2’, while client j has handwritten data labeled as ’3‘and ’4’, with no label overlap
between clients. This allocation method is commonly used and straightforward.
Furthermore, researchers commonly employ the Dirichlet distribution Dir(β), which
is based on the Bayesian prior distribution, to simulate the distribution differences in FL
data . In this context, the concentration parameter β plays a crucial role in controlling
the non-IIDness of the data. A smaller β value corresponds to higher data heterogeneity.
By manipulating β, researchers can flexibly generate various data distributions to
investigate the effects of non-IIDness on FL models.

[Link] distribution skew: Feature distribution skew refers to the variation in the
distribution of features, denoted as P (x) or input data, across different clients in FL.
This variation can lead to discrepancies in the representation of data for the same class
across different clients. To address feature distribution skew, researchers have explored
various methods. Some approaches involve adding noise to the dataset or using federated
datasets specifically designed to mitigate this issue. The Federated Extended MNIST
(FEMNIST) dataset is another example of a dataset created to address feature
distribution skew in [Link] includes 62 different handwritten characters,
comprising 10 numbers, 26 lowercase letters, and 26 uppercase letters. It partitions
the data from the Extended MNIST (EMNIST) dataset, which consists of characters
written by different authors with varying styles and degrees of sloppiness, thus skewing
the feature distribution.

[Link] skew: Quantity skew refers to the unequal amount of data that different clients have
in a FL setting. It is not feasible to ensure that every client has the same amount of data,and in
practice, the data is often randomly partitioned among clients. For instance, in, the CIFAR-
10datasetis partitioned among clients with different amounts of training data, ranging from400
to1600 examples per client. The amount of data each client has can significantly affect the
performance of the model, and this skew must be taken into account when designing and
evaluating FL algorithms.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 10

Model aggregation techniques in federated learning 2024-25

3.2.2 Cross-device and cross-silo

FL can be categorized into cross-device and cross-silo based on the difference in client size.
Cross-device FL is often employed in distributed mobile networks, where numerous similar
devices act as clients. Real-world examples of cross-device FL include federated systems based
on the Internet of Vehicles, where each vehicle independently collects its own driving data (such
as captured images and current coordinates). On the other hand, cross-silo FL involves large
organizations or institutions as clients, with fewer clients compared to cross-device FL. In this
case, different companies may collaborate to build a federated system, where each company
maintains local data and performs model training based on their respective databases

Fig 5 : The two primary network structures for FL: centralized FL and decentralized FL.
In a centralized FL system (a), there is a server positioned at the center, forming a star
network structure. Multiple clients connect to this central server for model aggregation
and synchronization. On the other hand, in a decentralized FL system (b), there is no
central server. Instead, clients directly communicate with each other in a peer-to-peer
(P2P) manner, creating a mesh network structure. Decentralized FL is specifically
designed to mitigate the presence of untrusted servers and offers advantages such as
increased resilience to network failures and communication delays.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 11

Model aggregation techniques in federated learning 2024-25

3.2.3 Network structure

During the initial stages of FL design, a common depiction involved a central server positioned at
the center, surrounded by multiple clients. In this setup, the central server was responsible for the
model aggregation phase, while each client performed local model training using its own data.
This configuration, known as centralized FL, typically employed a star topology. As illustrated in
Fig. 5 the central server was situated at the center, with all other clients connected to it. In
contrast, there exists a distinct federated configuration referred to as decentralized FL, which
adopts a mesh topology. In this structure, there is no central server. Each client in the system
conducts local model training based on its private data. Subsequently, clients select other clients
to communicate with, and local models are exchanged or fused in a peer-to-peer (P2P) manner.
This decentralized approach facilitates model aggregation and updates without relying on a central
server, thereby mitigating the presence of untrusted servers.

3.2.4 Data partition

Federated learning can be classified into three categories based on the distribution of
feature space and sample space: horizontal FL, vertical FL, and federated transfer learning.

• Horizontal Federated Learning

Horizontal FL, also known as sample-based FL, involves clients sharing the same feature
space but different sample spaces. This means that data from different clients may be sampled
from different objects, such as A, B, C, etc., but all share the same characteristics, such
as color, as shown in Fig. 6(a). For example, a study using FL to detect COVID-19
infection used chest CT images as training data for each client, which were sampled from
people of different ages and genders but had the same feature space.

• Vertical Federated Learning

Vertical FL, also referred to as feature-based FL, is a situation where the training data used
by all clients participating in training have the same sample space but different feature
spaces. This means that the data is sampled from the same object (object A), but different
features are distributed among different clients, as depicted in Fig. 6(b). The primary goal
of vertical FL is to align overlapping data samples between different clients, and it is
commonly used in scenarios where non- competing companies or organizations, such
as financial institutions, e-commerce platforms, and advertising companies with different

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 12

Model aggregation techniques in federated learning 2024-25

data characteristics, collaborate to train a shared learning model. By leveraging vertical

FL, these organizations can collectively develop a model that facilitates personalized
shopping experiences for online shoppers within the same region, sharing a common
sample space. For instance, financial institutions can provide prepaid funds to users when
they make purchases on online shopping plat- forms, while advertising companies can
offer personalized suggestions based on users’ purchasing behaviors. This collaborative
model allows for the integration of different data characteristics from various sources,
enabling delivery of enhanced personalized services to users

• Federated transfer learning

Federated transfer learning involves data from different participants that not only differ in
samples but also in feature spaces. Fig. 6(c) illustrates the schematic diagram of federated
transfer learning, where local data comes from different objects (object A, B, C...) with different
features (different colors). In this approach, common representations between different feature
spaces are typically learned from a limited set of common samples. These common
representations are then applied to the sample prediction task with only one-sided features. An
example of this is FedHealth, which is a framework for researching wearable healthcare using
federated transfer learning. In this framework, the cloud server trains the cloud model based on the
basic dataset. After obtaining the cloud model, transfer learning is performed in combination with
the client’s local data to establish a personalized local model.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 13

Model aggregation techniques in federated learning 2024-25

CHAPTER 4
Model aggregation in FL

4.1 Definition of model aggregation

In FL, model aggregation refers to summarizing model parameters from all parties in
each round of communication to form an updated global model. Privacy protection is
achieved by aggregating model parameters instead of raw training data. Model
aggregation can be divided into two types: parameter based aggregation and output-based
aggregation, based on the objects being aggregated. In parameter-based aggregation, train
able parameters of the local learning model, such as the weight parameters and gradients of
the deep neural network, are the objects of aggregation. In each round of communication,
local models are shared in parameters/gradients after iterative training based on their local
dataset

Output-based aggregation focuses on aggregating the representations of the model, such as the output
logits or compressed sketches. A notable example of output-based aggregation is the Fed mask approach
introduced by Li et al. Fed mask takes into account the limited computing resources of mobile devices in
FL. In this approach, each device learns a binary mask, and the server aggregates the overlapping binary
masks to enhance the computational efficiency of training. By aggregating the binary masks, the

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 14

Model aggregation techniques in federated learning 2024-25

computational burden on individual devices is reduced while still achieving effective model training.
When considering the form of aggregation, in centralized FL, the model aggregation is conducted by the
central server. The central server is responsible for collecting and merging the models from individual
clients to create an aggregated model. In contrast, in decentralized FL, model aggregation occurs through
peer-to-peer (P2P) communication. Typically, one party initiates the communication and collaborates
with neighboring clients or collaborators to aggregate their respective models. This decentralized
approach involves the exchange and fusion of models directly between the participating clients or
collaborators, without relying on a central server for aggregation.
[Link]: Fed Prox is an enhancement to the Fed Avg method aimed at mitigating the problem of local
optimization inherent in SGD-based approaches. The authors posit that performing numerous local
iterative training steps in Fed Avg may cause each client to prioritize achieving its local objective rather
than the global goal, leading to suboptimal convergence or model divergence. In FedProx, a proximal
term is incorporated into the objective function to regulate the influence of local models and ensure
convergence guarantees. Such term, defined as is the I2 norm of the local model
and global model. Here, µ ≥ 0 is the penalty constant of the proximal term, and Fed Prox is equal to Fed
Avg when µ = 0. The local model is pulled towards the global model through the constraints of the
proximal term. The subsequent model aggregation and global model update follow the same process as
Fed Avg.

2 FedNova: The FedNova algorithm, proposed by Wang et al. in , enhances the model aggregation
phase of the FedAvg algorithm to address non-IID-ness. The algorithm introduces a technique to
normalize and scale the local updates from each client based on its local iteration number, before
updating the global model. The update rule for FedNova is defined as follows:

where τeff denotes the effective iteration step. The local updates dt k are calculated by normalizing
the gradients with a non-negative vector and its l1-norm. In the case of using vanilla SGD as the
local solver, at k is a unit vector. The stack of all stochastic gradients received from client k at round
is denoted as .

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 15

Model aggregation techniques in federated learning 2024-25

[Link]: To address the client-drift problem resulting from data het erogeneity, the Scaffold
algorithm [34] suggests employing control variable technology, specifically variance reduction, in the
local update process. This algorithm incorporates client control variable ck and server control variable
c, with the server control variable being the average value of the control variables from all clients.
The disparity between local and global control variables is utilized to rectify gradient updates during
local training. The update procedure for the control variables in Scaffold is as follows:

where ’’+’’ denotes update and ηl is learning rate. e is the number of local update steps, and N
represents the total number of clients.

[Link]: MOON aims to minimize the discrepancy between local models and the global model by
incorporating a model con trastive loss as a regularization term. The model contrastive loss serves as a
metric for quantifying the dissimilarity between the local model and the global model. Its definition is as
follows:

The model-contrastive loss used in MOON is defined as a function of three terms: the global model wt
glob, the previous round model w prev k , and the current local model wt k. Also, sim represents the
cosine similarity, and temperature parameter τ is used to control the sharpness of the loss function. The
purpose of the model-contrastive loss is to encourage the local models to be close to the global model
while avoiding overfitting the current local data.
5. Zeno: To prevent Byzantine faults in FL, the Zeno algorithm uses a stochastic zero-order oracle to
score each candidate client. The scoring is based on the loss function, and is used to select high-scoring
clients for aggregation. More specifically, Zeno defines the Stochastic Descendant Score (SDS) for any
gradient update. The SDS score measures how much the candidate client contributes to reducing the
loss of the global model, while penalizing large updates that may cause instability. The high-scoring
clients are then selected for aggregation, and their updates are combined to form a new global model.
This approach allows Zeno to select reliable clients and improve the robustness of the learned global

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 16

Model aggregation techniques in federated learning 2024-25

model against Byzantine behavior.

5. Per-FedAvg: Per-Fed Avg combines model-agnostic meta-learning (MAML) with FL to produce
personalized local models. MAML first trains the initial parameters of the model, and then uses a small
amount of data to perform one or more gradient descents on the new task, enabling the model to achieve
good performance. For more information on MAML, please refer to . In Per-FedAvg, the local model
is up dated using one-step gradient descent of the loss function with the objective of finding an initial
model (i.e., meta model) for each client.

Fig. 7. The synchronous aggregation in FL. The diagram depicts the synchronous aggregation
process, where the server carries out model aggregation in each round after receiving updates
from all participating clients.
4.3. Taxonomy of model aggregation
As FL research advances, the possibilities for designing model aggregation methods seem
almost limitless. There are various aggregation techniques that serve different purposes, such as
enhancing the performance of the federated model, minimizing communication overhead, and
addressing data security and privacy concerns. This paper categorizes the aggregation techniques into
four types based on their aggregation form: synchronous, asynchronous, hierarchical, and robust
aggregation.
4.3.1. Synchronous aggregation
In synchronous aggregation-based FL research, model aggregation occurs after all client updates have
reached the server, and the latency or lag experienced on the client side is generally disregarded. Fig.7
illustrates the schematic diagram for synchronous aggregation. In algorithms like FedAvg, for example,

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 17

Model aggregation techniques in federated learning 2024-25

the server randomly selects clients to participate in training during each round of communication, and
their model parameters are included in the aggregation process. The synchronization occurs without
explicitly considering the latency or potential delays experienced by individual clients. Even though
the server performs synchronous aggregation of client updates, discrepancies can still arise during the
aggregation process, giving synchronous-based FL research its unique advantages. To tackle data
heterogeneity in the synchronous scenario, recent research works often focus on (1) Adding
regularization terms to constrain local models; (2) Adjusting aggregation weights to assign appropriate
shares for different clients; (3) Compressing the model to improve its expressiveness.
• Regularization terms introduction
In the case of non-IIDness, the optimization objective of the local model differs from that of the
global model, which can lead to increased convergence time for the global model or even cause it to
diverge. For example, in the FedDyn approach , a dynamic regularizer is suggested for each client in each
round to minimize the disparities between local models and the global model. This is due to the
fundamental difference between the minimum value of the local-device empirical loss and that of the
global empirical loss. In the FL system called FedDist, regularization terms are based on Euclidean
distances between the local models and the global model. On the other hand, in the FedAAR approach
introduced by , cosine distances are employed as a measure of dissimilarity between the local models and
the global model.
• Weights adjustment
When aggregating models using the parameter-weighted average method, the typical approach
is to determine the aggregation weights based on the amount of data each client possesses. However,
this approach can be disadvantageous for data-poor clients. Clients with larger amounts of data have a
greater influence on the quality of the global model, while those with smaller amounts of data may not
receive sufficient attention. To address this issue, re searchers have proposed optimizing the allocation
of aggregation weights. The aim is to ensure a fair distribution of weight by assigning appropriate
shares to different clients. This approach considers clients that are often overlooked, allowing global
models to learn additional knowledge from these previously neglected clients. For instance, in the
FedDisco approach introduced by the authors found that using dataset size as the sole aggregation
weight is suboptimal. Instead, they propose using the difference between local and global category
distributions as a complementary metric for aggregation weights. Similarly, in the work by ,the authors
suggest adaptively assigning different weights to clients based on their contribution in each round. In
their work, Wu et al. propose measuring the contribution of participating clients by contrasting the local
gradient vector with the global gradient vector. They suggest quantizing the weights using a nonlinear

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 18

Model aggregation techniques in federated learning 2024-25

mapping function. Another efficient approach for quantifying weights is based on a scoring system. In
studies such as , reputation scores are introduced for local models and utilized to scale the aggregation
weights. The reputation score of a client is computed based on performance metrics of the local model
in each round of training, providing an assessment of the direct contribution of the local model.
Furthermore, the learning mechanism, such as an attention mechanism, can be employed to learn
learnable parameters in the model for obtaining aggregated weights. Some researchers propose
incorporating an at tention mechanism during model aggregation, where an attention score is learned for
each client, and these attention scores serve as the weight factors. In the Fed LAW frame work, a
learnable approach to aggregation weights is proposed. The authors demonstrate that the L1 norm of the
aggregation weights can be less than 1, indicating that the aggregation weights can be adaptively
adjusted during the aggregation process.
• Knowledge distillation
Knowledge distillation, commonly used in ML, involves training a smaller model (student) to learn
from a larger model. In the context of FL, applying knowledge distillation techniques is referred to as
federated distillation. This technique has the potential to significantly reduce communication costs in FL.
The gen eral framework of knowledge distillation in FL is illustrated in Fig. 8. For instance, the FedGKT
framework proposed by trains smaller networks on edge nodes and employs knowledge distillation to
transmit the learned knowledge to the server. This approach helps reduce communication costs,
particularly when the local model is a large convolutional neural network. Knowledge distillation
techniques have been utilized in various FL architectures to enhance convergence speed and performance.
Some studies, such as, leverage knowledge generated on public datasets to facilitate convergence.
Distillation terms are also added to the local objective function in order to generate personalized models,
as demonstrated in. Furthermore, integrating local knowledge with predictive logits can improve the
performance of general distillation fusion models.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 19

Model aggregation techniques in federated learning 2024-25

Fig. 9 Asynchronous aggregation in FL. In the asynchronous aggregation pro cess, the server
carries out the aggregation in a set manner, as soon as updates from individual clients are received,
without requiring synchronization with all clients
4.3.2. Asynchronous aggregation
Asynchronous FL (AFL) has gained significant attention due to the growing influence of device
heterogeneity in federated systems. AFL allows clients to upload their local updates in a stag gered
manner, which helps mitigate the negative impact of device heterogeneity. In traditional FL, delays caused
by poor network signals or client crashes during training may lead to delays in uploading updates, thereby
increasing the waiting time for the server to receive updates from all clients. The primary goal of
asynchronous aggregation is to accelerate the training process. In fully asynchronous aggregation, the
aggregation takes place as soon as the server receives the local updates from each client, allowing each
client to train independently without waiting for other clients to complete their updates. This
asynchronous approach enhances the efficiency and scalability of FL by reducing the waiting time and
enabling clients to contribute their updates at their own pace. A schematic diagram of asynchronous
aggregation is shown in Fig. 9. Recent research has proposed further improvements, such as (i) dynamic
fusion and (ii) evaluation of the client-side model, optionally for aggregation. In addition, the semi-
asynchronous aggregation between synchronous and asynchronous also brings about the optimization of
the training process.
• Dynamic fusion of local models
Dynamic fusion of local models has emerged as a popular method in FL to reduce overall training time.
While fully asynchronous aggregation performs model aggregation as updates arrive, this approach can
result in low-accuracy global models in each round, necessitating multiple rounds of training to achieve

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 20

Model aggregation techniques in federated learning 2024-25

satisfactory accuracy. To overcome this challenge, researchers have proposed dynamic fusion techniques
based on interval time windows or the number of clients. For instance, FedPA addresses the trade-off
between training time and model performance by dynamically determining the number of aggregations
in each round. Another approach involves handling stale models in the aggregation process , where local
models with staleness exceeding a threshold are excluded from the aggregation. In terms of adaptive
deadline determination, methods such as have been proposed. These methods calculate the expected time
for each round based on the computing resources and channel conditions of each mobile device, enabling
the adaptation of deadlines to improve the overall FL process. In addition to the aforementioned client
selection and dynamic aggregation methods, several other strategies have been proposed to enhance
asynchronous aggregation. These strategies include excluding clients who have not completed their tasks
in the current aggregation round and evaluating the decision to aggregate based on a client’s historical
performance and current status. In the uplink and downlink processes are planned to ensure a balanced
influence and share among all clients. Furthermore, some researchers have proposed employing temporal
weight decay strategies to achieve effective asynchronous aggregation.

• Clients evaluation
The evaluation of client performance plays a significant role in the domain of AFL. Due to varying signal
strengths and limited network resources, not all clients can effectively participate in the aggregation
process. Moreover, the statistical heterogeneity of local models can negatively impact the global model.
Therefore, prioritizing clients with better communication capabilities, higher prediction accuracy, or
models more aligned with the optimal global model is crucial. For instance, in the Eiffel system , a mobile
edge computing framework, clients are selected for aggregation based on relevant metrics such as data
size, computing power, and last update time. An overall index is calculated to determine the priority of
each client. Other methods for client selection include sorting clients based on the gradient update norm,
utilizing radial-basis function , assessing model uncertainty, and considering the Mutual Information ratio
between ground truth and model predictions. Researchers have also proposed bandit learning algorithms
and Lyapunov optimization techniques to address the asynchronous client selection problem. Some
studies employ the Sequential Kalman Filter to rank the parameters uploaded by clients. Additionally, the
quality of clients can be evaluated using various approaches, as demonstrated in works. These evaluation
methods contribute to the selection of suitable clients for aggregation, enhancing the performance and
effectiveness of asynchronous aggregation in FL.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 21

Model aggregation techniques in federated learning 2024-25

• Semi- asynchronous
Semi-asynchronous aggregation serves as an intermediary aggregation method between synchronous and
asynchronous aggregation. In synchronous aggregation, the server must wait for all resulting client
models to arrive before performing aggregation, leading to potential long waiting times. On the other
hand, fully asynchronous aggregation addresses issues like device heterogeneity but may result in
frequent model transfers, consuming significant communication resources. To address these challenges,
re searchers have proposed semi-asynchronous FL mechanisms such as FedSA. With a predefined
communication budget, FedSA allows the server to perform a certain number of aggregations based on
the order of client model arrivals in each round, thus optimizing the trade-off between waiting time and
resource consumption. Similarly SAFA is a semi-asynchronous FL protocol that introduces the
hyperparameter ‘‘lag tolerance’’ to coordinate lagging clients and compensate for the arrival order of
client models. Additionally, proposed a semi-asynchronous aggregation strategy that fixes client selection
in the first round and subsequently selects clients based on priority. Semi-asynchronous aggregation
strikes a balance between synchronous and asynchronous aggregation, effectively improving the round
efficiency of the FL process while minimizing resource waste. It offers a practical compromise to leverage
the benefits of both synchronous and asynchronous approaches in FL.

4.3.3. Hierarchical aggregation

In the context of edge learning, the presence of a large number of edge devices, such as IoT devices, can
significantly impact the efficiency of FL due to frequent model aggregation and the result ing high
communication overhead . To tackle this challenge, researchers have proposed a hierarchical aggregation
approach that incorporates an edge layer to partially aggregate local models from closely related client
devices before further aggregation on the cloud server. This hierarchical approach aims to re duce
communication overhead and the number of model transfer rounds by introducing multiple aggregation
centers. Fig. 10 illustrates the architecture of hierarchical FL, where the hierarchy optimization and client
similarity clustering are two key research focuses.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 22

Model aggregation techniques in federated learning 2024-25

Fig. 10 Hierarchical aggregation in FL. The figure depicts the hierarchical aggregation process
through the federated IoV: vehicles are clustered into different clusters, and upload local models to
the corresponding edge servers (i.e. roadside units), which then upload the models to the cloud for
further global model updates
• Hierarchical optimization
Improving the hierarchical structure of Hierarchical Federated Learning (HFL) can effectively address
resource allocation and communication efficiency challenges. Various approaches have been proposed to
enhance the hierarchical architecture of HFL and optimize its performance. One example is the FogL
architecture proposed in , which adopts a tree-based federated network structure. In this architecture, end
devices are represented as leaves, and the master server acts as the root. By employing multilevel and
multi-stage tree aggregation techniques, FogL mitigates the risk of system overload and improves
resource allocation efficiency. In the context of the Internet of Vehicles (IoV), HFL research has explored
the utilization of roadside units as the middle layer and the road traffic cloud as the cloud aggregator. This
hierarchical approach leverages the infrastructure of the IoV to facilitate communication and coordination
among vehicles. However, t is worth noting that traditional HFL approaches may encounter performance
degradation for highly mobile users in hierarchical wireless networks, as discussed. To address this
challenge, the Mobile-Aware Collaborative Federated Learning (MACFL) algorithm is proposed.
MACFL allows mobile users to download the cluster model from the nearest edge server for local training,
and if they roam to another cell, they can upload updates to a different edge server, thus improving
communication efficiency. Incentive mechanisms can also be introduced to enhance communication
efficiency in HFL. For instance,a data owner competition framework is proposed where participants
compete for cluster head qualification, equivalent to edge aggregators. Rewards are distributed based on

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 23

Model aggregation techniques in federated learning 2024-25

their contributions, incentivizing active participation and efficient communication. Another approach is
MaxQ , which applies game theory to improve matching mechanisms in HFL, enabling efficient
collaboration and resource allocation.
• Clients clustering
The idea of clustering in Hierarchical Federated Learning (HFL) has gained significant attention in recent
research as it offers a promising approach for improving system performance and resource utilization.
Clustering involves grouping clients with similar characteristics or data distributions into clusters, and
performing model aggregation within each cluster. In this context, several clustering-based approaches
have been proposed in the literature . Lin et al. divide clients into clusters based on their communication
capabilities and leverage device-to-device (D2D) communication within each cluster. Periodic global
aggregations are then performed to update the global model. The FedSim framework utilizes the k-means
algorithm to measure similarity between clients and guide the aggregation process. Different distance
metrics, such as L1 (Manhattan distance), L2 (Euclidean distance), and cosine distance, have been
employed to quantify the similarity between clients in clustering-based approaches. Another line of
research focuses on clustering clients based on the distribution of their local data . For instance, the HPFL-
CN framework proposes aggregating edge devices with similar environmental data distributions and
efficiently training personalized models for each cluster using a hierarchical architecture. In the work by
Bao et al. clients are clustered into non-overlapping coalitions based on the distance between their data
distributions and the volume of their data. Each client collaborates only with clients that have similar data
distributions, promoting effective model training. Clients with less data tend to collaborate with a larger
number of other clients to compensate for their limited data.
4.3.4. Robust aggregation
Compared to centralized ML, FL provides greater data privacy and security guarantees. However, recent
research has identified several security vulnerabilities in FL. For example, inference attacks can be
launched by an attacker to infer the data distribution of participants by analyzing the parameters of local
models, while backdoor attacks can be initiated by malicious actors by introducing bad clients to the
global model. These safety risks have spurred researchers to devise strategies to enhance the robustness
of FL models. To ensure secure aggregation throughout the FL process, researchers recommend the use
of various encryption techniques, such as differential privacy and homomorphic encryption. Differential
privacy involves adding random noise to the output to prevent attackers from reverse-engineering
sensitive data, while homo morphic encryption enables basic functional operations to be performed on
encrypted data. In addition, the use of decentralized model aggregation is suggested, for example, by lever

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 24

Model aggregation techniques in federated learning 2024-25

aging blockchain technology or gossip principles. This decentralized approach eliminates the central
server from the system, thereby preventing single points of failure. Other approaches include optimizing
contract mechanisms , using robust stochastic model aggregation, electing a small committee, rather than
assigning sanitization factors or avoiding small group domination. Furthermore, strategies like multi-
party computation (MPC), and trusted execution environment (TEE) demonstrate the ongoing effort to
address security vulnerabilities in FL and enhance the privacy protection of the system.
• Homomorphic encryption-based
In homomorphic encryption (HE) based schemas, the users encrypt their models using the same public
key, which allows the central server to add the encrypted models together using the additive homomorphic
property of the underlying crypto-system. The ownership of the secret key is a crucial point in public-key
crypto-systems used for FL aggregation. Following the taxonomy in, there are three settings of secret key
management. In the first setting, the secret key is shared among all users but kept confidential against the
central server, making the global model public to all participants. Many different crypto-systems have
been adopted in this setting: RSA , Paillier, lattice-based, BGN, ElGamal. In the Second setting, the secret
key is known only by the central server to protect the privacy of the global model . The drawback of this
approach is that the server can decrypt the encrypted models sent by users, thus breaking their model
privacy. Other privacy-preserving methodologies are required to improve privacy guarantees, such as
masking models or the use of a trusted party to manage the secret key. The third one can be seen as a way
to improve the security of the first setting, where a set of users share the same secret key. More than a
threshold number of users must cooperate to decrypt an encrypted message. Crypto-systems such as
threshold Paillier , ElGa mal are used in this setting. This setting provides a higher security guarantee
than the first setting, as the central server must collude with more users to break the security.
• MPC-based
Several works have explored the use of Multi-Party Computation (MPC) to enable privacy-preserving
aggregation and FL training. In these schemes, users distribute their locally trained models to a set of
selected users/servers referred to as agents, who subsequently aggregate them to construct a new global
model. In the authors propose the utilization of a Fast Fourier Transform (FFT) based secret sharing
scheme instead of the traditional Shamir secret sharing scheme. Alternatively, verifiable sharing schemes
have been utilized in. Furthermore, propose an aggregation strategy based on a two-step process where
users first elect a committee, which then receives and aggregates the models shared by the users. Other
method ologies rely on sharing the models between two or more servers to facilitate privacy-preserving
aggregation and FL training.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 25

Model aggregation techniques in federated learning 2024-25

• Blockchain-based

The utilization of blockchain technology, which is a decentralized and non-tamperable distributed a

wide range of advantages for various research fields such as healthcare, finance, and education. In
recent years, more researchers have been exploring the combination of FL with blockchain technology.
In Fig. 11, a blockchain-based architecture for FL is illustrated where a client (e.g., medical devices)
first downloads the current global model from the blockchain for initialization. Subsequently, the client
encrypts and uploads the trained model to the blockchain, which is then downloaded by the miner
server. The server aggregates the models and broadcasts the new global model to other nodes. In a
recent study, each aggregation node conducts a quality test on the local model and broadcasts the
reputation evaluation to the blockchain network. The reward distribution is based on the combination of
client contributions and reputation. This reputation-based reward distribution algorithm, along with
blockchain technology, provides quality assurance for the model.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 26

Model aggregation techniques in federated learning 2024-25

• TEE-based
A Trusted Execution Environment (TEE) is a secure area in the main processor that enables the storage,
processing, and protection of sensitive data and code within a trusted and isolated environment. TEEs
represent a secure counterpart to the Rich Execution Environment (REE), which is the standard operating
system that the device is running. Within FL frameworks, TEEs can be successfully used for secure
aggregation: the users encrypt their locally trained models and transfer them to the REE. The TEE then
receives the encrypted models from the REE, decrypts them, and aggregates them. The output is returned
to the REE for distribution to all users. Several strategies have been proposed to enhance the security and
privacy of FL systems utilizing TEEs. These include distributing trust among several TEEs applying DP
techniques to perturb users’ models before uploading them to the TEE , and utilizing a TEE to add
randomness by shuffling users’ models before uploading them to the central server. Furthermore, a novel
approach for secure aggregation in FL using TEEs is to deploy ML training algorithms inside TEEs
themselves. This approach provides an additional layer of security by ensuring the confidentiality of data
and code.
4.4. The proposed taxonomy
To provide a comprehensive overview of the emerging research methods in FL, we have classified them
into four categories and provided detailed explanations. The taxonomy is depicted in Fig. 12, and further
information can be found in Table 3, which presents the specific details of the taxonomy along with
corresponding literature references. In general, synchronous aggregation is effective in achieving good
model performance when the number of devices in the federated system is relatively small. However, as
the number of clients increases, client heterogeneity becomes more pronounced, leading to longer training
times as clients wait for each other. In such scenarios, asynchronous aggregation becomes a more
favorable choice. Hierarchical aggregation, on the other hand, is particularly suitable for large-scale IoT
applications like the Internet of Vehicles (IoV), where a significant number of devices require
coordination. This approach helps to reduce communication overhead and the number of model transfer
rounds. For applications with high system security requirements, the robust aggregation method is
preferred. This approach addresses security concerns by incorporating tech niques such as parameter
encryption or blockchain technology, which enhance the overall security of the federated system. By
categorizing the research methods and considering their specific characteristics and benefits, FL
practitioners can make informed decisions on selecting the most appropriate model aggregation approach
for their specific requirements and constraints.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 27

Model aggregation techniques in federated learning 2024-25

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 28

Model aggregation techniques in federated learning 2024-25

CHAPTER 5
Applications of aggregation methods

FL’s unique advantages have made it an attractive solution for a variety of fields, such as smart
transportation, financial business, and healthcare. FL can handle data islands while providing strong
privacy protection. As FL gains more attention in real-life applications, including smart healthcare, smart
transportation, smart city, smart industry, etc. In practical applications, selecting the appropriate
aggregation method is crucial since it has a significant impact on the functionality and efficiency of the
designed FL architecture. In the following discussion, we will explore the reasons and benefits of
choosing different aggregation methods for various industries from the perspective of practical
application.
5.1. Model aggregation in smart healthcare
In the healthcare industry, ensuring user privacy is of paramount importance. While the rapid
development of ML has greatly promoted the advancement of smart healthcare, the leak age of medical
data has also caused significant problems for people. To address this issue, multiple means of enhancing
privacy protection can be employed in FL based on robust aggregation, thereby helping to establish a safe
and secure smart medical system. For example, Kumar et al. propose the MediSecFed system, a security
framework for FL in hostile environments. The authors demonstrate that the MediSecFed system is also
robust to poisoning attacks through experimental evaluations on real-world pneumonia datasets. In, the
authors propose an FL framework for protecting medical privacy using blockchain technology and
homomorphic encryption methods. Evaluation experiments on medical images from CT scanners
demonstrate that the framework can strike a balance between privacy and accuracy. In addition, the
synchronization-based aggregation scheme helps to improve the prediction accuracy of the FL model.
Choudhury et al. conducted a study of electronic health data to predict adverse drug reactions. Their
experimental results show that the FL model performs similarly to centralized ML and avoids the
challenges associated with the latter.
5.2. Model aggregation in smart transportation
ML technology has been commonly used to improve transportation, and recently, FL has been gaining
attention in this field. By surveying the relevant literature, we found that several different aggregation
methods are widely adopted by researchers in the field of intelligent transportation, thanks to the different
benefits provided by each aggregation method. In FL studies based on the IoV, hierarchical and

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 29

Model aggregation techniques in federated learning 2024-25

asynchronous aggregation schemes are commonly used due to the large number of vehicles involved and
the need for full coordination. Furthermore, roadside units in IoV can provide intermediate support for
staged aggregation is a FL-IoV framework based on hierarchical aggregation, which realizes dynamic
map fusion technology without data labels. In this framework, roadside units provide labels for local
training, and cloud servers perform model aggregation, vehicle edge computing was studied using FL,
and a client selection approach was used to improve the accuracy and efficiency of model aggregation.
Specifically, by evaluating the local image quality and computing power, a good local DNN model is
selected and sent to the central server. In an asynchronous aggregation method is used for local model
aggregation of vehicles in IoV. Because in each round of communication, a large number of vehicles
upload model parameters to the server in the uplink communication phase, which will cause huge
communication pressure and prolong the training time, they based their framework on the rules of partial
vehicle participation, achieving faster convergence in fewer communication rounds.
5.3. Model aggregation in smart city
FL can be used in smart cities to dynamically control environ mental pollution, which can help protect
the lives and health of urban residents . FL can be used to train machine learning models on data collected
from sensors, such as air quality sensors, distributed throughout a city. By aggregating these models, FL
can provide insights into environmental pollution levels and help improve decision-making on how to
reduce pollution and protect public health. Extensive research has been conducted on the application of
FL in smart cities, and asynchronous aggregation has been found to be a popular method for coordinating
the aggregation of client models in FL. In FL based on asynchronous aggregation, the aggregation of
client models can be dynamically coordinated to improve the operating efficiency of the federated system.
The study by Gao et al. proposes an asynchronous FL model aggregation method called n-softsync for air
quality prediction, which limits the number of local models when aggregation to reduce time overhead.
In the model aggregation step, only n nodes are allowed to upload the latest local model in each round;
the remaining nodes that have not uploaded their local models use their old models for the aggregation.
Another study proposes an asynchronous FL framework for urban environment perception that considers
regional characteristics of monitoring points and uses an asynchronous aggregation strategy to address
different bandwidth requirements and to reduce latency. proposes an edge-intermediary-cloud FL
architecture to achieve prediction tasks in urban environments. In this framework, environmental data are
mapped to different complex network domains. Similarity clustering is performed by extracting low-level
feature representations of each edge server. The intermediary is responsible for coordinating the model
training of the edge servers in each cluster, and periodically uploading the weights to the cloud server for
model aggregation.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 30

Model aggregation techniques in federated learning 2024-25

5.4. Model aggregation in smart industry

In the field of the Industrial Internet of Things (IIoT), research related to FL pays more attention to the
robustness of the system. This is because Industrial Internet of Things (IIoT) involves a large number of
important industrial devices, and if the nodes of the federal network are attacked or damaged, the
industrial system may be paralyzed. Thus, FL based on robust aggregation is more commonly used in
IIoT research . Traditional FL can have a single point of failure when the central server is damaged. To
address this issue, researchers have proposed different methods for secure and decentralized FL. For
instance, a serverless FL framework driven by blockchain was proposed in for distributed ML in smart
grid analysis. The authors used asymmetric cryptography to encrypt models and verified the effectiveness
of the proposed method using power grid simulation datasets. In a framework based on blockchain and
with Intel Software Guard Extensions (SGX) is proposed for simulating the scenario of FL in smart
warehouses. Each of these blockchain nodes hosts a trusted SGX processor for secure aggregation of
models. In this framework, each blockchain node can verify the authenticity of the aggregated model, and
the blockchain consensus mechanism is run to ensure the integrity of the model.
5.5. Model aggregation in other fields
FL has found applications in other domains, such as finance, education, network security, E-commerce
recommender system, natural disaster prediction etc. The efficient model aggregation has brought great
advances to those industries. For example, Imteaj et al. utilize asynchronous FL to predict customers’
financial distress. In this framework, local models learn about customers’ personal information and past
financial status, and accurate predictions are achieved by considering clients’ local models at different
epochs. Furthermore, with the continuous development of network technology and the popularization of
network applications, research on network security issues based on FL has become a research hotspot.
Zhao et al. proposed an intrusion detection system called DAFL. In addition to the inherent data protection
advantages of FL, the dynamic filtering and weighting strategy achieves less communication overhead
and higher detection performance. In recent years, online course systems have gradually become popular
in the education industry and schools, and data leakage risks have also received a lot of attention. An
integrated system of multimedia course recommendation and data-secure FL is proposed in the study,
which keeps a copy of the recommended model on each user device and coordinates the division of model
aggregation communication rounds. Experiments on datasets containing real multimedia courses
demonstrate effective operation in a privacy-preserving mode. As a conclusion, since different application
domains have different characteristics and situations, there are generally one or two main aggregation
methods adapted to meet application requirements. methods in different domains.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 31

Model aggregation techniques in federated learning 2024-25

CHAPTER 6
Challenges And Future Directions
FL has led to the development of several aggregation methods, but there are still some significant
challenges that need to be addressed. Now we will discuss these challenges in detail, focusing on
three aspects: statistical heterogeneity, communication bottlenecks, and secure aggregation. We will
also highlight potential future trends in addressing these challenges.
6.1. Statistical heterogeneity
In FL, a significant challenge in model aggregation is caused by statistical heterogeneity. The
FedAvg algorithm per forms model aggregation by taking a weighted average of local model
parameters from all participating parties. However, this may cause the aggregated global model to
converge poorly or even diverge. To address this issue, some studies have employed the Bayesian
non-parametric mechanism to perform model aggregation through neuron matching and a
combination of local models, as seen in PFNM and Claici et al. In addition, Shukla et al. proposed
the Infogain FedMA algorithm, which uses an information-gain-based sampling method to select
the parameters to be aggregated, paired with probabilistic federated neural matching. However, these
methods have only been applied to relatively simple neural network models, and generalizing them
to complex network models to improve applicability remains a challenge that needs to be further
explored.
6.2. Communication bottlenecks
Model aggregation in FL faces another significant challenge, namely the communication problem.
This problem directly affects the aggregation speed, which slows down the overall training progress.
In a typical federated system, there are usually multiple clients, such as hundreds or thousands of
devices in an IoT-based federated system. During model aggregation, a large number of clients need
to upload their local updates to the same network, which can cause severe communication
congestion problems due to limited network bandwidth. This is also a major challenge faced by FL
in enterprise implementation. While training simple models in FL can reduce the communication
burden, the explosive growth of data in the internet age means that this may not be sufficient to meet
practical application needs. Therefore, training large network models has become a focus of several
studies. Correspondingly, maximizing the use of limited resources and improving aggregation
efficiency under such conditions is an urgent problem that needs to be solved. There are proposed
solutions to address the communication bottleneck in FL. One such solution is FL based on over-

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 32

Model aggregation techniques in federated learning 2024-25

the-air computation (AirComp), which can achieve fast model aggregation. AirComp is a non-
orthogonal multiple access (NOMA) technique that uses the waveform superposition characteristics
of multiple access channels to perform combined calculation of data transmitted by multiple clients.
Ni et al. proposed to use intelligent reflective surfaces (IRS), which is a technique to reconfigure the
wireless propagation environment, to improve signal distortion caused by AirComp. Another
proposed solution is to deploy multiple relays to assist signal transmission and im prove the
performance of aerial model aggregation, as suggested by Lin et al. Additionally, the sixth
generation (6G) wireless communication is expected to be an effective method to address the
communication bottleneck in FL. Compared with previous generations of wireless communications
(4G and 5G), 6G has higher data transmission rates, wider frequency bands, and wider network
coverage . The 6G era is expected to help FL address its model aggregation difficulties, and in turn,
FL can promote the integration of 6G into more IoT industries and AI services.
6.3. Secure aggregation
In FL, security is a critical issue that needs attention, especially in model aggregation. In centralized
FL, the server is assumed to be honest but curious, and the clients are honest. However, the presence
of attackers can easily disrupt this assumption. In FL, the attacker can take active and passive attacks.
Active attacks include model poisoning attacks. Generally speaking, in a model poisoning attack,
the attacker can modify the local model before uploading it to the central server, thereby affecting
the aggregated global model. The idea of filtering out suspect local models for evaluating client-side
updates has emerged in several studies. When the vast majority of clients are honest, filter-based
approaches can lead to good aggregated results. But when there is more than one dangerous client,
there may be a Sybil attack by multiple attackers which will cause greater damage to the model. At
this time, the filtering-based method may also cause the loss of client information, which is a method
that is not worth the candle. How to ensure safe aggregation without losing information is a matter
of balance. On the other hand, passive attack means that the attacker does not change the training
process of FL, but makes inferences by observing updates, such as the inference attack. In an
inference-based attack, the model parameters are reversely deduced, which will lead to the leakage
of private data information. Both participating clients and malicious centralized servers in FL have
the potential to launch inference attacks. Currently, researchers are exploring the potential benefits
of blockchain technology, such as decentralization, traceability, and irreversibility, in addressing
security. Finally, it is essential to educate both clients and developers about the security risks
associated with FL and how to mitigate them.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 33

Model aggregation techniques in federated learning 2024-25

CHAPTER 7
CONCLUSION
In this paper, we have conducted an in-depth and comprehensive survey on the topic of model aggregation
within the framework of Federated Learning (FL), an emerging distributed machine learning paradigm
designed to enhance data privacy and reduce reliance on centralized data storage. As FL continues to gain
traction in both academic research and practical deployments, model aggregation has become a
cornerstone of its architecture, directly influencing the efficiency, scalability, and robustness of federated
systems.
Our motivation for undertaking this work stemmed from the noticeable lack of a systematic taxonomy
and detailed analysis of model aggregation strategies in FL. To address this gap, we began by outlining
our research methodology and providing a foundational overview of FL, followed by an exploration of
the significance of model aggregation in this context. We proposed a novel and structured taxonomy that
categorizes model aggregation techniques into four primary types: synchronous, asynchronous,
hierarchical, and robust aggregation. Each category was analyzed in terms of its operational mechanisms,
advantages, trade-offs, and applicability to different FL scenarios Furthermore, we illustrated the real-
world impact and benefits of effective model aggregation by examining use cases in domains such as
smart healthcare, where sensitive patient data can remain decentralized, and smart transportation, where
edge devices can collaboratively learn traffic patterns without sharing raw data. These examples
demonstrate how tailored aggregation strategies can lead to improved model performance, reduced
communication costs, and enhanced resilience to data heterogeneity and adversarial threats.
In addition to presenting the current landscape, we also identified several critical challenges facing model
aggregation in FL,including issues related to system heterogeneity, straggler effects, communication
efficiency, security vulnerabilities, and robustness against malicious participants. To stimulate further
research, we proposed future directions, such as the integration of adaptive aggregation mechanisms,
cross-device and cross-silo coordination, privacy-preserving techniques, and the application of advanced
optimization and learning theory in the design of aggregation [Link] the best of our knowledge, this
is the first work to present a detailed and categorized taxonomy of model aggregation in FL. Our survey
not only consolidates existing knowledge but also highlights open questions and research opportunities,
providing a valuable reference for both newcomers and seasoned researchers in the field. We hope that
this work will serve as a foundation and inspiration for further innovations and improvements in federated
learning, particularly in optimizing model aggregation strategies to meet the demands of future intelligent
systems.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 34

Model aggregation techniques in federated learning 2024-25

CHAPTER 7
REFERENCES
[1] A. Paleyes, R.-G. Urma, N.D. Lawrence, Challenges in deploying machine learning: a survey
of case studies, ACM Comput. Surv. (2020).
[2] M.I. Jordan, T.M. Mitchell, Machine learning: Trends, perspectives, and prospects, Science 349
(6245) (2015) 255–260.
[3] J. Konečn` y, H.B. McMahan, F.X. Yu, P. Richtárik, et al., Federated learning: Strategies for
improving communication efficiency, 2016, arXiv preprint.
[4] C. Pappas, D. Chatzopoulos, S. Lalis, M. Vavalis, Ipls: A framework for decentralized federated
learning, in: IFIP Networking Conference, IEEE, 2021, pp. 1–6.
[5] S. Ji, J. Zhang, Y. Zhang, Z. Han, C. Ma, LAFED: A lightweight authentication mechanism for
blockchain-enabled federated learning system, Future Gener. Comput. Syst. 145 (2023) 56–67.
[6] T. Li, A.K. Sahu, A. Talwalkar, V. Smith, Federated learning: Challenges, methods, and future
directions, IEEE Signal Process. Mag. 37 (3) (2020) 50–60.
[7] T. Li, A.K. Sahu, M. Zaheer, M. Sanjabi, et al., Federated optimization in heterogeneous
networks, in: Proceedings of Machine Learning and Systems, Vol. 2, 2020, pp. 429–450.
[8] W. Shi, J. Cao, Q. Zhang, Y. Li, L. Xu, Edge computing: Vision and challenges, IEEE Int. Things
J. 3 (5) (2016) 637–646.
[9] W.Y.B. Lim, N.C. Luong, D.T. Hoang, Y. Jiao, et al., Federated learning in mobile edge
networks: A comprehensive survey, IEEE Commun. Surv. Tutor. 22 (3) (2020) 2031–2063.
[10] R. Kumar, J. Kumar, A.A. Khan, H. Ali, et al., Blockchain and homomorphic encryption based
privacy-preserving model aggregation for medical images, Comput. Med. Imaging Graph. 102
(2022) 102139.
[11] S. Wang, F. Liu, H. Xia, Content-based vehicle selection and resource allocation for federated
learning in iov, in: Wireless Communications and Networking Conference Workshops, IEEE, 2021,
pp. 1–7.
[12] X. Chen, G. Zhu, Y. Deng, Y. Fang, Federated learning over multihop wireless networks with
in-network aggregation, IEEE Trans. Wireless Commun. 21 (6) (2022) 4622–4634.
Sattler, S. Wiedemann, K.-R. Müller, W. Samek, Robust and communication-efficient federated
learning from non-iid data, IEEE Trans. Neural Netw. Learn. Syst. 31 (9) (2019) 3400–3413.

[Link] ECE, BLDEA’s V.P. Dr.P.G. Halakatti CET 35