Deep Leakage from Gradients (DLG) – Zhu et al.

(2019)

Overview
Zhu et al. (2019) introduced Deep Leakage from Gradients (DLG), a seminal work that first
demonstrated the feasibility of reconstructing sensitive training data directly from shared
gradients in collaborative learning systems. The attack operates by optimizing dummy inputs
and labels such that the gradients they produce closely match those received from the
target model. Through iterative optimization, the dummy inputs gradually converge towards
the original training data, thereby exposing private information.

Key Contributions

1. Established that sharing gradients, even without explicit access to raw data, can lead
to severe privacy leakage.

2. Proposed a gradient inversion framework that reconstructs both inputs and labels
solely from gradient information.

3. Provided experimental validation across multiple datasets, highlighting that even

small-scale models are vulnerable.

Strengths and Impact

DLG was the first to shift the privacy discussion in federated and distributed learning away
from parameter updates toward the inherent risks in gradient sharing. By demonstrating
near-exact image recovery in vision tasks, it raised awareness that gradients themselves can
be sensitive data. This insight has since served as the foundation for a large body of
subsequent research on gradient inversion attacks.

Limitations
Despite its novelty, DLG faces practical constraints. The method struggles with larger batch
sizes, as overlapping gradient signals reduce the clarity of reconstruction. It also requires
access to precise gradient updates at each iteration, an assumption that may not always hold
in real-world federated learning environments where gradients are often aggregated,
compressed, or obfuscated. Additionally, optimization is computationally expensive and can
converge to inaccurate reconstructions when data dimensionality increases.

Relevance to Present Work

DLG serves as the conceptual cornerstone of privacy attack research in federated learning.
While its assumptions limit direct applicability in production settings, the work highlights a
critical vulnerability that necessitates the development of stronger defensive strategies. For
this reason, DLG remains essential background for evaluating subsequent improvements
(such as iDLG) and corresponding countermeasures (such as secure aggregation and
homomorphic encryption).
Improved Deep Leakage from Gradients (iDLG) – Zhao et al. (2020)

Overview
Building upon Zhu et al.’s DLG, Zhao et al. (2020) proposed Improved Deep Leakage from
Gradients (iDLG), a more efficient gradient inversion method that addressed key
shortcomings of the original attack. The central innovation of iDLG lies in its ability to
analytically infer ground-truth labels from gradient signs, thereby bypassing the label-
guessing stage required in DLG. This simplification reduces computational complexity and
enhances reconstruction accuracy.

Key Contributions

1. Introduced a closed-form approach to recover ground-truth labels directly from

gradients, eliminating the need for iterative label optimization.

2. Demonstrated significantly faster and more stable reconstructions compared to DLG,

especially for small-batch settings.

3. Validated the attack across various tasks, showing improved fidelity of data recovery
from gradients.

Strengths and Impact

iDLG advanced the field by showing that label information is directly encoded in gradient
structures, revealing an even deeper privacy risk. Its efficiency not only improved the
practicality of gradient inversion but also emphasized that defenses must consider label
recovery in addition to input reconstruction. The work has been widely cited as evidence
that naive gradient sharing in federated learning is inherently unsafe, influencing both attack
methodologies and defensive design.

Limitations
Despite its efficiency gains, iDLG shares several weaknesses with its predecessor. The
method still performs poorly on large-batch training, where gradient overlap dilutes per-
sample signal. Furthermore, the assumption of unprotected, unaggregated gradient access
remains unrealistic for many practical deployments. Reconstruction quality also declines for
high-dimensional data and complex models, underscoring the challenges of scaling such
attacks.

Relevance to Present Work

For research on privacy-preserving federated learning, iDLG serves as an essential bridge
between proof-of-concept vulnerabilities (DLG) and more sophisticated gradient inversion
frameworks. By clarifying that label information itself is highly vulnerable, it strengthens the
case for integrating secure aggregation (to obscure gradients collectively) and homomorphic
encryption (to prevent individual gradient inspection). Thus, iDLG directly motivates the
defensive mechanisms this thesis seeks to combine in a unified framework.
Inverting Gradients – How Easy is it to Break Privacy in Federated Learning? (Geiping et al.,
2020)

Introduction

Federated Learning (FL) allows clients to collaboratively train a model by sharing gradients or
parameter updates rather than raw data. Earlier research, notably DLG (Zhu et al., 2019),
demonstrated that gradients can be inverted to recover private data, but mainly under
simple conditions such as small batch sizes and low-complexity images. Geiping et al. revisit
this problem and ask: Can gradient inversion attacks succeed against realistic, large-scale
models and complex data? Their work, titled Inverting Gradients, pushes the boundaries of
privacy attacks in FL by showing that even modern deep networks on natural images are
vulnerable.

Methodology

shared gradients ∇W from a model F(x;W) trained on private input (x, y), the attacker
The proposed method is based on gradient inversion with strong regularization. Given

attempts to reconstruct x by finding a dummy input x′ whose gradients ∇W′ match ∇W.
Unlike DLG, which optimizes both dummy input and labels simultaneously, Geiping et al. fix
the ground-truth labels and focus on reconstructing inputs.

Their innovation lies in the objective function:

– ∇W||²).
 The reconstruction loss matches the dummy gradients to the observed ones (||∇W′

 A Total Variation (TV) prior is added to encourage natural-looking smoothness and

reduce noise.

 Optimization uses advanced solvers (e.g., L-BFGS, Adam) for stability.

The method is tested across CNN architectures, including ResNet and VGG, and datasets
such as CIFAR-10, ImageNet, and CelebA.

Key Contributions

1. Generalization of Gradient Inversion Attacks: Demonstrates privacy leakage even

with high-capacity networks and complex, real-world data.

2. Total Variation Regularization: Introduces image priors to improve reconstruction

fidelity, especially for natural images.

3. Scalability: Shows attacks work for larger models and deeper architectures, not just
toy datasets.

4. New Evaluation Metrics: Proposes quantitative measures for assessing fidelity of

reconstructed samples.
Results & Observations

 High-fidelity reconstructions were achieved, with images closely resembling private

training data.

 Unlike DLG, which struggled beyond MNIST or small-batch data, this method reliably
recovered CIFAR-10 and ImageNet samples.

 The attack remains effective even when gradients are aggregated from multiple
samples, although fidelity decreases as batch size grows.

 Experiments highlight that realistic FL scenarios, including complex image tasks,

remain at risk.

Strengths

 Robust methodology: Use of TV regularization makes reconstructions visually

coherent, bridging the gap between theory and practical attacks.

 Wide applicability: Demonstrated on modern networks (ResNet, VGG) and complex

datasets.

 Strong empirical evidence: Repeated experiments confirm stability across settings.

Limitations & Challenges

 Batch size dependence: Larger batches reduce attack accuracy, though partial
reconstructions still occur.

 Computational cost: Optimization with regularization is slower than DLG.

 Defense gap: The work reveals vulnerabilities but does not propose strong
countermeasures; defenses like secure aggregation remain necessary.

Relevance to Federated Learning Privacy

This paper significantly advanced the understanding of privacy risks in FL. While DLG first
showed that gradients can leak data, Inverting Gradients proved that this threat extends to
real-world, high-resolution, and large-scale tasks. It establishes that sharing raw gradients is
fundamentally insecure, motivating defenses such as differential privacy, gradient
perturbation, or secure aggregation.

Secure Aggregation for Federated Learning (Bonawitz et al., 2016 & 2017)

Introduction

While Federated Learning (FL) enables collaborative model training without centralizing raw
data, a major vulnerability lies in the exposure of individual model updates. Even if raw data
never leaves client devices, updates (gradients or parameters) can be reverse-engineered to
leak sensitive information. Recognizing this, Bonawitz et al. introduced the concept of
Secure Aggregation (SA), a cryptographic protocol that allows the server to learn only the
aggregated model update while concealing each participant’s individual contribution. Their
2016 paper presented the first practical secure aggregation protocol, and their 2017 work
extended this framework to operate at Google’s production scale, thereby demonstrating its
feasibility in real-world FL deployments.

Methodology (2016: Practical Secure Aggregation)

The 2016 work proposed a lightweight, fault-tolerant cryptographic protocol enabling secure
summation of updates from thousands of clients. The protocol relied on the following
principles:

1. Pairwise Random Masks – Each pair of clients establishes a shared secret mask via
Diffie–Hellman key exchange. These masks are added to each client’s model update
before submission. When aggregated, the masks cancel out, leaving only the true
sum.

2. Additive Secret Sharing – If clients drop out mid-protocol (a common problem in

mobile environments), the remaining masks can be reconstructed using shares
distributed earlier. This ensures dropout resilience.

3. Multi-Round Protocol Design – The protocol is executed in several stages (setup, key
exchange, mask generation, aggregation, and unmasking) to handle unreliable client
participation.

4. Security Guarantee – The server learns nothing about individual updates, only the
sum. The threat model assumes an honest-but-curious server that follows the
protocol but tries to infer extra information.

The authors emphasize scalability and robustness, showing the protocol can tolerate high
dropout rates while remaining computationally efficient.

Methodology (2017: Production Deployment at Google)

Building on the 2016 protocol, the 2017 paper addressed deployment challenges at scale,
adapting secure aggregation to real-world federated learning systems on mobile devices.

Key enhancements included:

 Communication Efficiency: Optimizations to reduce bandwidth usage, since mobile

networks are unstable and costly.

 Scalable Key Management: Efficient distribution and recovery of pairwise keys, even
with tens of thousands of devices.
  Dropout Tolerance at Scale: Protocol modifications ensured robustness when a large
fraction of devices (sometimes >50%) dropped out during training rounds.

 System Integration: Secure aggregation was embedded into Google’s FL

infrastructure, proving feasibility for tasks like mobile keyboard prediction.

This marked the first demonstration of SA in a production-grade federated system, bridging

the gap between theory and large-scale deployment.

Key Contributions

1. 2016 – First practical SA protocol that was both fault-tolerant and efficient for large
populations.

2. 2017 – Demonstrated real-world feasibility of SA, deploying at Google scale with

thousands of mobile clients.

3. Established cryptographic rigor + system-level optimizations as essential for

preserving privacy in FL.

4. Set the stage for subsequent privacy-preserving approaches that combined SA with
differential privacy or homomorphic encryption.

Results & Observations

 Efficiency: The 2016 protocol required only light cryptographic operations (e.g.,
modular arithmetic, Diffie–Hellman exchanges), making it feasible for smartphones.

 Dropout Robustness: The scheme successfully tolerated client churn, a critical

requirement in mobile environments.

 Deployment Success: By 2017, secure aggregation was fully integrated into Google’s
FL pipeline, proving scalability to real-world applications like Gboard’s next-word
prediction.

 Privacy Strength: The server could only observe aggregated updates, closing off
avenues for gradient inversion attacks like DLG/iDLG.

Strengths

 Pioneering and practical cryptographic solution for protecting FL updates.

 Highly scalable, with explicit consideration of dropout and unreliable clients.

  Transitioned from theory (2016) to industrial-scale deployment (2017), a rare
achievement in privacy research.

Limitations & Challenges

 Threat model: Assumes an honest-but-curious server; stronger adversaries (e.g.,

collusion between server and subset of clients) are not addressed.

 Computational overhead: While efficient, SA still adds latency compared to plain FL.

 Limited scope: SA hides updates but does not prevent inference attacks on the
aggregated model itself (e.g., membership inference).

Relevance to Federated Learning Privacy

The combined contributions of Bonawitz et al. form the foundation of modern secure
federated learning. The 2016 protocol proved that secure aggregation is possible in practice,
while the 2017 deployment demonstrated its viability in production systems. Together, these
works established SA as a baseline privacy-preserving mechanism in FL, inspiring
subsequent research that integrates SA with differential privacy, homomorphic encryption,
and robust aggregation to further strengthen defenses.

Private Federated Learning on Vertically Partitioned Data via Entity Resolution and
Additively Homomorphic Encryption

(Hardy, Henecka, Ivey-Law, Nock, Patrini, Smith, Brian Thorne – 2017-arXiv, “Hardy et al.”)
arXiv

Introduction

Hardy et al. address a setting in federated learning that is different but relevant: vertical
partitioning of data. In this scenario, multiple data providers each hold different sets of
features about the same entities, with only one provider holding the label (target variable),
and the providers may not even know which entities are shared across them. The goal is to
jointly train a linear (logistic regression) model across all features without exposing raw data
or revealing which entities overlap. The work becomes particularly relevant to your thesis
because it uses additively homomorphic encryption (HE) to secure messages and model
updates, under an honest-but-curious adversary model. This aligns with part of your
proposed defense (HE + SA) against reverse-engineering attacks in Federated Learning.

Methodology / Approach
  Two-Phase Protocol: Hardy et al. split the solution into two phases:

1. Entity Resolution Phase

 Since the data providers may not know which entities correspond to
each other (i.e. which rows map across providers), they run a privacy-
preserving entity resolution protocol. This allows matching common
entities without revealing non-shared entities or data in the clear.

2. Federated Training Phase with HE

 Once matched, logistic regression is trained “vertically”: i.e. feature

subsets are held separately, but the computations for the model are
done collaboratively.

 Providers exchange encrypted values computed using additive

homomorphic encryption. HE allows addition of ciphertexts (and
scalar multiplication), such that the server (or other providers) can
aggregate necessary sums without decrypting individual values.

 Honest-but-Curious Adversary Model: Assumes no party deviates from the protocol

maliciously, but adversaries may try to inspect messages received or shared to infer
private inputs.

 Impact of Entity Resolution Errors: An important nuance: the paper doesn’t assume
perfect matching between entities. The authors analyze how errors in entity
matching degrade performance, generalization, loss, margins etc.

Key Contributions

 First to combine privacy-preserving entity resolution with vertically partitioned

federated learning under HE, allowing training without either revealing feature sets
or entity alignment.

 Demonstrates that training under HE in this vertically partitioned context can achieve
accuracy comparable to a non-private baseline (where all data is centrally held), even
with scaling to millions of entities and hundreds of features.

 Provides a formal characterization of how mismatches in entity resolution impact

model performance (on classification, margins, etc.).

 Shows that additively homomorphic encryption is practically usable in these

federated settings, not just theoretically clean but with “good enough” performance
and communication costs.
Results & Observations

 Experiments show that their HE-based model has nearly the same performance
(accuracy / loss) as a non-private logistic regression that would see all data.

 When entity resolution is imperfect, drops in accuracy, margin, generalization are

analyzed: small error rates in matching incur modest loss, but large errors degrade
performance significantly.

 Scalability: implemented with datasets involving up to millions of entities, and with

feature sizes in the hundreds — demonstrates that HE isn’t automatically prohibitive
in cost.

Strengths

 Uses a realistic data partitioning scenario (vertical split) which many works ignore;
most FL literature assumes horizontal partitioning.

 HE usage is well integrated, not just “encrypt everything and pay huge cost.” They
carefully evaluate trade-offs.

 The formal treatment of entity resolution errors is rare and useful; it shows how
robust the learning is under imperfect matching.

Limitations & Challenges

 Honest-but-Curious Model: doesn’t address more powerful adversaries (e.g. parties

colluding, or malicious clients).

 Linear Models Only: focuses on logistic regression, not deeper / modern neural
network architectures. Could be harder to scale HE to deep networks with many
parameters.

 Vertical Partitioning: not directly applicable to many FL settings (especially in cyber

threat detection) that use horizontal partitioning of data (each client has both
features and labels).

 Communication and computational overhead of HE, while acceptable, still non-

negligible. For large-scale deep model training, the overhead may still be
problematic.

Relevance to Your Thesis

  This paper illustrates that HE can be used to hide sensitive data during training,
which supports your idea of integrating HE into your framework.

 The vertical partitioning scenario is different, but many of the same concerns apply:
encrypted updates, privacy in message exchange, trade-offs between accuracy and
overhead.

 While they use logistic regression, their findings provide insight into how HE may
behave (in terms of performance loss, overhead) when used with more complex
models (your cyber threat detection model).

 The handling of entity resolution errors parallels the problem of noisy or imperfect
data in real-world cybersecurity settings.

Summary

Hardy et al.’s work provides a strong example of how HE can be effectively used in federated
learning to ensure privacy without massive accuracy loss. It addresses some of the practical
concerns: secure computation, encrypted messages, scalable datasets, feature partitioning,
imperfect alignment. For your thesis, it represents one of the best case studies to compare
with: you can draw from its methodology, results, and limitations to justify using HE + SA,
and anticipate what the trade-offs might be when applying them in your cyber threat
detection FL framework.