A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 1

A Survey on Cache Methods in Diffusion

Models: Toward Efficient Multi-Modal Generation
Jiacheng Liu1 , Xinyu Wang1,2 , Yuqi Lin1 , Zhikai Wang1 , Peiru Wang1 , Peiliang Cai1 ,
Qinming Zhou1,2 , Zhengan Yan1 , Zexuan Yan1 , Zhengyi Shi1 , Chang Zou1 , Yue Ma1,3 , Linfeng Zhang1†
1 Shanghai Jiao Tong University 2 Tsinghua University 3 The Hong Kong University of Science and Technology
: Cache4Diffusion
Awesome-Diffusion-Acceleration-Cache
arXiv:2510.19755v3 [[Link]] 1 Nov 2025

Abstract—Diffusion Models have emerged as a cornerstone paradigm following GANs [3] and VAEs [4]. With the rise
of contemporary generative AI, owing to their exceptional of the Diffusion Transformer (DiT) [5] architecture, new-
generation quality and controllability. However, their inherent generation models such as FLUX [6] and Qwen-Image [7] in
“multi-step iterations” and “complex backbone networks” infer-
ence paradigm results in prohibitive computational overhead the field of image generation have shown strong generative ca-
and significant generation latency, which has become a critical pabilities. At the same time, in the field of video generation, a
bottleneck hindering their deployment in real-time interactive ap- large amount of industrial investment has promoted the devel-
plications. Although existing acceleration techniques have made opment of large-scale video diffusion models, producing sev-
some progress, they often face challenges such as limited applica- eral models with hundreds of billions of parameters, including
bility, high training costs, or a degradation in generation quality.
Against this backdrop, Diffusion Caching presents a promising open-source models such as Wan2.1 [2] and Hunyuan [8], and
technical pathway as a training-free, architecture-agnostic, and closed-source models such as Sora2.0 [9], Movie-Gen [10],
efficient inference paradigm. Its core mechanism lies in accurately and Seaweed [11]. These models have successively broken
identifying and reusing the intrinsic computational redundancies records in terms of generative quality, diversity, and control-
within the diffusion inference process. Through feature-level lability, showcasing remarkable generative potential.
cross-step reuse and inter-layer scheduling, it effectively reduces
the computational load without altering the model parameters. However, this leap in performance comes at the cost of
This paper systematically reviews the theoretical foundations rapidly escalating computational complexity and model scale.
and technological evolution of Diffusion Caching and proposes Owing to their “step-by-step denoising” generation mech-
a unified framework for its classification and analysis. Through anism, diffusion models require multiple iterative forward
a comparative analysis of representative methods, we indicate passes through deep neural networks during sampling, making
that Diffusion Caching exhibits a clear evolutionary trajectory
from “static reuse” to “dynamic prediction”. This trend not only them inherently computation-intensive. This issue becomes
enhances the flexibility of caching mechanisms in addressing particularly pronounced in high-resolution image generation
the computational demands of diverse generation tasks but also and long-sequence video synthesis, where computational de-
holds great potential for deep integration with other mainstream mands grow exponentially. For instance, generating a single
acceleration techniques, such as sampling optimization and model 1328 × 1328 image with Qwen-Image [7] involves approx-
distillation, to jointly construct a unified and efficient inference
framework for future multimodal and interactive applications. imately 1.29 × 104 TFLOPs of computation, resulting in a
Through this systematic review and forward-looking analysis, latency of up to 127 seconds per image on NVIDIA H20
this paper aims to provide researchers with a clear technological GPU. Such substantial computational costs severely limit the
roadmap for Diffusion Caching. We argue that this efficient in- feasibility of diffusion models in real-time creation and large-
ference paradigm will become a key enabling technology, driving scale production applications.
the advancement of generative AI towards real-time performance
and widespread adoption, thereby injecting new vitality into Traditional acceleration efforts have primarily focused on
the theoretical construction and practical implementation of optimizing numerical solvers [12] and introducing model dis-
“Efficient Generative Intelligence”. tillation techniques [13]. While these approaches can achieve
Index Terms—Diffusion Models, Cache-based Acceleration, partial speedup, they often struggle to balance acceleration and
Inference Acceleration, Feature Cache generation fidelity. Excessive reduction of sampling steps may
lead to accumulated discretization errors, thereby degrading
image quality. On the other hand, distillation and optimizing-
I. I NTRODUCTION based methods require additional computational and anno-
A. Background tation resources, making them less flexible and harder to
In recent years, Diffusion Models (DMs) have achieved generalize across models.
groundbreaking progress in the field of generative artificial Against this background, the high computational complexity
intelligence , particularly in image and video generation [1], and inference latency of diffusion models have become key
[2]. They have emerged as the most representative generative bottlenecks for practical deployment. Generating a single high-
resolution image can take tens of seconds to minutes, while
† Corresponding author. Email: zhanglinfeng@[Link] producing a video may take several hours. This high latency
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 2

BWCache
OminiCache
SpeCa
FoCa
FastCache

RainFusion
BlockDance
TaylorSeer
Paper Number

ToCa CacheQuant
TeaCache
FORA AdaCache
∆-DiT
Approximate VCUT
Caching DiTFastAttn
DeepCache FasterDiffusion
BlockCaching

Time

Fig. 1. The development trend of Diffusion Caching The line chart illustrates the rapid growth in the number of related works from 2024 to the present.
Representative works from each period are highlighted.

not only reduces system throughput but also fails to meet immersion of interaction deteriorate dramatically, preventing
the demands of real-time interaction and low-cost deployment. the model from integrating effectively into real-time systems.
Therefore, achieving efficient inference without compromising Second, when deploying diffusion models on on-device or
generation quality has become a crucial challenge that must be edge systems, the limitation of computational resources further
addressed for diffusion models to transition from theoretical magnifies inference latency. Even with high-end consumer
research to real-world applications. GPUs (e.g., NVIDIA RTX 4090), producing a high-resolution
image can take tens of seconds, while mid- and low-tier
hardware suffers from even more severe delays. Such strong
B. Challenges reliance on hardware capabilities imposes a high entry thresh-
Despite the paradigmatic breakthroughs achieved by DMs in old, hindering the scalability and accessibility of diffusion
terms of generation quality and controllability [14], their com- models in real-world applications.
putationally intensive generative mechanism still faces severe Finally, in large-scale deployment on the cloud, generation
efficiency bottlenecks during inference. The most prominent latency directly affects the economic efficiency and scalability
issue is the excessively long generation latency. of services. The long inference time per generation request
The inherent computational design of diffusion models, reduces system throughput and concurrency, forcing service
combining multi-step iterations with complex network archi- providers to allocate additional computational resources to
tectures, is the primary source of [Link] generation process, maintain responsiveness. This not only increases energy con-
the model need to progressively denoise high-dimensional sumption and operational costs but also undermines the com-
noise, where each step involves extensive parameter computa- mercial viability of diffusion-based services.
tion and nonlinear mapping. High-fidelity synthesis typically Overall, high generation latency has become the primary
requires tens of denoising steps (typically 20 to 50), and each bottleneck for diffusion models to move from laboratory
step requires a full forward propagation through multi-layer research to practical applications. It not only limits the appli-
deep networks such as U-Net or DiT. Therefore, inference cation of the model in real-time and interactive scenarios but
time scales approximately linearly or even super-linearly with also raises the economic threshold for large-scale deployment
sequence length, image resolution, and network depth. For and hinders the popularization of the model on personal and
instance, on NVIDIA H20 GPU, generating a single 2K- edge computing devices. Therefore, how to efficiently reduce
resolution image often takes several minutes, while synthe- end-to-end generation latency—while maintaining generation
sizing a 720p video of around 129 frames may take several quality and model stability, has become a central challenge in
hours. As resolution and temporal length increase, generation current research on accelerating diffusion model inference.
latency grows exponentially. This problem is evident not only
in academic benchmarks but also in practical applications.
Firstly, in interactive generation tasks, latency directly af- C. Motivation
fects user experience. Real-time applications such as virtual To address the high inference latency of diffusion models,
dressing, real-time game scene generation, and live back- existing research primarily explores two technical directions:
ground replacement are extremely sensitive to response de- (1) reducing the number of sampling steps (Step Reduction),
lays. When the latency between user input and model output and (2) lowering the computational cost per-step (Single-
reaches several seconds, or even minutes, the naturalness and Step Cost Reduction). Both directions theoretically aim to
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 3

reduce the overall computational load during inference, yet in leads to substantial waste. Diffusion Caching leverages such
practice they must balance trade-offs among speed, generation temporal correlation and invariance to enable computational
quality, and adaptability, making it difficult to achieve an ideal reuse and efficient inference.
equilibrium. This limitation provides a direct motivation for This concept has been well validated in autoregressive
exploring new acceleration paradigms. generation with large language models (LLMs), where KV-
Cache has become a standard component, providing multi-
• Reducing the number of sampling steps
fold acceleration without loss of quality. By analogy, caching
This type of method usually models the diffusion pro-
in diffusion models offers two core advantages: (1)Training-
cess as a numerical integration problem of ordinary
free nature: Diffusion Caching operates purely at inference
differential equations (ODE) or stochastic differential
time, requiring no additional training or fine-tuning; (2) Or-
equations (SDE), and reduces the number of sampling
thogonality and composability: it can be combined with
times by expanding the integration step length through
other acceleration techniques, such as step reduction or model
high-order numerical solvers, such as DPM-Solver [12],
compression, to achieve complementary benefits.
UniPC [15]. The core idea is to simulate the denoising
Therefore, Diffusion Caching should not be viewed as a
trajectory with larger time steps to achieve sampling com-
replacement for existing acceleration strategies but rather as a
pression. However, when the number of sampling steps is
low-cost, highly compatible, and easily deployable optimiza-
reduced to a certain threshold (for example, less than 10
tion technique. It offers a new research direction and theoret-
steps), discretization errors accumulate rapidly, leading to
ical foundation for overcoming the inference bottlenecks of
missing details, structural distortion, and visual artifacts
diffusion models without compromising generation quality.
in the generated results. In addition, another type of
distillation method, such as consistency distillation, pro-
gressive distillation, approximates the multi-step behavior D. Contributions
of the original model by training lightweight models, Currently, this field lacks a comprehensive survey and uni-
thus completing generation in fewer steps. Although these fied theoretical framework, and significant differences remain
methods can achieve higher acceleration, they have high across existing studies in terms of principle interpretation,
training costs, limited adaptability, and are sensitive to the strategy design, and applicability. To fill this gap, the main
target model and task conditions, which are not conducive contributions of this work are as follows:
to general deployment.
1) The first systematic summary of the theory and practice
• Lowering the computational cost per-step
of diffusion caching. We explain the core idea of diffu-
This direction focuses on reducing the computational bur-
sion caching from a principled perspective: identifying and
den of each forward pass through model compression or
reusing computational redundancies in the diffusion infer-
system-level optimization. Model compression techniques
ence process to reduce repeated computation and achieve
such as quantization [16] and pruning [17] can lower
acceleration. We interpret the mathematical essence, ap-
computation and memory costs but often sacrifice gener-
plicable conditions, and constraints of caching from the
ation quality and require complex retraining procedures.
dual perspectives of numerical analysis and neural network
Lightweight architecture design can reduce computa-
computation graphs. By reviewing existing research, we
tion structurally, but it is constrained by limited model
clarify the unique position of diffusion caching within the
capacity and dependence on pretrained ecosystems, mak-
diffusion model acceleration ecosystem, providing a unified
ing it difficult to maintain both fidelity and generalization.
theoretical framework for future studies.
Meanwhile, system-level optimization techniques, such
2) A unified taxonomy and analytical framework for dif-
as FlashAttention [18], TensorRT acceleration, improve
fusion caching. We construct the first systematic clas-
operator efficiency and memory scheduling to enhance
sification and analytical framework for diffusion caching
hardware utilization. While they can yield performance
along three dimensions: trigger condition, reuse granu-
gains, these methods rely heavily on specific platforms
larity, and update strategy, revealing the intrinsic logic
and hardware support, which limits their portability and
and technological evolution among different methods. We
generality. Moreover, their performance is often sensitive
categorize existing approaches into two levels: Static
to input distribution and model architecture, increasing
and Dynamic Caching. Within the dynamic caching
the complexity of system tuning and maintenance.
paradigm, we further identify four representative strategies:
In this context, Diffusion Caching emerges as a promising Timestep-Adaptive, Layer-Adaptive, Predictive, and Hy-
optimization paradigm based on a different principle. Unlike brid Caching. Through systematic comparison and anal-
methods that modify model architecture or require retraining, ysis, we reveal the technological trajectory of diffusion
caching aims to identify and eliminate computational redun- caching from “Static Reuse” to “Dynamic Prediction.”.
dancy during inference. In the iterative denoising process of 3) Evolution trends and future research directions of dif-
diffusion models, many intermediate results can be reused. fusion caching. Building upon a comprehensive review
For example, under fixed text conditions, the key and value of existing work, we summarize common challenges and
matrices in Cross-Attention layers remain constant across potential breakthroughs in diffusion caching. Existing meth-
timesteps, while feature maps between adjacent timesteps ods still face limitations in terms of cache consistency,
often change slowly. Recomputing these redundant results generalization, memory efficiency, and cross-platform
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 4

adaptability, particularly in high-resolution image and In image generation, research primarily focuses on com-
long-sequence video generation tasks. Looking forward, we pressing U-Net and DiT architectures. SPD [34] applies struc-
anticipate multi-dimensional development trends: on one tured pruning to reduce channels and attention heads. TM-
hand, diffusion caching mechanisms can be combined with SD [35] and TFusion [36] merge or fuse tokens to reduce self-
other inference techniques to achieve multi-level acceler- attention cost, while DFA [37] targets attention computation
ation; on the other hand, the concept of diffusion caching optimization in DiT-based models. IBTM [38] and ToMA [39]
can be extended to complex generation tasks, providing new employ token-merging mechanisms to enhance efficiency for
scalable avenues for efficient generation. both image and video generation. DC-Gen [?] connects a
In summary, this paper not only provides the first systematic deeply compressed autoencoder with diffusion models, reduc-
overview of diffusion caching from theoretical, methodologi- ing the redundancy in latent space.
cal, and application perspectives but also proposes a unified Studies for video emphasize reducing spatiotemporal com-
analytical framework and future directions. We believe that plexity. VSA [40] introduces trainable sparse attention to
diffusion caching will become a cornerstone technology for lower inter-frame computation, and SvDiT [41] incorporates
future Efficient Generation, offering a new research paradigm sparse spatiotemporal attention for efficient video synthesis.
and engineering perspective for practical deployment and MegaTTS-3 [42] applies sparse alignment to strengthen latent
sustainable optimization of diffusion models. DiT representations for zero-shot speech generation. DC-
VideoGen [?] extends DC-Gen in video generation, reducing
the number of latent space tokens while preserving high
II. R ELATED M ETHODS
reconstruction quality.
A. Distillation
Distillation methods accelerate diffusion sampling by train- C. Parallel Sampling & Hardware Acceleration
ing a student model to approximate the teacher’s denoising Parallel acceleration methods improve efficiency by intro-
process, compressing the long iterative trajectory into only ducing concurrency at the operator, timestep, or task level,
a few or even a single step. Progressive Distillation [19] minimizing redundant memory access and transforming the
reduces hundreds of sampling steps to several through iterative inherently sequential sampling process into parallel execution.
halving with minimal quality loss. Consistency Models [20] This greatly reduces latency and increases throughput. FA [44]
enforce prediction agreement across timesteps, enabling one- minimizes memory I/O overhead through I/O-aware on-chip
step or few-step high-quality sampling. GD-Distill [21] in- tiling, while FA2 [45] further optimizes workload partitioning
tegrates classifier-free and non-classifier-free diffusion into and scheduling, approaching the theoretical efficiency limit of
the student model to reduce computation. LCM [22] per- matrix multiplication.
forms distillation in latent space, achieving high-quality results In parallel sampling, SpecDiff [46] performs draft–verify
within 1–4 steps. MCM [23] unifies diffusion and consistency generation by producing multiple candidates simultaneously,
frameworks, supporting flexible-step sampling with improved avoiding strictly sequential sampling. SpecSampling [47] ex-
stability. RG-LCM [24] incorporates human preference signals tends this idea to continuous diffusion, enabling fully paral-
during distillation, surpassing the teacher in perceptual quality lelized inference without auxiliary draft models. APD [48]
for one-step generation. dynamically allocates parallel computation during decod-
In video generation, ADL [25] fuses multiple teacher mod- ing, effectively lowering latency and improving efficiency.
els to accelerate synthesis. MoCM [26] and AVDM² [27] STADI [49] introduces step–block partitioning and schedul-
enhance temporal and visual coherence through mo- ing in heterogeneous GPU environments to achieve balanced
tion–appearance disentanglement and distribution-matching workload distribution.
objectives. VIP [28] and DAP [29] introduce preference- At the hardware and system level, DF [50] achieves
guided or adversarial optimization, enabling one-step video near-linear multi-GPU scaling for high-resolution generation
generation with improved fidelity and diversity. through image partitioning and asynchronous communication.
PF [51] combines inter-layer pipelining with block-level paral-
B. Pruning & Sparsification lelism to enable efficient inference in Diffusion Transformers.
RAIN [52] introduces frame-level parallel attention and cross-
Pruning and sparsification reduce computational and mem-
temporal noise-layer updates, efficiently supporting continuous
ory cost by removing redundant parameters, sparsifying
and long-duration video generation.
weights, or compressing intermediate representations. DC [30]
performs joint compression via pruning, quantization, and cod-
ing, greatly lowering parameter and bandwidth requirements D. Sampler Optimization
with minimal quality loss. PCNN [31] accelerates inference Sampler optimization methods aim to reduce the number
through channel- and filter-level pruning, while LTH [32] re- of iterations during the reverse inference phase of diffusion
veals trainable sparse subnetworks within over-parameterized models through improved numerical integration and timestep
models, inspiring structured and unstructured compression in scheduling. DDPM [53] establishes the foundational denoising
later diffusion architectures. RTL [33] improves sparse net- framework using variational inference to model the reverse
work trainability through reset-and-retrain strategies, providing Markov chain. DDIM [54] introduces deterministic sampling
a foundation for modern sparse diffusion designs. by reformulating the process into a non-Markovian form,
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 5

enabling faster high-quality generation. SDE [55] unifies diffu- generation paradigm that achieves exceptional performance in
sion under stochastic differential equations, and IDDPM [56] multimodal content generation tasks including images [62],
further incorporates learnable variance and refined timestep videos, and audio. To understand the mechanisms of subse-
scheduling to enhance stability in few-step sampling. quent caching acceleration techniques, this section elucidates
Building on these foundations, PNDM [57] applies pseudo- the theoretical foundations of diffusion models.
numerical integration to maintain stability with limited steps, The core idea of diffusion models can be summarized as a
while DPM-Solver [12] and DPM-Solver++ [58] employ high- bidirectional stochastic process consisting of the forward dif-
order ODE solvers to achieve high-quality results within fusion process and the reverse generation process. In the for-
about ten iterations. UniPC [59] unifies prediction–correction ward diffusion process, given a real data distribution q(x0 ),
and multi-step solvers into a general framework for accurate data samples x0 are transformed into pure noise xT through
few-step generation. Recent works such as RAPID³ [60] the gradual addition of Gaussian noise. This process can be
and S4S [61] introduce learning-based and reinforcement- understood as gradually converting complex data distributions
driven optimization, jointly tuning step size, solver order, into simple standard Gaussian distributions until the structural
and scheduling to enable end-to-end adaptive sampling with information of the original data is completely masked by noise.
minimal steps. The reverse generation process is the temporal inversion of
the forward process, which learns a neural network to progres-
E. Cache Acceleration sively recover meaningful data structure from pure noise. This
process learns how to remove noise, achieving a mapping from
Although methods such as distillation, pruning and sparsi-
simple distributions to complex data distributions. This design
fication, parallel acceleration, and sampler optimization have
offers key advantages. The forward process provides a clear
substantially reduced the inference cost of diffusion models,
training objective through noise prediction. Meanwhile, the
each still faces intrinsic limitations. Distillation requires ex-
reverse process uses learnable neural networks to approximate
tensive retraining on teacher-generated trajectories, demand-
the denoising distribution, avoiding the complexity of adver-
ing high computational cost and often leading to quality
sarial training [3] or variational inference [4] in traditional
degradation under extremely few-step sampling due to in-
generative models.
sufficient distribution coverage. Pruning and sparsification
mainly reduce per-step computation, but the iterative denoising In mathematical formalization, the forward diffusion pro-
process remains unchanged, leaving overall latency largely cess is modeled as a fixed Markov chain, where each step
unimproved. Moreover, structured pruning and token sparsifi- adds a small amount of Gaussian noise to the data. Given a
cation can weaken representational capacity, causing semantic data point x0 ∼ q(x0 ), the forward process is defined as
misalignment or loss of fine details under high compression T
ratios. Parallel and hardware-based acceleration methods
Y
q(x1:T |x0 ) := q(xt |xt−1 ), (1)
depend heavily on specific architectures, kernel implementa- t=1
tions, and communication strategies. While they achieve near-
linear scaling on multi-GPU systems, their benefits diminish where the transition kernel is
under limited hardware, and large-scale parallelism often p
q(xt |xt−1 ) = N (xt ; 1 − βt xt−1 , βt I). (2)
introduces synchronization overhead and consistency issues
across timesteps or regions. Sampler optimization reduces Here βt ∈ (0, 1) is a predefined noise schedule parameter
the number of steps through advanced numerical solvers and controlling the noise strength added at each step. Through the
scheduling, yet still fundamentally relies on iterative integra- reparameterization trick [4], we can directly sample xt at any
tion, making computational cost scale linearly with sequence time t fromQx0 without iterative steps. Defining αt := 1 − βt
length or resolution. These approaches also tend to be sensitive t
and ᾱt := s=1 αs , we have
to guidance strength, error accumulation, and distribution √
shifts, limiting their robustness and general applicability. q(xt |x0 ) = N (xt ; ᾱt x0 , (1 − ᾱt )I), (3)
To overcome these constraints, Diffusion Caching intro-
duces a complementary perspective by explicitly storing and i.e.,
√ √
reusing intermediate activations or tokens during inference. xt = ᾱt x0 + 1 − ᾱt ϵ, (4)
Without additional training or architectural modification, it di-
rectly avoids redundant computations across timesteps, thereby where ϵ ∼ N (0, I). When T is sufficiently large and the noise
mitigating the inherent inefficiency of stepwise iteration. schedule is reasonable, ᾱT ≈ 0, making q(xT |x0 ) ≈ N (0, I),
i.e., the final state approximates a Gaussian distribution [53].
III. TAXONOMY OF C ACHE ACCELERATION The reverse process aims to learn the temporal inversion of
the forward process, i.e., starting from the noise distribution
A. Diffusion Theory
p(xT ) = N (0, I) and progressively denoising to recover the
Diffusion models have emerged as one of the most in- data distribution. The reverse process is also modeled as a
fluential generative models in recent years, with their core Markov chain
concepts rooted in the physical diffusion process. These mod- T
els simulate the random diffusion phenomenon of particles in Y
pθ (x0:T ) := p(xT ) pθ (xt−1 |xt ), (5)
thermodynamics, constructing a forward diffusion and reverse t=1
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 6

TABLE I
C OMPARISON OF D IFFUSION M ODEL ACCELERATION T ECHNIQUES .

Category Core Mechanism / Representative Advantages Limitations

Technique Work
Reduces sampling steps
Student model learns teacher’s Requires costly retraining; quality drops
GD-Distill [21], LCM [22], MCM [23], dramatically while maintaining
Distillation denoising mapping to compress long at extreme few-step generation; weak
RG-LCM [24], DAP [29]. quality; flexible for
sampling trajectories. cross-domain generalization.
image/video generation.
Limited real speedup due to hardware
Reduce redundant parameters sparsity support; potential loss of fine
Remove redundant parameters or
Pruning & SPD [34], TM-SD [35], [36], VSA [40], and computation with minimal details or temporal stability;
weights; structured/unstructured
Sparsification DFA [37], ToMA [39]. quality loss; compatible with task-specific robustness; incompatible
sparsity in U-Net or DiT.
existing architectures. with efficient attention variants; cannot
mitigate iterative sampling latency.
Strong hardware dependency and poor
Significantly reduce inference
portability; communication overhead
latency through kernel
Introduce concurrency at FA2 [45], SpecDiff [46], scales poorly with multi-GPU setups;
Parallel optimization, distributed
operator/timestep/task level; SpecSampling [47], APD [48], limited benefit on single-GPU systems;
Acceleration scheduling, and speculative
multi-GPU and pipelined execution. STADI [49]. parallelization may break
decoding; achieve real-time
temporal/spatial consistency and cause
generation on high-end GPUs.
quality instability.
Greatly reduce sampling steps Limited by iterative nature; sensitive to
Optimize numerical integration and
Sampler PNDM [57], DPM-Solver [12], while preserving quality; score and guidance variations; may lose
timestep scheduling to reduce
Optimization UniPC [59], RAPID³ [60], S4S [61]. broadly applicable to existing details or stability under few steps;
diffusion iterations.
diffusion models. weak cross-model robustness.

where each reverse transition kernel is parameterized as for t = T, T − 1, . . . , 1: use the trained network to predict
noise ϵ̂ = ϵθ (xt , t), compute the denoised mean
pθ (xt−1 |xt ) = N (xt−1 ; µθ (xt , t), Σθ (xt , t)). (6) 1

βt

µ= √ xt − √ ϵ̂ , (9)
αt 1 − ᾱt
In practice, the covariance matrix Σθ (xt , t) is set to fixed
values or time-dependent functions [56], while the mean sample the next state xt−1 = µ + σt z (where z ∼ N (0, I)),
µθ (xt , t) is modeled through neural networks. A key insight of and finally return the generated sample x0 . Standard DDPM
DDPM [53] is reparameterizing the mean as noise prediction: requires complete T -step iteration (usually T = 1000), leading
  to slow inference speed. Subsequent works such as DDIM [54]
1 βt and DPM-Solver [12] reduce the number of steps to dozens
µθ (xt , t) = √ xt − √ ϵθ (xt , t) , (7)
αt 1 − ᾱt through deterministic sampling or high-order numerical meth-
ods, significantly improving speed while maintaining quality.
where ϵθ (xt , t) is a neural network that predicts the noise ϵ From a broader theoretical perspective, beyond the DDPM
added to x0 at time t. This parameterization transforms the viewpoint, the theoretical framework of diffusion models can
complex distribution modeling problem into a relatively simple also be unified and generalized through Score Matching and
noise regression problem [53]. Stochastic Differential Equations (SDEs) [55]. Although these
Based on the above definitions of forward and reverse perspectives differ in mathematical form, they ultimately all
processes, the training objective of diffusion models can be point to the core idea of learning the data distribution gradient
clearly constructed. The core is to minimize the KL divergence (the score function) through neural networks, and provide a
between the reverse process and the true posterior distribution. theoretical basis for the design of more flexible samplers, such
Through variational lower bound (ELBO) [4] derivation, the as ODE solvers.
final training loss can be simplified to In recent years, Flow Matching [63] has emerged as a
further theoretical development of diffusion models, providing
L = Et∼U [1,T ],x0 ∼q(x0 ),ϵ∼N (0,I) λ(t)∥ϵ − ϵθ (xt , t)∥2 , (8)
 
a more concise and efficient training paradigm building upon
DDPM. Unlike traditional DDPM which relies on stochas-
where λ(t) is a weighting function and xt is computed from tic diffusion processes and complex variational lower bound
x0 and ϵ through the reparameterization trick. The intuitive derivations, Flow Matching directly learns deterministic Con-
meaning of this loss function is to train the network to predict tinuous Normalizing Flows (CNF) [64] from noise distribution
the noise added to clean images at given noise levels. The to data distribution. Specifically, given a target probability
specific training process includes sampling clean images x0 path pt (x) and its corresponding generating vector field ut (x),
from the dataset, randomly sampling time step t and noise Flow Matching trains a neural network vθ to regress the target
ϵ, computing xt through the forward process, training the velocity field:
network ϵθ to predict noise ϵ, and finally computing prediction
LF M (θ) = Et,pt (x) ∥vθ (x, t) − ut (x)∥2 ,
 
error and backpropagating to update parameters [53]. (10)
After training, the process of generating new samples is where t ∼ U [0, 1]. Since the global probability path pt (x) is
sampling from the reverse Markov chain. First, sample from difficult to construct directly, Conditional Flow Matching [63]
the prior distribution xT ∼ N (0, I), then iteratively denoise only requires defining conditional probability paths pt (x|x1 )
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 7

DeepCache, FasterDiffusion, PAB,

Static-caching
FORA, Δ-DiT, FasterCache, ToCa

TeaCache, VCUT, LazyDiT, T-GATE V1/V2,

Timestep-Adaptive Caching
Chipmunk, MagCache, EasyCache
Caching methods

BlockCache, HarmoniCa, AdaCache,

Layer-Adaptive Caching
FEB-Cache, DBCache, Foresight

Dynamic-caching

TaylorSeer, AB-Cache, HiCache, FoCa,

Predictive-caching
FreqCa, DiCache, FastCache

ClusCa, SpeCa, OmniCache, HyCa, TokenCache,

Hybrid Caching
FISEdit, FlexCache, RainFusion, ProfilingDiT, BlockDance

Diffusion Cache
EEdit, ControlNet, EVCtrl,
Image and Video Editing
Follow-Your-Emoji-Faster

3D Generation Hash3D

Audio Generation DiTReducio

Applications Super-Resolution HiCache

Genie 3,
World Models
GameCraft, Matrix-Game 2.0

Lumina-DIMOO,
Diffusion Large Language Models
dLLM-Cache

AI for Science AB-Cache

Fig. 2. Overview of Diffusion Cache Framework The upper panel illustrates our unified taxonomy that categorizes caching methods into Static and
Dynamic paradigms, with Dynamic Caching further divided into Timestep-Adaptive, Layer-Adaptive, Predictive, and Hybrid strategies. The lower panel shows
applications across diverse generative tasks.

and corresponding conditional velocity fields ut (x|x1 ) at the B. Caching Acceleration Techniques
sample level, thus avoiding explicit modeling of global paths
and greatly simplifying the training process. Rectified Flow 1) U-Net Architecture: The U-Net architecture adopts a
[65], as an important variant of Flow Matching, learns straight- symmetric encoder-decoder design, capturing contextual in-
line interpolation paths from data x0 to noise x1 , reducing the formation through a contracting path and achieving precise
number of ODE solver steps and further improving sampling localization through an expanding path [67]. Given noisy input
efficiency. Flow Matching has gradually become a mainstream xt ∈ RH×W ×C , U-Net consists of three core components:
approach in the diffusion model field and has been widely encoder, bottleneck layer, and decoder. The encoder extracts
adopted in large-scale generative models including Stable hierarchical feature pyramids {henc enc enc
1 , h2 , . . . , hL } through
Diffusion 3 [66] and FLUX [6], providing a more flexible the- consecutive downsampling operations, while the decoder re-
oretical foundation for subsequent model architecture design constructs output through skip connection mechanisms, con-
and caching optimization techniques. catenating encoder features with upsampled features, where
Concat(·, ·) represents the feature concatenation operation.
The caching opportunities in U-Net arise from the evolu-
tionary differences of features across layers in the temporal di-
From a computational perspective, the Iterative Inference mension. Empirical studies [68] reveal that high-level features
Mechanism of diffusion models is inherently a highly redun- henc
L encode global semantic information and exhibit signif-
dant process. Due to the continuous and progressive nature of icant temporal consistency between adjacent denoising steps,
denoising steps in the temporal dimension, feature represen- while low-level features henc
1 are responsible for detail texture
tations between adjacent time steps tend to be highly similar, reconstruction and demonstrate higher sensitivity to time step
and the model’s intermediate layer activations and attention changes. This evolutionary difference across layers provides
structures also exhibit significant temporal correlations. This theoretical justification for designing layer-selective caching
computational overhead means that there are many reusable strategies. Slowly-changing high-level semantic features can
intermediate results in the multi-step denoising process. Based be cached and reused while maintaining dynamic updates
on this characteristic, researchers have proposed the Diffusion of low-level texture features, thereby significantly reducing
Caching mechanism. By caching and reusing computational computational overhead while preserving generation quality.
results with high correlation between adjacent time steps, it 2) Transformer Architecture: Diffusion Transformers (DiT)
effectively reduces unnecessary calculations without changing [5] shift the image processing paradigm from convolution op-
the model structure and parameters, thereby significantly im- erations to attention mechanisms, adopting Vision Transformer
proving inference efficiency and scalability. [69] design principles. Given latent representation zt , DiT first
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 8

decomposes it into non-overlapping patch sequences This strategy effectively skips redundant forward computations
for these N − 1 steps, yielding an (N − 1)-fold computational
xt = Patchify(zt ) ∈ RNp ×d , (11) acceleration within each cache reuse interval.
The acceleration effect of caching strategies stems from
where Np = (H/p) × (W/p) is the number of patches and
significant reduction in computational complexity. Based on
p is the patch size. The overall architecture consists of L
standard algorithmic complexity theory [71], consider the
DiT blocks in sequence (where L denotes the total number of
complete inference process with T total denoising steps, where
layers), with each DiT block gl (l ∈ {1, 2, . . . , L}) adopting
a caching strategy performs full computation at m steps while
Transformer design but injecting time step and class conditions
the remaining (T − m) steps reuse cached features. This
through Adaptive Layer Normalization (AdaLN) mechanisms:
reduces the total computational complexity from O(T · C1 ) to
gl = AdaLN(MLP(AdaLN(Attention(x), t, c)), t, c), (12) O(m · C1 + (T − m) · C2 ), where C1 denotes the complexity
of full forward computation and C2 denotes the complexity
where of cache retrieval. Since cache access is typically orders of
magnitude faster than full computation (i.e., C2 ≪ C1 ), this
AdaLN(x, t, c) = γ(t, c) · LN(x) + β(t, c), (13) yields an acceleration factor of approximately T /m.
However, naive caching strategies face fundamental chal-
and parameters γ, β are computed from time step t and
lenges when dealing with the dynamic evolution of features.
condition c through MLP networks [5], [70].
The denoising process in diffusion models is inherently dy-
Unlike U-Net’s spatial hierarchy, DiT’s optimization space
namic, with feature representations continuously changing
is primarily manifested in two aspects. First, the temporal sta-
along the temporal dimension—remaining relatively stable
bility of attention patterns, where token-wise attention weight
over short intervals but exhibiting significant drift as time ac-
matrices Aij = softmax(QK T ) exhibit relatively stable pat-
cumulates. Directly reusing historical features therefore leads
terns during specific stages of the denoising process, providing
to cumulative errors over multiple inference steps. This char-
possibilities for caching attention results. Second, each DiT
acteristic necessitates that caching strategies strike a balance
block gl typically comprises multiple computational modules
l l between acceleration gains and error control, which constitutes
including self-attention FSA , cross-attention FCA (when ap-
the core design objective of subsequent caching methods.
plicable for conditional generation), and feed-forward network
l
FM LP . These components demonstrate different sensitivities
to timesteps, with FCAl
having higher caching potential due to C. Static Caching Methods
its dependence on relatively stable conditional information. Static Caching uses a fixed reuse strategy in which caching
To establish a unified analytical framework, this subsection is performed at predefined layers or timesteps and remains
formally defines caching acceleration techniques. The essence constant across all inference runs, independent of content
of caching acceleration techniques lies in identifying and or input. This approach is simple to implement and highly
exploiting computational redundancy in inference processes. stable, but it lacks flexibility as it does not dynamically adjust
The caching acceleration techniques in diffusion models according to content changes during the process.
share similar design principles with the KV-Cache mechanism As one of the earliest works to introduce caching mech-
in large language models (LLMs). In LLM autoregressive anisms into diffusion models, DeepCache [68] utilizes the
generation, the Key and Value matrices of already-generated observation of the U-Net structure. It takes advantage of the
tokens are fixed and unchanging, and KV-Cache avoids re- fact that the Upsampling layer features change little during
dundant computation on historical sequences by caching and the continuous denoising process, and it preserves the com-
reusing these matrices. Diffusion model caching strategies putations for the Downsampling layers while directly reusing
similarly aim to reduce redundant computation through reusing the upsampling features. DeepCache adopts a fixed-interval
historical computation results. strategy, performing a complete computation of the entire U-
Specifically, we define the caching operation for any com- Net at a specific time step t. During this computation, it caches
t
putational module (whether U-Net’s convolutional layers or the upsampling layer features Um+1 (·) generated by the main
DiT’s Attention and MLP blocks) as follows. Let N denote the branch, specifically caching the feature results after the m+1-
cache reuse interval, indicating that cached features computed th upsampling block layer:
at step t − N are reused for the subsequent N − 1 timesteps. t
Fcache t
← Um+1 (·) (16)
For the l-th layer of the network, the caching process first
computes and stores the output at timestep t: In the subsequent time steps, the newly computed downsam-
t−1
pling layer features Dm (·) are concatenated with the cached
Ctl := F l (xt ), (14) t
upsampling layer features Fcache retrieved from the cache, and
used as the input for the m-th upsampling block:
where F l denotes the l-th layer’s forward function and xt is
the input at time t. Within the following N − 1 steps, these t−1 t−1 t


Um ← Concat Dm (·), Fcache (17)
cached features are directly reused for intermediate timesteps
{t − 1, t − 2, . . . , t − (N − 1)}: DeepCache leverages the “laziness” of the upsampling layer
feature changes, using caching to avoid redundant computa-
F l (xt−k ) ≈ Ctl , k ∈ {1, 2, . . . , N − 1}. (15) tions. At the same time, it retains the frequent updates of the
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 9

downsampling layer features to ensure that the quality and the major computational cost of the DiT model by introducing
details of the generated images are progressively improved. a universal caching mechanism for the Self-Attention and MLP
FasterDiffusion [72] quantitatively measures the inter-step layers. With a fixed interval N , after a full computation, all the
feature differences across layers of a U-Net based model corresponding output features of the model layers are cached.
and observes that the encoder features exhibit significantly Initially, the model performs a full computation for all layers
smaller magnitude and variance of change compared to those of the DiT block. For each layer k, the computed Attention
k
in the decoder. This finding reveals the stability of encoder Features are stored in Fattn , and the computed MLP Features
k
representations and the dynamic nature of decoder features. are stored in Fmlp . For the next N − 1 time steps, until
Based on this core insight, FasterDiffusion introduces Encoder the next cycle begins, the model reuses the cached features
Propagation. Specifically, the diffusion process is divided into instead of recomputing them. This means that during these
k
key timesteps and non-key timesteps. At a key timestep t, the steps, for any layer k, the model retrieves features from Fattn
k
model performs a full forward pass and stores the multi-level and Fmlp , avoiding the computational cost of full computation
outputs of the encoder E. During the subsequent K non-key for each step. FORA is a simple and effective cache mechanism
timesteps t − 1, t − 2, . . . , t − k + 1, the model completely proposed for any DiT-based model. It provides a static caching
skips the computation of the encoder E, while the decoder paradigm for DiT-based models and has inspired subsequent
D reuses the cached encoder features from timestep t. Since improvements in this area.
multiple decoder steps share identical encoder inputs, their Considering the differences among different DiT layers, ∆-
computations can be executed in parallel, thereby breaking DiT [75] delves deeper into the correlation between different
the traditional sequential constraint and achieving substantial layers of DiT and image generation. The study finds that the
inference acceleration. front layers of DiT are related to the overall contours of the
To compensate for the slight texture detail loss potentially generated image, while the rear layers are related to the details.
caused by encoder propagation, FasterDiffusion further intro- Based on this insight, ∆-DiT caches the rear layers during the
duces a lightweight mechanism called Prior Noise Injection. early generation stages (contour phase) and caches the front
In the late stage of generation (t < τ ), a small proportion α layers during the later stages (detail phase), achieving adaptive
of the initial noise latent zT is injected into the current latent acceleration that aligns with the generation process.
variable zt , i.e., Unlike the residual form of U-Net, directly caching the
zt = zt + α · zT . (18) output feature maps of DiT layers would completely lose the
information from the previous sampling step xt−1 that the
This simple operation effectively enhances the high-frequency block depends on. To address the information loss caused
details of generated images while incurring almost no addi- by direct caching, ∆-DiT propose ∆-Cache, which caches
tional computational cost [72]. the “increment” (residual) of the features instead of their
Unlike the U-Net architecture, the DiT architecture does not absolute values. Specifically, for k-th layer of DiT, ∆-Cache
have the structural characteristics of upsampling and down- caches F k (xt ) − xt . In the next step t − 1, the approximation
sampling layers, which makes the DeepCache method based calculation can be represented as xt−1 + (F k (xt ) − xt ). This
on structural features inapplicable to the DiT-based generative method incorporates the previous sampling result xt−1 into the
models. Methods represented by PAB [73] focus on the trend calculation, avoiding information loss. ∆-DiT is particularly
of changes in the attention mechanism during the denoising suitable for DiT’s isotropic architecture, as the input and
process. They find that different types of attention outputs have output feature maps of each block are of consistent scale,
different redundancies. Specifically, spatial attention shows the making it easier to compute the difference. However, ∆-
largest variation, involving high-frequency elements such as DiT also has limitations, as the caching mechanism is overly
edges and textures; temporal attention exhibits mid-frequency aggressive, skipping a large number of DiT layers at once.
changes related to motion and dynamics in videos; and cross- This high acceleration ratio can lead to content degradation.
modal attention is the most stable, linking text with video FasterCache [76] further optimizes time redundancy intro-
content, similar to low-frequency signals that reflect textual duced by the unconditional output of CFG calculations. It
semantics. Based on the redundancy of different types of atten- utilizes a weight function w(t) that linearly increases from
tion, PAB designs the Pyramid Attention Broadcast to reduce 0 to 1 with each time step t, to blend the cached features
t t+2
unnecessary attention computations. It customizes different (such as Fcache and Fcache ), thereby preserving subtle changes
broadcast ranges for each attention type and MLP layers based during the denoising process. Additionally, CFG-Cache is
on their stability and output variation rate. Attention types with introduced, which leverages frequency domain decomposition.
more changes and fluctuations are assigned smaller broadcast It uses high-pass (HPF) and low-pass (LPF) filters to cache
ranges, while those with fewer changes are given larger ranges, the difference between conditional and unconditional outputs.
forming a “pyramid” strategy. In subsequent inference steps, the unconditional output can
FORA [74] provides a more comprehensive analysis of the be efficiently computed using the cached frequency domain
diffusion process, finding that the output images of consecutive information and adaptive weights w1 (t) and w2 (t), avoiding
time steps during sampling exhibit significant visual similarity, the need for full unconditional forward propagation.
which is highly correlated with the similarity of hierarchical The recently proposed FORA and ∆-DiT directly applied
features. Based on the constancy of the Diffusion model struc- prior caching methods to DiT, but they did not fully analyze
ture throughout the entire sampling process, FORA addresses or leverage the specific characteristics of the Transformer
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 10

architecture. ToCa [77] first investigates how caching tech- 1) Timestep-Adaptive Caching Methods: Diffusion Models
niques impact DiT in the token level, revealing that different achieve high-fidelity generation through a gradual denoising
tokens exhibit varying sensitivity to feature caching. This process, yet this multi-step inference inherently involves sub-
difference arises from two main factors: first, different tokens stantial temporal redundancy. Early caching methods (e.g.,
have different redundancies in the time dimension; second, the DeepCache and FORA) typically adopt fixed-interval static
same caching error on different tokens can lead to significantly strategies, where cached features from one forward pass are
different biases in the final generated output. ToCa provides reused for k consecutive timesteps. Although such strategies
a fine-grained caching strategy for tokens within the same reduce computation to some extent, they implicitly assume that
layer and across different layers, allowing for the accurate feature variations are uniformly distributed across timesteps.
selection of tokens suitable for feature caching with minimal However, empirical studies reveal that the evolution of fea-
computational cost. tures in the diffusion process is highly non-uniform: features
Furthermore, ToCa [77] defines four scoring metrics for vary smoothly and are highly reusable during early stages,
token selection from two perspectives: Temporal Redundancy while rapid changes occur in later stages, leading to reduced
and Error Propagation. These scores can be obtained without correlation. Consequently, fixed-interval reuse often results in
introducing additional computational overhead. The caching accuracy degradation and generation quality loss.
score for each token is defined as: Timestep-Adaptive Caching methods dynamically adjust
caching activation and refresh timing based on the varying
4
X stability and dynamics of features across diffusion stages.
S (xi ) = λj sj (xi ) (19) By estimating model outputs, internal feature dynamics, or
j=1 temporal stability online, these approaches flexibly determine
when to compute and when to reuse, significantly reducing
The tokens with the lowest scores are selected as caching
redundant computation while maintaining visual fidelity.
candidates:
As representative method, TeaCache [78] introduces an
input-side signal-based dynamic change estimation mecha-
ICache = arg min {S (x1 ) , S (x2 ) , . . . , S (xn )} nism. It computes the difference between noise-conditioned
{i1 ,i2 ,...,iR%×N } features modulated by timestep embeddings to predict output
⊆{1,2,...,n}
variations between adjacent timesteps, thereby deciding when
(20)
to refresh cached features. TeaCache defines the relative L1
In ToCa, the overall caching ratio is first set as a global
difference between adjacent outputs Ot and Ot+1 as:
hyperparameter R0 , which controls the allowable caching
scale during inference. However, a single global ratio cannot ∥Ot − Ot+1 ∥1
L1rel (O, t) = . (22)
adequately address the varying needs across different layers, ∥Ot ∥1 + ∥Ot+1 ∥1
token types, and time steps. To this end, ToCa introduces a Since input-level differences and actual output variations differ
multidimensional dynamic adjustment mechanism based on in scale, polynomial fitting is used for correction:
the global ratio, where the specific caching ratio for the l-th
layer, token type type, and time step t is defined as: ŷ = a0 + a1 x + a2 x2 + · · · + an xn , (23)
where ai are the fitted coefficients. During inference, the
Rl,t,type = R0 × rl × rtype × rt (21) model accumulates corrected estimates and performs a full
computation only when the cumulative variation exceeds a
By leveraging the differences between tokens, ToCa threshold δ:
achieves significantly better acceleration performance in image b −1
tX
and video generation compared to existing caching methods, ˆ rel (O, t) < δ.
L1 (24)
with almost no loss in generation quality. It provides a new t=ta
approach for token-wise optimization in the DiT-based model. This change-driven refresh paradigm allows TeaCache to
substantially reduce redundant computation while maintain-
ing visual fidelity. Owing to its excellent trade-off between
D. Dynamic Caching Methods
efficiency and generation quality, TeaCache has become one
Unlike the fixed strategy of Static Caching, Dynamic of the most influential and widely adopted caching methods
Caching introduces an error checking mechanism, represent- in diffusion acceleration research and practical systems.
ing the evolution of caching technology towards refinement. Another research direction recognizes that different
At each layer or step of inference, it dynamically decides diffusion stages exhibit distinct computational demands.
whether to perform computation, update the cache, or di- VCUT [79] systematically analyzes the role of cross-attention
rectly use the cache based on predefined metrics (such as in Stable Video Diffusion and finds that due to the global
feature similarity, L1-Norm, etc.). This adaptability enables pooling property of CLIP embeddings, cross-attention con-
“on-demand computation” and achieves a higher acceleration tributes minimally—particularly in mid-to-late stages. Based
ratio. The adaptive adjustment of Dynamic Caching can be on this, VCUT divides the diffusion process into a semantic
expanded along multiple dimensions, which will be introduced binding stage and a quality refinement stage: the former
separately below. ensures semantic consistency, while the latter focuses on
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 11

detail enhancement. During the semantic binding stage, VCUT proposes a unified amplitude decay law. It shows that the
computes conditional linear outputs as: magnitude ratio of adjacent residuals
1 c,m ∥rt ∥2
Lτ (y) + Lc,m


M= ∅ , m ∈ [1, l], (25) rt = vθ (xt , t) − xt , γt = (29)
2 ∥rt−1 ∥2
and reuses these cached features in later stages, effectively monotonically decreases over time, with nearly constant di-
removing redundant cross-attention computation. This stage- rectional consistency (token-level cosine distance ≈ 0). Thus,
based reuse strategy achieves substantial computational sav- skip-step error can be modeled as a geometric decay:
ings without compromising semantic consistency.
t
In contrast to heuristic or stage-based estimation, Lazy- Y
εskip (t̂, t) = 1 − γi . (30)
DiT [80] explicitly learns when to skip computations across
i=t̂+1
timesteps. Its key insight is that, during diffusion inference,
certain modules (e.g., attention or feed-forward layers) pro- By maintaining an accumulated error Et and refreshing when
duce nearly identical outputs between adjacent timesteps and it exceeds a threshold, MagCache achieves robust performance
can thus be reused. LazyDiT introduces a linear predictor across models and prompts without offline fitting.
before each Transformer layer to learn a similarity function Finally, EasyCache [85] emphasizes a fully online self-
based on a first-order Taylor approximation: correction mechanism. Analysis of DiT inference trajectories
Φ Φ reveals a nearly constant relative transformation rate:
f (Yℓ,t−1 , Yℓ,t ) ≈ ⟨WℓΦ , Zℓ,t
Φ
⟩. (26)
∥vt − vt−1 ∥
If the predicted similarity exceeds a threshold, the computation kt = , (31)
∥xt − xt−1 ∥
for that layer is skipped and the cached output is reused.
During training, a lazy loss is introduced to encourage the indicating strong local linearity in diffusion processes. Hence,
model to learn optimal skipping behavior: the transformation vector from the last full computation ∆i =
vi − xi can approximate future outputs:
L B L B
1 XX 1 XX
Llazy,t = ρattn (1−sattn )
l,t b +ρ feed (1−sfeed
l,t )b . v̂t = xt + ∆i . (32)
B B
l=1 b=1 l=1 b=1
(27) An accumulated deviation indicator
Experiments demonstrate that LazyDiT maintains comparable t
generation quality even when skipping approximately 50%
X ki ∥xn − xn−1 ∥
Et = εn , εn ≈ × 100% (33)
of layer computations, offering a fine-grained and learnable n=i+1
∥vn−1 ∥
framework for timestep-adaptive acceleration.
At a deeper level of temporal decomposition, the T-GATE is used to monitor caching error in real time. The cache is
series reveals distinct functional roles of self-attention and refreshed only when Et ≥ τ , forming a purely online gating
cross-attention across diffusion stages. Studies show that cross- mechanism free from offline calibration. EasyCache dynami-
attention dominates semantic planning in early stages but cally adapts computational intensity according to model state,
contributes little later, while self-attention becomes critical for maintaining a stable balance between inference efficiency and
fine-grained detail synthesis. Accordingly, T-GATE V1 [81] generative quality.
caches cross-attention outputs after the semantic stage: 2) Layer-Adaptive Caching Methods: In diffusion mod-
n o els, different network layers exhibit distinct temporal behav-
F = 21 (C∅m,i + Ccm,i ) i ∈ [1, l] , (28) iors during the generation process. Shallow layers primarily
capture low-level textures and local spatial details, showing
and reuses them in subsequent stages. T-GATE V2 [82] relatively smooth and stable variations across timesteps. In
further introduces a phase-based policy: adopting a “warm- contrast, deeper layers encode high-level semantics and global
up–interval reuse” schedule for attention, skipping compu- structures, whose representations evolve more rapidly and
tations in early steps and performing full calculations later, irregularly over time. Such layer-wise diversity in temporal
achieving a balanced trade-off between speed and fidelity. dynamics results in non-uniform feature stability across the
Chipmunk [83] redefines the granularity of timestep redun- network, implying that a single, fixed caching policy cannot
dancy by extending caching to the activation level. By observe optimally accommodate all layers.
that 5%–25% of activations account for 70%–90% of total To address this issue, Layer-Adaptive Caching Methods
variation across timesteps, they propose a sparse incremental aim to determine both where to compute and where to reuse
computation framework: performing full computation (“dense within the model, adaptively adjusting the caching and update
steps”) intermittently and caching key indices, while in frequency of each layer according to its dynamic characteris-
“sparse steps” recomputing only high-contribution activations tics—such as gradient magnitude, feature difference, or sim-
and reusing the rest. With a hardware-friendly column-sparse ilarity metrics. Compared with timestep-adaptive approaches,
implementation, Chipmunk significantly accelerates inference layer-adaptive caching focuses on the structural heterogeneity
while preserving generation quality. within the network, providing fine-grained control over com-
Transitioning from empirical fitting to physical modeling, putational allocation and achieving efficient acceleration while
MagCache [84] investigates residual evolution dynamics and maintaining generation stability and fidelity.
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 12

Cache Me if You Can [86] first proposed Block Caching, adaptivity. Video diffusion models must process both inter-
whose core idea is to exploit the redundancy of internal frame temporal continuity and large-scale spatial features, with
computations within the denoising network. Instead of treating significant motion variations across regions. AdaCache refor-
the denoising U-Net as a black box, the researchers analyzed mulates caching as a content-adaptive dynamic scheduling
the behavioral changes of its internal blocks during itera- problem. Specifically, it measures the residual variation:
tive inference. They found that the outputs of U-Net blocks
clt = ∥rtl − rt−k
l
∥1 , (37)
vary smoothly across timesteps, with different patterns across
layers, but with overall minimal inter-step differences. They to quantify changes in the residuals of layer l across diffusion
defined the relative absolute variation between blocks as: steps, and introduces a motion regularization term
∥Ci (xt , st ) − Ci (xt−1 , st−1 )∥1 ∂mlt
L1rel (i, t) = , (34) mgtl = , (38)
∥Ci (xt , st )∥1 ∂t
where Ci denotes the core computation output of the i- to characterize motion intensity in latent space. These are
th block. Based on this metric, Block Caching caches the combined into a composite metric c̃lt = αclt + βmgtl , enabling
block output computed at timestep ta and reuses it until the aggressive caching in static regions and frequent updates in
cumulative variation exceeds a threshold δ: dynamic ones. This strategy achieves content-aware hierarchi-
b −1
tX tb
X cal scheduling for video diffusion, significantly accelerating
L1rel (i, t) ≤ δ < L1rel (i, t). (35) inference while maintaining high generation quality.
t=ta t=ta FEB-Cache [89] develops a frequency-domain-driven
This method achieves fine-grained block-level scheduling: caching mechanism to mitigate exposure bias. It shows that
computationally expensive blocks (e.g., SpatialTransformer) Attention and MLP layers exhibit complementary spectral
are cached when variation is small, while detail-sensitive sensitivities: Attention emphasizes low-frequency structures,
blocks (e.g., ResBlock) are refreshed more frequently. To while MLP captures high-frequency details, making unified
alleviate feature misalignment artifacts, the authors introduced caching suboptimal and prone to error amplification. FEB-
a lightweight scale-shift adjustment mechanism, applying Cache introduced dynamic noise scaling:
(
timestep-dependent scaling and shifting to cached features, a · εt , t > T0 ,
and employing a student–teacher distillation process to en- εt → (39)
b · εt , t ≤ T0 ,
sure feature distribution consistency. Overall, Block Caching
significantly accelerates inference while maintaining or even and constructed a frequency-oriented cache table that priori-
improving visual quality. tizes caching MLP modules in early stages and Attention mod-
However, non-learning caching methods that rely on fixed ules in later stages to align with the stage-specific frequency
rules or thresholds still suffer from distribution mismatch composition of exposure bias. By comparing the error of three
between training and inference. To mitigate this limitation, caching states: NoCache, AllCache, and StageSpecific, the op-
HarmoniCa [87] reconceptualizes learning-based caching from timal strategy is selected to dynamically balance acceleration
a system-level perspective, enabling adaptive and efficient fea- and quality across stages.
ture reuse. The method pointed out that traditional Learning- In contrast to methods requiring retraining or frequency
to-Cache frameworks optimize only randomly sampled single statistics, DBCache [90] proposed a training-free Dual Block
timesteps during training, preventing the model from learn- Caching framework. Inspired by U-Net architecture design,
ing the true temporal dependencies and error accumulation DBCache divides the Transformer block stack of DiT into
patterns observed during inference. To overcome this, Har- three functional segments: a front section (controlled by Fn )
moniCa introduced Stepwise Denoising Training (SDT), which acting as a “probe” that performs full computation to capture
performs full denoising trajectories from Gaussian noise xT to residual signals for comparison with the previous step; a mid-
the final image x0 during training, enabling the student model dle section serving as the main caching region, skipping com-
to perceive the historical influence of caching decisions under putation and reusing cached outputs when residual changes fall
teacher guidance. Furthermore, it proposed the Image Error below a threshold residual diff threshold; and a rear section
Proxy Objective (IEPO) as follows: (controlled by Bn ) functioning as a “corrector” that always
N −1 recomputes to fuse and correct possible deviations. This
(t) (t) “probe–decision–correction” loop enables structured caching
X
LIEPO = λ(t) LMSE + β rt,i , (36)
i=0 that can be flexibly applied to DiT models.
(t)
Foresight [91] introduces a foresight-driven dynamic
where λ(t) = ∥x0 − x0 ∥2F dynamically reflects the quality caching framework that leverages layer-wise feature hetero-
(t)
gap between generated and target images, LMSE maintains P geneity in video diffusion models. Since shallow layers exhibit
teacher–student consistency, and the regularization term rt,i high temporal similarity while deep layers vary more rapidly,
encourages the model to reuse cached results through learnable Foresight estimates per-layer variation thresholds during a
caching weights. This objective optimizes for final image warm-up stage to guide adaptive caching throughout inference.
quality and eliminates the training–inference distribution gap. W P
!
For the more complex scenario of video generation, Ada- x
X 1 1 X i i 2
λl = (x (t) − xl (t − 1)) , (40)
Cache [88] further extended caching with spatiotemporal 10W −t P i=1 l
t=W −2
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 13

where P denotes the number of feature elements. During similarity pattern” in diffusion models from the perspective of
inference, the reuse or recomputation of each layer is decided numerical integration. This work models the denoising process
in real time based on the dynamic reuse metric δl (t) and layer- of diffusion models as a numerical solution problem for
specific threshold: ordinary differential equations, discovering linear relationships
( between outputs of adjacent steps:
t+1 C(xl ), if δl (t) ≤ γλl ,
xl = (41)
Compute, otherwise. ϵ̂θ (xtλ (λo ) , tλ (λo )) ≈ r · ϵ̂θ (xtλ (λs ) , tλ (λs )) (43)
3) Predictive Caching Methods: The iterative denoising where the scaling factor r has the specific form:
process of diffusion models can be interpreted as a continuous αt h
dynamic system, where the hidden feature evolution over r= (44)
3e−h αt h − 2σλt eλo (eh −1)
timesteps forms a trajectory that can be approximated by
numerical prediction [92]. Predictive Caching Methods lever- Furthermore, AB-Cache extends this relationship to k-th order
age this insight by extending the traditional caching paradigm Adams-Bashforth methods, establishing more general linear
from simple reuse (Cache-Then-Reuse) to forward prediction recursive relationships:
of future feature states (Cache-Then-Forecast). Instead of k
X
relying solely on stored historical features, these methods ϵ̂θ (xtλ (λn ) , tλ (λn )) ≈ ci ϵ̂θ (xtλ (λn−i ) , tλ (λn−i )) (45)
leverage cached representations from previous timesteps to i=1
explicitly forecast future features over multiple steps, using The method approximates the integral equation of the denois-
numerical solvers or derivative-based predictors. ing process
TaylorSeer [92] serves as a pioneering work in the pre-
Z λt
dictive caching domain, systematically challenging the the-
oretical foundation of the traditional “Cache-Then-Reuse” I= e−τ ϵ̂θ (xtλ (τ ) , tλ (τ ))dτ (46)
λs
paradigm. Existing cache-based acceleration methods for dif-
fusion models commonly assume high similarity between achieving the transformation from heuristic caching to nu-
adjacent timestep features, but this assumption often fails at merical integration methods and mathematically formalizing
high acceleration ratios, leading to rapid degradation of feature existing observational phenomena.
similarity as caching frequency increases. Through in-depth HiCache [94] tackles the numerical instability of high-order
theoretical analysis and empirical studies, TaylorSeer discov- predictive caching by identifying and overcoming fundamental
ered a key insight: while the similarity of feature values them- limitations of traditional Taylor expansion in diffusion feature
selves may decay, the evolutionary trajectory of features in the prediction. It reveals that although Taylor polynomials provide
temporal dimension exhibits high regularity and predictability. accurate local approximations, their power-function basis leads
Through Principal Component Analysis (PCA), researchers to instability when modeling oscillatory features, causing ex-
observed that both features and their temporal derivatives trapolation overshoot and error amplification in long-timestep
(i.e., the “velocity” of feature changes along trajectories) or high-order predictions. Moreover, Taylor expansion lacks
present stable patterns. Based on this discovery, TaylorSeer awareness of local geometric structures, limiting its ability to
proposed the revolutionary “Cache-Then-Forecast” paradigm, capture complex nonlinear dynamics in diffusion processes.
remodeling the problem as mathematical prediction of feature To address these issues, HiCache adopts Hermite polynomi-
trajectories. als as the prediction basis, leveraging their mathematical rigor
The core idea of TaylorSeer is to treat cached features as and physical relevance. As orthogonal functions describing
discrete samples along a continuous feature trajectory and harmonic oscillators in quantum mechanics, Hermite polyno-
model their evolution via Taylor series expansion. To estimate mials inherently handle oscillatory signals effectively.
higher-order derivatives without additional computation, it Furthermore, HiCache introduces a contraction factor σ to
employs finite difference approximations, allowing high-order control polynomial decay, softly suppressing high-order terms
feature dynamics to be inferred from a few fully computed while preserving expressiveness and mitigating numerical os-
timesteps. The prediction formula is: cillations. The prediction is formulated as:
m N
X ∆i F(xl ) t
X ∆ i Ft
Fpred,m (xlt−k ) = F (xlt ) + (−k)i (42) F̂t−k = Ft + · H̃i (−k) (47)
i=1
i! · Ni i=1
i!

where ∆i F(xlt ) is computed through finite difference methods, where H̃i (x) = σ i Hi (σx) represents Hermite polynomials
N is the sampling interval, and m is the order of Taylor expan- with contraction factor.
sion. The significance of TaylorSeer lies not only in achieving While previous predictive caching methods primarily focus
the paradigm shift from passive reuse to active prediction, but on short-term feature extrapolation, they often suffer from
more importantly in establishing the theoretical foundation for accumulated errors and instability under large step sizes.
the entire predictive caching field, inspiring subsequent series FoCa [95] first explicitly treats feature caching as a numerical
of acceleration techniques based on numerical methods. integration problem for feature ODEs. Traditional methods
As a follower of TaylorSeer, AB-Cache [93] provides (direct reuse or single-step Taylor extrapolation) only use
mathematical explanations for the widely observed “U-shaped local difference information from the most recent step, prone
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 14

to extrapolation overshoot, noise amplification, and quality variation ∆tprobe to trigger recomputation and linear interpola-
collapse during long jumps and “stiff ” phases. tion for multi-step cache fusion. FastCache [97] adopts a dual
FoCa’s technical innovation is embodied in its carefully strategy: it identifies static tokens via temporal significance
designed two-stage process. First, in the prediction phase, scoring and approximates them using linear transformation
it employs the second-order Backward Difference Formula Hst = Wc Xst + bc ; when relative change δt,l satisfies statisti-
(BDF2) for multi-step extrapolation: cal criteria, it skips transformer blocks and replaces them with
learnable linear transformations Ht,l = Wl Ht,l−1 +bl . These
4 1 2h b′
F̂k+1 = Fk − Fk−1 + F (48) methods demonstrate that intelligent computation reuse across
3 3 3 k
feature, token, and block levels can reduce computational
This design fully utilizes two-step historical information, pro- overhead while maintaining generation quality, providing com-
viding stronger noise resistance and numerical stability com- plementary approaches to numerical prediction paradigms.
pared to single-step methods. Subsequently, in the correction 4) Hybrid Caching Methods: Existing caching paradigms,
phase, it employs the Heun method for
trapezoidal integration such as timestep-adaptive and predictive caching, typically
correction: Fk+1 = Fk + h2 Fbk′ + Fbk+1
′
Although this correc- focus on a single optimization dimension—either temporal
tion step has minimal computational overhead, it significantly scheduling or numerical forecasting. However, diffusion in-
suppresses extrapolation overshoot and cumulative errors. ference involves complex interactions across timesteps, net-
FreqCa introduces a frequency-domain perspective to ad- work layers, and feature dynamics, making single dimensional
dress the inherent trade-off between feature similarity and caching strategies insufficient to fully exploit the intrinsic
temporal continuity in diffusion model caching. Through de- redundancy of the process.
tailed analysis, it reveals that the low- and high-frequency Hybrid Caching Methods address these limitations by
components of diffusion features exhibit distinct temporal jointly modeling multiple dimensions: timestep, network hi-
dynamics: low-frequency components maintain high similarity erarchy, and feature dynamics, within a unified framework.
across adjacent timesteps, whereas high-frequency compo- By integrating complementary strategies such as adaptive
nents demonstrate strong temporal smoothness. Motivated by scheduling, predictive estimation, and structural selection,
this observation, FreqCa performs frequency decomposition these methods dynamically coordinate where and when to
using a generic frequency transform D(·): compute/reuse across both spatial and temporal domains.
zt = zlow high
where zlow/high


= Plow/high D(zt ) . (49) This unified design enables flexible and robust cache reuse,
t + zt , t
significantly improving stability and generation quality under
The low-frequency components ztlow exhibit high cross-step aggressive acceleration settings.
similarity and are thus directly reused: Caching mechanisms that depend solely on temporal sim-
ilarity often fail under high acceleration ratio, as feature
ẑtlow = zt−k
low
, k ∈ {1, 2, . . . , K}. (50)
discrepancies between adjacent timesteps expand rapidly with
In contrast, the high-frequency components zthigh
display larger step sizes. To address this issue, ClusCa [98] introduces
smooth temporal evolution and are predicted using a second- spatial token similarity as an orthogonal complement to tempo-
order Hermite polynomial: ral similarity. After completing a full computation, the model
performs K-Means clustering on the token representations
∆s ′ of the final layer, obtaining K clusters where each cluster
ĥi (st ) = hi (st−1 ) + [hi (st−1 ) + h′i (st )] . (51)
2 represents a spatial region. During cache reuse, only a subset
This frequency-decoupled design bridges the advantages of the of representative tokens within each cluster are recomputed,
“Cache-Then-Reuse” and “Cache-Then-Forecast” paradigms, and the results are fused with the cached features from the
establishing a unified caching framework that achieves effi- previous timestep to approximate updates for other tokens
cient computation without compromising generation fidelity. within the same cluster:
Furthermore, FreqCa introduces the Cumulative Residual (
Feature (CRF) mechanism, which aggregates features across F(xi ), i ∈ ICompute
C(xi ) = (53)
residual connections to drastically reduce memory consump- γ · µ(i) + (1 − γ)C(xi ), i ∈/ ICompute
tion. Formally, CRF is defined as:
where µ(i) is the mean value of computed feature of tokens
L
X in ICompute , formally expressed as:
ϕL (xt ) = xt + Fl (h(l) ), (52)
P
j∈ICompute F(xj )
l=1
µ(i) = P . (54)
which reduces memory complexity from O(L) to O(1) and j∈ICompute [Ij = Ii ]
minimizes the number of frequency (inverse) transformations
required during inference. This spatiotemporal fusion mechanism significantly im-
FreqCa achieves up to 7.14× inference acceleration and proves cache utilization, reducing computation while even
99% memory savings on Qwen-Image, while maintaining enhancing generation quality.
comparable generation quality. Beyond spatial aggregation, another class of hybrid strate-
Beyond numerical prediction methods, DiCache [96] em- gies approaches caching from the Forecast-Then-Verify per-
ploys shallow features as lightweight probes, using feature spective, aiming to balance extrapolation efficiency and error
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 15

control in multi-step inference. Inspired by speculative decod- and attention to recompute only modified areas, while using
ing in large language models, SpeCa [99] constructs a dual- asynchronous pipelined caching to reduce memory usage. For
phase closed-loop framework of prediction and verification. video generation, FlexCache [103] identifies strong cross-
Specifically, a lightweight draft predictor (e.g., TaylorSeer) frame feature correlations, applying keyframe selection and
extrapolates the next k steps via a Taylor-series expansion: linear interpolation for two-stage compression. It further em-
m ploys object–background decoupled retrieval and an LRBU
X ∆i F(xl ) t
Fpred (xlt−k ) = F(xlt ) + (−k)i , (55) (Least Recently Beneficial Used) replacement policy to im-
i=1
i! N i prove cache hit rate and timeliness. Together, these methods
embody the hybrid principle of “multi-dimensional coordina-
where ∆i denotes the i-th order finite difference. The model
tion + dynamic adaptivity.”
then computes the relative error between the predicted and true
From a structural standpoint, RainFusion [104] optimizes
values through a lightweight verifier:
video diffusion acceleration through sparse attention. It iden-
∥Fpred (xlt−k ) − F (xlt−k )∥ tifies that conventional binary partitioning of attention heads
ek = , (56) (spatial vs. temporal) overlooks a third category: textural
∥F(xlt−k )∥
heads, which are crucial for modeling high-frequency se-
and compares it with a dynamic threshold τt . If ek ≤ τt , mantics. RainFusion thus introduces a ternary attention clas-
the prediction is accepted; otherwise, the model rolls back sification and an Adaptive Recognition Module (ARM) to
to the last verified state and performs a full computation for dynamically determine head types online. The ARM evaluates
correction. This design acts as a “firewall”, enabling aggressive the recall ratio under masked attention:
step extrapolation while maintaining stability. The theoretical S(Q′ , K ′ , M ′ )
acceleration ratio can be approximated as R′ = , (59)
S(Q′ , K ′ , Minit )
1 and selects the corresponding sparsity pattern when R′ exceeds
S≈ , (57)
(1 − α) + γ a threshold; otherwise, it employs checkerboard sampling to
where α is the prediction acceptance rate and γ is the preserve detail. RainFusion achieves low-cost adaptive cou-
verification cost ratio (typically much smaller than 1). SpeCa pling of sparsity and caching, significantly improving effi-
effectively suppresses error accumulation under high accelera- ciency without degrading fidelity.
tion, offering a reliable extrapolation mechanism for inference. From another perspective, ProfilingDiT [105] approaches
In contrast to heuristic methods relying on local feature caching from a semantic perspective, revealing long-term at-
similarity, OmniCache [100] revisits cache scheduling from a tention biases between foreground and background across DiT
global sampling-trajectory perspective. Empirical observations layers. Based on this, it proposes a dual-granularity adaptive
reveal that feature trajectories of diffusion models exhibit a caching framework. Across layers, it classifies modules into
“boomerang” shape in low-dimensional space, implying that a foreground set F and background set B according to the
model states evolve more smoothly in certain low-curvature foreground attention ratio:
stages, making them suitable for cache reuse. OmniCache Nhigh∩f g
thus introduces a curvature-guided global scheduling strategy, Rattn = . (60)
Nf g
reusing caches during low-curvature phases to minimize inter-
Across timesteps, it dynamically adjusts the cache interval:
ference, while employing a noise correction mechanism:
s − s0
qθ (xt , t) = ϵ̃θ (xt , t) − ϵθ (xt , t) Ts = Tmax − (Tmax − Tmin ) · , (61)
(58) S − s0
qθ (xt−1 ,t − 1) ≈ γt−1 qθ (xt , t), realizing a progressive strategy of coarse caching in early
where qθ denotes cache-induced noise. The model selectively stages and fine updates later.
applies low-pass or high-pass filtering to this noise depending BlockDance [106] analyzes the structural heterogeneity of
on the sampling stage—preserving global structure in early DiT and observes that shallow and middle-layer features
phases and enhancing fine-grained detail later. Notably, this change minimally during late denoising stages, while deep-
method requires no additional training, combining offline layer features continue to evolve significantly. This reflects the
calibration with online reuse. “structure-first, detail-later” principle of diffusion generation:
In the task- and structure-adaptive direction, researchers early stages construct global layouts, and later ones refine
have further integrated caching with importance estimation, textures. Accordingly, BlockDance proposes a hierarchical
regional sparsity computation, and multi-frame compression. caching strategy that maintains full computation during early
TokenCache [101] targets the DiT architecture by using a stages for quality assurance, while reusing shallow features
lightweight predictor to evaluate token importance and aggre- at fixed frequencies in later stages to eliminate redundancy.
gating tokens into blocks, where only low-importance blocks Further, BlockDance-Ada introduces a lightweight decision
undergo cache reuse, thereby improving inference efficiency network that leverages latent variable zp and text embedding
without sacrificing detail. FISEdit [102] focuses on image c to dynamically assess content complexity and generate an
editing tasks, leveraging an automatic masking mechanism instance-specific reuse strategy u. Its objective is defined as:
PS−ρ
(based on latent differences and OTSU thresholding) to locate ut
affected regions. It employs pixel-level sparse convolution R(u) = C(u) + λQ(u), C(u) = 1 − t=1 . (62)
S−ρ
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 16

balancing acceleration gain and generation quality, thereby temporal redundancy, or feature consistency to realize task-
enabling instance-level adaptive caching. specific computation reuse and acceleration.
HyCa [107] introduces a novel perspective on predictive
caching by addressing the key limitation of prior approaches A. Acceleration in Image and Video Editing Tasks
that assume uniform temporal dynamics across all hidden
dimensions of DiTs. In practice, diffusion features exhibit In controllable image and video generation tasks, a large
highly heterogeneous temporal behaviors—some dimensions amount of input such as background areas usually remains
undergo oscillatory or multimodal evolution, while others vary unchanged, but consumes a lot of computation. By leveraging
smoothly and predictably over time. To capture this intrinsic spatial and temporal redundancy, these areas can be efficiently
heterogeneity, HyCa models feature evolution as a mixture cached and reused. For image editing, caching intermediate
of ordinary differential equations (ODEs), enabling distinct features of unedited regions and reusing them in subsequent
subsets of feature dimensions to follow independent dynamic iterations is a natural acceleration strategy. EEdit [108] adopts
trajectories. this idea by caching and selectively updating intermediate re-
Concretely, HyCa performs unsupervised clustering of fea- sults of self-attention and MLP layers based on spatial locality
ture dimensions based on dynamic descriptors such as accel- (SLoC), thereby avoiding redundant computation without com-
eration ratios and curvature ratios, grouping dimensions with promising the fidelity of edited regions. When control signals
similar temporal behaviors. Given a solver set S, each cluster (e.g., from ControlNet [109]) are introduced, similar redun-
c is assigned its optimal solver s⋆c by minimizing the mean dancies exist across both spatial and temporal dimensions.
next-step prediction error within the cluster: For instance, EVCtrl [110] divides the network into global
" # and local functional regions, caching the global uncontrolled
C 2 features and recomputing only the locally controlled ones
X 1 X (sc ,d) (d)
min F̂t+1 − Ft+1 , (63) while performing sparse temporal updates at key frames or
{sc ∈S}C
c=1 c=1
|c| 2
d∈c denoising steps. This approach achieves more than a 2× speed-
(s ,d)
c up in tasks like CogVideo-ControlNet with negligible quality
where F̂t+1 denotes the predicted feature value of dimension
loss. In controllable video generation, adjacent frames also ex-
d at timestep t+1 using solver sc . Remarkably, these clustering
hibit high temporal similarity. Follow-Your-Emoji-Faster [111]
assignments remain stable across prompts, resolutions, and
caches features at key time steps and reconstructs intermediate
timesteps, enabling a “One-Time Choosing, All-Time Solving”
frame features via Taylor interpolation, avoiding full inference
inference paradigm in which solver selection is performed
for each frame. Furthermore, it employs facial landmarks to
once offline and reused throughout inference without addi-
guide update frequency in spatial regions, minimizing cumu-
tional overhead. By combining complementary solvers, HyCa
lative errors. This face-guided interpolation caching achieves
balances stability and adaptability, aligning caching dynamics
around 2.6× lossless acceleration while preserving fine details.
with heterogeneous feature evolution. This mixture-of-ODE
Collectively, these studies highlight the plug-and-play nature
framework offers a unified, training-free approach that im-
and strong generality of cache mechanisms in editing tasks.
proves both efficiency and fidelity in diffusion inference.
Overall, hybrid caching methods achieve fine-grained and
intelligent cache reuse by integrating cooperative mechanisms B. Acceleration for 3D Generation Tasks
across temporal, spatial, structural, and semantic dimensions. In 3D generation, both neighboring camera views and
Rather than relying on simple timestep skipping, they dy- adjacent time steps contain strong redundancy, enabling cache-
namically adjust reuse patterns guided by feature evolution, based acceleration without retraining. Hash3D [112] employs
structural divergence, and content semantics. Consequently, a hash-bucket-based feature reuse strategy that shares features
these strategies balance stability and visual fidelity under high between nearby viewpoints, reducing inconsistencies from
acceleration, providing new theoretical and practical pathways independent noise sampling and improving smoothness and
for accelerating diffusion model inference. coherence of generated 3D models. This approach achieves
up to a 4× speed-up in DreamGaussian (from 2 minutes to
IV. A PPLICATIONS OF C ACHE ACCELERATION 30 seconds) and a 3× acceleration in Zero-123 (from 20
minutes to 7 minutes). In text-to-3D generation, the speed-
With the rapid development of diffusion models in multi- up ranges from 1.5× to 1.9×, while PSNR and SSIM show
modal generation tasks, cache mechanisms have emerged as an slight improvements and LPIPS exhibits minor degradation.
essential strategy for improving inference efficiency. The core Importantly, CLIP-G scores remain comparable to the base-
idea is to reuse intermediate features across time steps or spa- line, indicating preserved semantic consistency.
tial regions to avoid redundant computation, thereby achieving
significant acceleration without sacrificing generation quality.
Cache strategies have demonstrated strong adaptability and C. Acceleration in Audio Generation Tasks
generality across various downstream applications, including Speech generation tasks can also benefit from training-
image and video editing, 3D generation, speech synthesis, free cache acceleration. DiTReducio [113] identifies two types
super-resolution reconstruction, world model construction, dis- of redundancy: temporal redundancy where model outputs
crete diffusion models and AI for Science category. Depending across adjacent denoising steps are highly similar, and branch
on the task, cache techniques often exploit spatial locality, redundancy where conditional and unconditional branches in
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 17

CFG exhibit near-identical outputs. By computing only one under the GPQA [122] benchmark, this achieves an 8.08×
branch and reconstructing the other using cached residuals, speed-up, reducing FLOPs per token from 22.07T to 2.73T.
DiTReducio achieves more accurate results than direct reuse. As the diffusion paradigm continues to converge with diverse
For TTS caching, a progressive calibration process is em- fields, cache-based acceleration techniques are expected to
ployed to determine whether the reuse-induced loss is below gain increasing attention and broader adoption.
a predefined threshold, thereby ensuring high cache precision.
Experiments show that this approach reduces computation by G. Acceleration in AI for Science
50% and improves inference speed by 30%.
In molecular geometry generation, flow-matching models
typically rely on ODE solvers requiring multiple iterative
D. Acceleration in Super-Resolution Tasks steps. Since molecular coordinates and atom types evolve
Image super-resolution, as one of the canonical diffusion smoothly across iterations, intermediate neural features exhibit
model applications, also exhibits temporal redundancy in its temporal redundancy that can be exploited via predictive
denoising process, allowing for training-free cache-based ac- caching. The AB-Cache [123] achieves with more than 3× ac-
celeration. HiCache [94] advances this direction by introduc- celeration while getting competitive results compared to 100-
ing a Hermite-polynomial-based interpolation caching scheme step full baselines across multiple metrics, including energy,
on the InfDiT model. The method achieves a theoretical strain, validity, and molecular stability on GEOM-Drugs [124]
acceleration of up to 5.93× and a practical wall-clock speed- dataset. This demonstrates that cache-based acceleration holds
up of 2.43×, while outperforming TaylorSeer [92] in SSIM substantial promise for AI4Science domains, offering efficient
and PSNR on the NTIRE 2025 [114] benchmark. and accurate computation for scientific generative modeling
tasks.
E. Acceleration in World Models
Recently, video-based world models have become a cor- V. F UTURE P ERSPECTIVES OF DIFFUSION C ACHING
nerstone of generative video modeling and interactive content In recent years, cache-based acceleration techniques have
generation. These models often follow a “base diffusion model emerged as a promising solution for speeding up diffusion
training + control signal integration → autoregressive distil- model inference by reusing intermediate representations across
lation” paradigm. High-fidelity diffusion backbones (e.g., Wan successive steps. However, this approach inherently incurs a
2.1/2.2 [2]) are trained on diverse video data and integrated trade-off between computational redundancy and memory
with external control signals such as mouse, keyboard, or overhead. Specifically, three core challenges remain to be
environment parameters. Subsequently, the model is distilled addressed: first, the substantial memory footprint required to
into a few-step autoregressive generator for efficient infer- store intermediate activations, second, potential degradation
ence. During this process, caching modules [78], [115] are in generative quality caused by cache-induced approximation
integrated into diffusion backbones to cache cross-timestep errors and third, a lack of theoretical grounding and systematic
features and reduce computational costs. Furthermore, when integration with complementary acceleration strategies. In the
the model is distilled to only 4–10 inference steps, accuracy following, we discuss each of these challenges in turn and
degradation and temporal instability can be alleviated via outline potential directions for future research.
Diffusion Caching, which reuses intermediate features from
adjacent time steps. This enables significant computational A. Memory Consumption Challenge
savings without modifying model parameters or structure,
ensuring both efficiency and temporal consistency in world Cache-based mechanisms can substantially accelerate the
model generation, as demonstrated in systems like Google diffusion process by reusing intermediate representations and
Genie 3 [116], GameCraft [117], and Matrix-Game 2.0 [118]. thereby eliminating redundant computations. However, this ef-
ficiency gain comes at the cost of increased memory consump-
tion. In particular, activations that would normally be released
F. Acceleration in Discrete Diffusion Models after each step must be retained across successive diffusion
Interestingly, even for diffusion-based discrete diffusion steps for either direct reuse or partial recomputation, leading
models, caching can effectively exploit hidden redundancies to to a significant GPU memory overhead. While such caching
achieve acceleration. This includes unified multimodal under- strategies effectively enhance computational efficiency, they
standing and generation, and discrete diffusion language mod- also place a substantial burden on the memory subsystem.
els (dLLMs). Lumina-DIMOO [119] achieves state-of-the-art Large diffusion models today often contain billions of
performance in unified multimodal understanding, generation, parameters, requiring over 10 GB of GPU memory solely
and editing tasks, while improving sampling efficiency via for parameter storage. For instance, Hunyuan-Image 3.0 [1]
a Max-Logit-based Cache (ML-Cache) mechanism, doubling comprises an enormous 80B parameters, which cannot even
sampling efficiency. Similarly, in the rapidly evolving diffusion fit within a single 80 GB GPU. When cache storage is
language modeling domain, dLLM-Cache [120] accelerates taken into account, the overall memory footprint increases
inference by combining long-interval prompt caching, short- substantially. In particular, cache memory consumption typ-
interval response caching, adaptive partial updates, and a V- ically scales linearly with the token sequence length. In
verify mechanism. On the LLaDA-8B [121] Instruct model a 512 × 512 image generation task, the KV-cache alone
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 18

can demand an additional 4–8 GB of memory. Under long- mismatches; in medical imaging or scientific visualization,
sequence or memory-constrained settings, this often results even slight distortions in tumor boundaries or vessel structures
in Out-of-Memory (OOM) errors. Moreover, state-of-the-art can significantly impact diagnostic accuracy. Consequently,
methods such as TaylorSeer further exacerbate this issue by although cache-based diffusion acceleration is highly practical
caching activations for every network layer, causing memory for amusement-level AIGC tasks such as visual effects, artistic
usage to grow linearly with network depth and introducing an creation, and digital humans, its applicability to high-precision
additional 7–8 GB of GPU overhead. generation remains limited. Current cache mechanisms are
Such substantial memory demands present serious chal- better suited for applications with moderate detail constraints,
lenges for hardware deployment. On mobile devices (e.g., while fields such as medical imaging, security recognition,
smartphones or tablets) with only 4-12 GB of unified mem- and engineering design still require solutions that balance
ory, it becomes infeasible to load both model parameters acceleration with generation quality.
and cache data concurrently. In embodied AI and robotics
applications—where power consumption and chip area are
C. Theoretical Limitations and Future Directions
tightly constrained—the high memory requirements of caching
frequently exceed available hardware capacity. Even in high- 1) Lack of Theoretical Frameworks: Although cache meth-
resolution image generation or video synthesis tasks on high- ods have achieved remarkable success in accelerating diffusion
end GPUs (e.g., NVIDIA A100 80 GB), cache usage scales model inference, with some SOTA approaches such as Tay-
approximately quadratically with sequence length, creating lorSeer reaching nearly 5× acceleration. Their development
critical memory bottlenecks. Furthermore, in multi-task con- remains largely empirical. Existing techniques are primarily
current inference scenarios, memory contention among parallel engineering-driven explorations rather than theories grounded
jobs exacerbates this problem and degrades overall throughput. in mathematical rigor. The distributional deviation introduced
To alleviate these issues, researchers have proposed several during cached inference lacks interpretable and verifiable
memory-efficient caching strategies. A representative example theoretical characterization. Current quality assurance largely
is FreqCa [125], which introduces a Memory-Efficient Feature relies on empirical parameter tuning rather than controllable,
Caching scheme. Traditional cache mechanisms store all fea- theoretically guided analysis, which is inefficient and lacks
tures from both the attention and feed-forward (FFN) layers, generalization across different tasks such as image generation,
leading to exorbitant memory costs (e.g., ToCa requires more video prediction, and 3D modeling.
than 10 GB on FLUX). FreqCa, inspired by the residual- Some studies have attempted to investigate the theoretical
network interpretation that residual connections capture an underpinnings of cache-induced errors. For instance, ERTA
ensemble of features across layers, proposes the Cumulative Cache [126] categorized these errors into two main types: Fea-
Residual Feature (CRF) caching strategy. By retaining only ture Shift and Step Amplification, providing preliminary insight
the cumulative residual feature vector, this method compresses into how caching affects the diffusion process. However, such
the 2 × L layer features into a single representation, reducing analyses remain heuristic and lack systematic mathematical
cache memory usage by up to 99%. Moreover, it cuts the derivation or unified theoretical frameworks. Their conclusions
number of frequency-decomposition operations by roughly often depend on empirical observations rather than first-
2L, such that the total latency introduced by caching remains principle reasoning from diffusion dynamics, thus providing
below 0.01% of the overall diffusion process. As a result, Fre- limited theoretical guidance for future algorithm design.
qCa markedly enhances the memory efficiency and practical Furthermore, little attention has been given to understand-
deployability of cache-based acceleration techniques. ing how caching interacts with different sampling strategies
such as DDIM or Flow Matching. The absence of a unified
framework for analyzing how cache strategies can be adapted
B. Generation Quality Degradation to diverse samplers, minimize error propagation, and enhance
Cache-based acceleration improves inference efficiency by sampling efficiency has restricted their generalization across
reusing historical intermediate results, thereby reducing redun- generation paradigms. This theoretical gap not only limits
dant computation. However, this reuse inevitably introduces the adaptability of caching in various diffusion settings but
cache-induced errors, which can lead to the loss of fine- also constrains its integration with other efficient inference
grained information during the diffusion process. As some techniques. Therefore, future research should develop a more
intermediate computations are skipped or approximated, the rigorous, unified theory of cache-induced error analysis, quan-
model’s ability to capture complex feature dependencies is titatively characterizing its effect on diffusion dynamics and
weakened, resulting in degraded generation quality. Common exploring the compatibility of cache strategies with different
artifacts include texture blurring, edge distortion, and loss of sampling approaches. Such theoretical advancements would
micro-structures. At higher acceleration ratios, these errors lay a solid foundation for high-performance and reliable
accumulate progressively, leading to noticeable deterioration diffusion inference.
in visual fidelity. 2) Integration with Other Acceleration Strategies: Cache
Such quality degradation is particularly problematic in strategies possess strong flexibility and orthogonality, making
detail-sensitive or high-precision tasks. For example, in fa- them highly compatible with other model acceleration tech-
cial generation or recognition, minor inaccuracies in local niques. When integrated with mainstream methods such as
regions (e.g., eye corners or nose wings) can cause feature model distillation, pruning, and quantization, caching can
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 19

further enhance inference speed while maintaining generation manner. In contrast, Diffusion Caching offers a new training-
quality and reducing memory footprint, achieving a balanced free and model-agnostic approach to speeding up inference. Its
trade-off between performance and efficiency. main idea is to identify and reuse redundant computations in
Recent works have explored the synergy between caching the diffusion process, thus cutting computational costs without
and other acceleration techniques. For example, DaTo [127] harming output quality. This enables a lightweight, efficient,
combines caching with token pruning to jointly improve and composable form of inference optimization. Especially
inference speed and memory efficiency through dynamic token with the rise of the “Cache-Then-Forecast” approach, which
selection and feature reuse. QuantCache [128] systematically represented by TaylorSeer, diffusion caching has achieved
studies the fusion of caching and quantization, achieving lossless acceleration on mainstream models with high speedup
significant memory savings without sacrificing output quality. ratios, showing great promise and broad applicability. Looking
Jenga [129] introduces an integrated framework that combines ahead, as model sizes grow and generative tasks become more
TeaCache [119] with variable-resolution inference, enabling multimodal and interactive, diffusion caching may become a
multi-scale acceleration with improved efficiency. These works key part of next-generation efficient generative frameworks.
collectively demonstrate that cache mechanisms not only ex- It could evolve into a standard module in diffusion inference
tend their own acceleration potential but also complement pipelines and drive joint optimization across algorithmic and
other optimization paradigms, offering promising solutions system levels. This will help support efficient generation for
for achieving high-quality generation under limited memory images, videos, 3D content, and even world models. From a
budgets. broader view, the idea of Diffusion Caching may become an
Nevertheless, achieving effective coordination among mul- important path toward achieving truly scalable and sustainable
tiple acceleration techniques remains a major challenge. Most generative intelligence.
existing methods still face a trade-off between inference speed
and generation quality: aggressively pursuing speed may am- R EFERENCES
plify approximation errors and degrade output fidelity, whereas [1] S. Cao, H. Chen, P. Chen, Y. Cheng, Y. Cui, X. Deng, Y. Dong,
prioritizing quality preservation often reduces computational K. Gong, T. Gu, X. Gu et al., “Hunyuanimage 3.0 technical report,”
efficiency. When multiple acceleration mechanisms are ap- arXiv preprint arXiv:2509.23951, 2025.
[2] T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C.-W. Xie, D. Chen, F. Yu,
plied jointly, their induced errors may compound, thereby H. Zhao, J. Yang et al., “Wan: Open and advanced large-scale video
exacerbating distributional deviations from the original model. generative models,” arXiv preprint arXiv:2503.20314, 2025.
Therefore, it is essential to design principled integration frame- [3] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley,
S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial net-
works that can mitigate or counterbalance these accumulated works,” Communications of the ACM, vol. 63, no. 11, pp. 139–144,
errors rather than allowing them to linearly aggregate. The 2020.
core issue lies in the insufficient theoretical understanding of [4] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv
preprint arXiv:1312.6114, 2013.
error propagation and interaction across different acceleration [5] W. Peebles and S. Xie, “Scalable diffusion models with transformers,”
mechanisms. Future research should aim to establish a uni- in Proceedings of the IEEE/CVF international conference on computer
fied analytical framework that quantitatively models these vision, 2023, pp. 4195–4205.
[6] Black Forest Labs, “FLUX.1 [dev],” 2024, accessed: 2024-10-10.
interactions and identifies the optimal conditions for deep [Online]. Available: [Link]
integration among caching, quantization, pruning, and other 1-dev
acceleration strategies. Such efforts will not only guide the [7] C. Wu, J. Li, J. Zhou, J. Lin, K. Gao, K. Yan, S.-m. Yin, S. Bai,
X. Xu, Y. Chen et al., “Qwen-image technical report,” arXiv preprint
design of robust and efficient diffusion inference systems but arXiv:2508.02324, 2025.
also foster theoretical advancements that bridge engineering [8] W. Kong, Q. Tian, Z. Zhang, R. Min, Z. Dai, J. Zhou, J. Xiong,
practice with diffusion theory. X. Li, B. Wu, J. Zhang et al., “Hunyuanvideo: A systematic framework
for large video generative models,” arXiv preprint arXiv:2412.03603,
2024.
VI. C ONCLUSION [9] T. Brooks, B. Peebles, C. Holmes, W. DePue, Y. Guo, L. Jing,
D. Schnurr, J. Taylor, T. Luhman, E. Luhman et al., “Video generation
In summary, Diffusion Models (DMs) have achieved re- models as world simulators,” OpenAI Blog, vol. 1, no. 8, p. 1, 2024.
[10] A. Polyak, A. Zohar, A. Brown, A. Tjandra, A. Sinha, A. Lee, A. Vyas,
markable progress in both generation quality and controllabil- B. Shi, C.-Y. Ma, C.-Y. Chuang et al., “Movie gen: A cast of media
ity, becoming one of the key foundations of modern generation foundation models,” arXiv preprint arXiv:2410.13720, 2024.
models. However, the high computational cost and long infer- [11] T. Seawead, C. Yang, Z. Lin, Y. Zhao, S. Lin, Z. Ma, H. Guo, H. Chen,
L. Qi, S. Wang et al., “Seaweed-7b: Cost-effective training of video
ence time remain key challenges that hinder their widespread generation foundation model,” arXiv preprint arXiv:2504.08685, 2025.
adoption in real-world applications. In scenarios such as real- [12] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “Dpm-solver: A
time interaction, on-device deployment, or large-scale online fast ode solver for diffusion probabilistic model sampling in around
10 steps,” in Advances in Neural Information Processing Systems
generation, the heavy inference load significantly limits the (NeurIPS), 2022.
model’s practicality, scalability, and cost-efficiency. Existing [13] T. Salimans and J. Ho, “Progressive distillation for fast sampling of
acceleration methods, including model distillation, pruning, diffusion models,” arXiv preprint arXiv:2202.00512, 2022.
[14] Y. Ma, K. Feng, Z. Hu, X. Wang, Y. Wang, M. Zheng, X. He, C. Zhu,
sampler optimization, and system-level operator acceleration, H. Liu, Y. He et al., “Controllable video generation: A survey,” arXiv
have shown some success in specific cases. However, these preprint arXiv:2507.16869, 2025.
methods often suffer from limited applicability, degraded [15] W. Zhao, L. Bai, Y. Rao, J. Zhou, and J. Lu, “Unipc: A unified
predictor-corrector framework for fast sampling of diffusion models,”
generation quality, or high training costs, making it difficult Advances in Neural Information Processing Systems, vol. 36, pp.
to achieve efficient and stable acceleration in a generalizable 49 842–49 869, 2023.
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 20

[16] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, [43] Y. Li, B. Zhang, H. Xu, and L. Wang, “Dc-gen: Post-training diffusion
and D. Kalenichenko, “Quantization and training of neural networks acceleration with deeply compressed latent space,” arXiv preprint
for efficient integer-arithmetic-only inference,” in Proceedings of the arXiv:2509.25180, 2025.
IEEE conference on computer vision and pattern recognition, 2018, [44] T. Dao, D. Y. Fu, S. Ermon, A. Rudra, and C. Ré, “Flashattention: Fast
pp. 2704–2713. and memory-efficient exact attention with io-awareness,” in Advances
[17] S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights in Neural Information Processing Systems (NeurIPS), 2022.
and connections for efficient neural network,” Advances in neural [45] T. Dao, “Flashattention-2: Faster attention with better parallelism and
information processing systems, vol. 28, 2015. work partitioning,” in International Conference on Learning Represen-
[18] T. Dao, D. Fu, S. Ermon, A. Rudra, and C. Ré, “Flashattention: Fast tations (ICLR), 2024.
and memory-efficient exact attention with io-awareness,” Advances in [46] J. K. Christopher, B. R. Bartoldson, T. Ben-Nun, M. Cardei,
neural information processing systems, vol. 35, pp. 16 344–16 359, B. Kailkhura, and F. Fioretto, “Speculative diffusion decoding: Ac-
2022. celerating language generation through diffusion,” in Conference of
[19] T. Salimans and J. Ho, “Progressive distillation for fast sampling of the North American Chapter of the Association for Computational
diffusion models,” in International Conference on Learning Represen- Linguistics (NAACL), 2025.
tations (ICLR), 2022. [47] V. De Bortoli, A. Galashov, A. Gretton, and A. Doucet, “Accel-
[20] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever, “Consistency models,” erated diffusion models via speculative sampling,” arXiv preprint
in International Conference on Machine Learning (ICML), 2023. arXiv:2501.05370, 2025.
[21] C. Meng, Y. Song, J. Song, and S. Ermon, “On distillation of guided [48] D. Israel, G. Van den Broeck, and A. Grover, “Accelerating diffusion
diffusion models,” in IEEE/CVF Conference on Computer Vision and llms via adaptive parallel decoding,” arXiv preprint arXiv:2506.00413,
Pattern Recognition (CVPR), 2023, pp. 14 297–14 306. 2025.
[22] S. Luo, Y. Tan, L. Huang, J. Li, and H. Zhao, “Latent consistency [49] H. Liang, J. Zhou, Z. Zhou, X. Zhang, and X. Chen, “Stadi: Fine-
models: Synthesizing high-resolution images with few-step inference,” grained step-patch diffusion parallelism for heterogeneous gpus,” arXiv
arXiv preprint arXiv:2310.04378, 2023. preprint arXiv:2509.04719, 2025.
[23] J. Heek and T. Salimans, “Multistep consistency models,” arXiv [50] M. Li, T. Cai, J. Cao, Q. Zhang, H. Cai, J. Bai, Y. Jia, M.-Y. Liu,
preprint arXiv:2403.06807, 2024. K. Li, and S. Han, “Distrifusion: Distributed parallel inference for high-
[24] Y. Wu, Y. Li, S. Luo, F. Bao, and C. Li, “Reward guided latent resolution diffusion models,” arXiv preprint arXiv:2402.19481, 2024.
consistency distillation,” arXiv preprint arXiv:2403.11027, 2024. [51] J. Fang, J. Pan, J. Wang, A. Li, and X. Sun, “Pipefusion: Patch-
[25] X. Yang, W. Hong, J. Li et al., “Animatediff-lightning: Cross-model level pipeline parallelism for diffusion transformers inference,” arXiv
diffusion distillation,” arXiv preprint arXiv:2403.12706, 2024. preprint arXiv:2405.14430, 2024.
[26] Z. Zhao, J. Liu, H. Wang et al., “Motion consistency model: Accelerat- [52] Z. Shu, R. Feng, Y. Cao, and Z.-J. Zha, “Rain: Real-time animation of
ing video diffusion with disentangled motion-appearance distillation,” infinite video stream,” arXiv preprint arXiv:2412.19489, 2024.
arXiv preprint arXiv:2406.06890, 2024.
[53] J. Ho, A. N. Jain, and P. Abbeel, “Denoising diffusion probabilis-
[27] H. Xu, B. Zhang, W. Shao et al., “Accelerating video diffusion models
tic models,” in Advances in Neural Information Processing Systems
via distribution matching,” arXiv preprint arXiv:2412.05899, 2024.
(NeurIPS), 2020.
[28] C. Sun, Y. Wang, R. Gao et al., “V.i.p.: Iterative online prefer-
[54] J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit mod-
ence distillation for efficient video diffusion models,” arXiv preprint
els,” in International Conference on Learning Representations (ICLR),
arXiv:2508.03254, 2025.
2021.
[29] J. Huang, Z. Zhou, and L. Wang, “Diffusion adversarial post-training
[55] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon,
for one-step video generation,” arXiv preprint arXiv:2501.08316, 2025.
and B. Poole, “Score-based generative modeling through stochastic
[30] S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing
differential equations,” in International Conference on Learning Rep-
deep neural networks with pruning, trained quantization and huffman
resentations (ICLR), 2021.
coding,” in International Conference on Learning Representations
(ICLR), 2016. [56] A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion proba-
bilistic models,” in Proceedings of the 38th International Conference
[31] H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning
on Machine Learning (ICML), vol. 139, 2021, pp. 8162–8171.
convolutional neural networks for resource efficient inference,” in
International Conference on Learning Representations (ICLR), 2017. [57] X. Liu, S. Gu, Z. Lu, R. Timofte, and L. V. Gool, “Pseudo nu-
[32] J. Frankle and M. Carbin, “The lottery ticket hypothesis: Finding merical methods for diffusion models on manifolds,” arXiv preprint
sparse, trainable neural networks,” in International Conference on arXiv:2202.09778, 2022.
Learning Representations (ICLR), 2019. [58] C. Lu, F. Bao, J. Chen, C. Li, and J. Zhu, “Dpm-solver++: Fast solver
[33] T. Chen, Z. Yao, and Z. Wang, “Rigging the lottery: Making all tickets for guided sampling of diffusion probabilistic models,” arXiv preprint
winners,” in International Conference on Learning Representations arXiv:2211.01095, 2022.
(ICLR), 2020. [59] W. Zhao, L. Bai, Y. Rao, J. Zhou, and J. Lu, “Unipc: A unified
[34] G. Fang, X. Ma, and X. Wang, “Structural pruning for diffusion mod- predictor-corrector framework for fast sampling of diffusion models,” in
els,” in Advances in Neural Information Processing Systems (NeurIPS), Advances in Neural Information Processing Systems (NeurIPS), 2023.
2023. [60] S. Chen, Y. Li, Y. Liu, and L. Wang, “Rapid³: Tri-level rein-
[35] Y. Hu, H. Zhou, W. Qi et al., “Token merging for fast stable diffusion,” forced acceleration policies for diffusion transformer,” arXiv preprint
arXiv preprint arXiv:2303.17604, 2023. arXiv:2509.22323, 2025.
[36] J. Li, X. Zhang et al., “Token fusion: Bridging the gap between token [61] J. Kim, M. Park, J. Lee, J. Kim, H. J. Chang, and Y. Uh, “S4s: Solving
pruning and token merging,” arXiv preprint arXiv:2312.01026, 2023. for a diffusion model solver,” arXiv preprint arXiv:2502.17423, 2025.
[37] Z. Li, Q. Zhang, W. Liu, and J. Zhu, “Ditfastattn: Atten- [62] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer,
tion compression for diffusion transformer models,” arXiv preprint “High-resolution image synthesis with latent diffusion models,” in
arXiv:2406.08552, 2024. Proceedings of the IEEE/CVF conference on computer vision and
[38] Y. Liu, J. Chen, Y. Zhang, Y. Li, J. Zhang, and Y. Xu, “Importance- pattern recognition, 2022, pp. 10 684–10 695.
based token merging for efficient image and video generation,” arXiv [63] Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow
preprint arXiv:2411.16720, 2024. matching for generative modeling,” arXiv preprint arXiv:2210.02747,
[39] H. Wang, S. Zhang, W. Shao, and L. Wang, “Toma: Token merge with 2022.
attention for diffusion models,” arXiv preprint arXiv:2509.10918, 2025. [64] R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural
[40] Z. Zhang, B. Zhang, T. Cai, and L. Wang, “Vsa: Faster video diffusion ordinary differential equations,” Advances in neural information pro-
with trainable sparse attention,” arXiv preprint arXiv:2505.13389, cessing systems, vol. 31, 2018.
2025. [65] X. Liu, C. Gong, and Q. Liu, “Flow straight and fast: Learning
[41] H. Liu, R. Huang, and L. Wang, “Sparse-vdit: Unleashing the power to generate and transfer data with rectified flow,” arXiv preprint
of sparse attention to accelerate video diffusion transformers,” arXiv arXiv:2209.03003, 2022.
preprint arXiv:2506.03065, 2025. [66] P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini,
[42] R. Huang, J. Li, Y. Ren, J. Liu, and Z. Zhao, “Megatts 3: Sparse Y. Levi, D. Lorenz, A. Sauer, F. Boesel et al., “Scaling rectified
alignment enhanced latent diffusion transformer for zero-shot speech flow transformers for high-resolution image synthesis,” in Forty-first
synthesis,” arXiv preprint arXiv:2502.18924, 2025. international conference on machine learning, 2024.
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 21

[67] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional net- [90] [Link], “cache-dit: A unified, flexible and training-free
works for biomedical image segmentation,” in International Confer- cache acceleration framework for diffusers.” 2025, open-source
ence on Medical image computing and computer-assisted intervention. software available at [Link] [Online].
Springer, 2015, pp. 234–241. Available: [Link]
[68] X. Ma, G. Fang, and X. Wang, “Deepcache: Accelerating diffusion [91] M. Adnan, N. Kurella, A. Arunkumar, and P. J. Nair, “Foresight:
models for free,” in Proceedings of the IEEE/CVF conference on Adaptive layer reuse for accelerated and high-quality text-to-video
computer vision and pattern recognition, 2024, pp. 15 762–15 772. generation,” arXiv preprint arXiv:2506.00329, 2025.
[69] A. Dosovitskiy, “An image is worth 16x16 words: Transformers for [92] J. Liu, C. Zou, Y. Lyu, J. Chen, and L. Zhang, “From reusing to
image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020. forecasting: Accelerating diffusion models with taylorseers,” arXiv
[70] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. preprint arXiv:2503.06923, 2025.
Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” [93] Z. Yu, Z. Zou, G. Shao, C. Zhang, S. Xu, J. Huang, F. Zhao, X. Cun,
Advances in neural information processing systems, vol. 30, 2017. and W. Zhang, “Ab-cache: Training-free acceleration of diffusion
[71] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction models via adams-bashforth cached feature reuse,” arXiv preprint
to algorithms. MIT press, 2022. arXiv:2504.10540, 2025.
[72] S. Li, T. Hu, J. van de Weijer, F. S. Khan, T. Liu, L. Li, S. Yang, [94] L. Feng, S. Zheng, J. Liu, Y. Lin, Q. Zhou, P. Cai, X. Wang, J. Chen,
Y. Wang, M.-M. Cheng, and J. Yang, “Faster diffusion: Rethinking the C. Zou, Y. Ma et al., “Hicache: Training-free acceleration of diffusion
role of the encoder for diffusion model inference,” Advances in Neural models via hermite polynomial-based feature caching,” arXiv preprint
Information Processing Systems, vol. 37, pp. 85 203–85 240, 2024. arXiv:2508.16984, 2025.
[73] X. Zhao, X. Jin, K. Wang, and Y. You, “Real-time video generation [95] S. Zheng, L. Feng, X. Wang, Q. Zhou, P. Cai, C. Zou, J. Liu, Y. Lin,
with pyramid attention broadcast,” arXiv preprint arXiv:2408.12588, J. Chen, Y. Ma et al., “Forecast then calibrate: Feature caching as ode
2024. for efficient diffusion transformers,” arXiv preprint arXiv:2508.16211,
[74] P. Selvaraju, T. Ding, T. Chen, I. Zharkov, and L. Liang, “Fora: Fast- 2025.
forward caching in diffusion transformer acceleration,” arXiv preprint [96] J. Bu, P. Ling, Y. Zhou, Y. Wang, Y. Zang, D. Lin, and J. Wang,
arXiv:2407.01425, 2024. “Dicache: Let diffusion model determine its own cache,” arXiv preprint
[75] P. Chen, M. Shen, P. Ye, J. Cao, C. Tu, C.-S. Bouganis, Y. Zhao, arXiv:2508.17356, 2025.
and T. Chen, “δ-dit: A training-free acceleration method tailored for [97] D. Liu, Y. Yu, J. Zhang, Y. Li, B. Lengerich, and Y. N. Wu, “Fast-
diffusion transformers,” arXiv preprint arXiv:2406.01125, 2024. cache: Fast caching for diffusion transformer through learnable linear
[76] Z. Lv, C. Si, J. Song, Z. Yang, Y. Qiao, Z. Liu, and K.-Y. K. Wong, approximation,” arXiv preprint arXiv:2505.20353, 2025.
“Fastercache: Training-free video diffusion model acceleration with [98] Z. Zheng, X. Wang, C. Zou, S. Wang, and L. Zhang, “Compute only
high quality,” arXiv preprint arXiv:2410.19355, 2024. 16 tokens in one timestep: Accelerating diffusion transformers with
[77] C. Zou, X. Liu, T. Liu, S. Huang, and L. Zhang, “Accelerating cluster-driven feature caching,” arXiv preprint arXiv:2509.10312, 2025.
diffusion transformers with token-wise feature caching,” arXiv preprint [99] J. Liu, C. Zou, Y. Lyu, F. Ren, S. Wang, K. Li, and L. Zhang, “Speca:
arXiv:2410.05317, 2024. Accelerating diffusion transformers with speculative feature caching,”
[78] F. Liu, S. Zhang, X. Wang, Y. Wei, H. Qiu, Y. Zhao, Y. Zhang, Q. Ye, arXiv preprint arXiv:2509.11628, 2025.
and F. Wan, “Timestep embedding tells: It’s time to cache for video
[100] H. Chu, W. Wu, G. Fen, and Y. Zhang, “Omnicache: A trajectory-
diffusion model,” in Proceedings of the Computer Vision and Pattern
oriented global perspective on training-free cache reuse for diffusion
Recognition Conference, 2025, pp. 7353–7363.
transformer models,” arXiv preprint arXiv:2508.16212, 2025.
[79] A. Taghipour, M. Ghahremani, M. Bennamoun, A. M. Rekavandi,
[101] J. Lou, W. Luo, Y. Liu, B. Li, X. Ding, W. Hu, J. Cao, Y. Li, and
Z. Li, H. Laga, and F. Boussaid, “Faster image2video generation: A
C. Ma, “Token caching for diffusion transformer acceleration,” arXiv
closer look at clip image embedding’s impact on spatio-temporal cross-
preprint arXiv:2409.18523, 2024.
attentions,” IEEE access, 2025.
[102] Z. Yu, H. Li, F. Fu, X. Miao, and B. Cui, “Accelerating text-to-image
[80] X. Shen, Z. Song, Y. Zhou, B. Chen, Y. Li, Y. Gong, K. Zhang, H. Tan,
editing via cache-enabled sparse diffusion inference,” in Proceedings
J. Kuen, H. Ding et al., “Lazydit: Lazy learning for the acceleration
of the AAAI Conference on Artificial Intelligence, vol. 38, no. 15, 2024,
of diffusion transformers,” in Proceedings of the AAAI Conference on
pp. 16 605–16 613.
Artificial Intelligence, vol. 39, no. 19, 2025, pp. 20 409–20 417.
[81] W. Zhang, H. Liu, J. Xie, F. Faccio, M. Z. Shou, and J. Schmidhuber, [103] D. Sun, H. Tian, T. Lu, and S. Liu, “Flexcache: Flexible approximate
“Cross-attention makes inference cumbersome in text-to-image diffu- cache system for video diffusion,” arXiv preprint arXiv:2501.04012,
sion models,” arXiv e-prints, pp. arXiv–2404, 2024. 2024.
[82] H. Liu, W. Zhang, J. Xie, F. Faccio, M. Xu, T. Xiang, M. Z. Shou, [104] A. Chen, B. Dong, J. Li, J. Lin, K. Tian, Y. Yao, and G. Wang, “Rain-
J.-M. Perez-Rua, and J. Schmidhuber, “Faster diffusion via temporal fusion: Adaptive video generation acceleration via multi-dimensional
attention decomposition,” arXiv preprint arXiv:2404.02747, 2024. visual redundancy,” arXiv preprint arXiv:2505.21036, 2025.
[83] A. Silveria, S. V. Govande, and D. Y. Fu, “Chipmunk: Training- [105] X. Ma, Y. Liu, Y. Liu, X. Wu, M. Zheng, Z. Wang, S.-N. Lim, and
free acceleration of diffusion transformers with dynamic column- H. Yang, “Model reveals what to cache: Profiling-based feature reuse
sparse deltas,” in ES-FoMo III: 3rd Workshop on Efficient Systems for for video diffusion models,” arXiv preprint arXiv:2504.03140, 2025.
Foundation Models, 2025. [106] H. Zhang, T. Gao, J. Shao, and Z. Wu, “Blockdance: Reuse structurally
[84] Z. Ma, L. Wei, F. Wang, S. Zhang, and Q. Tian, “Magcache: similar spatio-temporal features to accelerate diffusion transformers,”
Fast video generation with magnitude-aware cache,” arXiv preprint in Proceedings of the Computer Vision and Pattern Recognition Con-
arXiv:2506.09045, 2025. ference, 2025, pp. 12 891–12 900.
[85] X. Zhou, D. Liang, K. Chen, T. Feng, X. Chen, H. Lin, Y. Ding, [107] S. Zheng, G. Chen, Q. Zhou, Y. Lin, L. He, C. Zou, P. Cai, J. Liu, and
F. Tan, H. Zhao, and X. Bai, “Less is enough: Training-free video L. Zhang, “Let Features Decide Their Own Solvers: Hybrid Feature
diffusion acceleration via runtime-adaptive caching,” arXiv preprint Caching for Diffusion Transformers,” 2025.
arXiv:2507.02860, 2025. [108] Z. Yan, Y. Ma, C. Zou, W. Chen, Q. Chen, and L. Zhang, “Eedit:
[86] F. Wimbauer, B. Wu, E. Schoenfeld, X. Dai, J. Hou, Z. He, Rethinking the spatial and temporal redundancy for efficient image
A. Sanakoyeu, P. Zhang, S. Tsai, J. Kohler et al., “Cache me if editing,” arXiv preprint arXiv:2503.10270, 2025.
you can: Accelerating diffusion models through block caching,” in [109] L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control
Proceedings of the IEEE/CVF Conference on Computer Vision and to text-to-image diffusion models,” in Proceedings of the IEEE/CVF
Pattern Recognition, 2024, pp. 6211–6220. international conference on computer vision, 2023, pp. 3836–3847.
[87] Y. Huang, Z. Wang, R. Gong, J. Liu, X. Zhang, J. Guo, X. Liu, and [110] Z. Yang, Y. Ma, Y. Zhang, S. Mo, D. Liu, and L. Zhang, “Evc-
J. Zhang, “Harmonica: Harmonizing training and inference for better trl: Efficient control adapter for visual generation,” arXiv preprint
feature caching in diffusion transformer acceleration,” arXiv preprint arXiv:2508.10963, 2025.
arXiv:2410.01723, 2024. [111] Y. Ma, Z. Yan, H. Liu, H. Wang, H. Pan, Y. He, J. Yuan, A. Zeng,
[88] K. Kahatapitiya, H. Liu, S. He, D. Liu, M. Jia, C. Zhang, M. S. Ryoo, C. Cai, H.-Y. Shum et al., “Follow-your-emoji-faster: Towards efficient,
and T. Xie, “Adaptive caching for faster video generation with diffusion fine-controllable, and expressive freestyle portrait animation,” arXiv
transformers,” arXiv preprint arXiv:2411.02397, 2024. preprint arXiv:2509.16630, 2025.
[89] Z. Zou, J. Huang, H. Yu, and F. Zhao, “Feb-cache: Frequency-guided [112] X. Yang, S. Liu, and X. Wang, “Hash3d: Training-free acceleration
exposure bias reduction for enhancing diffusion transformer caching,” for 3d generation,” in Proceedings of the Computer Vision and Pattern
Available at SSRN 5584552. Recognition Conference, 2025, pp. 21 481–21 491.
A SURVEY ON CACHE METHODS IN DIFFUSION MODELS: TOWARD EFFICIENT MULTI-MODAL GENERATION 22

[113] Y. Huo, Z. Jiang, Z. Tang, Q. Hong, and Z. Zhao, “Ditreducio: A

training-free acceleration for dit-based tts via progressive calibration,”
arXiv preprint arXiv:2509.09748, 2025.
[114] Z. Chen, K. Liu, J. Gong, J. Wang, L. Sun, Z. Wu, R. Timofte, Y. Zhang
et al., “Ntire 2025 challenge on image super-resolution (x4): Methods
and results,” in CVPRW, 2025.
[115] Q. Song, X. Wang, D. Zhou, J. Lin, C. Chen, Y. Ma, and X. Li, “Hero:
Hierarchical extrapolation and refresh for efficient world models,”
arXiv preprint arXiv:2508.17588, 2025.
[116] Genie 3. [Online]. Available: [Link]
[117] J. Li, J. Tang, Z. Xu, L. Wu, Y. Zhou, S. Shao, T. Yu, Z. Cao, and Q. Lu,
“Hunyuan-gamecraft: High-dynamic interactive game video generation
with hybrid history condition,” arXiv preprint arXiv:2506.17201, 2025.
[118] X. He, C. Peng, Z. Liu, B. Wang, Y. Zhang, Q. Cui, F. Kang, B. Jiang,
M. An, Y. Ren et al., “Matrix-game 2.0: An open-source, real-time, and
streaming interactive world model,” arXiv preprint arXiv:2508.13009,
2025.
[119] Y. Xin, Q. Qin, S. Luo, K. Zhu, J. Yan, Y. Tai, J. Lei, Y. Cao, K. Wang,
Y. Wang et al., “Lumina-dimoo: An omni diffusion large language
model for multi-modal generation and understanding,” arXiv preprint
arXiv:2510.06308, 2025.
[120] Z. Liu, Y. Yang, Y. Zhang, J. Chen, C. Zou, Q. Wei, S. Wang, and
L. Zhang, “dllm-cache: Accelerating diffusion large language models
with adaptive caching,” arXiv preprint arXiv:2506.06295, 2025.
[121] S. Nie, F. Zhu, Z. You, X. Zhang, J. Ou, J. Hu, J. Zhou, Y. Lin, J.-
R. Wen, and C. Li, “Large language diffusion models,” arXiv preprint
arXiv:2502.09992, 2025.
[122] D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani,
J. Michael, and S. R. Bowman, “GPQA: A graduate-level google-proof
q&a benchmark,” in First Conference on Language Modeling, 2024.
[Online]. Available: [Link]
[123] J. Sommer, J. Rachwan, N. Fleischmann, S. Gunnemann, and
B. Charpentier, “Predictive feature caching for training-free
acceleration of molecular geometry generation,” 2025. [Online].
Available: [Link]
[124] S. Axelrod and R. Gómez-Bombarelli, “Geom, energy-annotated
molecular conformations for property prediction and molecular
generation,” Scientific Data, vol. 9, no. 1, p. 185, 2022. [Online].
Available: [Link]
[125] J. Liu, P. Cai, Q. Zhou, Y. Lin, D. Kong, B. Huang, Y. Pan, H. Xu,
C. Zou, J. Tang et al., “Freqca: Accelerating diffusion models via
frequency-aware caching,” arXiv preprint arXiv:2510.08669, 2025.
[126] X. Peng, H. Liu, C. Yan, R. Ma, F. Chen, X. Wang, Z. Wu, S. Liu,
and M. Lin, “Ertacache: Error rectification and timesteps adjustment
for efficient diffusion,” arXiv preprint arXiv:2508.21091, 2025.
[127] E. Zhang, B. Xiao, J. Tang, Q. Ma, C. Zou, X. Ning, X. Hu, and
L. Zhang, “Token pruning for caching better: 9 times acceleration on
stable diffusion for free,” arXiv preprint arXiv:2501.00375, 2024.
[128] J. Wu, Z. Li, Z. Hui, Y. Zhang, L. Kong, and X. Yang, “Quantcache:
Adaptive importance-guided quantization with hierarchical latent and
layer caching for video generation,” arXiv preprint arXiv:2503.06545,
2025.
[129] Y. Zhang, J. Xing, B. Xia, S. Liu, B. Peng, X. Tao, P. Wan, E. Lo,
and J. Jia, “Training-free efficient video generation via dynamic token
carving,” arXiv preprint arXiv:2505.16864, 2025.