[July 2025]We release our code for inference and evaluation.
In this paper, we observe that the ratio of probabilities at different noise levels serves as a good indicator of artifacts during the reverse diffusion process.
Leveraging this insight, we propose Self-Guidance (SG) —a plug-and-play guidance method to modulate the output distribution at each denoising step via incorporating the difference between the output probability distributions computed at both current time
Moreover, such guidance can be implemented without incurring additional inference cost, we refer to as SG-prev, which approximates SG by directly reusing the output from the immediately previous timestep.
Conpared to existing diffusion guidance strategy, SG has multiple advantages:
- architecture independence: SG is independent of neural architectures, meaning that it can be deployed freely on diffusion and flow models based on different neural architectures regardless of whether it is a UNet-based model or a Transformer-based model.
- requiring no specific training: SG does not require any additional model training and can be used in a plug-and-play manner.
- compatibility with other well-established guidance methods: SG is compatible with other guidance on both UNet and DiT-based diffusion models for text-to-image and text-to-video generation tasks.
You can try a demo in demo.ipynb.
git clone https://github.com/maple-research-lab/Self-Guidance.git
cd Self-Guidance
pip install -e . from models.stable_diffusion_3.pipeline_sd_3 import StableDiffusion3SGPAGPipeline
pipe = StableDiffusion3SGPAGPipeline.from_pretrained(
"stabilityai/stable-diffusion-3-medium",
torch_dtype=torch.float16,
)
device="cuda"
pipe = pipe.to(device)
prompts = ["a photograph of an astronaut riding a horse"]output = pipe(
prompts,
width=1024,
height=1024,
num_inference_steps=28,
guidance_scale=7.5,
pag_scale=0.7,
self_guidance_scale=3,
self_guidance_shift_t=10,
self_guidance_type="sg"
).images[0]output = pipe(
prompts,
width=1024,
height=1024,
num_inference_steps=28,
guidance_scale=7.5,
pag_scale=0.7,
self_guidance_scale=3,
self_guidance_shift_t=10,
self_guidance_type="sg_prev",
sg_prev_max_t=500
).images[0]from models.flux.pipeline_flux import FluxSGPipeline
pipe = FluxSGPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev",
torch_dtype=torch.bfloat16,
)
device="cuda"
pipe = pipe.to(device)
prompts = ["a photograph of an astronaut riding a horse"]output = pipe(
prompts,
width=1024,
height=1024,
num_inference_steps=28,
guidance_scale=3.5,
self_guidance_scale=1,
self_guidance_shift_t=10,
self_guidance_type="sg"
).images[0]output = pipe(
prompts,
width=1024,
height=1024,
num_inference_steps=28,
guidance_scale=3.5,
self_guidance_scale=3,
self_guidance_shift_t=10,
self_guidance_type="sg_prev",
sg_prev_max_t=500
).images[0]from models.cogvideox.pipeline_cogvideox import CogVideoXSGPipeline
pipe = CogVideoXSGPipeline.from_pretrained(
"THUDM/CogVideoX-5b",
torch_dtype=torch.bfloat16,
)
device="cuda"
pipe = pipe.to(device)
prompts = ["A panda sits on a wooden stool in a serene bamboo forest."]output = pipe(
prompts,
width=720,
height=480,
num_inference_steps=50,
guidance_scale=6,
self_guidance_scale=3,
self_guidance_shift_t=10,
self_guidance_type="sg"
).frames[0]output = pipe(
prompts,
width=720,
height=480,
num_inference_steps=50,
guidance_scale=6,
self_guidance_scale=3,
self_guidance_shift_t=10,
self_guidance_type="sg_prev",
sg_prev_max_t=500
).frames[0]accelerate launch inference/coco.py \
--config config/sd3.yaml \
--local_data_path data/coco/coco_val5000_prompts.txt \
--output runs/sd3/coco \
--seed 0 accelerate launch inference/hps.py \
--config config/sd3.yaml \
--local_data_path data/hps \
--output runs/sd3/hps \
--seed 0 accelerate launch inference/vbench.py \
--config config/cogvideox.yaml \
--local_data_path data/vbench \
--output runs/cogvideox/vbench \
--seed 0 python evaluation/evaluate.py \
--coco_data_path /path/to/coco/results \
--hps_data_path /path/to/hps/results \
--prompt_path data/coco/coco_val5000_prompts.txt \
--fid --clip --aes --hpsEvaluation Metrics
If this repository helps with your work, please consider giving a star and citation:
@article{li2024self,
title={Self-Guidance: Boosting Flow and Diffusion Generation on Their Own},
author={Li, Tiancheng and Luo, Weijian and Chen, Zhiyang and Ma, Liyuan and Qi, Guo-Jun},
journal={arXiv preprint arXiv:2412.05827},
year={2024}
}