Problem

Traditional image forgery techniques, exacerbated by advanced AI models like Stable Diffusion (SD), pose significant threats to digital media credibility. Existing image forgery localization methods struggle due to reliance on costly annotated data and their inability to keep pace with emerging manipulation technologies. They also fail to fully exploit the powerful image understanding capabilities of large pre-trained models, limiting their robustness and generalization, and facing a data acquisition bottleneck.

Method

The proposed SDiFL framework is the first to integrate both the image generation and perceptual capabilities of Stable Diffusion into an image forensic framework for efficient and accurate forgery localization. It theoretically demonstrates that SD's multi-modal architecture can be conditioned on forgery-related information. Building on this, SDiFL specifically leverages Stable Diffusion V3 (SD3) by treating high-frequency image forgery residuals, extracted using Spatially Rich Model (SRM) filters, as an explicit modality. This modality is fused into the latent space during training, enhancing localization performance while preserving the rich semantic information from the input image. The model is trained to minimize a loss function comprising a latent space matching loss and an image-space localization loss, derived from maximizing the Evidence Lower Bound.

Results

Extensive experiments show that SDiFL achieves state-of-the-art performance, with up to 12% improvements over current models on widely used benchmarking datasets for traditional forgeries (NIST16, Coverage, CASIA1). It also demonstrates unprecedented performance on diffusion-generated datasets like CocoGlide and GRE, significantly outperforming baselines. Furthermore, the model exhibits strong generalization to real-world forgeries, including natural scenes (ACDTamp) and domain-unseen book cover images (PS-boundary, PS-arbitrary). SDiFL also proves robust against various post-processing operations such as Gaussian noise, JPEG compression, resizing, and complex degradations introduced by online social networks (OSNs), maintaining superior performance even under severe perturbation conditions.

Implications

This pioneering work provides the first theoretical and practical integration of generative image models like Stable Diffusion into image forensics, offering a new paradigm for forgery localization. The demonstrated state-of-the-art performance, robustness, and strong generalization ability across diverse forgery types and real-world conditions highlight SDiFL's significant practical applicability. Future work will focus on developing computationally efficient variants through knowledge distillation and exploring the integration of SD's noise-adding and denoising processes to further enhance performance and simplify the pipeline.