Revisiting the Perception-Distortion Trade-off with Spatial-Semantic Guided Super-Resolution

Dan Wang1, Haiyan Sun 2, Shan Du 3, Z. Jane Wang3, Zhaochong An4, Serge Belongie 4, Xinrui Cui 2
1University of California, San Diego 2University of North Texas 3University of British Columbia 4University of Copenhagen
PDF
Model visualization

Perception-distortion trade-off in GAN-based, Diffusion-based, and Ours: GAN-based methods reduce distortion but produce blurry textures, while diffusion-based methods generate perceptually sharp yet hallucinated details. By integrating spatial-grounded textual guidance, SpaSemSR improves reconstruction fidelity (PSNR, SSIM in (a)), while semantic-enhanced visual guidance enhances perceptual quality (CLIP-IQA, MUSIQ, MANIQA in (b)), resulting in a better perception-distortion trade-off compared with GAN-based (c) and diffusion-based models (d).

Abstract

Image super-resolution (SR) aims to reconstruct high resolution images with both high perceptual quality and low distortion, but is fundamentally limited by the perception–distortion trade-off. GAN-based SR methods reduce distortion but still struggle with realistic fine-grained textures, whereas diffusion-based approaches synthesize rich details but often deviate from the input, hallucinating structures and degrading fidelity. This tension raises a key challenge: how to exploit the powerful generative priors of diffusion models without sacrificing fidelity. To address this, we propose SpaSemSR, a spatial–semantic guided diffusion framework with two complementary guidances. First, spatial-grounded textual guidance integrates object-level spatial cues with semantic prompts, aligning textual and visual structures to reduce distortion. Second, semantic-enhanced visual guidance with a multi-encoder design and semantic degradation constraints unifies multimodal semantic priors, improving perceptual realism under severe degradations. These complementary guidances are adaptively fused into the diffusion process via spatial-semantic attention, suppressing distortion and hallucination while retaining the strengths of diffusion models. Extensive experiments on multiple benchmarks show that SpaSemSR achieves a superior perception-distortion balance, producing both realistic and faithful restorations.

BibTeX

@misc{wang2026SpaSemSR,
      title={Revisiting the Perception-Distortion Trade-off with Spatial-Semantic Guided Super-Resolution}, 
      author={Dan Wang and Haiyan Sun and Shan Du and Z. Jane Wang and Zhaochong An and Serge Belongie and Xinrui Cui},
      year={2026},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
}