Efficient Deblurring for Shaken and Partially Saturated Images

Try the online demo: http://willow-fd.rocq.inria.fr/unshake/

Overview

One common feature of “shaken” images is the presence of saturated pixels. These are caused when the radiance of the scene exceeds the range of the camera’s sensor, leaving bright highlights clipped at the maximum output value (e.g. 255 for an 8-bit image). To anyone who has attempted to take hand-held photographs at night, this effect should be familiar as the conspicuous bright streaks left by electric lights, such as in the images below. These bright pixels, with their clipped values, violate the assumption made by many algorithms that the image formation process is linear, and as a result can cause obtrusive artifacts in the deblurred images. This can be seen in the example images below. In this paper, we propose a non-blind deblurring algorithm that takes account of saturated pixels, and is able to greatly reduce the artifacts they cause in the deblurred results.

We also demonstrate an efficient approximation for spatially-varying blur, extending the work of Hirsch et al. [4] to handle our previously-proposed global model for spatially-varying camera shake blur [5]. This approximation, based on the local uniformity of the blur, allows the forward model for spatially-varying blur to be computed significantly faster than the exact model, while retaining the benefits of the global parameterization. Note that independently of our work, Hirsch et al. [6] have proposed a similar method to combine a global model of spatially-varying blur with their locally-uniform approximation.

Examples

Click to enlarge images. Navigate through with cursor keys.

• Blurry image with saturation

• Deblurred with Richardson-Lucy [1,2]

• Deblurred with algorithm of Krishnan & Fergus [3]

• Deblurred with proposed method

• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 

Efficient Approximation of the Global Blur Model

The images below show a synthetic blur kernel for our previously proposed model [5]. The PSF is visualized at various points across the image, using the exact forward model and the approximation at increasingly fine levels of discretization. The approximation quickly approaches the exact model, and at the finest discretization shown it is almost identical to the exact model.

• 

  Global blur kernel

• 

  PSF, exact forward model

• 

  PSF, approximate forward model,
  3 × 4 patches

• 

  PSF, approximate forward model,
  6 × 8 patches

• 

  PSF, approximate forward model,
  12 × 16 patches

Paper

O. Whyte, J. Sivic and A. Zisserman
Deblurring Shaken and Partially Saturated Images
IEEE Color and Photometry in Computer Vision Workshop (2011), in conjunction with ICCV 2011
PDF  | Abstract | BibTeX | PPT Slides (19MB)

O. Whyte, J. Sivic and A. Zisserman
Deblurring Shaken and Partially Saturated Images
International Journal of Computer Vision, 2014
PDF  | Abstract | BibTeX | Journal page

Images

The images from the CPCV 2011 paper: Download (25MB)

The results of our algorithm on the images of Cho et al. [7]: Download (16MB)

Code

A package of Matlab code for non-blind deblurring of blurry images with clipped / saturated pixels.
Version 0.1 (20-Nov-2011): Download (7MB) | Readme

Code

This package contains code to perform fast blind deblurring of images degraded by camera shake, using the MAP algorithm described in our IJCV 2012 paper, and the fast approximation of spatially-varying blur described in our CPCV 2011 paper. 

Version 1.0 (21-Sep-2014): Download (18MB) | Readme 

 

Comparison to the Method of Cho et al.

The images below show the results of our non-blind deblurring algorithm on images provided by Cho et al. [7]. The (spatially-invariant) blur kernels for the images, along with the results of their algorithm, are provided by the authors online here.

Download the uncompressed images above, or at this link.

Click to enlarge images. Navigate through with cursor keys.

• Blurry Image

• Result from Cho et al. [7]

• Our result

• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 

References

[1] W. H. Richardson. Bayesian-Based Iterative Method of Image Restoration. Journal of the Optical Society of America, 62(1), 1972.

[2] L. B. Lucy. An iterative technique for the rectification of observed distributions. Astronomical Journal, 79(6), 1974.

[3] D. Krishnan and R. Fergus. Fast Image Deconvolution using Hyper-Laplacian Priors. In Proc. NIPS, 2009.

[4] M. Hirsch, S. Sra, B. Scholkopf, and S. Harmeling. Efficient Filter Flow for Space-Variant Multiframe Blind Deconvolution. In Proc. CVPR, 2010.

[5] O. Whyte, J. Sivic, A. Zisserman, and J. Ponce. Non-uniform Deblurring for Shaken Images. In Proc. CVPR, 2010.

[6] M. Hirsch, C. J. Schuler, S. Harmeling, and B. Schölkopf. Fast Removal of Non-uniform Camera Shake. In Proc. ICCV, 2011.

[7] S. Cho, J. Wang, and S. Lee. Handling Outliers in Non-blind Image Deconvolution. In Proc. ICCV, 2011.

Last updated: 14th December 2011