Report

Hashim Mir, Peter Xu, and Peter van Beek University of Waterloo • Digital cameras are equipped with contrast-based autofocus • Contrast-detection autofocus makes use of a focus measure • maps an image to a value that represents degree of focus of image • Speed and accuracy of autofocus crucial to user satisfaction • search for an image with maximal focus according to focus measure • choice of focus measure is an important factor in performance of autofocus February 4, 2014 Focus Measures for Digital Photography 2 • Previous studies on focus measures: • small number of benchmarks images in their evaluation, • directed at microscopy and not digital cameras, or • based on ad hoc evaluation criteria • Here: • extensive empirical evaluation of focus measures for digital photography • advocate using three standard statistical measures of performance— precision, recall, and mean absolute error—as evaluation criteria February 4, 2014 Focus Measures for Digital Photography 3 • • • • • • Introduction Focus Measures Related Work Experimental Methodology Experimental Results Conclusion February 4, 2014 Focus Measures for Digital Photography 4 • Many focus measures have been proposed in the literature • Let f (x, y) be the luminance or grayscale at pixel (x, y) in an image of size M × N pixels • Example: squared gradient focus measure for an image acquired when the lens is at position p is given by, M 1N 2 ( p) = ( f ( x , y 1) f ( x , y )) 2 x=0 y=0 February 4, 2014 Focus Measures for Digital Photography 5 (a) February 4, 2014 (b) Focus Measures for Digital Photography 6 (a) February 4, 2014 (b) Focus Measures for Digital Photography 7 • We evaluated more than 30 focus measures in our study • The focus measures can be classified under six types: 1. 2. 3. 4. 5. 6. first-order differentiation second-order differentiation image histogram image statistics correlation data compression February 4, 2014 Focus Measures for Digital Photography 8 • Example: squared gradient • Example: focus measure based on 3×3 Sobel operators, 1 2 1 0 0 0 1 2 1 (horizontal) February 4, 2014 1 0 1 2 0 2 1 0 1 (vertical) Focus Measures for Digital Photography 9 • Example: focus measure based on 3×3 Laplacian operator, 1 1 1 February 4, 2014 1 8 1 1 1 1 Focus Measures for Digital Photography 10 • Example: variance focus measure is given by, ( p) = 1 MN M 1N 1 ( f ( x, y ) ) 2 x=0 y=0 where µ is the mean luminance of the image February 4, 2014 Focus Measures for Digital Photography 11 • Example: Vollath’s F4 focus measure is given by, M 2 N 1 ( p) = f ( x , y ) f ( x 1, y ) x=0 y=0 M 31 N 1 f ( x , y ) f ( x 2, y ) x=0 y=0 February 4, 2014 Focus Measures for Digital Photography 12 setting citation conclusions Groen, Young & Ligthart, 1985 squared gradient Microscopy applications Firestone et al. 1991 vertical Brenner, variance Santos et al. 1997 Vollath's F4, F5 Liu, Wang & Sun 2007 variance Subbarao and Tyan 1998 Laplacian Digital Ng, Neow & Ang 2001 photography Shih 2007 Yousefi et al. 2011 February 4, 2014 first-order 3×3 Sobel, variance first-order 3×3 Sobel & Prewitt Vollath's F4 Focus Measures for Digital Photography 13 • Step 1: Implemented a camera remote control application • Step 2: Using app, gathered 25 sets of benchmark images • cover a range of common photography settings • 4,303 jpeg images (23 × 167 + 2 × 231, one for each focus position) • Canon EOS 550D/Rebel T2i camera February 4, 2014 Focus Measures for Digital Photography 14 • Step 3: Determined ground truth • true lens position (or image) where an object is in best focus • Step 4: Implemented more than 30 focus measures • extensive survey of the literature • Step 5: Given a benchmark set of images and a focus measure • applied focus measure to each jpeg image in benchmark • focus measure predicts where an object will be in best focus • given predictions of focus measure and ground truth, calculated precision, recall, and mean absolute error February 4, 2014 Focus Measures for Digital Photography 15 ground truth true tp (true positive) fn (false negative) true predicted false precision = February 4, 2014 tp tp fp false fp (false positive) tn (true negative) recall = Focus Measures for Digital Photography tp tp fn 16 squared gradient February 4, 2014 Laplacian Focus Measures for Digital Photography 17 squared gradient precision = reca ll = February 4, 2014 3 30 3 31 Laplacian precision = reca ll = Focus Measures for Digital Photography 4 40 4 4 0 18 absolute error = p p where p is lens position of image in maximal focus pis lens position that focus measure predicts • MAE chosen over mean squared error • MAE prefers errors that are occasionally large but small on average • mean square error prefers errors that avoid large errors while still possibly being quite unsatisfactory overall February 4, 2014 Focus Measures for Digital Photography 19 Focus measure Brenner squared gradient 3 × 3 difference 3 × 3 Sobel 3 × 3 Scharr 3 × 3 Roberts 3 × 3 Prewitt Gaussian vertical sq. gradient vertical Brenner February 4, 2014 precision 100.00 100.00 98.00 98.00 98.00 98.00 98.00 95.00 91.33 91.33 recall 99.00 99.00 97.00 97.00 97.00 97.00 97.00 92.00 92.00 90.00 Focus Measures for Digital Photography MAE 0.00 0.00 0.02 0.02 0.02 0.02 0.02 0.05 0.55 0.23 20 Focus measure precision Laplacian of Gaussian 98.00 5 × 5 vertical Sobel 96.67 5 × 5 Laplacian 96.67 5 × 5 cross Sobel 90.33 3 × 3 cross Sobel 89.67 3 × 3 vertical Sobel 88.73 3 × 3 Laplacian 85.00 5 × 5 horizontal Sobel 83.33 3 × 3 horizontal Sobel 73.27 February 4, 2014 recall 99.00 99.00 99.00 93.00 93.00 96.00 91.00 88.00 78.00 Focus Measures for Digital Photography MAE 0.32 0.65 0.46 0.66 1.18 3.94 1.41 1.08 2.95 21 Focus measure normalized variance variance threshold pixel count threshold content power February 4, 2014 precision 43.57 32.33 4.24 2.13 0.53 recall 53.00 35.00 18.00 4.00 2.00 Focus Measures for Digital Photography MAE 26.36 20.80 52.38 36.31 43.80 22 Focus measure autocorrelation Vollath's F4 Vollath's F5 February 4, 2014 precision 80.40 61.74 33.00 recall 99.00 87.00 35.00 Focus Measures for Digital Photography MAE 26.87 24.60 19.00 23 • Results: • some popular focus measures perform poorly when applied to autofocusing in digital photography or to more representative sets of images • simple focus measures based on taking the first derivative of an image perform exceedingly well in digital photography, although care must be taken in which direction derivative is taken April 8th 2013 Combinatorial Problems in Compiler Optimization 24 • The speed and accuracy of a digital camera's contrast-based autofocus algorithm are crucial to user satisfaction • We advocate using three standard statistical measures of performance—precision, recall, and mean absolute error—as accuracy criteria • Our experimental results indicate that two simple focus measures based on taking first derivative of an image perform exceedingly well in digital photography February 4, 2014 Focus Measures for Digital Photography 25