A Universal Approach to Designing an Image Interpolator with an Image Smoothing Filter

Show more

1. Introduction

As known, the process of decimation or down-sampling is an effective way aid often used to reduce image sizes, thus reducing the amount of information transmitted through the communication channels and the local storage requirements, while trying to preserve as much as possible the image quality. Conversely, the reverse procedure of this, referred to as interpolation or up-sampling, is useful in restoring the original high resolution image from its decimated version or for resizing or zooming a digital image. Decimation and interpolation are used for several purposes in many practical applications, such as progressive image transmission systems, image zooming, photographic enlarging, image reconstruction, optical scanners, high resolution printer, and in multi-media applications which require browsing or retrieval of images from the internet or image and video databases. These problems are further aggravated in the case of color images which usually require larger storage capacity and processing time.

A number of conventional interpolation techniques have been proposed in the literature to increase the spatial resolution of an image [1] [2] [3] . Often, image interpolation is performed by pixel replication in a small neighborhood of each existing pixel. This is equivalent to a first-order linear filter. Obviously, such a scheme is very simple, but can hardly provide a satisfactory solution. In particular, the resulting blocking artifacts and blurring are quite annoying. The performance can be slightly improved by employing higher-order linear filters [4] [5] [6] . However, it seems that there do not exist good criteria for the design of optimal linear interpolators. Also, such an interpolator can hardly provide a satisfactory solution for interpolating noisy images. As known, linear interpolation is generally preferred not for the performance but for computational simplicity whereas adaptive methods aim to avoid these problems by analyzing the local structure of the source image and using different interpolation functions with different areas of support. Several algorithms [7] [8] [9] [10] have been proposed to improve the subjective quality of the interpolated images by imposing more accurate models. However, few of them concern the problem of dealing with noisy images. In this paper, the novelty of this research is that a universal approach is proposed to design an image interpolator with any one image smoothing filter, thereby not only interpolating a down-sampled image but also preserving the characteristics of the used filter.

Following the above introduction, this paper is organized as follows. Section 2 describes in detail the approach to designing an interpolator with a smoothing filter. Section 3 illustrates some examples of the approach proposed here, the interpolators designed with a mean filter, a median filter or a probability filter. Experimental results and evaluations are shown to highlight the approach in Section 4 and finally some concluding remarks are drawn in Section 5.

2. Approach to Designing an Interpolator

In this paper, we propose a universal approach for simultaneous image interpolation and smoothing by utilizing the smoothing filter coupled with a pyramidal decomposition shown as in Figure 1; furthermore, the approach includes three steps to accomplish interpolation.

Step 1: Any one pixel in a down-sampled image is decomposed into four pixels; for example, ${g}_{i,j}\left(k,l\right)$ are obtained from ${g}_{i,j}$ , the (k, l) of which are (−1, −1), (−1, 1), (1, −1), and (1, 1), respectively.

Step 2:

1)
${g}_{i,j}\left(-1,-1\right)$ is determined with the four original pixels,
${g}_{i,j}$ ,
${g}_{i-1,j-1}$ ,
${g}_{i-1,j}$ , and
${g}_{i,j-1}$ .

2) ${g}_{i,j}\left(-1,1\right)$ is determined with the four original pixels, ${g}_{i,j}$ , ${g}_{i-1,j+1}$ , ${g}_{i-1,j}$ , and ${g}_{i,j+1}$ .

3) ${g}_{i,j}\left(1,-1\right)$ is determined with the four original pixels, ${g}_{i,j}$ , ${g}_{i+1,j-1}$ , ${g}_{i+1,j}$ , and ${g}_{i,j-1}$ .

Figure 1. The pyramidal decomposition for interpolation. (a) Diagram of pyramidal decomposition, (b) The notation and relation of interpolated and original pixels, (c) ${g}_{i,j}$ decomposed into ${g}_{i,j}\left(-1,-1\right)$ , ${g}_{i,j}\left(-1,1\right)$ , ${g}_{i,j}\left(1,-1\right)$ , ${g}_{i,j}\left(1,1\right)$ .

4) ${g}_{i,j}\left(1,1\right)$ is determined with the four original pixels, ${g}_{i,j}$ , ${g}_{i+1,j+1}$ , ${g}_{i+1,j}$ , and ${g}_{i,j+1}$ .

Step 3: The interpolated pixel is determined by using the smoothing filter performed on the four original pixels. If odd pixels are needed, ${g}_{i,j}$ is repeatedly added in convenience; for example, a median filter is applied.

3. Interpolator Derived from Smoothing Filters

In this section, we illustrate the approach discussed in Sec. 2 with interpolators derived from a mean filter, a median filter, or a probability filter.

3.1. Interpolator Based on the Mean Filter

The interpolator derived from the mean filter can be expressed as Equation (1).

${g}_{i,j}\left(k,l\right)=\left({g}_{i,j}+{g}_{i+k,j}+{g}_{i,j+l}+{g}_{i+k,j+l}\right)/4$ (1)

where (k, l) = (−1, −1), (−1, 1), (1, −1), (1, 1) shown as in Figure 1.

3.2. Interpolator Based on the Median Filter

Similarly, the interpolator derived from the median filter can be expressed as Equation (2).

${g}_{i,j}\left(k,l\right)=median\left\{{g}_{i,j},{g}_{i+k,j},{g}_{i,j+l},{g}_{i+k,j+l}\right\}$ (2)

where (k, l) = (−1, −1), (−1, 1), (1, −1), (1, 1).

3.3. Interpolator Based on the Probability Filter

Originally, a probability filter [11] is designed for image smoothing or integration with variable weights which are interpreted as probabilities of respective pixel values in a local neighborhood of the current estimated pixel. The algorithm of this filter is explained in more detail as follows. ${f}^{*}\left(i,j\right)$ , the restored pixel at (i, j), is defined as

${f}^{*}\left(i,j\right)={\displaystyle \underset{k=-n}{\overset{n}{\sum}}{\displaystyle \underset{l=-n}{\overset{n}{\sum}}p\left(i+k,j+l\right)g\left(i+k,j+l\right)}}$ (3)

where $p\left(i+k,j+l\right)$ , the probability function of a (2n+1) by (2n+1) mask, is given as below:

$\begin{array}{l}p\left(i+k,j+l\right)\\ =\mathrm{exp}\left\{-{\left[g\left(i+k,j+l\right)-\stackrel{\xaf}{g}\left(i,j\right)\right]}^{2}/2{T}_{1}^{2}\right\}/Norm1,\text{\hspace{0.17em}}\text{\hspace{0.17em}}k,\text{\hspace{0.17em}}l=-n,\cdots ,-1,0,1,\cdots ,n\end{array}$ (4)

and

$Norm1={\displaystyle \underset{k=-n}{\overset{n}{\sum}}{\displaystyle \underset{l=-n}{\overset{n}{\sum}}\mathrm{exp}\left\{-{\left[g\left(i+k,j+l\right)-\stackrel{\xaf}{g}\left(i,j\right)\right]}^{2}/2{T}_{1}^{2}\right\}}}$ (5)

${T}_{1}=\frac{1}{{\left(2n+1\right)}^{2}}{\displaystyle \underset{k=-n}{\overset{n}{\sum}}{\displaystyle \underset{l=-n}{\overset{n}{\sum}}\left|g\left(i+k,j+l\right)-\stackrel{\xaf}{g}\left(i,j\right)\right|}}$ (6)

$\stackrel{\xaf}{g}\left(i,j\right)$ and $g\left(i+k,j+l\right)$ are the average of all pixels in the mask centered at (i, j) and the gray level of the pixel at $g\left(i+k,j+l\right)$ , respectively.

For the purpose of image interpolation with noise removal, this probability filter is modified as follows. According the pyramidal decomposition as Figure 1, the new interpolated pixel, ${g}_{i,j}\left(k,l\right)$ , yielded from the original pixel, ${g}_{i,j}$ , can be defined as below:

${g}_{i,j}\left(k,l\right)=\left({p}_{i,j}{g}_{i,j}+{p}_{i+k,j}{g}_{i+k,j}+{p}_{i,j+l}{g}_{i,j+l}+{p}_{i+k,j+l}{g}_{i+k,j+l}\right)/Norm2$ (7)

where

${p}_{i,j}=w\times \mathrm{exp}\left[-{\left({g}_{i,j}-\stackrel{\xaf}{g}\right)}^{2}/2{T}_{2}^{2}\right]$ (8)

${p}_{i+k,j}=\mathrm{exp}\left[-{\left({g}_{i+k,j}-\stackrel{\xaf}{g}\right)}^{2}/2{T}_{2}^{2}\right]$ (9)

${p}_{i,j+l}=\mathrm{exp}\left[-{\left({g}_{i,j+l}-\stackrel{\xaf}{g}\right)}^{2}/2{T}_{2}^{2}\right]$ (10)

${p}_{i+k,j+l}=\mathrm{exp}\left[-{\left({g}_{i+k,j+l}-\stackrel{\xaf}{g}\right)}^{2}/2{T}_{2}^{2}\right]$ (11)

$Norm2=\left({p}_{i,j}+{p}_{i+k,j}+{p}_{i,j+l}+{p}_{i+k,j+l}\right)/4$ (12)

$\stackrel{\xaf}{g}=\left({g}_{i,j}+{g}_{i+k,j}+{g}_{i,j+l}+{g}_{i+k,j+l}\right)/4$ (13)

${T}_{2}=\left(\left|{g}_{i,j}-\stackrel{\xaf}{g}\right|+\left|{g}_{i+k,j}-\stackrel{\xaf}{g}\right|+\left|{g}_{i,j+l}-\stackrel{\xaf}{g}\right|+\left|{g}_{i+k,j+l}-\stackrel{\xaf}{g}\right|\right)/4$ (14)

and w, a weight number, is chosen to be 1.5 here and (k, l) = (−1, −1), (−1, 1), (1, −1), (1, 1) shown as in Figure 1.

As previous discussion, the interpolator associated with the probability filter, (8)~(15), employs the characteristics of interpolating an image with noise removal. Furthermore, the difference of the mean filtering, median filtering, and probability filtering has been discussed and illustrated as described in [11] .

4. Results and Discussions

In this section, the interpolators are performed on two categories, grey-level and color images, including the following three cases: 1) case 1: noiseless images, 2) case 2: images with impulse noise, and 3) case 3: images with Gaussian white noise. The results of interpolated images through five kinds of interpolators (a bilinear interpolator, a spline interpolator, and three filters designed in Sec. 3,) are to be compared with each other. The mean square error (M.S.E.) is adopted for the measure of evaluation. A 50 × 80 image of clown was used to be interpolated and obtained a “100 × 160” interpolated image.

Figure 2 demonstrates case 1. As shown in Figure 2, less difference is visualized but there is some difference among them in MSEs which are 106.9361, 106.9361, 21.5693, 22.4969, and 21.5758 forFigures 2(d)-(h) corresponding to Figure 2(b), respectively. Figure 3 demonstrates case 2. Obviously, Figure 3(g) and Figure 3(h) are improved with less noise; MSEs are 157.1737, 188.4837, 29.6231, 25.9398, and 26.2405 for Figures 3(d)-(h) corresponding to Figure 3(b), respectively. Thus, the interpolation using median or probability filtering is suggested for interpolating images with impulse noise. Figure 4 demonstrates case 3. As shown in Figure 4, Figure 4(f) and Figure 4(h) are better than other; MSEs are 153.8532, 175.6014, 25.7838, 26.7301, and 25.7365 forFigures 4(d)-(h) corresponding to Figure 4(b), respectively. As can be seen, the interpolation using mean or probability filtering is recommended for interpolating images with Gaussian white noise.

As can be seen, a bilinear or a spline interpolation demonstrates a good visual quality as other three interpolation do in noiseless images; however, when an image with unknown noise is to be processed, the interpolation with the proposed approach cooperated with probability filtering is best choice whereas a bilinear or a spline interpolation enhances noise.

Figure 2. Comparison among varied interpolators for a down-sampled image without noise. (a) 200 × 320 original image, (b) 100 × 160 down-sampled image, (c) 50 × 80 down-sampled image, and 100 × 160 interpolated images using (d) bilinear method, (e) spline method, (f) mean filtering, (g) median filtering, and (h) probability filtering, respectively.

Figure 3. Comparison among varied interpolators for a down-sampled image with 5% impulse noise. (a) 200 × 320 original image, (b) 100 × 160 down-sampled image, (c) 50 × 80 down-sampled and noisy image, and 100 × 160 interpolated images using (d) bilinear method, (e) spline method, (f) mean filtering, (g) median filtering, and (h) probability filtering, respectively.

Figure 4. Comparison among varied interpolators for a down-sampled image with PSNR = 10 dB Gaussian white noise. (a) 200 × 320 original image, (b) 100 × 160 down-sampled image, (c) 50 × 80 down-sampled and noisy image, and 100 × 160 interpolated images using (d) bilinear method, (e) spline method, (f) mean filtering, (g) median filtering, and (h) probability filtering, respectively.

5. Conclusion

This paper has developed the approach to designing an interpolator based on a smoothing filter. Such a universal approach can generalize the application of the image smoothing filter for the purpose of interpolation and smoothing simultaneously. According to the above discussion, the interpolation using probability filtering is optimal either for noiseless images or for noisy images.

Acknowledgements

The work is substantially supported by a grant from the MOST in Taiwan (Project No. MOST 106-2221-E-236-007).

References

[1] Lehmann, T.M., Gönner, C. and Spitzer, K. (1999) Survey: Interpolation Methods in Medical Image Processing. IEEE Transactions on Medical Imaging, 18, 1049-1074.

https://doi.org/10.1109/42.816070

[2] Vaquero, D., Turk, M., Pulli, K. Tico, M. and Gelfand, N. (2010) A Survey of Image Retargeting Techniques. SPIE Optical Engineering Applications, 7798, Article ID: 779814.

https://doi.org/10.1117/12.862419

[3] Zhen, R.W. and Stevenson, R.L. (2015) Color Image and Video Enhancement: Chapter 2-Image Demosaicing.

[4] Gulati, T. and Sinha, H.P. (2014) Interpolating Low Resolution MRI Images Using Polynomial Based Interpolation. International Journal of Engineering Trends and Technology, 10, 626-631.

https://doi.org/10.14445/22315381/IJETT-V10P325

[5] Thevenaz. T.B. and Unser, M. (2004) Linear Interpolation Revitalized. IEEE Transaction on Image Processing, 13, 710-719.

https://doi.org/10.1109/TIP.2004.826093

[6] Meijering, E.H.W., Zuiderveld, K.J. and Viergever, M.A. (1999) Image Reconstruction by Convolution with Symmetrical Piecewise nth-Order Polynomial Kernels. IEEE Transactions on Image Processing, 8, 192-201.

https://doi.org/10.1109/83.743854

[7] Carrato, S., Ramponi, G. and Marsi, S. (1996) A Simple Edge-Sensitive Image Interpolation Filter. Proceedings of IEEE on International Conference of Image Processing, 3, 711-714.

https://doi.org/10.1109/ICIP.1996.560778

[8] Li, X. and Orchard, M. (2001) New Edge Directed Interpolation. IEEE Transactions on Image Processing, 10, 1521-1527.

https://doi.org/10.1109/83.951537

[9] Kirshner, H. and Porat, M. (2009) On the Role of Exponential Splines in Image Interpolation. IEEE Transactions on Image Processing, 18, 2198-2208.

https://doi.org/10.1109/TIP.2009.2025008

[10] Lee, J.S., Wen, J.H., Ainsworth, T.L., Chen, K.S. and Chen, A.J. (2009) Improved Sigma Filter for Speckle Filtering of SAR Imagery. IEEE Transactions on Geoscience and Remote Sensing, 47, 202-213.

https://doi.org/10.1109/TGRS.2008.2002881

[11] Pan, M.-C. and Lettington, A.H. (1998) Smoothing Images by a Probability Filter. Proceedings of IEEE International Joint Symposia on Intelligence and Systems, Rockville, MD, 23 May 1998, 343-346.