Dehaze Enhancement Algorithm Based on Retinex Theory for Aerial Images Combined with Dark Channel

Show more

1. Introduction

Under foggy conditions, aerial images are severely degraded. Processing based on machine vision for image detection and recognition depends on the sharpness of the images captured by the imaging equipment. Therefore, the study of image sharpness has become one of the hot research topics in machine vision.

Image defogging is a method to remove unfavorable factors in an image that are unclear due to bad weather or a shooting device. When removing the effects of severe weather, such as rain and snow [1], haze [2], etc., leading to severe image quality, some researchers have used physical models [3] to study the causes of image degradation. He Kaiming (HE Kai-ming) proposed a method of image dehazing based on dark channel prior [4]. In order to reduce the complexity, Huang Darong uses wavelet thresholds to estimate the atmospheric light, then uses the atmospheric light to calculate the transmission map, and then uses the atmospheric light to restore the image [5]. Rajiv Kapoor uses the guidance filter to refine the transmission image based on the defogging of the dark channel. The limited contrast adaptive histogram equalization (CLAHE) is used to improve visibility and effectively reduce the halo phenomenon [6]. Other researchers have analyzed foggy images through the method of image enhancement [7]. The main reason for image quality is due to the decrease in contrast and brightness. Early image enhancement was enhanced by histogram equalization [8]. With different reasons for image degradation, Land and McCann proposed the retinal theory; Jobson et al. proposed the retinal enhancement algorithm (Retinex); Reitnex theory has better results in low contrast image applications, but due to the single-scale Retinex (Single Scale Retinex (SSR) algorithm [9] cannot take into account the contradiction between large-scale and small-scale at the same time in enhancing pictures. Later, on the basis of SSR, a multi-scale Retinex (Multi-Scale Reinex, MSR) theory was proposed [10]. Because the enhanced image cannot maintain color consistency, they also proposed a multi-scale Retinex (Multi-Scale with color recovery) Retinex with Color Restore (MSRCR) algorithm [11], which takes into account the dynamic range while maintaining color consistency. In 2017, LIU [12] obtained the MSR algorithm of more than 3 scales through mathematical derivation, which has a better effect on balancing dynamic range and color consistency. In order to further maintain the image edge details and texture details, PU proposed a fractional variational frame theory on the Retinex framework, thereby improving the texture details of the picture [13]. In the past 5 years, with the deepening of machine learning research, neural network-based defogging technology has also emerged. Among them, the defogging algorithm based on adversarial neural network [14] has excellent performance in defogging, but the data is not easy. Processing, and compared with the traditional defogging algorithm, the algorithm is not good in processing speed.

However, different algorithms perform differently in different foggy images. Based on the dark channel priors, they exhibit blockiness and inconsistent colors in aerial image dehazing, and the lack of dimming, while the typical Retinex algorithm shows the performance after fog removal. The image showed insufficient contrast and details were lost. Based on the Retinex algorithm model, this article combined the atmospheric scattering model to effectively estimate the fog-free incident light component and the fog-free reflected light component, and finally restored the fog-free image by the Retinex model.

2. Reitnex Theory

According to the Retinex physical model, the image acquired by the imaging device can be expressed as:

$S\left(x,y\right)=R\left(x,y\right)\times L\left(x,y\right)$ (1)

Among them, $S\left(x,y\right)$ is the foggy image, $L\left(x,y\right)$ is the incident light component, $R\left(x,y\right)$ is the reflected light component

To reduce the amount of calculation, convert the formula to the logarithmic domain, that is:

$r\left(x,y\right)=\mathrm{log}\left(x,y\right)=\mathrm{log}S\left(x,y\right)-\mathrm{log}L\left(x,y\right)$ (2)

The core of traditional Retinex is to accurately estimate the incident light component. Generally, the input foggy image and Gaussian filter function are used to estimate the convolution operation, where:

$F\left(x,y\right)=\lambda {\text{e}}^{-\frac{{x}^{2}+{y}^{2}}{{\sigma}^{2}}}$ (3)

$\lambda $ is a normalization constant such that:

$\iint F\left(x,y\right)\text{d}x\text{d}y}=1$ (4)

The single-scale Retinex (SSR) defogging operation can be expressed as:

$\mathrm{log}R\left(x,y\right)=\mathrm{log}S\left(x,y\right)-\mathrm{log}\left[S\left(x,y\right)\ast F\left(x,y\right)\right]$ (5)

Because single-scale Retinex (SSR) does not consider the influence of various scales on the defogging when estimating the incident light component, detailed information cannot be avoided. Therefore, researchers have proposed the concept of multi-scale, using three large, medium, and small scales to estimate the incident light component, and finally the idea of image fusion. The weighted average of the defogging results at each scale is within the dynamic range. Reduce fog while reducing loss of detail information. Multi-scale Retinex (MSR) mathematical expression is as follows:

${R}_{i}\left(x,y\right)={\displaystyle \underset{n=1}{\overset{3}{\sum}}{\omega}_{n}\left\{\mathrm{log}{S}_{i}\left(x,y\right)-\mathrm{log}\left[{S}_{i}\left(x,y\right)\ast {F}_{n}\left(x,y\right)\right]\right\}}$ (6)

where $\omega $ is the weight value.

Multi-scale Retinex with color recovery (MSRCR) adds a color recovery factor to eliminate the color distortion of the image after defogging. Its mathematical form is as follows:

${R}_{i}\left(x,y\right)={C}_{i}\left(x,y\right){\displaystyle \underset{n=1}{\overset{3}{\sum}}{\omega}_{n}\left\{\mathrm{ln}{S}_{i}\left(x,y\right)-\mathrm{ln}\left[{S}_{i}\left(x,y\right)\ast {F}_{n}\left(x,y\right)\right]\right\}}$ (7)

Traditional MSRCR uses a pair of gain and offset to perform linear transformations to transform the output domain of multi-scale Retinex (MSR) into the display domain. Its mathematical form is as follows:

${R}^{\text{msrcr}}\left(x,y\right)={C}_{i}\left(x,y\right){R}^{\text{msr}}\left(x,y\right)$ (8)

3. Atmospheric Scattering Model

McCartney et al. Described the causes of image degradation based on atmospheric scattering phenomena and constructed an atmospheric scattering model. The phenomenon described by the atmospheric physical scattering model is shown in Figure 1.

During the transmission of a beam of parallel light, its energy must be scattered due to the small particles in the atmosphere. If the length of the scattering medium is x, the mathematical expression of its attenuation model is:

${E}_{\alpha}\left(d,x\right)={E}_{0}\left(\lambda \right){\text{e}}^{-\beta \left(\lambda \right)dx}$ (10)

where ${E}_{\alpha}\left(d,x\right)$ is attenuated light, ${E}_{0}\left(\lambda \right)$ is incident parallel light, ${\text{e}}^{-\beta \left(\lambda \right)dx}$ is attenuation index, and d is called depth of field distance.

The atmospheric light intensity received by the imaging device after the atmospheric light passes through the transmission path is:

${E}_{\beta}\left(d,\lambda \right)={E}_{\infty}\left(\lambda \right)\left(1-{\text{e}}^{-\beta \left(\lambda \right)dx}\right)$ (11)

the total light intensity is:

$E\left(d,\lambda \right)={E}_{\alpha}\left(d,\lambda \right)+{E}_{\beta}\left(d,\lambda \right)={E}_{0}\left(\lambda \right){\text{e}}^{-\beta \left(\lambda \right)dx}+{E}_{\infty}\left(\lambda \right)\left(1-{\text{e}}^{-\beta \left(\lambda \right)dx}\right)$ (12)

let ${E}_{0}\left(\lambda \right)=J\left(x\right)$ , ${\text{e}}^{-\beta \left(\lambda \right)dx}=t\left(x\right)$ , ${E}_{\infty}\left(\lambda \right)={A}_{0}$ . Then the simplified calculation formula is as follows:

$I\left(x\right)=J\left(x\right)t\left(x\right)+{A}_{0}\left(1-t\left(x\right)\right)$ (13)

Dark Channel Prior Theory

Kaiming-He et al. Conducted statistics on the dark channels of a large number of fog-free images without sky areas, and found that the pixels of each area have

Figure 1. Atmospheric scattering model.

a gray value of 0 or near 0 in at least one color channel. Based on this prior knowledge, put forward the dark channel theory.

${J}^{dark}\left(x\right)=\underset{y\in \Omega \left(x\right)}{\text{min}}\underset{c\in \left\{r,g,b\right\}}{\mathrm{min}}{J}^{c}\left(y\right)$ (14)

Among them, ${J}^{c}\left(y\right)$ represents the R, G, and B color channels of the foggy image, $\Omega \left(x\right)$ is a window centered on x; c is the three color channels. In the non-sky area of the fog-free image, the gray value of the dark channel is very low, that is ${J}^{dark}\left(x\right)\to 0$ . According to the atmospheric scattering model formed by the fog image, if the fog is to be removed, two parameters need to be taken-the atmospheric light value. A and transmittance $\stackrel{\u02dc}{t}\left(x\right)$ . Kaiming-He et al. first assumed that the atmospheric light value A is a known quantity. Based on this assumption, the dark channel operation can be obtained:

$\underset{c\in \left\{r,g,b\right\}}{\mathrm{min}}\frac{{I}^{c}(y)}{A}=t\left(x\right)\underset{c\in \left\{r,g,b\right\}}{\mathrm{min}}\frac{{J}^{c}\left(y\right)}{A}+1-t\left(x\right)$ (15)

Take the minimum value of the brightness in the local area of the three RGB channels and redefine the transmittance. The mathematical form is as follows:

$\underset{c\in \left\{r,g,b\right\}}{\mathrm{min}}\underset{y\in \Omega \left(x\right)}{\mathrm{min}}\frac{{I}^{c}\left(y\right)}{{A}^{c}}=\stackrel{\u02dc}{t}\left(x\right)\underset{c\in \left\{r,g,b\right\}}{\mathrm{min}}\underset{y\in \Omega \left(x\right)}{\mathrm{min}}\frac{{J}^{c}\left(y\right)}{{A}^{c}}+1-\stackrel{\u02dc}{t}\left(x\right)$ (16)

According to prior knowledge:

$\stackrel{\u02dc}{t}\left(x\right)=1-\underset{c\in \left\{r,g,b\right\}}{\mathrm{min}}\left(\underset{y\in \Omega \left(x\right)}{\mathrm{min}}\left(\frac{{J}^{c}\left(y\right)}{{A}^{c}}\right)\right)$ (17)

Due to the presence of various small particles in the sky area leading to scattering phenomena, adjustment parameters are introduced to maintain the authenticity of the image after defogging, which is:

$\stackrel{\u02dc}{t}\left(x\right)=1-\omega \underset{c\in \left\{r,g,b\right\}}{\mathrm{min}}\left(\underset{y\in \Omega \left(x\right)}{\mathrm{min}}\left(\frac{{J}^{c}\left(y\right)}{{A}^{c}}\right)\right)$ (18)

After the atmospheric light value A and transmittance are known, a restored fog-free image can be obtained, namely:

$J\left(x\right)=\frac{I\left(x\right)-A}{\mathrm{max}\left(t\left(x\right),{t}_{0}\right)}+A$ (19)

The result after defogging through the dark channel prior algorithm is shown in Figure 2.

4. Retinex Defogging Algorithm Combined with Dark Channel Model

Because the Retinex algorithm model is based on the principle of constant color enhancement and defogging, there is no analysis of the essential reason for the formation of fog. By adjusting the contrast and brightness of the image to achieve the effect of defogging enhancement, over-enhancement will occur. The dark channel prior defogging algorithm fully analyzes the cause of the formation

Figure 2. Prior defogging results of dark channel.

of fog and understands the nature of its formation. However, due to insufficient estimation of transmittance, the image processed by the dark channel prior defogging algorithm will appear darker. Holo effect and other issues. This section fully combines the advantages of the two defogging models. Based on the Retinex model, the dark channel defogging algorithm is used to analyze the nature of the fog, while maintaining higher contrast and brightness while defogging. The processing flow is as follows Figure 3.

The dark channel prior principle is developed based on the atmospheric scattering model, and the Retinex model is similar to the atmospheric scattering model. The Retinex model is equivalent to the attenuation model in the atmospheric scattering model. Atmospheric scattering model to estimate the incident light in the Retinex algorithm is a combination of the two algorithms. The mathematical forms of the two models are as follows:

$\{\begin{array}{l}I\left(x\right)=J\left(x\right)\times t\left(x\right)+A\times \left(1-t\left(x\right)\right)\\ J\left(x\right)=R\left(x\right)\times L\left(x\right)\end{array}$ (20)

where $I\left(x\right)$ is the fogged image obtained by the imaging device, $R\left(x\right)$ is the incident light component of the image after defogging, and $L\left(x\right)$ is the reflected light component after defogging. Assuming that the defogged images of the two models can simultaneously meet the images collected under sunny weather conditions, there are:

$I\left(x\right)=R\left(x\right)L\left(x\right)t\left(x\right)+A\left(1-t\left(x\right)\right)$ (21)

Among them, $R\left(x\right)$ is a fog-free incident light component, and $L\left(x\right)$ is a fog-free reflected light component. From the foggy image $I\left(x\right)$ , $R\left(x\right)L\left(x\right)$ can be effectively separated to achieve the purpose of defogging.

The Retinex model of the foggy image according to the above formula can be

Figure 3. Retinex + DCP defogging process.

expressed as:

${J}^{-1}\left(x\right)={R}^{-1}\left(x\right)\times {L}^{-1}\left(x\right)$ (22)

It is known from Retinex theory that the foggy incident light component ${R}^{-1}\left(x\right)$ can be obtained by Gaussian blurring of foggy images. Retinex can meet the dynamic range compression, enhance the details and maintain the color balance. Then the incident light component ${R}^{-1}\left(x\right)$ of the foggy image can be obtained by the following formula:

${R}^{-1}\left(x\right)={\displaystyle \underset{n=1}{\overset{3}{\sum}}\frac{1}{n}I\left(x\right)}\ast F\left(x\right)$ (23)

where $F\left(x\right)$ is the Gaussian surround function and n is the number of scales.

The fog reflection map can be obtained by the principle of atmospheric light scattering, namely:

${R}^{-1}\left(x\right)=R\left(x\right){t}_{\alpha}\left(x\right)-{A}_{\text{0}}\left(1-{t}_{\alpha}\left(x\right)\right)$ (24)

Solve the fog-free reflection map $R\left(x\right)$ through the fog reflection map ${R}^{-1}\left(x\right)$ , and use the dark channel dehazing principle to determine the atmospheric transmittance ${t}_{\alpha}\left(x\right)$ and the atmospheric light value ${A}_{\text{0}}$ . The experimental results show that when ${A}_{0}=1$ , the calculation amount can be reduced, and at the same time, a good Defog effect. According to the dark channel defogging theory, its transmittance is obtained:

${t}_{\alpha}\left(x\right)=1-\underset{c\in \left\{r,g,b\right\}}{\mathrm{min}}\left(\underset{y\in \Omega \left(x\right)}{\mathrm{min}}\left({R}^{-1}\left(x\right)\right)\right)$ (25)

The Retinex model is used to separate foggy reflected light components. The foggy reflected light component can be obtained by logarithmic transformation, which has the form:

${L}^{-1}\left(x\right)={\text{e}}^{\mathrm{log}{L}^{-1}}={\text{e}}^{\left[\mathrm{log}{J}^{-1}\left(x\right)-\mathrm{log}{R}^{-1}\left(x\right)\right]}$ (26)

where ${t}_{\beta}\left(x\right)$ is obtained from the fog reflection light component obtained by it:

${t}_{\beta}\left(x\right)=1-\underset{c\in \left\{r,g,b\right\}}{\mathrm{min}}\left(\underset{y\in \Omega \left(x\right)}{\mathrm{min}}\left({L}^{-1}\left(x\right)\right)\right)$ (27)

Finally, through the Retinex physical model, the components of the fog-free incident light and the fog-free reflected light are multiplied to obtain a fog-free image after defogging.

5. Experimental Results and Evaluation

In order to verify the effectiveness of the algorithm proposed in section 3, this algorithm uses Microsoft Visual Studio 2015 development platform and Open CV tripartite library. The computer's configuration is 4GB, operating system: Windows 10 (64-bit) for verification. The algorithm of this paper is used to enhance the defogging. The processing result and the corresponding gray histogram are shown in the following Figure 4.

5.1. Subjective Evaluation

Compare the pros and cons of the effect after image processing. Because the

Figure 4. Experimental results of the algorithm in this paper.

subjective evaluation is simple and fast, it can reflect the first sense of the processed image for various tasks, which will help researchers to take advantage of the masses in subsequent research. It is beneficial for related personnel to have better visual effects when applying these images. The experimental images in this paper use the characteristics of low fog density, low contrast and overall graying. It can be seen from the histogram display that the histogram of the algorithm results in this paper is more evenly distributed and the gray levels are more obvious, which makes the recognition of ground objects very high. However, the gray level of the original image in this experiment is concentrated in a certain area, and the image shows underexposure, and the graying phenomenon is serious.

As shown in Figure 5, the image obtained by the algorithm in this paper has higher contrast and color vividness. Effectively avoid the under-estimation of the dark channel algorithm in the sky region and the phenomenon of Retinex over-enhancement. And it effectively removes the interference of fog, which greatly improves the visual effect of the overall image.

As shown in this paper, various image dehazing methods are used for horizontal comparison, including the method of Fattal et al. in the literature [3], the method of Tarel et al. in the literature [15], and the standard dark channel dehazing algorithm, NASA The defogging enhancement algorithm based on Retinex theory, by reproducing its algorithm, is compared with the results of the algorithm in this paper. The results are shown in Figure 6, and Figure 6(a) is the original image. From left to right, the defogging results of different algorithms. The experimental images in this paper are all with high fog concentration and low visibility. As a whole, the Fattal algorithm has average defogging ability, and the sky area is completely distorted. The Tarel algorithm still has a large amount of fog remaining, and does not show a good defogging effect. The HE algorithm performs well in defogging, but the overall color of the defogging image obtained is dark, as shown in Figure 6(d-3) and Figure 6(d-4). The image itself has low brightness and poor visual effects. The Retinex algorithm uses excellent image performance in this paper, but still cannot retain the authenticity of the sky area. At the same time, the contrast shown in Figure 6(e-2) and Figure 6(e-4) is too high, and there is an over-enhancement phenomenon. There is a

Figure 5. Sky area comparison.

Figure 6. Algorithm comparison results.

phenomenon of color distortion. The algorithm proposed in this paper performs well in defogging performance, with higher color fidelity and rich levels.

5.2. Objective Evaluation

The SSIM evaluation factor is the result of multi-information fusion. The mean, variance, and covariance of the image are required. The brightness, contrast, and structure of the image are integrated to form a comprehensive SSIM evaluation index.

By comparing the evaluation factors of the images of the results of different algorithms, the quality of the image can be objectively evaluated. The higher the average gradient and the SSIM score, the higher the quality of the image. The original images in the figure are selected as a (1,2), a (1,3), a (1,5) and evaluated as comparative images. The results are shown in Table 1.

According to Table 1, it can be concluded that the average gradient (avgGradient) and structural similarity (SSIM) of the algorithm in this paper exceed the other algorithms in scores. The image processed by this algorithm maintains the original image structure while dehazing and has more details Information, at the same time, it has a higher global and local contrast, removes the graying phenomenon of the original image, the color distortion is small, and there is no color distortion and halo in the sky area, while avoiding the phenomenon of excessive enhancement of the Retinex algorithm, Effectively improve the visual effect of the image.

6. Conclusion

Aiming at the phenomenon that the single Retinex defogging algorithm has

Table 1. Algorithm comparison evaluation index results.

insufficient defogging capacity and over-enhancement, based on the channel-based prior defogging algorithm, the brightness of the aerial image after defogging is low, and the sky area has poor performance of color shift and halo phenomenon. In this article, according to the characteristics of the two models, the aerial image is separated into foggy incident light component and foggy reflected light component. Using the dark channel defogging algorithm, the atmospheric light value and transmittance of the fog incident light component and the fog reflected light component are re-estimated, respectively, to obtain new fog-free incident light component and fog-free reflected light component. The fog-free image obtained by Retinex physical imaging model has higher contrast and brightness. Experiments on the algorithm proposed in this article verify the effectiveness of the proposed algorithm. Through subjective and objective evaluation with a variety of existing defogging algorithms, the algorithms presented in this article have significantly improved the defogging capabilities of aerial images, with higher contrast and color saturation.

References

[1] Li, S., Ren, W., Zhang, J., et al. (2019) Single Image Rain Removal via a Deep Decomposition-Composition Network. Computer Vision and Image Understanding.

https://doi.org/10.1016/j.cviu.2019.05.003

[2] Lu, H., Li, Y., Nakashima, S., et al. (2016) Single Image Dehazing through Improved Atmospheric Light Estimation. Multimedia Tools and Applications, 75, 17081-17096.

https://doi.org/10.1007/s11042-015-2977-7

[3] Fattal, R. (2008) Single Image Dehazing. ACM Transactions on Graphics, 27, 1-9.

https://doi.org/10.1145/1360612.1360671

[4] He, K.M., Sun, J. and Tang, X. (2011) Single Image Haze Removal Using Dark Channel Prior. IEEE Transcations on Pattern Analysis & Machine Intelligence, 33, 2341-2353.

https://doi.org/10.1109/TPAMI.2010.168

[5] Huang, D.R., et al. (2014) An Improved Image Clearness Algorithm Based on Dark Channel Prior. Proceedings of the 33rd Chinese Control Conference, Nanjing, 28-30 July 2014.

https://doi.org/10.1109/ChiCC.2014.6896219

[6] Rajiv, K., et al. (2019) Fog Removal in Images Using Improved Dark Channel Prior and Contrast Limited Adaptive Histogram Equalization. Multimedia Tools and Applications, 78, 23281-23307.

https://doi.org/10.1007/s11042-019-7574-8

[7] Wang, W.C. and Yuan, X.H. (2017) Recent Advances in Image Dehazing. IEEE/CAA Journal of Automatica Sinica, 4, 410-436.
https://doi.org/10.1109/JAS.2017.7510532

[8] Sonali, S.S., Singh, A.K., Ghrera, S.P. and Elhoseny, M. (2018) An Approach for De-Noising and Contrast Enhancement of Retinal Fundus Image Using CLAHE. Optics and Laser Technology, 110, 87-98.
https://doi.org/10.1016/j.optlastec.2018.06.061

[9] Jobson, D.J., Rahman, Z.-U., Woodell, G., et al. (1997) Properties and Performance of a Center/Surround Retinex. IEEE Transactions on Image Processing, 6, 451-462.

https://doi.org/10.1109/83.557356

[10] Patil, M.D.V., Sutar, M.S.G. and Mulla, M.A.N. (2013) Automatic Image Enhancement for Better Visualization Using Retinex Technique. International Journal of Scientific and Research Publications, 3, No. 6.

[11] Livingston, M.A., Garrett, C.R. and Ai, Z. (2011) Image Processing for Human Understanding in Low-Visibility. Naval Research Lab Information Technology DIV, Washington DC.

https://doi.org/10.21236/ADA609988

[12] Liu, C.J., Cheng, I., Zhang, Y. and Basu, A. (2017) Enhancement of Low Visibility Aerial Images Using Histogram Truncation and an Explicit Retinex Representation for Balancing Contrast and Color Consistency. Journal of Photogrammetry and Remote Sensing, 128, 16-26.

https://doi.org/10.1016/j.isprsjprs.2017.02.016

[13] Pu, Y.F., Siarry, P., Chatterjee, A., et al. (2017) A Fractional-Order Variational Framework for Retinex: Fractional-Order Partial Differential Equation-Based Formulation for Multi-Scale Nonlocal Contrast Enhancement with Texture Preserving. IEEE Transactions on Image Processing, 27, 1214-1229.
https://doi.org/10.1109/TIP.2017.2779601

[14] Suárez, P.L., et al. (2018) Deep Learning Based Single Image Dehazing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, 18-22 June 2018. https://doi.org/10.1109/CVPRW.2018.00162

[15] Tarel, J.P. and Nicolas, H. (2009) Fast Visibility Restoration from a Single Color or Gray Level Image. IEEE 12th International Conference on Computer Vision, ICCV 2009, Kyoto, Japan, 27 September- 4 October 2009.
https://doi.org/10.1109/ICCV.2009.5459251