Multi-Scale Fusion of Enhanced Hazy Images Using Particle Swarm Optimization and Fuzzy Intensification Operators

— Dehazing from a single image is still a challenging task, where the thickness of the haze depends on depth information. Researchers focus on this area by eliminating haze from the single image by using restoration techniques based on haze image model. Using haze image model, the haze is eliminated by estimating atmospheric light, transmission, and depth. A few researchers have focused on enhancement based methods for eliminating haze from images. Enhancement based dehazing algorithms will lead to saturation of pixels in the enhanced image. This is due to assigning fixed values to the parameters used to enhance an image. Therefore, the enhancement based methods fail in the proper tuning of the parameters. This can be overcome by optimizing the parameters that are used to enhance the images. This paper describes the research work carried to derive two enhanced images from a single input hazy image using particle swarm optimization and fuzzy intensification operators. The two derived images are further fused using multi-scale fusion technique. The objective evaluation shows that the entropy of the haze eliminated images is comparatively better than the state-of-the-art algorithms. Also, the fog density is measured using an evaluator known as fog aware density evaluator (FADE), which considers all the statistical parameters to differentiate a hazy image from a highly visible natural image. Using this evaluator we found that the density of the fog is less in our proposed method when compared with enhancement based algorithms used to eliminate haze from images.


I. INTRODUCTION
Weather condition varies, based on the size and number of the particles in space. According to Nayar et.al as in [1], the inclement weather condition is due to large particle size and more concentration of the particles in space. There are two types of weather condition namely (i) static and (ii) dynamic weather conditions. Haze and fog are static whereas rain and snow are dynamic. Dispersed aerosol in the atmosphere, suspended as small particles, forms haze. Due to the increase in humidity in the atmosphere, the denser haze becomes fog. Both haze and fog decrease the perceptual quality of the image. To increase the visibility of the degraded images affected due to haze and fog many dehazing algorithms were developed based on enhancement and restoration based methods.
Restoration based methods use the haze image model which is given by Koschmieder Eq. (1), )) , (1) where H(x,y) is the hazy image, H f (x,y) is the dehazed or haze-free image, A is the atmospheric light and t(x,y) is transmission which is given in Eq.
(2) as follows, Transmission t(x, y) is the light that is reaching the image sensors after reflecting from the scene and its value varies between 0 and 1. Transmission depends on scattering coefficient b and depth d (x,y).
From the Eq. (1), the first term known as attenuation term H f (x,y)t(x,y) will produce low contrast image and the second term known as airlight term A(1-t(x,y)) will cause whiteness to the image will blur the hazy image. Improving the contrast and colour correction will remove the haze from the image. By estimating the transmission (t) and atmospheric light (A) we can restore the haze-free (H f ) image.
Enhancement based methods mainly focus on enhancing the image along with colour correction. Most of the image enhancement based method uses Histogram equalization, single and multi-scale Retinex theory and wavelet. But these methods fail to preserve the colour fidelity. Also, a saturation of picture element occurs due to over enhancement of hazy images.
Research work on de-hazing started initially with multiple images. According to Narasimhan et.al [2], scene depth discontinuity can be detected by determining the pixel intensity variation of different images of the same scene taken at different weather conditions. Similarly, Schechner et.al [3] developed a method by capturing multiple images taken at different degree of polarization.
Later, many researchers started working using a single hazy image. Fattal's [4] method was based on restoration using independent component analysis (ICA) to estimate the transmission to remove haze from single color image. Tarel's [5] method was based on calculating the airlight, which is assumed to be a percentage between local standard deviation and the local mean of whiteness. Kaiming He et al. [6] introduced "dark channel prior" which was a turning point in this research work, and most of the researcher's followed Kaiming He's work which showed very good results comparatively. But, the problem with the halos near the edges in the image was not completely removed. So, Kaiming [6] introduced guided filter to remove the halos, but not succeeded completely because guided filter being a local filter. So, Padmini et al. [7] removed the halos completely by using guided filter followed by L0 gradient minimization filter, which is a global filter. Similarly, many other researchers in this field removed halos in the image by using global filters. Recently, many researchers started using metaheuristic algorithms in dehazing as in Guo [8] and Zhang [9]. A complete review of the literature work in this field of de-hazing can be referred from Tripathi et al. [10], Padmini et al. [11] and Singh [12].

II. MATERIALS AND METHOD
Multi-scale fusion based de-hazing was introduced by Ancuti et al. [13], using two images derived from a single hazy image. First derived image was the white balancing of hazy image for color correction and the second derived image was the enhancement of hazy image by subtracting the hazy image from the average intensity of the image or by using simple Histogram equalization technique. The two images are further fused using multi-scale fusion technique. This method was comparatively better when compared with other de-hazing methods because it uses point processing when compared with other methods which use the patchbased method so that the halos in the image can be reduced. In general, the problem with the enhancement based methods is the proper tuning of the parameters. This can be overcome by optimizing the parameters by using metaheuristic optimization algorithms.
Metaheuristic methods are search based methods, which tunes the parameter to get an optimized value in the proposed method, the global search optimization algorithm. Particle Swarm Optimization (PSO) method is used to enhance the hazy input image to obtain one of the derived input images.
The other derived image from the hazy input image is obtained by fuzzy intensification, which mainly enhances as well as performs color correction. The two derived image is further applied with luminance, chrominance and saliency weight map as in [13], which is further fused using multiscale fusion technique as shown in

A. Derived Input Image(1) Using PSO
The hazy input image is enhanced by using particle swarm optimization technique. Let the input hazy image be H(x,y) and the enhanced derived image using PSO be P(x,y) as explained in [14], [15] which is specified in Eq.
where a and C are constants in the range a∊(0, 1.5) and C∊(0, 1) and local mean m for an nxn window is given by Eq. (4)  (4) and K(x,y) depends on global mean and global variance which is given by Eq.
Where d and b are constants in the range d ∊ (0.5, 1.5) and b ∊ (0, total mean/2) and G is the global mean which can be written as in Eq. (6) (6) and global variance σ is specified by the Eq. (7) Multi-scale Fusion The objective function, which is known as fitness function, is formulated is given in Eq. (8) ) where E(I s ) is the edge intensity of the image after applying the Sobel edge detector. n edges is the number of edges above threshold in Sobel operation. E(I e ) is the entropy of the enhanced image.
B. Introduction to PSO PSO is initiated with random particles which are known as a swarm. The particles search and find the optimal solution, which is updated for every iteration. In every iteration, each particle is updated with two optimal values. The best solution of each particle "pbest" is updated. The other solution, "gbest" is determined to be the best out of all particles obtained. After determining the "pbest" and "gbest" the particle is moved to a new position by finding the position and velocity. The velocity is determined from the following Eq. (9), where w is the inertia weight used to balance the local and global search, and c 1 and c 2 are positive acceleration constant. The parameters r 1 and r 2 are random values in the range 0 and 1. p(t) is the best solution obtained from a particle, whereas g(t) is the global best solution obtained so far. The position can be determined from the following Eq. (10), Many researchers have worked on enhancing the images using the optimization technique like PSO as in [14] and [15], which maximize the fitness function as given in Equation (8). Compared with enhancement techniques like Histogram equalization and contrast stretching, Particle swarm optimization technique shows better results. So, in this paper, one of the derived input hazy images is enhanced using PSO. Also, in Ancuti, [13] the derived enhanced image can cause problems by making the pixels darker or brighter by subtracting the pixel intensity from average intensity, which can be overcome by PSO.

C. Derived Input Image(2) Using Fuzzy Intensification:
The other derived image is obtained using a fuzzy intensification operator which is mainly used to restore the colour that was lost because of the air-light as described in equation (1). In Fuzzy intensification as explained in [16], the input colour hazy image is disintegrated into three layers namely R, G, B and for each layer membership function is computed as follows, Membership for red channel Mf R is specified in Eq. (11) )) min( ) (max( )) min( ( r r r r Mf R − − = (11) Similarly, membership for green channel Mf G, is specified in Eq. (12) )) min( ) (max( )) min( ( g g g g Mf G − − = (12) and Membership for blue channel Mf B, is specified in Eq.
)) min( ) (max( where r, g, b indicates the pixel value in the respective channel and min, max indicates the minimum and maximum pixel value in the channel. So the membership function will convert the pixels in the range 0 to 1. Further, fuzzy intensification is applied as explained in [16], is shown in Eq. (14) for the red channel.
Similarly, Equation (15) and (16) indicates fuzzy intensification applied for green and blue channels, Where τ R, τ G, τ B are predetermined scalars with values 0.5, 0.6 and 0.4. Further, the fuzzy intensification is tuned using the formula as given in Eq. (17), Eq. (18) and Eq. (19) Zeta (ζ) is a tuning parameter which is chosen as 0.5. Thus the second derived image is obtained which is free from colour cast. The derived image (1) which was enhanced using PSO and the derived image (2) enhanced and colour corrected using fuzzy intensification operator are further processed before fusing together by applying luminance, chrominance and saliency weight map as explained in [13].

D. Applying Weight Maps to Derived Images:
Before multi-scale fusion, the derived images are applied with luminance, chrominance, and saliency weight maps explained in [13] is specified in Eq. (20) as follows, where L is the average of R, G, B channels.
Chrominance weight map is computed as in Equation (21), where s(x) indicates the saturation of each pixel and s max indicates maximum saturation value. s(x) and s max are obtained from HSI colour space. σ indicates standard deviation and its default value is chosen as 0.3. Saliency weight map is computed as in Eq. (22), where I blur is the blurred derived input image and I μ is the average or mean pixel value of the input image. Finally, the resultant weight map is obtained by multiplying the luminance, chrominance and saliency weight maps. Further, a normalized weight map W N is obtained from the resultant weight map.

E. Multi-scale Fusion Using Weighted Gaussian Pyramid and Laplacian Pyramid
Since direct fusion of the images causes halos, multi-scale fusion technique is adopted as in [13]. Here each derived input image is initially multiplied by normalized weight map and further the image is decomposed into the Gaussian pyramid at different scales. Similarly, the Laplacian pyramid is obtained from the input derived image. Finally, the weighted Gaussian pyramid and the Laplacian pyramid for each derived image is multiplied and finally added.
The computation for multi-scale fusion is computed as in Eq. (23), where MS f is the final multi-scale fused haze free output image, G l is Gaussian pyramid and L l is the Laplacian pyramid with same levels l, W N is normalized weight map and I K is the derived input image with index K=1 and 2.

III. RESULTS AND DISCUSSION
The results are discussed with respect to subjective and objective evaluation. For subjective evaluation, two hazy images, a doll, and a mountain image are taken and compared with the haze-free image computed using Ancuti's method [13] and our proposed method as shown in fig 2.   Fig 2. (a) and (d) are input hazy image, (b) and (e) are haze-free images obtained by computing Ancuti's method [13], (c) and (f) are haze-free images obtained by our proposed method.
From the images, as pointed by the highlighted red square region shown in fig 2, indicates that the details are clearly visible in our method. Apart from subjective evaluation, the objective evaluation was performed. Table I. shows the comparison of a fitness function calcualated using Eq. (8), for enhancing the derived hazy image using Histogram equalization and Particle swarm optimization techinique as explained in Eq. (3). In the case of PSO, the number of particles is considered as 20 and the iteration performed is 20. Around 18 hazy images were tested using MATLAB 2013 and the average of the two methods shows that fitness function using PSO as explained in Eq. (3), shows maximum when compared with Histogram equalization. For objective evaluation, parameters like entropy and fog density measurement using FADE software [17] was performed. Entropy is a measure of randomness based on statistics that can be used to describe the texture of the image. Entropy is defined as given in Eq. (24) where p is the normalized histogram counts. Entropy was tested for 18 images and founded that the entropy for a dehazed image using our proposed method as shown better results than Ancuti's method. A higher value of entropy indicates that details are clearly visible thereby enhancing the hazy images.  Table II shows the comparison of the entropy of a dehazed image of Ancuti's method with our method. The average value considering for all the 18 images shows that the entropy value of our method using PSO is comparatively higher than Ancuti's method. Fig 3 shows the comparison of entropy in the form of a bar chart for 18 hazy images which was tested for both the methods. From these results, we can conclude that by using PSO to enhance the image the entropy comparatively increases. Our results are tested on software known as Fog Aware Density Evaluator (FADE) [17], which is a fog density prediction model based on natural scene statistics. This software is modeled in such a way that it predicts the fog density from a single image, without any reference fog-free image of the same scene. It does not depend on salient objects in a scene nor any information related to the camera. Normally in the literature of fog removal algorithms, the fog density is predicted by estimating the transmission map, which is depth dependent. But this software predicts the fog density without estimating transmission map. Table III shows fog density measurement using FADE. According to the results obtained after testing on foggy images, our method shows the fog density is comparatively less than Ancuti's method. Fig 4. shows the comparison of fog density for different hazy images in the form of the bar chart.

IV. CONCLUSIONS
The Multi-scale fusion of enhanced hazy images using PSO and fuzzy intensification operators shows improved entropy, and the fog density is comparatively reduced, which is measured using FADE evaluator. Fog removal algorithms find wide applications in the driver assistance system, in automatic robotic navigation systems and surveillance cameras. Since most of the fog removal algorithms are used in the driver assistance system and automatic robotic navigation, the execution time of these algorithms should be considerably reduced as possible. Since particle swarm optimization is an iterative algorithm, the computation time of our proposed method is high. Our future work will be focused on reducing computation time.