Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front. Comput. Sci.    2023, Vol. 17 Issue (2) : 172702    https://doi.org/10.1007/s11704-022-1205-7
RESEARCH ARTICLE
Underwater image enhancement by maximum-likelihood based adaptive color correction and robust scattering removal
Bo WANG1(), Zitong KANG1, Pengwei DONG1, Fan WANG1, Peng MA1, Jiajing BAI1, Pengwei LIANG1, Chongyi LI2
1. School of Physics and Electronic-Electrical Engineering, Ningxia University, Yinchuan 750021, China
2. School of Computer Science and Engineering, Nanyang Technological University, Singapore 639798, Singapore
 Download: PDF(29491 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Underwater images often exhibit severe color deviations and degraded visibility, which limits many practical applications in ocean engineering. Although extensive research has been conducted into underwater image enhancement, little of which demonstrates the significant robustness and generalization for diverse real-world underwater scenes. In this paper, we propose an adaptive color correction algorithm based on the maximum likelihood estimation of Gaussian parameters, which effectively removes color casts of a variety of underwater images. A novel algorithm using weighted combination of gradient maps in HSV color space and absolute difference of intensity for accurate background light estimation is proposed, which circumvents the influence of white or bright regions that challenges existing physical model-based methods. To enhance contrast of resultant images, a piece-wise affine transform is applied to the transmission map estimated via background light differential. Finally, with the estimated background light and transmission map, the scene radiance is recovered by addressing an inverse problem of image formation model. Extensive experiments reveal that our results are characterized by natural appearance and genuine color, and our method achieves competitive performance with the state-of-the-art methods in terms of objective evaluation metrics, which further validates the better robustness and higher generalization ability of our enhancement model.

Keywords underwater image enhancement      adaptive color correction      background light estimation     
Corresponding Author(s): Bo WANG   
Just Accepted Date: 19 October 2021   Issue Date: 17 March 2022
 Cite this article:   
Bo WANG,Zitong KANG,Pengwei DONG, et al. Underwater image enhancement by maximum-likelihood based adaptive color correction and robust scattering removal[J]. Front. Comput. Sci., 2023, 17(2): 172702.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-022-1205-7
https://academic.hep.com.cn/fcs/EN/Y2023/V17/I2/172702
Fig.1  The flowchart of the proposed method
Fig.2  Adaptive color correction for underwater images. (a1) Greenish image. (b1) Bluish image. (c1) Greenish-blue image. (d1) Bluish-green image. (a2), (b2), (c2) and (d2) Color compensation using red, green, and blue channels. (a3), (b3), (c3) and (d3) The results of our proposed adaptive color correction. The corresponding RGB histograms are shown under each result
Fig.3  Underwater imaging model
Fig.4  Background light estimation using results of the proposed adaptive color correction. (a1)?(a4) Original images. (b1)?(b4) The locations of estimated background light (red and yellow dots signify DCP-based and our proposed methods respectively). (c1)?(c4) RGB color of the corresponding red dots. (d1)?(d4) RGB color of the corresponding yellow dots
Fig.5  Background light estimation map. (a1) Original images. (a2)?(a4) The gradient maps of H, S, V component in HSV color space. (b1) Absolute difference map. (b2)?(b4) The modified gradient maps of (a2)?(a4) respectively. The yellow dot denotes the location of estimated background light in bottom estimation map
Fig.6  Enhanced transmission map using the piece-wise affine transform. The original image is shown in (a1). The result of adaptive color correction is shown in (a2). The result of transmission map estimation when using Eq. (14) is shown in (b1). While utilizing our piece-wise affine transform generates higher contrast, as shown in (b2). And the corresponding histograms of transmission map are displayed on the right side
Fig.7  Comparison to classic color constancy, Gao 2019 [44] and Fu 2020 [2] approaches. The original images taken with seven different cameras are shown in the first row. The result of our proposed method (ACC) are shown in the last row
Fig.8  Qualitative comparisons on Berman dataset. The original images taken in different locations with varying water properties contain three different color charts (red rectangle) and the corresponding standard color charts are shown below. From left to right are original images, the results of Drews 2016 [5], Fu 2017 [33], Peng 2017 [26], Peng 2018 [29], Fu 2020 [2], and our proposed method
Method Non-reference metrics
UCIQE↑ Entropy↑
Scene A Scene B Scene C Average Scene A Scene B Scene C Average
Drews 2016 [5] 0.6959 0.5485 0.6691 0.6212 6.7810 4.4696 6.0323 5.7610
Fu 2017 [33] 0.6080 0.4380 0.6599 0.5686 7.2340 5.3133 6.7848 6.4441
Peng 2017 [26] 0.6355 0.4663 0.5772 0.5597 7.3876 5.1223 6.4637 6.3245
Peng 2018 [29] 0.6536 0.3771 0.4791 0.5033 7.3589 4.9163 6.3101 6.1951
Fu 2020 [2] 0.6428 0.5867 0.6542 0.6279 7.1995 6.4094 6.7895 6.7995
Ours 0.6883 0.6016 0.6803 0.6567 7.7000 6.8094 7.1749 7.2281
Tab.1  Results of non-reference metrics in terms of UCIQE and Entropy. The best results are highlighted in red and the second best results are marked in blue
Fig.9  Qualitative comparisons on UIEB dataset (890 images). From left to right are original images, the results of Drews 2016 [5], Fu 2017 [33], Peng 2017 [26], Peng 2018 [29], Gao2019 [44], Fu 2020 [2], our proposed method and reference images
Method Full-reference metrics Non-reference metrics
PSNR↑ SSIM↑ MSE(×103)↓ HCC↑ UCIQE↑ Entropy↑
Drews 2016 [5] 12.4737 0.6173 4.3301 0.1631 0.5896 6.5381
Fu 2017 [33] 19.4887 0.8421 1.1258 0.4277 0.5761 7.2849
Peng 2017 [26] 17.9254 0.7821 1.5937 0.3334 0.5972 7.1622
Peng 2018 [29] 13.4894 0.7306 3.6482 0.1105 0.5947 7.1338
Fu 2020 [2] 21.3105 0.8779 0.6567 0.3577 0.6363 7.3254
Ours 19.9451 0.8466 0.8545 0.4290 0.6454 7.6090
Tab.2  Quantitative evaluation of results on UIEB dataset. The best results are highlighted in red and the second best results are marked in blue
Fig.10  Qualitative comparisons on UIEB dataset (challenging-60 images). From left to right are original images, the results of Drews 2016 [5], Fu 2017 [33], Peng 2017 [26], Peng 2018 [29], Fu 2020 [2], and our proposed method
Fig.11  Qualitative comparisons on high turbid data. From left to right are original images, the results of Drews 2016 [5], Fu 2017 [33], Peng 2017 [26], Peng 2018 [29], Fu 2020 [2], and our proposed method
Fig.12  Average scores of TCWT. From left to right are the results of original images (Std = 0.2822), Drews 2016 [5] (Std = 0.2051), Fu 2017 [33] (Std = 0.2401), Peng 2017 [26] (Std = 0.2978), Peng 2018 [29] (Std = 0.3305), Fu 2020 [2] (Std = 0.2205), our proposed method (Std = 0.2265), and reference (Std = 0.2375)
Method Non-reference metrics
UCIQE↑ Entropy↑
Challcnging-60 High-turbid-30 Average Challcnging-60 High-turbid-30 Average
Drews 2016 [5] 0.5309 0.5049 0.5179 5.8613 5.5584 5.7099
Fu 2017 [33] 0.5445 0.5128 0.5287 6.9774 6.6771 6.8273
Peng 2017 [26] 0.5830 0.5104 0.5467 6.9195 6.5152 6.7174
Peng 2018 [29] 0.5589 0.5129 0.5359 6.8922 6.3590 6.6256
Fu 2020 [2] 0.6028 0.6072 0.6050 6.9457 7.0245 6.9851
Ours 0.6271 0.6170 0.6221 7.3745 7.5897 7.4821
Tab.3  Quantitative evaluation of results on UIEB dataset (challenging-60 images) and high turbid data. The best results are highlighted in red and the second best results are marked in blue
Fig.13  Interest points detection and SIFT feature matching. For the left-side pair ((a1) and (a3), (b1) and (b3)), Harris corner detections only find 8 and 2 interest points that can be correctly matched when using SIFT features based on original image pairs. For the right-side pair ((a2), (a4), (b2), (b4)), there are 80 and 90 interest points found and correctly matched when using Harris corner detection and SIFT feature matching respectively based on dehazing image pairs. (a) The first pair of underwater images; (b) The second pair of underwater images
Fig.14  The visual comparison of salient region detection. Compared with the left-side pair ((a1) and (a3), (b1) and (b3)), salient regions can be more accurately detected using the results of our dehazing method ((a2) and (a4), (b2) and (b4)), where the region is enclosed with red rectangle. (a) The first pair of underwater images; (b) The second pair of underwater images
Fig.15  Failure cases for underwater image dehazing. From left to right are original images, the results of Drews 2016 [5], Fu 2017 [33], Peng 2017 [26], Peng 2018 [29], Fu 2020 [2], and our proposed method
Fig.16  Ablation study of the effect of correct background light estimation in our proposed method
Method Full-reference Metrics Non-reference Metrics
PSNR↑ SSIM↑ UCIQE↑ Entropy↑
w/o Ours 18.9534 0.8109 0.6394 7.4720
Ours 19.9451 0.8466 0.6454 7.6090
Tab.4  Quantitative evaluation of ablation study on UIEB dataset in terms of PSNR, SSIM, UCIQE and Entropy metrics respectively. The best results are highlighted in red
  
  
  
  
  
  
  
  
1 C Li , C Guo , W Ren , R Cong , J Hou , S Kwong , D Tao . An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 2020, 29: 4376– 4389
2 X Fu , X Cao . Underwater image enhancement with global-local networks and compressed-histogram equalization. Signal Processing: Image Communication, 2020, 86: 115892
3 P Jr Drews, Nascimento E do, F Moraes, S Botelho, M Campos. Transmission estimation in underwater single images. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. 2013, 825−830
4 A Galdran , D Pardo , A Picón , A Alvarez-Gila . Automatic red-channel underwater image restoration. Journal of Visual Communication and Image Representation, 2015, 26: 132– 145
5 P L J Drews , E R Nascimento , S S C Botelho , M F M Campos . Underwater depth estimation and image restoration based on single images. IEEE Computer Graphics and Applications, 2016, 36( 2): 24– 35
6 Y T Peng, P C Cosman. Single image restoration using scene ambient light differential. In: Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP). 2016, 1953−1957
7 H Lu , Y Li , X Xu , J Li , Z Liu , X Li , J Yang , S Serikawa . Underwater image enhancement method using weighted guided trigonometric filtering and artificial light correction. Journal of Visual Communication and Image Representation, 2016, 38: 504– 516
8 C Li , J Guo , C Guo , R Cong , J Gong . A hybrid method for underwater image correction. Pattern Recognition Letters, 2017, 94: 62– 67
9 P Zhuang , C Li , J Wu . Bayesian retinex underwater image enhancement. Engineering Applications of Artificial Intelligence, 2021, 101: 104171
10 W Zhang , L Dong , T Zhang , W Xu . Enhancing underwater image via color correction and Bi-interval contrast enhancement. Signal Processing: Image Communication, 2021, 90: 116030
11 L Chen , Z Jiang , L Tong , Z Liu , A Zhao , Q Zhang , J Dong , H Zhou . Perceptual underwater image enhancement with deep learning and physical priors. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31( 8): 3078– 3092
12 C Li , S Anwar , J Hou , R Cong , C Guo , W Ren . Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Transactions on Image Processing, 2021, 30: 4985– 5000
13 H Wen, Y Tian, T Huang, W Gao. Single underwater image enhancement with a new optical model. In: Proceedings of the 2013 IEEE International Symposium on Circuits and Systems (ISCAS). 2013, 753−756
14 S Emberton, L Chittka, A Cavallaro. Hierarchical rank-based veiling light estimation for underwater dehazing. In: Proceedings of the British Machine Vision Conference (BMVC). 2015, 125.1−125.12
15 Y Zhou , Q Wu , K Yan , L Feng , W Xiang . Underwater image restoration using color-line model. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 29( 3): 907– 911
16 K Iqbal, M Odetayo, A James, R A Salam, A Z H Talib. Enhancing the low quality images using unsupervised colour correction method. In: Proceedings of the 2010 IEEE International Conference on Systems, Man and Cybernetics. 2010, 1703−1709
17 A S A Ghani , N A M Isa . Enhancement of low quality underwater image through integrated global and local contrast correction. Applied Soft Computing, 2015, 37: 332– 344
18 A S A Ghani , N A M Isa . Underwater image quality enhancement through integrated color model with Rayleigh distribution. Applied Soft Computing, 2015, 27: 219– 230
19 Y Wang, J Zhang, Y Cao, Z Wang. A deep CNN method for underwater image enhancement. In: Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP). 2017, 1382−1386
20 A Dudhane , P Hambarde , P Patil , S Murala . Deep underwater image restoration and beyond. IEEE Signal Processing Letters, 2020, 27: 675– 679
21 J Islam , Y Xia , J Sattar . Fast underwater image enhancement for improved visual perception. IEEE Robotics and Automation Letters, 2020, 5( 2): 3227– 3234
22 N Carlevaris-Bianco, A Mohan, R M Eustice. Initial results in underwater single image dehazing. In: Proceedings of the Oceans 2010 MTS/IEEE Seattle. 2010, 1−8
23 J Y Chiang , Y C Chen . Underwater image enhancement by wavelength compensation and dehazing. IEEE Transactions on Image Processing, 2012, 21( 4): 1756– 1769
24 H M Lu , Y J Li , L F Zhang , S Serikawa . Contrast enhancement for images in turbid water. Journal of the Optical Society of America A, 2015, 32( 5): 886– 893
25 C Y Li , J C Guo , R M Cong , Y W Pang , B Wang . Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Transactions on Image Processing, 2016, 25( 12): 5664– 5677
26 Y T Peng , P C Cosman . Underwater image restoration based on image blurriness and light absorption. IEEE Transactions on Image Processing, 2017, 26( 4): 1579– 1594
27 D Berman, T Treibitz, S Avidan. Diving into haze-lines: color restoration of underwater images. In: Proceedings of the British Machine Vision Conference (BMVC). 2017, 1−12
28 Y Wang , H Liu , L P Chau . Single underwater image restoration using adaptive attenuation-curve prior. IEEE Transactions on Circuits and Systems I: Regular Papers, 2018, 65( 3): 992– 1002
29 Y T Peng , K Cao , P C Cosman . Generalization of the dark channel prior for single image restoration. IEEE Transactions on Image Processing, 2018, 27( 6): 2856– 2868
30 C Ancuti, C O Ancuti, T Haber, P Bekaert. Enhancing underwater images and videos by fusion. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. 2012, 81−88
31 X Fu, P Zhuang, Y Huang, Y Liao, X P Zhang, X Ding. A Retinex-based enhancing approach for single underwater image. In: Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP). 2014, 4572−4576
32 S Zhang , T Wang , J Dong , H Yu . Underwater image enhancement via extended multi-scale Retinex. Neurocomputing, 2017, 245: 1– 9
33 X Fu, Z Fan, M Ling, Y Huang, X Ding. Two-step approach for single underwater image enhancement. In: Proceedings of the 2017 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS). 2017, 789−794
34 C O Ancuti , C Ancuti , C De Vleeschouwer , P Bekaert . Color balance and fusion for underwater image enhancement. IEEE Transactions on Image Processing, 2018, 27( 1): 379– 393
35 C O Ancuti , C Ancuti , C De Vleeschouwer , M Sbert . Color channel compensation (3C): a fundamental pre-processing step for image enhancement. IEEE Transactions on Image Processing, 2020, 29: 2653– 2665
36 J Li , K A Skinner , R M Eustice , M Johnson-Roberson . WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robotics and Automation Letters, 2018, 3( 1): 387– 394
37 J Y Zhu, T Park, P Isola, A A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 2242−2251
38 C Li , J Guo , C Guo . Emerging from water: underwater image color correction based on weakly supervised color transfer. IEEE Signal Processing Letters, 2018, 25( 3): 323– 327
39 Y Guo , H Li , P Zhuang . Underwater image enhancement using a multiscale dense generative adversarial network. IEEE Journal of Oceanic Engineering, 2020, 45( 3): 862– 870
40 C Li , S Anwar , F Porikli . Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recognition, 2020, 98: 107038
41 D Berman, T Treibitz, S Avidan. Non-local image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1674−1682
42 B Wang , C Li . A visual hierarchical framework based model for underwater image enhancement. Frontiers of Computer Science, 2019, 13( 3): 665– 667
43 B Wang , B Wei , Z Kang , L Hu , C Li . Fast color balance and multi-path fusion for sandstorm image enhancement. Signal, Image and Video Processing, 2021, 15( 3): 637– 644
44 S B Gao , M Zhang , Q Zhao , X S Zhang , Y J Li . Underwater image enhancement using adaptive retinal mechanisms. IEEE Transactions on Image Processing, 2019, 28( 11): 5580– 5595
45 Y Xu , J Wen , L Fei , Z Zhang . Review of video and image defogging algorithms and related studies on image restoration and enhancement. IEEE Access, 2016, 4: 165– 188
https://doi.org/10.1109/ACCESS.2015.2511558
46 M Yang , A Sowmya . An underwater color image quality evaluation metric. IEEE Transactions on Image Processing, 2015, 24( 12): 6062– 6071
47 G D Finlayson, E Trezzi. Shades of gray and colour constancy. In: Proceedings of the Twelfth Color Imaging Conference: Color Science and Engineering Systems, Technologies, and Applications. 2004, 37−41
48 E H Land . The Retinex theory of color vision. Scientific American, 1977, 237( 6): 108– 128
49 G Buchsbaum . A spatial processor model for object colour perception. Journal of the Franklin Institute, 1980, 310( 1): 1– 26
50 J van de Weijer , T Gevers , A Gijsenij . Edge-based color constancy. IEEE Transactions on Image Processing, 2007, 16( 9): 2207– 2214
51 H H Kareem , E G Daway , H G Daway . No reference quality of the hazy images depending on transmission component estimation. IIUM Engineering Journal, 2019, 20( 2): 70– 77
52 C Harris, M Stephens. A combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference. 1988, 147−151
53 D G Lowe . Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60( 2): 91– 110
54 M M Cheng , N J Mitra , X L Huang , P H S Torr , S M Hu . Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37( 3): 569– 582
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed