Turbidity-adaptive underwater image enhancement method using image fusion

doi:10.1007/s11465-021-0669-8

Front. Mech. Eng.

2022, Vol. 17

Issue (3) : 13 https://doi.org/10.1007/s11465-021-0669-8

RESEARCH ARTICLE

Turbidity-adaptive underwater image enhancement method using image fusion

Bin HAN¹, Hao WANG¹, Xin LUO¹(

), Chengyuan LIANG¹, Xin YANG², Shuang LIU¹, Yicheng LIN¹

¹. School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China
². Guangdong Intelligent Robotics Institute, Dongguan 523808, China

Download: PDF(9137 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Clear, correct imaging is a prerequisite for underwater operations. In real freshwater environment including rivers and lakes, the water bodies are usually turbid and dynamic, which brings extra troubles to quality of imaging due to color deviation and suspended particulate. Most of the existing underwater imaging methods focus on relatively clear underwater environment, it is uncertain that if those methods can work well in turbid and dynamic underwater environments. In this paper, we propose a turbidity-adaptive underwater image enhancement method. To deal with attenuation and scattering of varying degree, the turbidity is detected by the histogram of images. Based on the detection result, different image enhancement strategies are designed to deal with the problem of color deviation and blurring. The proposed method is verified by an underwater image dataset captured in real underwater environment. The result is evaluated by image metrics including structure similarity index measure, underwater color image quality evaluation metric, and speeded-up robust features. Test results exhibit that the method can correct the color deviation and improve the quality of underwater images.

Keywords turbidity underwater image enhancement image fusion underwater robots visibility

Corresponding Author(s): Xin LUO

Just Accepted Date: 15 March 2022 Issue Date: 04 November 2022

Cite this article:

Bin HAN,Hao WANG,Xin LUO, et al. Turbidity-adaptive underwater image enhancement method using image fusion[J]. Front. Mech. Eng., 2022, 17(3): 13.

URL:

https://academic.hep.com.cn/fme/EN/10.1007/s11465-021-0669-8
https://academic.hep.com.cn/fme/EN/Y2022/V17/I3/13

Fig.1 Images of standard color plate in different underwater environments.

Fig.2 Histogram of underwater images with different turbidity.

Fig.3 Turbidity detection results of different underwater images.

Fig.4 Comparative results of histograms before and after white balance.

Fig.5 Comparison of histogram equalization, gray world, and our method.

Fig.6 Underwater images and transmission maps with different methods.

Fig.7 Comparative results of original images and sharpened images.

Fig.8 Three weight maps used in image fusion.

Fig.9 Algorithm flowchart of turbidity-adaptive underwater image enhancement method.

Fig.10 Image comparison of other papers and our experiment. The images in the first row are cited from Refs. [5,33,43,44] with permission from Elsevier.

Fig.11 Equipment and parameter setting of our experiments.

Fig.12 Different turbid waters in our experiments: (a) soil waters changing from light to severe, (b) blue waters changing from light to severe, and (c) red waters changing from light to severe.

Fig.13 Images captured at Songshan Lake. (a) Platform and devices, (b) images captured with different objects at Songshan Lake, and (c) images of the shoal of Songshan Lake.

Fig.14 Comparison of different enhancement methods of other papers: gray world [46], contrast limited adaptive histogram equalization (CLAHE) [45], and underwater dark channel prior (UDCP) [35].

Fig.15 Comparison of different enhancement methods in our experiments: gray world [46], CLAHE [45], and UDCP [35].

Fig.16 Comparison of different enhancement methods of real underwater images, namely, gray world [46], CLAHE [45], and UDCP [35]: (a) results of the shoal of Songshan Lake and (b) results of Songshan Lake.

Fig.17 (a) SSIM and (b) UCIQE results of different methods. UCIQE is obtained from all underwater images including images of our experiment, images of other papers, and real underwater images captured from a lake. SSIM is taken from all underwater images of our experiments. The original images of clean water are selected as reference of other processed images.

Fig.18 SURF results of original images and our method.

A	Ambient light
A^c	Ambient light of color channel c
B	Blurred image
$B i$ (i = 0,1,2,3)	$B 0$ is the image without filtering. $B 1$ , $B 2$ and $B 3$ are blurred images filtered by $G 1$ , $G 2$ , and $G 3$ , respectively
$B ∞ (x) c$	Value of infinite pixel $x$ of color channels $c$ of ambient light image
c	Color channel (R, G, B) of image
$c 1$ , $c 2$ , $c 3$	$c 1$ , $c 2$ , and $c 3$ are the weight coefficients set as $c 1 = 0.4680$ , $c 2 = 0.2745$ , and $c 3 = 0.2576$
$C i 1$ , $C i 2$	Two other channels in addition to attenuation channel
$C i 1 ¯$ , $C i 2 ¯$	Average of $C i 1$ and $C i 2$ , respectively
$C j ¯$	Mean of attenuation channel
$C j c$	Compensation channel
$C v a r$	Standard deviation of chroma
$d (x)$	Object distance of pixel $x$
$D e v i a t i o n c$	Deviation of color channel $c$
$D e v i a t i o n level$	Color deviation level
$D e v i a t i o n max$	Maximum deviation of image
$E UCIQE$	Value of UCIQE of image
$F (x)$	Fusion image
$G$	Gaussian differential filter
G_i (i = 1,2,3)	Gaussian differential filter with different filter ratio
$I$	Image to be sharpened
$I (x)$	Pixel value at $x$ of image $I$
$I c (x)$	Pixel value at $x$ of color channel $c$ of degraded image
$I k (x)$	Number k input image
$I sharp$	Sharpened image
$I v a r$	Variance of channels
$J$	Undegraded image
$J c (x)$	Pixel value at $x$ of color channel $c$ of original image
$J dark$	Dark channel images
K	Amount of input images
$L$ , $A$ , $B$	Channels of Lab color space
L₁, L₂	Thresholds for color deviation level detecting, and they are set as 40 and 60, respectively
$L ¯$ , $A ¯$ , $B ¯$	Means of channels of Lab color space
$L con$	Contrast of luminance
m	Constant parameter for compensation, it is set as 0.18
$M e a n c$	Mean of color channel $c$
$M e a n sum$	Mean of all color channels of image
n	Constant parameter for compensation, it is set as 0.15
N	Linear normalization operator, also named histogram stretching in the literature
S	Sharpened image in normalized unsharp masking method
$S aver$	Average of saturation
$T (x)$	Pixel value at $x$ of transmission map
T₁, T₂	Variance thresholds for turbidity level detecting, and they are set as 9 and 28, respectively
$T c (x)$	Pixel value at $x$ of color channel $c$ of transmission map
$T u r b i d i t y level$	Turbidity level of image
$w$	Compensation level
$W k$	Normalized weight map
W_L	Laplacian weight map
W_S	Saliency weight map
W_T	Saturation weight map
x	Localization of pixel
y	Pixel localization of region $Ω (x)$
$α$	Attenuation coefficient of water
$β$	Special parament in normalized unsharp masking method
$η$	Parameter to adjust the dehaze level
δ	A small regularization term that ensures that each input contributes to the output, and it is set as 0.1
$Ω (x)$	Local region centered at pixel $x$
$σ$	Filter ratio

Table A1 Underwater image evaluation based on UCIQE

Table A2 Average of UCIQE of Table A1

Fig.A1 Transmission maps of guide filter based on different ratios.

Fig.A2 Intermediate results of images 1–10.

Fig.A3 Intermediate results of images 11–19.

1	M Boudhane, B Nsiri. Underwater image processing method for fish localization and detection in submarine environment. Journal of Visual Communication and Image Representation, 2016, 39: 226–238 https://doi.org/10.1016/j.jvcir.2016.05.017
2	Y H Lin, C M Yu, C Y Wu. Towards the design and implementation of an image-based navigation system of an autonomous underwater vehicle combining a color recognition technique and a fuzzy logic controller. Sensors, 2021, 21(12): 4053 https://doi.org/10.3390/s21124053
3	J G Liu, Y C Wang, B Li, S G Ma. Current research, key performances and future development of search and rescue robots. Frontiers of Mechanical Engineering, 2007, 2(4): 404–416 https://doi.org/10.1007/s11465-007-0070-2
4	T C Li, J L Wang, K N Yao. Visibility enhancement of underwater images based on active polarized illumination and average filtering technology. Alexandria Engineering Journal, 2022, 61(1): 701–708 https://doi.org/10.1016/j.aej.2021.06.007
5	P X Zhuang, C Y Li, J M Wu. Bayesian retinex underwater image enhancement. Engineering Applications of Artificial Intelligence, 2021, 101: 104171 https://doi.org/10.1016/j.engappai.2021.104171
6	Z Liang, Y F Wang, X Y Ding, Z T Mi, X P Fu. Single underwater image enhancement by attenuation map guided color correction and detail preserved dehazing. Neurocomputing, 2021, 425: 160–172 https://doi.org/10.1016/j.neucom.2020.03.091
7	C Y Li, S Anwar, J H Hou, R M Cong, C L Guo, W Q Ren. Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Transactions on Image Processing, 2021, 30: 4985–5000 https://doi.org/10.1109/TIP.2021.3076367
8	C Ancuti, C O Ancuti, T Haber, P Bekaert. Enhancing underwater images and videos by fusion. In: Proceedings of 2012 IEEE CVPR Conference. 2012, 81–88
9	C O Ancuti, C Ancuti, C De Vleeschouwer, P Bekaert. Color balance and fusion for underwater image enhancement. IEEE Transactions on Image Processing, 2018, 27(1): 379–393 https://doi.org/10.1109/TIP.2017.2759252
10	Y Wang, J Zhang, Y Cao, Z F Wang. A deep CNN method for underwater image enhancement. In: Proceedings of 2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017, 1382–1386 https://doi.org/10.1109/ICIP.2017.8296508
11	Y C Guo, H Y Li, P X Zhuang. Underwater image enhancement using a multiscale dense generative adversarial network. IEEE Journal of Oceanic Engineering, 2020, 45(3): 862–870 https://doi.org/10.1109/JOE.2019.2911447
12	R S Liu, X Fan, M Zhu, M J Hou, Z X Luo. Real-world underwater enhancement: challenges, benchmarks, and solutions under natural light. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(12): 4861–4875 https://doi.org/10.1109/TCSVT.2019.2963772
13	X Y Fu, X Y Cao. Underwater image enhancement with global–local networks and compressed-histogram equalization. Signal Processing: Image Communication, 2020, 86: 115892 https://doi.org/10.1016/j.image.2020.115892
14	M Yang, K Hu, Y X Du, Z Q Wei, Z B Sheng, J T Hu. Underwater image enhancement based on conditional generative adversarial network. Signal Processing: Image Communication, 2020, 81: 115723 https://doi.org/10.1016/j.image.2019.115723
15	Q Jiang, Y F Zhang, F X Bao, X Y Zhao, C M Zhang, P D Liu. Two-step domain adaptation for underwater image enhancement. Pattern Recognition, 2022, 122: 108324 https://doi.org/10.1016/j.patcog.2021.108324
16	C Y Li, C L Guo, W Q Ren, R M Cong, J H Hou, S Kwong, D C Tao. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 2020, 29: 4376–4389 https://doi.org/10.1109/TIP.2019.2955241
17	Y Q Liu, Y Y Chen, X M Fang. A review of turbidity detection based on computer vision. IEEE Access: Practical Innovations, Open Solutions, 2018, 6: 60586–60604 https://doi.org/10.1109/ACCESS.2018.2875071
18	S S Chen, L S Han, X Z Chen, D Li, L Sun, Y Li. Estimating wide range total suspended solids concentrations from MODIS 250-m imageries: an improved method. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 99: 58–69 https://doi.org/10.1016/j.isprsjprs.2014.10.006
19	X H Zhang. Water quality turbidity detection based on image recognition system design and implementation. In: Proceedings of 2016 the First International Conference on Real Time Intelligent Systems. 2018, 613: 63–70
20	M O’Byrne, F Schoefs, V Pakrashi, B Ghosh. An underwater lighting and turbidity image repository for analysing the performance of image-based non-destructive techniques. Structure and Infrastructure Engineering, 2018, 14(1): 104–123 https://doi.org/10.1080/15732479.2017.1330890
21	K Xie, W Pan, S X Xu. An underwater image enhancement algorithm for environment recognition and robot navigation. Robotics, 2018, 7(1): 14 https://doi.org/10.3390/robotics7010014
22	H F Hu, Y B Zhang, X B Li, Y Lin, Z Z Cheng, T G Liu. Polarimetric underwater image recovery via deep learning. Optics and Lasers in Engineering, 2020, 133: 106152 https://doi.org/10.1016/j.optlaseng.2020.106152
23	Y B Wang, J Cao, S Rizvi, Q Hao, Y M Fang. Underwater image restoration based on adaptive color compensation and dual transmission estimation. IEEE Access: Practical Innovations, Open Solutions, 2020, 8: 207834–207843 https://doi.org/10.1109/ACCESS.2020.3037362
24	W H Zhang, G Li, Z Q Ying. A new underwater image enhancing method via color correction and illumination adjustment. 2017 IEEE Visual Communications and Image Processing, 2017, 1–4 https://doi.org/10.1109/VCIP.2017.8305027
25	J S Jaffe. Computer modeling and the design of optimal underwater imaging systems. IEEE Journal of Oceanic Engineering, 1990, 15(2): 101–111 https://doi.org/10.1109/48.50695
26	K O Amer, M Elbouz, A Alfalou, C Brosseau, J Hajjami. Enhancing underwater optical imaging by using a low-pass polarization filter. Optics Express, 2019, 27(2): 621–643 https://doi.org/10.1364/OE.27.000621
27	Y J Li, H M Lu, K C Li, H Kim, S Serikawa. Non-uniform de-scattering and de-blurring of underwater images. Mobile Networks and Applications, 2018, 23(2): 352–362 https://doi.org/10.1007/s11036-017-0933-7
28	Y T Peng, P C Cosman. Underwater image restoration based on image blurriness and light absorption. IEEE Transactions on Image Processing, 2017, 26(4): 1579–1594 https://doi.org/10.1109/TIP.2017.2663846
29	J Huang, G X Liu. Multi-color space threshold segmentation and self-learning k-NN algorithm for surge test EUT status identification. Frontiers of Mechanical Engineering, 2016, 11(3): 311–315 https://doi.org/10.1007/s11465-016-0376-z
30	D Q Zhu, Z Q Liu, Y M Zhang. Underwater image enhancement based on colour correction and fusion. IET Image Processing, 2021, 15(11): 2591–2603 https://doi.org/10.1049/ipr2.12247
31	X Yang, H Li, R Chen. Underwater image enhancement with image colorfulness measure. Signal Processing: Image Communication, 2021, 95: 116225 https://doi.org/10.1016/j.image.2021.116225
32	Y Q Wang, X N Yu, D An, Y G Wei. Underwater image enhancement and marine snow removal for fishery based on integrated dual-channel neural network. Computers and Electronics in Agriculture, 2021, 186: 106182 https://doi.org/10.1016/j.compag.2021.106182
33	A Galdran, D Pardo, A Picón, A Alvarez-Gila. Automatic red-channel underwater image restoration. Journal of Visual Communication and Image Representation, 2015, 26: 132–145 https://doi.org/10.1016/j.jvcir.2014.11.006
34	K M He, J Sun, X O Tang. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341–2353 https://doi.org/10.1109/TPAMI.2010.168
35	H F Yu, X B Li, Q Lou, C B Lei, Z X Liu. Underwater image enhancement based on DCP and depth transmission map. Multimedia Tools and Applications, 2020, 79(27–28): 20373–20390
36	M H Zhang, J H Peng. Underwater image restoration based on a new underwater image formation model. IEEE Access: Practical Innovations, Open Solutions, 2018, 6: 58634–58644 https://doi.org/10.1109/ACCESS.2018.2875344
37	K M He, J Sun, X O Tang. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397–1409 https://doi.org/10.1109/TPAMI.2012.213
38	L S Wei, Z X Jiao. Research and application of visual location technology for solder paste printing based on machine vision. Frontiers of Mechanical Engineering, 2009, 4(2): 184–191 https://doi.org/10.1007/s11465-009-0034-9
39	S Lin, K C Chi, W T Li, Y D Tang. Underwater optical image enhancement based on dominant feature image fusion. Acta Photonica Sinica, 2020, 49(3): 310003 https://doi.org/10.3788/gzxb20204903.0310003
40	Y Kim, Y J Koh, C Lee. S Kim, C S Kim. Dark image enhancement based onpairwise target contrast and multi-scale detail boosting. In: Proceedings of 2015 IEEE International Conference on Image Processing. 2015, 1404–1408
41	D H Shen, M Zareapoor, J Yang. Multimodal image fusion based on point-wise mutual information. Image and Vision Computing, 2021, 105: 104047 https://doi.org/10.1016/j.imavis.2020.104047
42	H Bay, T Tuytelaars, L V Goo. Surf: speeded up robust features. In: Proceedings of 2006 the 9th European Conference on Computer Vision. 2006, 3951, 404–417 https://doi.org/10.1007/11744023_32
43	Y F Huang, M Y Liu, F Yuan. Color correction and restoration based on multi-scale recursive network for underwater optical image. Signal Processing: Image Communication, 2021, 93: 116174 https://doi.org/10.1016/j.image.2021.116174
44	Z T Mi, Y Y Li, Y F Wang, X P Fu. Multi-purpose oriented real-world underwater image enhancement. IEEE Access: Practical Innovations, Open Solutions, 2020, 8: 112957–112968 https://doi.org/10.1109/ACCESS.2020.3002883
45	Y K Chang, C Jung, P Ke, H Song, J Hwang. Automatic contrast-limited adaptive histogram equalization with dual gamma correction. IEEE Access: Practical Innovations, Open Solutions, 2018, 6: 11782–11792 https://doi.org/10.1109/ACCESS.2018.2797872
46	G Buchsbaum. A spatial processor model for object colour perception. Journal of the Franklin Institute, 1980, 310(1): 1–26 https://doi.org/10.1016/0016-0032(80)90058-7
47	M Yang, A Sowmya. An underwater color image quality evaluation metric. IEEE Transactions on Image Processing, 2015, 24(12): 6062–6071 https://doi.org/10.1109/TIP.2015.2491020

[1]	BIAN Hongyou, LIU Weijun, ZHAO Jibin. New kind of subarea-parallel scanning mode for laser metal deposition shaping[J]. Front. Mech. Eng., 2007, 2(4): 417-422.

Viewed

Full text

Abstract

Cited

Shared

Discussed