A pan-sharpening method based on the ADMM algorithm

doi:10.1007/s11707-019-0754-z

Front. Earth Sci.

2019, Vol. 13

Issue (3) : 656-667 https://doi.org/10.1007/s11707-019-0754-z

RESEARCH ARTICLE

A pan-sharpening method based on the ADMM algorithm

Yingxia CHEN^1,², Tingting WANG¹, Faming FANG¹, Guixu ZHANG¹(

)

¹. Department of Computer Science, East China Normal University, Shanghai 200062, China
². School of Computer Science, Yangtze University, Jingzhou 434023, China

Download: PDF(2112 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Pan-sharpening is a method of integrating low-resolution multispectral images with corresponding high-resolution panchromatic images to obtain multispectral images with high spectral and spatial resolution. A novel variational model for pan-sharpening is proposed in this paper. The model is mainly based on three hypotheses: 1) the pan-sharpened image can be linearly represented by the corresponding panchromatic image; 2) the low-resolution multispectral image is down-sampled from the high-resolution multispectral image through the down-sampling operator; and 3) the satellite image has the low-rank property. Three energy components corresponding to these assumptions are integrated into a variational framework to obtain a total energy function. We adopt the alternating direction method of multipliers (ADMM) to optimize the total energy function. The experimental results show that the proposed method performs better than other mainstream methods in spectral and spatial information preserving aspect.

Keywords pan-sharpening multispectral image panchromatic image variational framework energy function ADMM

Corresponding Author(s): Guixu ZHANG

Just Accepted Date: 24 July 2019 Online First Date: 16 September 2019 Issue Date: 15 October 2019

Cite this article:

Yingxia CHEN,Tingting WANG,Faming FANG, et al. A pan-sharpening method based on the ADMM algorithm[J]. Front. Earth Sci., 2019, 13(3): 656-667.

URL:

https://academic.hep.com.cn/fesci/EN/10.1007/s11707-019-0754-z
https://academic.hep.com.cn/fesci/EN/Y2019/V13/I3/656

Fig.1 The histogram of singular values of the 800 satellite images.

Fig.2 PSNR versus the three parameters (

σ

β

, and

μ

Method	UIQI	ERGAS	SCC	Q⁴	RMSE	RASE	QAVE	CC	PSNR
Wavelet	0.9361	3.3184	0.9195	0.7883	0.0500	13.2271	0.9376	0.9797	26.0147
P+ XS	0.9146	3.8601	0.9188	0.6890	0.0584	15.4288	0.9160	0.8934	24.6774
VWP	0.9451	3.0253	0.9490	0.7834	0.0456	12.0656	0.9463	0.9721	26.8130
AVWP	0.8989	4.1665	0.9131	0.6592	0.0630	16.6468	0.9010	0.8844	24.0174
SIRF	0.9719	2.6590	0.9561	0.8421	0.0400	10.5729	0.9657	0.8961	27.9601
PanNet	0.9114	2.1016	0.8754	0.7397	0.0647	17.0925	0.9121	0.8544	23.7879
Ours	0.9688	2.2133	0.9686	0.8498	0.0332	8.7649	0.9691	0.9955	29.5891
Reference	1	0	1	1	0	0	1	1	$∞$

Tab.1 Quantitative comparison corresponds to Fig. 3

Fig.3 Comparison of pan-sharpened results (source: QuickBird) on reduced-resolution image. (a) Ground truth. (b) PAN image. (c) LMS image. (d)–(j) Pan-sharpened images obtained by Wavelet, P+ XS, VWP, AVWP, SIRF, PanNet, and the proposed method, respectively. The PAN image has 400 × 400 pixels.

Fig.4 Comparison of pan-sharpened results (source: QuickBird) on reduced-resolution image. (a) Ground truth. (b) PAN image. (c) LMS image. (d)–(j) Pan-sharpened images obtained by Wavelet, P+ XS, VWP, AVWP, SIRF, PanNet, and the proposed method, respectively. The PAN image has 400 × 400 pixels.

Fig.5 Comparison of pan-sharpened results (source: QuickBird) on reduced-resolution image. (a) Ground truth. (b) PAN image. (c) LMS image. (d) - (j) Pan-sharpened images obtained by Wavelet, P+ XS, VWP, AVWP, SIRF, PanNet, and the proposed method, respectively. The PAN image has 400 × 400 pixels.

Method	UIQI	ERGAS	SCC	Q⁴	RMSE	RASE	QAVE	CC	PSNR
Wavelet	0.8675	2.2173	0.9323	0.7705	0.0327	8.6919	0.8688	0.9853	29.6993
P+ XS	0.8638	2.4935	0.9173	0.7099	0.0363	9.6459	0.8659	0.9103	28.7948
VWP	0.8769	2.0567	0.9482	0.7658	0.0301	7.9937	0.8793	0.9623	30.4267
AVWP	0.8608	2.5959	0.9127	0.7054	0.0377	9.9983	0.8629	0.9061	28.4831
SIRF	0.8975	3.1881	0.8767	0.7393	0.0471	12.5010	0.8841	0.5545	26.5427
PanNet	0.8254	2.9575	0.8847	0.7572	0.0445	11.8133	0.8290	0.8701	27.0342
Ours	0.9006	1.6749	0.9665	0.8405	0.0249	6.6207	0.9009	0.9954	32.0636
Reference	1	0	1	1	0	0	1	1	$∞$

Tab.2 Quantitative comparison corresponds to Fig. 4

Method	UIQI	ERGAS	SCC	Q⁴	RMSE	RASE	QAVE	CC	PSNR
Wavelet	0.9080	1.9118	0.9311	0.6882	0.0354	7.2914	0.9062	0.9861	29.0304
P+ XS	0.8940	2.0781	0.9302	0.6966	0.0377	7.7795	0.8934	0.9267	28.4676
VWP	0.9068	1.8145	0.9404	0.7050	0.0333	6.8725	0.9060	0.9659	29.5443
AVWP	0.8876	2.1993	0.9220	0.6822	0.0398	8.2038	0.8874	0.9239	28.0063
SIRF	0.9322	2.4184	0.8914	0.6777	0.0460	9.4892	0.9081	0.6496	26.7420
PanNet	0.8606	2.5447	0.8800	0.7004	0.0491	10.1236	0.8623	0.8641	26.1799
Ours	0.9329	1.5262	0.9590	0.7308	0.0286	5.8978	0.9301	0.9954	30.8728
Reference	1	0	1	1	0	0	1	1	$∞$

Tab.3 Quantitative comparison corresponds to Fig. 5

Tab.4 The computational costs comparison (s)

Fig.6 Comparison of pan-sharpened results (source: QuickBird) on full-resolution image. (a) LMS image. (b) PAN image. (c)–(j) Pan-sharpened images obtained by Wavelet, P+ XS, VWP, AVWP, PanNet, SIRF, and the proposed method, respectively. The PAN image has 512 × 512 pixels.

Fig.7 Comparison of pan-sharpened results (source: QuickBird) on full-resolution image. (a) LMS image. (b) PAN image. (c)–(j) Pan-sharpened images obtained by Wavelet, P+ XS, VWP, AVWP, PanNet, SIRF, and the proposed method, respectively. The PAN image has 512 × 512 pixels.

Fig.8 Zoomed-in red-square area. (a) Pan-sharpened image obtained by SIRF in Fig. 6. (b) Pan-sharpened image obtained by our proposed method in Fig. 6. (c) Pan-sharpened image obtained by SIRF in Fig. 7. (d) Pan-sharpened image obtained by our proposed method in Fig. 7.

1	L Alparone, S Baronti, A Garzelli, F Nencini (2004). A global quality measurement of pan-sharpened multispectral imagery. IEEE Geosci Remote Sens Lett, 1(4): 313–317 https://doi.org/10.1109/LGRS.2004.836784
2	L Alparone, L Wald, J Chanussot, C Thomas, P Gamba, L M Bruce (2007). Comparison of pansharpening algorithms: outcome of the 2006 GRS-S data-fusion contest. IEEE Trans Geosci Remote Sens, 45(10): 3012–3021 https://doi.org/10.1109/TGRS.2007.904923
3	B Aiazzi, L Alparone, S Baronti, A Garzelli (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Trans Geosci Remote Sens, 40(10): 2300–2312 https://doi.org/10.1109/TGRS.2002.803623
4	H Aly, J Sharma (2014). A regularized model-based optimization framework for pan-sharpening. IEEE Trans Image Process, 23(6): 2596–2608 https://doi.org/10.1109/TIP.2014.2316641
5	C Ballester, V Caselles, L Igual, J Verdera, B Roug (2006). A variational model for P+XS image fusion. Int J Comput Vis, 69(1): 43–58 https://doi.org/10.1007/s11263-006-6852-x
6	S Boyd, N Parikh, E Chu, B Peleato, J Eckstein (2010). Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn, 3(1): 1–122 https://doi.org/10.1561/2200000016
7	K Bredies, D A Lorenz (2008). Soft-thresholding. J Fourier Anal Appl, 14(5–6): 813–837 https://doi.org/10.1007/s00041-008-9041-1
8	J Cai, E Candes, Z Shen (2010). A singular value thresholding algorithm for matrix completion. SIAM J Optim, 20(4): 1956–1982 https://doi.org/10.1137/080738970
9	E Candes, X Li, Y Ma, J Wright (2011). Robust principal component analysis. J Assoc Comput Mach, 58(3): 1–37 https://doi.org/10.1145/1970392.1970395
10	C Chen, Y Li, W Liu, J Huang (2015). SIRF: simultaneous satellite image registration and fusion in a unified framework. IEEE Transactions on Image Processing, A Publication of the IEEE Signal Processing Society, 24(11): 4213
11	C Chen, Y Li, W Liu, J Huang (2014). Image fusion with local spectral consistency and dynamic gradient sparsity. IEEE Conference on Computer Vision & Pattern Recognition, 2760–2765
12	M Choi (2006). A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter. IEEE Trans Geosci Remote Sens, 44(6): 1672–1682 https://doi.org/10.1109/TGRS.2006.869923
13	X Ding, Y Jiang, Y Huang, J Paisley (2014). Pan-sharpening with a Bayesian nonparametric dictionary learning model. In: Proceeding of 17th International Conference of Artificial Intelligence and Statistics, 176–184
14	W Dong, G Shi, X Li (2013). Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans Image Process, 22(2): 700–711 https://doi.org/10.1109/TIP.2012.2221729
15	F Fang, F Li, C Shen, G Zhang (2013). A variational approach for pan-sharpening. IEEE Trans Image Process, 22(7): 2822–2834 https://doi.org/10.1109/TIP.2013.2258355
16	T Goldstein, B O’Donoghue, S Setzer, R Baraniuk (2014). Fast alternating direction optimization methods. SIAM J Imaging Sci, 7(3): 225–231 https://doi.org/10.1137/120896219
17	C Laben, B Brower (2000). Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. Websterny Uspenfieldny. USA: Eastman Kodak Company, 275–299
18	X Lu, J Zhang (2014). Panchromatic and multispectral images fusion based on modified GS-SWT. Geoscience & Remote Sensing Symposium (IGARSS), 2530–2533
19	G Masi, D Cozzolino, L Verdoliva, G Scarpa (2016). Pansharpening by convolutional neural networks. Remote Sens, 8(7): 594 https://doi.org/10.3390/rs8070594
20	M Metwalli, A Nasr, O Faragallah, E E Rabaie, A Abbas (2014). Efficient pan-sharpening of satellite images with the contourlet transform. Int J Remote Sens, 35(5): 1979–2002 https://doi.org/10.1080/01431161.2013.873832
21	M E Mezouar, N Taleb, K Kpalma, J Ronsin (2011). An IHS-based fusion for color distortion reduction and vegetation enhancement in IKONOS imagery. IEEE Trans Geosci Remote Sens, 49(5): 1590–1602 https://doi.org/10.1109/TGRS.2010.2087029
22	M Moller, T Wittman, A Bertozzi, M Burger (2013). A variational approach for sharpening high dimensional images. SIAM J Imaging Sci, 5(1): 150–178 https://doi.org/10.1137/100810356
23	M Moller, T Wittman, A Bertozzi (2008). Variational wavelet pan-sharpening. IEEE Trans Geosci Remote Sens, 1–9
24	J Park, K Kim, K Yang (2001). Image fusion using multiresolution analysis. IEEE Int Geosci Remote Se, 2(2): 864–866
25	J Pushparaj, A Hegde (2016). Evaluation of pan-sharpening methods for spatial and spectral quality. Appl Geomatics, 9(1): 1–12 https://doi.org/10.1007/s12518-016-0179-2
26	S Rahmani, M Strait, D Merkurjev, M Moeller, T Wittman (2010). An adaptive HIS pan-sharpening method. IEEE Geosci Remote Sens Lett, 7(4): 746–750 https://doi.org/10.1109/LGRS.2010.2046715
27	V Shah, N Younan, R King (2008). An efficient pan-sharpening method via a combined adaptive PCA approach and contourlets. IEEE Trans Geosci Remote Sens, 46(5): 1323–1335 https://doi.org/10.1109/TGRS.2008.916211
28	V Shah, N Younan, R King (2007). An adaptive PCA-based approach to pan-sharpening. Remote Sens, 6748: 1–9
29	A Toet, M Hogervorst (2003). Performance comparison of different gray level image fusion schemes through a universal image quality index. In: Kadar I (Ed.), Signal Processing, Sensor Fusion, and Target Recognition XII, SPIE-5096, 552–561
30	G Vivone, L Alparone, J Chanussot, M Mura, A Garzelli (2015). A critical comparison among pansharpening algorithms. IEEE Trans Geosci Remote Sens, 53(5): 2265–2586 https://doi.org/10.1109/TGRS.2014.2361734
31	Q Wang, D Yu, Y Shen (2009). An overview of image fusion metrics. IEEE Instrumentation & Measurement Technology Conference, Singapore, 918–923
32	S Wang, Z Zhang (2012). Colorization by Matrix Completion. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI'12), AAAI Press, 1169–1175
33	Z Wang, A Bovik (2002). A universal image quality index. IEEE Signal Process Lett, 9(3): 81–84 https://doi.org/10.1109/97.995823
34	Y Wei, Q Yuan, H Shen, L Zhang (2017). Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geosci Remote Sens Lett, 14(10): 1795–1799
35	J Yang, X Fu, Y Hu, Y Huang, X Ding, J Paisley (2017). PanNet: A deep network architecture for pan-sharpening. Proceedings of IEEE International Conference on Computer Vision.,1753–1761
36	D Yocky (1995). Image merging and data fusion by means of the discrete two-dimensional wavelet transform. J Opt Soc Am A Opt Image Sci Vis, 12(9): 1834–1841 https://doi.org/10.1364/JOSAA.12.001834
37	Q Yuan, Y Wei, X Meng, H Shen, L Zhang (2018). A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3): 978–989
38	B Zhang (2010). Study on image fusion based on different fusion rules of wavelet transform. International Conference on Advanced Computer Theory & Engineering, 3(7): 649–653
39	H Zhang, D Roy (2016). Computationally inexpensive Landsat 8 operational land imager (OLI) pansharpening. Remote Sens, 8(3): 180 https://doi.org/10.3390/rs8030180
40	J Zhou, D Civco, J Silander (1998). A wavelet transform method to merge Landsat TM and SPOT panchromatic data. Int J Remote Sens, 19(4): 743–757 https://doi.org/10.1080/014311698215973

Viewed

Full text

Abstract

Cited

Shared

Discussed