Data fusion in data scarce areas using a back-propagation artificial neural network model: a case study of the South China Sea

doi:10.1007/s11707-017-0652-1

Front. Earth Sci.

2018, Vol. 12

Issue (2) : 280-298 https://doi.org/10.1007/s11707-017-0652-1

RESEARCH ARTICLE

Data fusion in data scarce areas using a back-propagation artificial neural network model: a case study of the South China Sea

Zheng WANG^1,^2,³, Zhihua MAO^1,^2,³(

), Junshi XIA⁴, Peijun DU^1,²(

), Liangliang SHI^3,⁵, Haiqing HUANG³, Tianyu WANG³, Fang GONG³, Qiankun ZHU³

¹. School of Geographic and Oceanographic Sciences, Nanjing University, Nanjing 210023, China
². Collaborative Innovation Center for the South China Sea Studies, Nanjing University, Nanjing 210023, China
³. States Key Laboratory of Satellite Ocean Environment Dynamics, Second Institute of Oceanography, State Oceanic Administration, Hangzhou 310012, China
⁴. Research Center for Advanced Science and Technology, The University of Tokyo, Tokyo 153-8904, Japan
⁵. Ocean College, Zhejiang University, Hangzhou 310058, China

Download: PDF(5804 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

The cloud cover for the South China Sea and its coastal area is relatively large throughout the year, which limits the potential application of optical remote sensing. A HJ-charge-coupled device (HJ-CCD) has the advantages of wide field, high temporal resolution, and short repeat cycle. However, this instrument suffers from its use of only four relatively low-quality bands which can’t adequately resolve the features of long wavelengths. The Landsat Enhanced Thematic Mapper-plus (ETM+) provides high-quality data, however, the Scan Line Corrector (SLC) stopped working and caused striping of remote sensed images, which dramatically reduced the coverage of the ETM+ data. In order to combine the advantages of the HJ-CCD and Landsat ETM+ data, we adopted a back-propagation artificial neural network (BP-ANN) to fuse these two data types for this study. The results showed that the fused output data not only have the advantage of data intactness for the HJ-CCD, but also have the advantages of the multi-spectral and high radiometric resolution of the ETM+ data. Moreover, the fused data were analyzed qualitatively, quantitatively and from a practical application point of view. Experimental studies indicated that the fused data have a full spatial distribution, multi-spectral bands, high radiometric resolution, a small difference between the observed and fused output data, and a high correlation between the observed and fused data. The excellent performance in its practical application is a further demonstration that the fused data are of high quality.

Keywords data fusion South China Sea BP-ANN model

Corresponding Author(s): Zhihua MAO,Peijun DU

Just Accepted Date: 03 May 2017 Online First Date: 09 June 2017 Issue Date: 09 May 2018

Cite this article:

Zheng WANG,Zhihua MAO,Junshi XIA, et al. Data fusion in data scarce areas using a back-propagation artificial neural network model: a case study of the South China Sea[J]. Front. Earth Sci., 2018, 12(2): 280-298.

URL:

https://academic.hep.com.cn/fesci/EN/10.1007/s11707-017-0652-1
https://academic.hep.com.cn/fesci/EN/Y2018/V12/I2/280

Fig.1 Study area (a), sample area (b), validation area (c), and validation points (d) used in this study.

Fig.2 Details of validation sites. Sites 1 and 2 were different water bodies, sites 3 and 4 were farmland sites on a plain, sites 5 and 6 are forested mountainous areas and sites 7 and 8 were urban areas. All of the 8 validation sites were chosen from Google Earth.

Tab.1 Specifications of Landsat ETM+ and HJ-1A CCD satellite missions

Fig.3 A basic neural network model structure for data fusion between HJ-CCD and Landsat ETM+ satellite.

Fig.4 Flow chart of the overall fusion framework. E_e is the expected training errors. Parameters initialization include: maximum training times, learning accuracy,the number of hidden nodes, initial weights, a threshold value, initial learning rate, etc.

Fig.5 Comparison of bands 1?4 in the observed HJ-CCD data (left column, panels (a), (c), (e) and (g)) and the synthetic ETM+ data simulated by the ANN model (right column, panels (b), (d), (f), (h)).

Fig.6 Comparison between the ANN simulated ETM+ (left column, panels (a), (c), (e), (g), (i) and (k)) and the observed ETM+ data (right column, panels (b), (d), (f), (h), (j) and (l)).

Fig.7 The difference in absolute value between ANN simulated ETM+ and observed ETM+ data for different terrain objects. Panels (a) and (b) were different water bodies, panels (c) and (d) were farmland sites on a plain, panels (e) and (f) are forested mountainous areas and panels (g) and (h) were urban areas.

Fig.8 Scatter plots of observed and simulated ETM+ data. Correlation coefficients in each band are listed at the top left along with the sample size.

Fig.9 Qualitative comparison between results of BP-ANN model and other similar methods. (a) ETM+ SLC-Off image; (b) Nearest neighbor interpolation (NNI); (c) Global linear histogram match (GLHM); (d) Local linear histogram match (LLHM); (e) Back-propagation artificial neural network (BP-ANN); and (f) The high-resolution remote sensing image acquired from the Google Earth software of the same region. The acquisition day of the Google Earth image was on January 15, 2014, which was reasonably close to the ETM+ data acquisition time.

Methods	Sensor1	Sensor2	Coefficient of correlation, R
Data combination	MODIS	Landsat-TM	$0.78 ≪ r ≪ 0.89$
STARFM	MODIS	Landsat-TM	$0.85 ≪ r ≪ 0.91$
STDFM	MODIS	Landsat-ETM+	$0.90 ≪ r ≪ 0.94$
BP-ANN	HJ-CCD	Landsat-ETM+	r=0.9657

Tab.2 Comparison the reflectance with other multi-source remote sensing data fusion methods in the NIR band

Fig.10 Comparison of the vegetation index value (EVI) between the ANN model simulated ETM+ and observed ETM+ data. The abscissa is the EVI value computed from the ANN model simulated ETM+ data, and the ordinate is the EVI value computed from the observed ETM+ data.

Fig.11 Multi-year averages for data from each month precipitation and temperature in the study area. The averages for monthly precipitation and temperature datasets is from year 1981 to year 2010.

Fig.12 NDMI in the validation area. Different colors and NDMI values indicate different canopy moisture levels. Blue color with high NDMI value indicates no agriculture drought. By contrast, the red color with low NDMI value indicates agriculture drought.

Fig.13 Detailed comparison between the water body area extracted by MNDWI and high-resolution false color composite image. Panels (a), (c), (e) and (g) correspond to cases 1–4 extracted by MNDWI ,while panels (b), (d), (f) and (h) are the false color composite images using bands 3–5 of fused data.

1	Al-Sbou Y A (2012). Artificial neural networks evaluation as an image denoising tool. World Appl Sci J, 17(2): 218–227
2	Amici G,Dell'Acqua F,Gamba P,Pulina G (2004). A comparison of fuzzy and neuro-fuzzy data fusion for flooded area mapping using SAR images. Int J Remote Sens, 25(20): 4425–4430 https://doi.org/10.1080/01431160412331269634
3	Benediktsson J A, Swain P H, Ersoy O K (1989). Neural Network Approaches Versus Statistical Methods in Classification of Multisource Remote Sensing Data. In: 12th Canadian Symposium on Remote Sensing Geoscience and Remote Sensing Symposium, 489–492
4	Bernstein L S, Adler-Golden S M, Sundberg R L, Levine R Y, (2005). A new method for atmospheric correction and aerosol optical property retrieval for VIS-SWIR multi- and hyperspectral imaging sensors: QUAC (QUick atmospheric correction). Geoscience and Remote Sensing Symposium, 2005. IGARSS '05. Proceedings. 2005 IEEE International IEEE, 2005:3549–3552 https://doi.org/?10.1109/IGARSS.2005.1526613
5	Bossé É, Roy J, Paradis S (2000). Modeling and simulation in support of the design of a data fusion system. Inf Fusio10.1109/IGARSS. 2005.1526613n, 1(2): 77–87 https://doi.org/10.1016/S1566-2535(00)00016-6
6	Busetto L, Meroni M, Colombo R (2008). Combining medium and coarse spatial resolution satellite data to improve the estimation of sub-pixel NDVI time series. Remote Sens Environ, 112(1): 118–131 https://doi.org/10.1016/j.rse.2007.04.004
7	Chen F, Tang L, Wang C, Qiu Q (2011a). Recovering of the thermal band of Landsat 7 SLC-off ETM+ image using CBERS as auxiliary data. Adv Space Res, 48(6): 1086–1093 https://doi.org/10.1016/j.asr.2011.05.012
8	Chen J, Zhu X, Vogelmann J E, Gao F, Jin S (2011b). A simple and effective method for filling gaps in Landsat ETM+ SLC-off images. Remote Sens Environ, 115(4): 1053–1064 https://doi.org/10.1016/j.rse.2010.12.010
9	Chen Y, Deng L, Li J, Li X, Shi P (2006). A new wavelet‐based image fusion method for remotely sensed data. Int J Remote Sens, 27(7): 1465–1476 https://doi.org/10.1080/01431160500474365
10	Chen Z Y, Desai M, Zhang X P (1997). Feedforward neural networks with multilevel hidden neurons for remotely sensed image classification. In: International Conference on Image Processing, 2: 653–656
11	Daily M I, Farr T, Elachi C, Schaber G (1979). Geologic interpretation from composited radar and Landsat imagery. Photogramm Eng Remote Sensing, 45(8): 1109–1116
12	Ehlers M (1991). Multi sensor image fusion techniques in remote sensing. ISPRS J Photogramm Remote Sens, 46(1): 19–30 https://doi.org/10.1016/0924-2716(91)90003-E
13	Fan J, Zhao D, Wang J (2014). Oil Spill GF-1 Remote Sensing Image Segmentation Using an Evolutionary Feedforward Neural Network. In IEEE International Joint Conference on Neural Networks (IJCNN), 446–450
14	Faouzi N E, Leung H, Kurian A (2011). Data fusion in intelligent transportation systems: progress and challenges – A survey. Inf Fusion, 12(1): 4–10 https://doi.org/10.1016/j.inffus.2010.06.001
15	Farifteh J, Van der Meer F, Atzberger C, Carranza E J M (2007). Quantitative analysis of salt-affected soil reflectance spectra: a comparison of two adaptive methods (PLSR and ANN). Remote Sens Environ, 110(1): 59–78 https://doi.org/10.1016/j.rse.2007.02.005
16	Fiorella M, Ripple W J (1995). Determining successional stage of temperate coniferous forests with landsat satellite data. Photogramm Eng Remote Sensing, 59(2): 239–246
17	Gao B C (1996). NDWI—A normalized difference water index for remote sensing of vegetation liquid water from space. Remote Sens Environ, 58(3): 257–266 https://doi.org/10.1016/S0034-4257(96)00067-3
18	Gigli G, Bossé É, Lampropoulos G A (2007). An optimized architecture for classification combining data fusion and data-mining. Inf Fusion, 8(4): 366–378 https://doi.org/10.1016/j.inffus.2006.02.002
19	Hilker T, Wulder M A, Coops N C, Linke J, McDermid G, Masek J G, Gao F, White J C (2009). A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS. Remote Sens Environ, 113(8): 1613–1627 https://doi.org/10.1016/j.rse.2009.03.007
20	Hu Q, Wu W, Xia T, Yu Q, Yang P, Li Z, Song Q (2013). Exploring the use of Google Earth imagery and object-based methods in land use/cover mapping. Remote Sens, 5(11): 6026–6042 https://doi.org/10.3390/rs5116026
21	Jacobson A, Dhanota J, Godfrey J, Jacobson H, Rossman Z, Stanish A, Walker H, Riggio J (2015). A novel approach to mapping land conversion using Google Earth with an application to East Africa. Environ Model Softw, 72: 1–9 https://doi.org/10.1016/j.envsoft.2015.06.011
22	Karayiannis N B, Purushothaman G (1994). Fuzzy pattern classification using feedforward neural networks with multilevel hidden neurons. Paper presented at the IEEE International Conference on Neural Networks, 1994. IEEE World Congress on Computational Intelligence
23	Khaleghi B, Khamis A, Karray F O, Razavi S N (2013). Multisensor data fusion: a review of the state-of-the-art. Inf Fusion, 14(1): 28–44 https://doi.org/10.1016/j.inffus.2011.08.001
24	Kiema J B K (2002). Texture analysis and data fusion in the extraction of topographic objects from satellite imagery. Int J Remote Sens, 23(4): 767–776 https://doi.org/10.1080/01431160010026005
25	Lee Z, Carder K L (2000). Band-ratio or spectral-curvature algorithms for satellite remote sensing. Appl Opt, 39(24): 4377–4380 https://doi.org/10.1364/AO.39.004377
55	Liu R, Sun J, Wang J, Liao X (2011). Data quality evaluation of Chinese HJ CCD sensor. Advances in Earth Science, 26(9): 971–979 https://doi.org/10.1016/j.inffus.2016.12.001
26	Liu Y, Chen X, Peng H, Wang Z (2017a). Multi-focus image fusion with a deep convolutional neural network. Inf Fusion, 36: 191–207 https://doi.org/10.1016/j.inffus.2016.12.001
27	Liu Z, Blasch E, John V (2017b). Statistical comparison of image fusion algorithms: recommendations. Inf Fusion, 36: 251–260 https://doi.org/10.1016/j.inffus.2016.12.007
28	Maeda E E, Formaggio A R, Shimabukuro Y E, Arcoverde G F B, Hansen M C (2009). Predicting forest fire in the Brazilian Amazon using MODIS imagery and artificial neural networks. Int J Appl Earth Obs Geoinf, 11(4): 265–272 https://doi.org/10.1016/j.jag.2009.03.003
29	Mallick K, Bhattacharya B K, Patel N K (2009). Estimating volumetric surface moisture content for cropped soils using a soil wetness index based on surface temperature and NDVI. Agric Meteorol, 149(8): 1327–1342 https://doi.org/10.1016/j.agrformet.2009.03.004
30	Maxwell S K, Schmidt G L, Storey J C (2007). A multi-scale segmentation approach to filling gaps in Landsat ETM+ SLC-off images. Int J Remote Sens, 28(23): 5339–5356 https://doi.org/10.1080/01431160601034902
31	McFeeters S K (1996). The use of the normalized difference water index (NDWI) in the delineation of open water features. Int J Remote Sens, 17(7): 1425–1432 https://doi.org/10.1080/01431169608948714
32	Mehta A, Parihar A S, Mehta N (2015). Supervised Classification of Dermoscopic Images using Optimized Fuzzy Clustering based Multi-Layer Feed-Forward Neural Network. 2015 International Conference on Computer, Communication and Control (IC4)
33	Mohammdy M, Moradi H R, Zeinivand H, Temme A J A M, Pourghasemi H R, Alizadeh H (2014). Validating gap-filling of Landsat ETM+ satellite images in the Golestan Province, Iran. Arab J Geosci, 7(9): 3633–3638 https://doi.org/10.1007/s12517-013-0967-5
34	Mohan S, Mehta R L (1988). Combined Radar and Landsat data analysis for land use/cover studies over parts of the Punjab plains. J Indian Soc Remote Sens, 16(4): 33–36 https://doi.org/10.1007/BF02991875
35	Muskat J (1983). Geologic interpretations of Seasat-A radar images and Landsat MSS images of a portion of the southern Appalachian Plateau, Virginia, Kentucky, West Virginia. California State University Northridge
36	Nachouki G, Quafafou M (2008). Multi-data source fusion. Inf Fusion, 9(4): 523–537 https://doi.org/10.1016/j.inffus.2007.12.001
37	Nguyen H, Katzfuss M, Cressie N, Braverman A(2014). Spatio-temporal data fusion for very large remote sensing datasets. Technometrics, 56(2): 174–185 https://doi.org/10.1080/00401706.2013.831774
38	Novelli A, Tarantino E, Fratino U, Iacobellis V, Romano G, Gentile F (2016). A data fusion algorithm based on the Kalman filter to estimate leaf area index evolution in durum wheat by using field measurements and MODIS surface reflectance data. Remote Sens Lett, 7(5): 476–484 https://doi.org/10.1080/2150704X.2016.1154219
39	Sharma S C, Rajendran N, Grover A K, Srivastava G S (1990). Interpretation of Synthetic Aperture Radar (SAR) imagery for geological appraisal: a comparative study in Anantapur district of Andhra Pradesh. J Indian Soc Remote Sens, 18(4): 45–64 https://doi.org/10.1007/BF02997072
40	Sims D A, Rahman A F, Cordova V D, Elmasri B, Baldocchi D, Bolstad P, Flanagan L, Goldstein A, Hollinger D, Misson L (2008). A new model of gross primary productivity for North American ecosystems based solely on the enhanced vegetation index and land surface temperature from MODIS. Remote Sens Environ, 112(4): 1633–1646 https://doi.org/10.1016/j.rse.2007.08.004
41	Suliman S I (2016). Locally linear manifold model for gap-filling algorithms of hyperspectral imagery: proposed algorithms and a comparative study. Dissertation for Master Degree. Michigan State University, 1–73
42	Tedesco M, Pulliainen J, Takala M, Hallikainen M, Pampaloni P (2004). Artificial neural network-based techniques for the retrieval of SWE and snow depth from SSM/I data. Remote Sens Environ, 90(1): 76–85 https://doi.org/10.1016/j.rse.2003.12.002
43	Toutin T (1995). Intéegration de données multisources: comparaison de méthodes géométriques et radiométriques. Int J Remote Sens, 16(15): 2795–2811 https://doi.org/10.1080/01431169508954592
44	Turker M, San B T (2003). SPOT HRV data analysis for detecting earthquake-induced changes in Izmit, Turkey. Int J Remote Sens, 24(12): 2439–2450 https://doi.org/10.1080/0143116031000070427
45	Weckenmann A, Jiang X, Sommer K D, Neuschaefer-Rube U, Seewig J, Shaw L, Estler T (2009). Multisensor data fusion in dimensional metrology. CIRP Annals- Manufacturing Technology, 58(2): 701–721
46	Welch R, Ehlers M (1987). Merging multiresolution SPOT HRV and Landsat TM data. Photogramm Eng Remote Sensing, 53: 301–303
47	Wilson E H, Sader S A (2002). Detection of forest harvest type using multiple dates of Landsat TM imagery. Remote Sens Environ, 80(3): 385–396 https://doi.org/10.1016/S0034-4257(01)00318-2
48	Wu M Q, Wang J, Niu Z, Zhao Y Q, Wang C Y (2012). A model for spatial and temporal data fusion. J Infrared Millim W, 31(1): 80–84 https://doi.org/10.3724/SP.J.1010.2012.00080
49	Xu H Q (2005). A study on information extraction of water body with the modified normalized difference water index (MNDWI).J Remot Sens, 9(5): 589‒595 https://doi.org/10.11834/jrs.20050586
50	Zeng C, Shen H, Zhang L (2013). Recovering missing pixels for Landsat ETM+ SLC-off imagery using multi-temporal regression analysis and a regularization method. Remote Sens Environ, 131: 182–194 https://doi.org/10.1016/j.rse.2012.12.012
51	Zervas E, Mpimpoudis A, Anagnostopoulos C, Sekkas O, Hadjiefthymiades S (2011). Multisensor data fusion for fire detection. Inf Fusion, 12(3): 150–159 https://doi.org/10.1016/j.inffus.2009.12.006
52	Zha Y, Gao J, Ni S (2003). Use of normalized difference built-up index in automatically mapping urban areas from TM imagery. Int J Remote Sens, 24(3): 583–594 https://doi.org/10.1080/01431160304987
53	Zhang C, Li W, Travis D (2007). Gaps-fill of SLC-off Landsat ETM+ satellite image using a geostatistical approach. Int J Remote Sens, 28(22): 5103–5122 https://doi.org/10.1080/01431160701250416
54	Zhu X, Liu D, Chen J (2012). A new geostatistical approach for filling gaps in Landsat ETM+ SLC-off images. Remote Sens Environ, 124: 49–60 https://doi.org/10.1016/j.rse.2012.04.019

[1]	Tianjie LEI, Jie FENG, Cuiying ZHENG, Shuguang LI, Yang WANG, Zhitao WU, Jingxuan LU, Guangyuan KAN, Changliang SHAO, Jinsheng JIA, Hui CHENG. Review of drought impacts on carbon cycling in grassland ecosystems[J]. Front. Earth Sci., 2020, 14(2): 462-478.
[2]	Chao FU, Xinghe YU, Xue FAN, Yulin HE, Jinqiang LIANG, Shunli LI. Classification of mass-transport complexes and distribution of gashydrate-bearing sediments in the northeastern continental slope of the South China Sea[J]. Front. Earth Sci., 2020, 14(1): 25-36.
[3]	Yang DING, Zhigang YAO, Lingling ZHOU, Min BAO, Zhengchen ZANG. Numerical modeling of the seasonal circulation in the coastal ocean of the Northern South China Sea[J]. Front. Earth Sci., 2020, 14(1): 90-109.
[4]	Xiaoyin TANG, Shuchun YANG, Shengbiao HU. Thermal-history reconstruction of the Baiyun Sag in the deep-water area of the Pearl River Mouth Basin, northern South China Sea[J]. Front. Earth Sci., 2018, 12(3): 532-544.
[5]	Jiali CHEN, Pengju HU, Xing LI, Yang YANG, Jinming SONG, Xuegang LI, Huamao YUAN, Ning LI, Xiaoxia LÜ. Impact of water depth on the distribution of iGDGTs in the surface sediments from the northern South China Sea: applicability of TEX₈₆ in marginal seas[J]. Front. Earth Sci., 2018, 12(1): 95-107.
[6]	Xiaoyin TANG, Shuchun YANG, Junzhang ZHU, Zulie LONG, Guangzheng JIANG, Shaopeng HUANG, Shengbiao HU. Tectonic subsidence of the Zhu 1 Sub-basin in the Pearl River Mouth Basin, northern South China Sea[J]. Front. Earth Sci., 2017, 11(4): 729-739.
[7]	Shaolei TANG,Xiaofeng YANG,Di DONG,Ziwei LI. Merging daily sea surface temperature data from multiple satellites using a Bayesian maximum entropy method[J]. Front. Earth Sci., 2015, 9(4): 722-731.
[8]	Yong ZHANG, Xin SU, Fang CHEN, Lu JIAO, Hongchen JIANG, Hailiang DONG, Guochun DING. Abundance and diversity of candidate division JS1- and Chloroflexi-related bacteria in cold seep sediments of the northern South China Sea[J]. Front Earth Sci, 2012, 6(4): 373-382.
[9]	YANG Shouye, YIM Wyss W.-S., TANG Min, HUANG Guangqing. Burial of organic carbon and carbonate on inner shelf of the northern South China Sea during the postglacial period[J]. Front. Earth Sci., 2008, 2(4): 427-433.
[10]	SHAO Lei, LI Qianyu, QIAO Peijun, PANG Xiong, CHEN Changmin, SHI Hesheng. Late Oligocene sedimentary environments and provenance abrupt change event in the northern South China Sea[J]. Front. Earth Sci., 2008, 2(2): 138-146.
[11]	ZHANG Yulan, LONG Jiangping. Sporopollen and algae research of core B106 in the northern South China Sea and its paleoenvironmental evolution[J]. Front. Earth Sci., 2008, 2(2): 157-161.
[12]	YANG Wenguang, ZHENG Hongbo, WANG Ke, XIE Xin, CHEN Guocheng, MEI Xi. Sedimentary characteristics of terrigenous debris at site MD05-2905 in the northeastern part of the South China Sea since 36 ka and evolution of the East Asian monsoon[J]. Front. Earth Sci., 2008, 2(2): 170-176.

Viewed

Full text

Abstract

Cited

Shared

Discussed