Multi-sensor image registration by combining local self-similarity matching and mutual information

doi:10.1007/s11707-018-0717-9

Front. Earth Sci.

2018, Vol. 12

Issue (4) : 779-790 https://doi.org/10.1007/s11707-018-0717-9

RESEARCH ARTICLE

Multi-sensor image registration by combining local self-similarity matching and mutual information

Xiaoping LIU¹, Shuli CHEN¹, Li ZHUO¹(

), Jun LI¹, Kangning HUANG²

¹. Guangdong Provincial Key Laboratory of Urbanization and Geo-simulation, School of Geography and Planning, Sun Yat-Sen University, Guangzhou 510275, China
². School of Forestry and Environmental Studies, Yale University, New Haven, CT 06511, USA

Download: PDF(2286 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Automatic multi-sensor image registration is a challenging task in remote sensing. Conventional image registration algorithms may not be applicable when common underlying visual features are not distinct. In this paper, we propose a novel image registration approach that integrates local self-similarity (LSS) and mutual information (MI) for multi-sensor images with rigid/non-rigid radiometric and geometric distortions. LSS is a well-performing descriptor that captures common, local internal layout features for multi-sensor images, whereas MI focuses on global intensity relationships. First, potential control points are identified by using the Harris algorithm and screened based on the self-similarity of their local surrounding internal layouts. Second, a Bayesian probabilistic model for matching the ensemble of the LSS features is introduced. Third, a particle swarm optimization (PSO) algorithm is adopted to optimize the point and region correspondences for maximum self-similarity and MI and, ultimately, a robust mapping function. The proposed approach is compared with several conventional image registration algorithms that are based on the sum of squared differences (SSD), scale-invariant feature transforms (SIFT), and speeded-up robust features (SURF) through the experimental registration of pairs of Landsat TM, SPOT, and RADARSAT SAR images. The results demonstrate that the proposed approach is efficient and accurate.

Keywords automatic registration multi-sensor images local self-similarity mutual information particle swarm optimization

Corresponding Author(s): Li ZHUO

Just Accepted Date: 28 August 2018 Online First Date: 09 October 2018 Issue Date: 20 November 2018

Cite this article:

Xiaoping LIU,Shuli CHEN,Li ZHUO, et al. Multi-sensor image registration by combining local self-similarity matching and mutual information[J]. Front. Earth Sci., 2018, 12(4): 779-790.

URL:

https://academic.hep.com.cn/fesci/EN/10.1007/s11707-018-0717-9
https://academic.hep.com.cn/fesci/EN/Y2018/V12/I4/779

Fig.1 Flowchart of the proposed method.

Fig.2 (a) Corner points that were extracted from a TM image using the Harris algorithm. (b) Remaining corner points after less useful ones were removed.

Fig.3 Bayesian probabilistic model for matching ensembles of descriptors.

Fig.4 Likelihood maps (b, c) for matching two control points (circled in red) in reference image (a).

Fig.5 Multi-sensor images that were used to evaluate the LSS-MI registration method. (a) The TM (band 4398 × 763) image. (b) The SAR (hv) image (335 × 526). CPs that were extracted from remote-sensing images. (c) CPs that were extracted from the TM image. (d) CPs that were extracted from the SAR image.

Fig.6 Evolution of best solution during the run of PSOs.

Fig.7 (a) Landsat 5 band 4 image (30 m). (b) SPOT band 4 image (10 m). (c) Landsat 5 band 4 image (30 m). (d) SPOT Panchromatic band image (2.5 m).

Fig.8 (a) The aligned image. (b) The reference image. (c) Merged image of the aligned image and the reference image.

Fig.9 Comparison between the SURF-based, SSD-based methods and our proposed method. (a) Matching performance based on SURF. (b) Matching performance based on SSD. (c) Matching performance based on integrated LSS and MI.

Tab.1 Performances of SSD, SIFT, SURF, LSS and our proposed algorithms

1	AbdelSayed S, Ionescu D, Goodenough D (1995). Matching and registration method for remote sensing images. In: Proceedings of Geoscience and Remote Sensing Symposium. 2, 1029–1031
2	Arévalo V, González J (2008). Improving piecewise linear registration of high-resolution satellite images through mesh optimization. IEEE Trans Geosci Remote Sens, 46(11): 3792–3803 doi:10.1109/TGRS.2008.924003
3	Atousa T (2011). Local self-similarity as a dense stereo correspondence measure for thermal-visible video registration. In: Proceedings of the 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC, USA
4	Bay H, Ess A, Tuytelaars T, Van Gool L (2008). Speeded-up robust features (SURF). Comput Vis Image Underst, 110(3): 346–359 https://doi.org/10.1016/j.cviu.2007.09.014
5	Belongie S, Malik J, Puzicha J (2002). Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell, 24(4): 509–522 https://doi.org/10.1109/34.993558
6	Bentoutou Y, Taleb N (2005 a). A 3-D space‒time motion detection for an invariant approach image registration approach in digital subtraction angiography. Comput Vis Image Underst, 97(1): 30–50 https://doi.org/10.1016/j.cviu.2004.07.002
7	Bentoutou Y, Taleb N (2005 b). Automatic extraction of control points for digital subtraction angiography image enhancement. IEEE Trans Nucl Sci, 52(1): 238–246 https://doi.org/10.1109/TNS.2004.843120
8	Bentoutou Y, Taleb N, Chikr El Mezouar M, Taleb M, Jetto L (2002). An invariant approach for image registration in digital subtraction angiography. Pattern Recognit, 35(12): 2853–2865 https://doi.org/10.1016/S0031-3203(02)00016-X
9	Boiman O, Irani M (2007). Detecting irregularities in images and in video. Int J Comput Vis, 74(1): 17–31 https://doi.org/10.1007/s11263-006-0009-9
10	Borzi A, Bisceglie M D, Galdi C, Giangregorio G (2009). Robust registration of satellite images with local distortions. In: Proceedings of 2009 IEEE International Geoscience and Remote Sensing Symposium, 3: III-251–III-254
11	Bouchiha R, Besbes K (2013). Automatic remote-sensing image registration using SURF. International Journal of Computer Theory and Engineering, 5(1): 88–92 https://doi.org/10.7763/IJCTE.2013.V5.653
12	Brook A, Ben-Dor E (2011). Automatic registration of airborne and spaceborne image topology map matching with SURF processor algorithm. Remote Sens, 3(1): 65–82 https://doi.org/10.3390/rs3010065
13	Chen H M, Arora M K, Varshney P K (2003 a). Mutual information-based image registration for remote sensing data. Int J Remote Sens, 24(18): 3701–3706 https://doi.org/10.1080/0143116031000117047
14	Chen H M, Varshney P K, Arora M K (2003 b). Performance of mutual information similarity measure for registration of multitemporal remote sensing images. IEEE Trans Geosci Remote Sens, 41(11): 2445–2454
15	Clerc M, Kennedy J (2002). The particle swarm—explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput, 6(1): 58–73 https://doi.org/10.1109/4235.985692
16	Cole-Rhodes A A, Eastman R D (2011). Gradient descent approaches to image registration. In: Moigne J L, Netanyahu N S, Eastman R D, eds. Image Registration for Remote Sensing. Cambridge: Cambridge University,265–276
17	Cole-Rhodes A, Johnson K L, Moigne J L, Zavorin I (2003). Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient. IEEE Transactions on Image, 12(12): 1495–1511
18	Cole-Rhodes A, Johnson K, Le Moigne J (2012). Multiresolution registration of remote sensing images using stochastic gradient. In: Szu H H, Buss J R, eds. Wavelet and Independent Component Analysis Applications IX. SPIE Proceedings Vol. 4738, doi:10.1117/12.458727
19	Collignon A, Maes F, Delaere D, Vandermeulen D, Suetens P, Marchal G (1995). Automated multimodality image registration based on information theory. Inf Process Med Imaging, 3: 263–274
20	Farah I R, Boulila W, Ettabaâ K S, Solaiman B, Ahmed M B (2008). Interpretation of multisensor remote sensing images: multiapproach fusion of uncertain information. IEEE Trans Geosci Remote Sens, 46(12): 4142–4152 https://doi.org/10.1109/TGRS.2008.2000817
21	Goshtasby A, Stockman G C, Page C V (1986). A region-based approach to digital image registration with subpixel accuracy. IEEE Trans Geosci Remote Sens, GE-24(3): 390–399 https://doi.org/10.1109/TGRS.1986.289597
22	Greenfeld J S (2002). Matching GPS Observation to Location on a Digital Map. In: Proceedings of the 81st Annual Meeting of the Transportation Research Board,(3): 13
23	Harris C, Felsberg M (1988). A combined corner and edge detector. In: Proceedings of Fourth Alvey Vision Conference,147–151
24	Hasan M, Pickering M R , Jia X(2012). Robust automatic registration of multimodal satellite images using CCRE with partial volume interpolation. IEEE Trans Geosci Remote Sens, 50(10): 4050–4061
25	Hoyer P O (2004). Non-negative matrix factorization with sparseness n constraints. J Mach Learn Res, 5: 1457–1469
26	Jiao W (2012). Free Viewpoint Action Recognition based on Self-similarities. In: Proceedings of the 11th International Conference on Signal Processing (ICSP), 2, 1131–1134
27	Ken C (2009). Efficient Retrieval of Deformable Shape Classes using Local Self-Similarities. In: Proceedings of 2009 IEEE 12th International Conference on Computer Vision Workshops, 264–271
28	Kennedy J, Eberhart R C (1995). Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, 4, 1942–1948
29	Kennedy J, Eberhart R C (2001). Swarm Intelligence. San Francisco: Morgan Kaufmann Publisher
30	Kim J, Fessler J A (2004). Intensity-based image registration using robust correlation coefficients. IEEE Trans Med Imaging, 23(11): 1430–1444 https://doi.org/10.1109/TMI.2004.835313
31	Klein L A (2004). Sensor and Data Fusion: A Tool for Information Assessment and Decision Making. Bellingham: SPIE Press,8–10
32	Lee H K, Kim T C (2012). Local self-similarity based backprojection for image upscaling. In: Proceedings of 2012 IEEE International Symposium on Circuits and Systems (ISCAS), 1215–1218
33	Li H, Manjunath B S, Mitra S K (1995). A contour-based approach to multisensor image registration. IEEE Trans Image Process, 4(3): 320–334 https://doi.org/10.1109/83.366480
34	Liang J, Liu X, Huang K, Li X, Wang D, Wang X (2014). Automatic registration of multisensor images using an integrated spatial and mutual information (SMI) metric. IEEE Trans Geosci Remote Sens, 52(1): 603–615 https://doi.org/10.1109/TGRS.2013.2242895
35	Liu S, Du X Y, Zhang J H (2009). Structure extracting and matching based on similarity-pictorial structure model for microscopic images. In: Proceedings of International Conference on Artificial Intelligence, 3: 181–185
36	Lowe D G (2004). Distinctive image features from scale-invariant key points. Int J Comput Vis, 60(2): 91–110 https://doi.org/10.1023/B:VISI.0000029664.99615.94
37	Meskine F, Mezouar M C E, Taleb N (2010). A rigid image registration based on the non subsampled contourlet transform and genetic algorithms. Sensors (Basel), 10(9): 8553–8571 https://doi.org/10.3390/s100908553
38	Messerschmidt L, Engelbrecht A P (2004). Learning to play games using a PSO-based competitive learning approach. IEEE Trans Evol Comput, 8(3): 280–288 https://doi.org/10.1109/TEVC.2004.826070
39	Pratt W K (1974). Correlation techniques of image registration. IEEE Trans Aerosp Electron Syst, AES-10(3): 353–358 https://doi.org/10.1109/TAES.1974.307828
40	Ricardo G (2012). Landmark localisation in brain MR images using feature point descriptors based on 3D local self-similarities. In: Proceedings of the 9th IEEE International Symposium on Biomedical Imaging,1535–1538
41	Richards J A, Jia X (2006). Remote Sensing Digital Image Analysis (4th ed). Berlin: Springer-Verlag,56–58
42	Sedaghat A, Ebadi H (2015). Distinctive order based self-similarity descriptor for multi-sensor remote sensing image matching. ISPRS J Photogramm Remote Sens, 108: 62–71 https://doi.org/10.1016/j.isprsjprs.2015.06.003
43	Shechtman E, Irani M (2007). Matching local self-similarities across images and videos. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,1–8
44	Suri S, Reinartz P (2010). Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas. IEEE Trans Geosci Remote Sens, 48(2): 939–949 https://doi.org/10.1109/TGRS.2009.2034842
45	Taleb N, Bentoutou Y, Deforges O, Taleb A (2001). A 3-D space-time motion evaluation for image registration in digital subtraction angiography. Comput Med Imaging Graph, 25(3): 223–233 https://doi.org/10.1016/S0895-6111(00)00054-9
46	Viola P, Wells W M III (1997). Alignment by maximization of mutual information. Int J Comput Vis, 24(2): 137–154 https://doi.org/10.1023/A:1007958904918
47	Wachowiak M P, Smolikova R, Zheng Y, Zurada J M, Elmaghraby A S (2004). An approach to multimodal biomedical image registration utilizing particle swarm optimization. IEEE Trans Evol Comput, 8(3): 289–301 https://doi.org/10.1109/TEVC.2004.826068
48	Wolberg G, Zokai S (2000). Robust image registration using log-polar transform. In: Proceedings of IEEE International Conference on Image Processing, 1: 493–496
49	Wong A, Clausi D A (2007). ARRSI: automatic registration of remote sensing images. IEEE Trans Geosci Remote Sens, 45(5): 1483–1493 https://doi.org/10.1109/TGRS.2007.892601
50	Yang H, Hou X (2012). Local self-similarity based texture classification. In: Proceedings of the 5th International Congress on Image and Signal Processing (CISP),795–799
51	Yi Z, Chen Z, Yang X (2008). Multi-spectral remote image registration based on SIFT. Electron Lett, 44(2): 107–108 https://doi.org/10.1049/el:20082477
52	Zhang H G, Bai X, Zheng H X, Zhao H J, Zhou J, Cheng J, Lu H (2013). Hierarchical remote sensing image analysis via graph laplacian energy. IEEE Geosci Remote Sens Lett, 10(2): 396–400 https://doi.org/10.1109/LGRS.2012.2207087
53	Zheng H (2011). A novel approach for satellite image classification using local self-similarity. In: Proceedings of Geoscience and Remote Sensing Symposium (IGARSS), 2011 IEEE International,2888‒2891
54	Zitová B, Flusser J (2003). Image registration methods: a survey. Image Vis Comput, 21(11): 977–1000 https://doi.org/10.1016/S0262-8856(03)00137-9

[1]	Parham PAHLAVANI, Behnaz BIGDELI. A mutual information-Dempster-Shafer based decision ensemble system for land cover classification of hyperspectral data[J]. Front. Earth Sci., 2017, 11(4): 774-783.

Viewed

Full text

Abstract

Cited

Shared

Discussed