Machine learning-based crop recognition from aerial remote sensing imagery

doi:10.1007/s11707-020-0861-x

Front. Earth Sci.

2021, Vol. 15

Issue (1) : 54-69 https://doi.org/10.1007/s11707-020-0861-x

RESEARCH ARTICLE

Machine learning-based crop recognition from aerial remote sensing imagery

Yanqin TIAN¹, Chenghai YANG²(

), Wenjiang HUANG¹, Jia TANG¹, Xingrong LI¹, Qing ZHANG¹(

)

¹. Key Laboratory of Digital Earth Science, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China.
². USDA-Agricultural Research Service, Aerial Application Technology Research Unit, College Station, TX 77845, USA.

Download: PDF(6620 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Timely and accurate acquisition of crop distribution and planting area information is important for making agricultural planning and management decisions. This study employed aerial imagery as a data source and machine learning as a classification tool to statically and dynamically identify crops over an agricultural cropping area. Comparative analysis of pixel-based and object-based classifications was performed and classification results were further refined based on three types of object features (layer spectral, geometry, and texture). Static recognition using layer spectral features had the highest accuracy of 75.4% in object-based classification, and dynamic recognition had the highest accuracy of 88.0% in object-based classification based on layer spectral and geometry features. Dynamic identification could not only attenuate the effects of variations on planting dates and plant growth conditions on the results, but also amplify the differences between different features. Object-based classification produced better results than pixel-based classification, and the three feature sets (layer spectral alone, layer spectral and geometry, and all three) resulted in only small differences in accuracy in object-based classification. Dynamic recognition combined with object-based classification using layer spectral and geometry features could effectively improve crop classification accuracy with high resolution aerial imagery. The methodologies and results from this study should provide practical guidance for crop identification and other agricultural mapping applications.

Keywords machine learning crop recognition aerial imagery dynamic recognition static recognition

Corresponding Author(s): Chenghai YANG,Qing ZHANG

Online First Date: 30 March 2021 Issue Date: 19 April 2021

Cite this article:

Yanqin TIAN,Chenghai YANG,Wenjiang HUANG, et al. Machine learning-based crop recognition from aerial remote sensing imagery[J]. Front. Earth Sci., 2021, 15(1): 54-69.

URL:

https://academic.hep.com.cn/fesci/EN/10.1007/s11707-020-0861-x
https://academic.hep.com.cn/fesci/EN/Y2021/V15/I1/54

Fig.1 Location of the study area: (a) airborne RGB mosaic image of July 2017, (b) RGB image of study area, (c) calibration tarps, and (d) ground control point (GCP) measurement with GPS.

Fig.2 Airborne images for this study: (a) RGB image of June, (b) Color-infrared (CIR) composite of June, (c) NDVI image of June, (d) RGB image of July, (e) CIR composite of July, and (f) NDVI image of July.

Fig.3 Flowchart of the classification methods.

Fig.4 Global image segmentation results: (a) four-band image segmentation with 449 image objects and (b) six-band image segmentation with 370 image objects.

Fig.5 Image object merged results: (a) four-band image with 277 image objects and (b) six-band image with 227 image objects.

Tab.1 List of object features for classification modeling

Tab.2 Count and percentage by class type for 500 reference points

Fig.6 Feature optimization results of (a) four-band image and (b) six-band image based on three sets of object features: layer spectral features (L), layer spectral features and geometry features (LG), and layer spectral features, geometry features and texture features (LGT).

Tab.3 Best object features for different classification scenarios

Fig.7 Image classification results: 4= four-band image, 6= six-band image, PB= pixel-based classification, OB-L= object-based classification of the layer spectral features, OB-LG= object-based classification of the layer spectral and geometry features, and OB-LGT= object-based classification of the layer spectral, geometry and texture features.

Fig.8 Results of overall kappa and overall accuracy analysis by four classification methods: OKP = Overall Kappa, OA= Overall accuracy: 4= four-band image, 6= six-band image, PB= pixel-based classification, OB_L= object-based classification of the layer spectral features, OB_LG= object-based classification of the layer spectral and geometry features, and OB_LGT= object-based classification of the layer spectral, geometry and texture features.

Tab.4 Accuracy assessment results by class for four-band and six-band images based on four classification methods

Fig.9 Analysis of the importance of classification methods by class: BF= bare soil and fallow, CO= corn, CT= cotton, GR= grass, IM= impervious, SG= sorghum, SB= soybean, AKp1= the average Kappa coefficient of the two data sources for each class under four classification methods, PB= pixel-based classification, OB_L= object-based classification of the layer spectral features, OB_LG= object-based classification of the layer spectral and geometry features, and OB_LGT= object-based classification of the layer spectral, geometry and texture features.

Tab.5 Analysis of the importance of classification methods on the overall scale

Fig.10 Analysis of the importance of dynamic recognition by class: BF= bare soil and fallow, CO= corn, CT= cotton, GR= grass, IM= impervious, SG= sorghum, SB= soybean, and AKp2= the average Kappa coefficient of the four classification methods for each class under the two recognition methods.

1	M E Bauer, J E Cipra (1973). Identification of agricultural crops by computer processing of ERTS MSS data. In: The Proceedings of the Symposium on Significant Results Obtained from the Earth Resources Technology Satellite-1. New Carollton, IN, USA, 3, 205–212
2	U C Benz, P Hofmann, G Willhauck, I Lingenfelder, M Heynen (2004). Multi-resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready information. ISPRS J Photogramm Remote Sens, 58(3–4): 239–258 https://doi.org/10.1016/j.isprsjprs.2003.10.002
3	C Boryan, Z Yang, R Mueller, M Craig (2011). Monitoring US agriculture: the US Department of Agriculture, National Agricultural Statistics Service, Cropland Data Layer Program. Geocarto Int, 26(5): 341–358 https://doi.org/10.1080/10106049.2011.562309
4	L Breiman, J Friedman, C J Stone, R Olshen (1984). Classification and Regression Trees. New York: Wadsworth Inc
5	Y Cai, K Guan, J Peng, S Wang, C Seifert, B Wardlow, Z Li (2018). A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach. Remote Sens Environ, 210: 35–47 https://doi.org/10.1016/j.rse.2018.02.045
6	J Camargo Neto (2004). A combined statistical-soft computing approach for classification and mapping weed species in minimum-tillage systems. Dissertation for the Doctoral Degree. Lincoln: University of Nebraska
7	M S Chubey, S E Franklin, M A Wulder (2006). Object-based analysis of Ikonos-2 imagery for extraction of forest inventory parameters. Photogramm Eng Remote Sensing, 72(4): 383–394 https://doi.org/10.14358/PERS.72.4.383
8	R G Congalton (1991). A review of assessing the accuracy of classifications of remotely sensed data. Remote Sens Environ, 37(1): 35–46 https://doi.org/10.1016/0034-4257(91)90048-B
9	J M Damian, O H de C Pias, M R Cherubin, A Z da Fonseca, E Z Fornari, A L Santi (2020). Applying the NDVI from satellite images in delimiting management zones for annual crops. Sci Agric, 77(1): e20180055 https://doi.org/10.1590/1678-992x-2018-0055
10	D Dimov, F Löw, J H Uhl, S Kenjabaev, O Dubovyk, M Ibrakhimov, C Biradar (2019). Framework for agricultural performance assessment based on MODIS multitemporal data. J Appl Remote Sens, 13(2): 1 https://doi.org/10.1117/1.JRS.13.025501
11	L Drăguţ, O Csillik, C Eisank, D Tiede (2014). Automated parameterisation for multi-scale image segmentation on multiple layers. ISPRS J Photogramm Remote Sens, 88(100): 119–127 https://doi.org/10.1016/j.isprsjprs.2013.11.018 pmid: 24748723
12	eCognition (2019). User Guide.
13	E Farg, M N Ramadan, S M Arafat (2019). Classification of some strategic crops in Egypt using multi-remotely sensing sensors and time series analysis. Egypt J Remote Sens Space Sci, 22(3): 263–270 https://doi.org/10.1016/j.ejrs.2019.07.002
14	G M Foody (2009). Classification accuracy comparison: Hypothesis tests and the use of confidence intervals in evaluations of difference, equivalence and non-inferiority. Remote Sens Environ, 113(8): 1658–1663 https://doi.org/10.1016/j.rse.2009.03.014
15	M D Hossain, D Chen (2019). Segmentation for Object-Based Image Analysis (OBIA): a review of algorithms and challenges from remote sensing perspective. ISPRS J Photogramm Remote Sens, 150: 115–134 https://doi.org/10.1016/j.isprsjprs.2019.02.009
16	Q Hu, W B Wu, Q Song, M Lu, D Chen, Q Y Yu, H J Tang (2017). How do temporal and spectral features matter in crop classification in Heilongjiang Province, China? J Integr Agric, 16(02): 324–336 https://doi.org/10.1016/S2095-3119(15)61321-1
17	A R Huete (1988). A soil-adjusted vegetation index (SAVI). Remote Sens Environ, 25(3): 295–309 https://doi.org/10.1016/0034-4257(88)90106-X
18	G F Hughes (1968). On the mean accuracy of statistical pattern recognizers. IEEE Trans Inf Theory, 14(1): 55–63 https://doi.org/10.1109/TIT.1968.1054102
19	M E Jakubauskas, D R Legates, J H Kastens (2002). Crop identification using harmonic analysis of time-series AVHRR NDVI data. Comput Electron Agric, 37(1–3): 127–139 https://doi.org/10.1016/S0168-1699(02)00116-3
20	J R Jensen (2005). Introductory Digital Image Processing: A Remote Sensing Perspective. New Jersey: Prentice-Hall, Inc
21	C F Jordan (1969). Derivation of leaf-area index from quality of light on the forest floor. Ecological Society of America, 50(4): 663–666 https://doi.org/10.2307/1936256
22	B K Kenduiywo, D Bargiel, U Soergel (2016). Crop type mapping from a sequence of terrasar-X images with dynamic conditional random fields. ISPRS Annals of the Photogrammetry. Remote Sensing and Spatial Information Sciences, 3(7): 59–66
23	J F Knight, R S Lunetta, J Ediriwickrema, S Khorram (2006). Regional scale land cover characterization using MODIS-NDVI 250 m multi-temporal imagery: a phenology-based approach. GIsci Remote Sens, 43(1): 1–23 https://doi.org/10.2747/1548-1603.43.1.1
24	A S Laliberte, D M Browning, A Rango (2012). A comparison of three feature selection methods for object-based classification of sub-decimeter resolution UltraCam-L imagery. Int J Appl Earth Obs Geoinf, 15: 70–78 https://doi.org/10.1016/j.jag.2011.05.011
25	M J Lambert, P C S Traoré, X Blaes, P Baret, P Defourny (2018). Estimating smallholder crops production at village level from Sentinel-2 time series in Mali’s cotton belt. Remote Sens Environ, 216: 647–657 https://doi.org/10.1016/j.rse.2018.06.036
26	E Lichtblau, C J Oswald (2019). Classification of impervious land-use features using object-based image analysis and data fusion. Comput Environ Urban Syst, 75: 103–116 https://doi.org/10.1016/j.compenvurbsys.2019.01.007
27	F Löw, U Michel, S Dech, C Conrad (2013). Impact of feature selection on the accuracy and spatial uncertainty of per-field crop classification using Support Vector Machines. ISPRS J Photogramm Remote Sens, 85: 102–119
28	L Ma, M Li, X Ma, L Cheng, P Du, Y Liu (2017). A review of supervised object-based land-cover image classification. ISPRS J Photogramm Remote Sens, 130: 277–293 https://doi.org/10.1016/j.isprsjprs.2017.06.001
29	I Masialeti, S Egbert, B D Wardlow (2010). A comparative analysis of phenological curves for major crops in Kansas. GIsci Remote Sens, 47(2): 241–259 https://doi.org/10.2747/1548-1603.47.2.241
30	G E Meyer, T Hindman, K Laksmi (1999). Machine vision detection parameters for plant species identification. Proc SPIE, 3543: 327–335 https://doi.org/10.1117/12.336896
31	S Murmu, S Biswas (2015). Application of fuzzy logic and neural network in crop classification: a review. Aquatic Procedia, 4: 1203–1210 https://doi.org/10.1016/j.aqpro.2015.02.153
32	S W Myint, P Gober, A Brazel, S Grossman-Clarke, Q H Weng (2011). Per-pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery. Remote Sens Environ, 115(5): 1145–1161 https://doi.org/10.1016/j.rse.2010.12.017
33	J B Odenweller, K I Johnson (1984). Crop identification using Landsat temporal-spectral profiles. Remote Sens Environ, 14(1–3): 39–54 https://doi.org/10.1016/0034-4257(84)90006-3
34	O Mutanga, T Dube, O Galal(2017). Remote sensing of crop health for food security in Africa: potentials and constraints. Remote Sensing Applications: Society and Environment, 8: 231–239 https://doi.org/10.1016/j.rsase.2017.10.004
35	M Pal (2013). Hybrid genetic algorithm for feature selection with hyperspectral data. Remote Sens Lett, 4(7): 619–628 https://doi.org/10.1080/2150704X.2013.777485
36	M Pal, G M Foody (2010). Feature selection for classification of hyperspectral data by SVM. IEEE Trans Geosci Remote Sens, 48(5): 2297–2307 https://doi.org/10.1109/TGRS.2009.2039484
37	M A Peña, A Brenning (2015). Assessing fruit-tree crop classification from Landsat-8 time series for the Maipo Valley, Chile. Remote Sens Environ, 171: 234–244 https://doi.org/10.1016/j.rse.2015.10.029
38	J A Richards, X Jia (2006). Remote Sensing Digital Image Analysis, 3rd ed. Berlin: Springer-Verlag, 273–274.
39	A J Richardson, J H Everitt (1992). Using spectral vegetation indices to estimate rangeland productivity. Geocarto Int, 7(1): 63–69 https://doi.org/10.1080/10106049209354353
40	J W Rouse Jr, R H Haas, J A Schell, D W Deering (1974). Monitoring vegetation systems in the great plains with ERTS. In: Proceedings of the Third ERTS-1 Symposium NASA, NASA SP-351. Washington: 309–317
41	G Rondeaux, M Steven, F Baret (1996). Optimization of soil-adjusted vegetation indices. Remote Sens Environ, 55(2): 95–107 https://doi.org/10.1016/0034-4257(95)00186-7
42	T Sakamoto, A A Gitelson, A L Nguy-Robertson, T J Arkebauer, B D Wardlow, A E Suyker, S B Verma, M Shibayama (2012). An alternative method using digital cameras for continuous monitoring of crop status. Agric Meteorol, 154–155: 113–126 https://doi.org/10.1016/j.agrformet.2011.10.014
43	Y Shen, J Chen, L Xiao, D Pan (2019). Optimizing multiscale segmentation with local spectral heterogeneity measure for high resolution remote sensing images. ISPRS J Photogramm Remote Sens, 157: 13–25 https://doi.org/10.1016/j.isprsjprs.2019.08.014
44	S Siachalou, G Mallinis, M Tsakiri-Strati (2015). A hidden markov models approach for crop classification: linking crop phenology to time series of multi-sensor remote sensing data. Remote Sens, 7(4): 3633–3650 https://doi.org/10.3390/rs70403633
45	M Sibanda, A Murwira (2012). The use of multi-temporal MODIS images with ground data to distinguish cotton from maize and sorghum fields in smallholder agricultural landscapes of Southern Africa. Int J Remote Sens, 33(16): 4841–4855 https://doi.org/10.1080/01431161.2011.635715
46	H Song, C Yang, J Zhang, W C Hoffmann, D He, J A Thomasson (2016). Comparison of mosaicking techniques for airborne images from consumer-grade cameras. J Appl Remote Sens, 10(1): 016030 https://doi.org/10.1117/1.JRS.10.016030
47	J Torres-Sánchez, J M Peña, A I de Castro, F López-Granados (2014). Multi-temporal mapping of the vegetation fraction in early-season wheat fields using images from UAV. Comput Electron Agric, 103: 104–113 https://doi.org/10.1016/j.compag.2014.02.009
48	v T an Klompenburg, A Kassahun, C Catal (2020). Crop yield prediction using machine learning: A systematic literature review. Comput Electron Agric, 177: 105709 https://doi.org/10.1016/j.compag.2020.105709
49	T G van Niel, T R McVicar (2004). Determining temporal windows for crop discrimination with remote sensing: a case study in south-eastern Australia. Comput Electron Agric, 45(1–3): 91–108 https://doi.org/10.1016/j.compag.2004.06.003
50	G Waldhoff, U Lussem, G Bareth (2017). Multi-Data Approach for remote sensing-based regional crop rotation mapping: a case study for the Rur catchment, Germany. Int J Appl Earth Obs Geoinf, 61: 55–69 https://doi.org/10.1016/j.jag.2017.04.009
51	P Wang, E Fan, P Wang (2021). Comparative analysis of image classification algorithms based on traditional machine learning and deep learning. Pattern Recognit Lett, 141: 61–67 https://doi.org/10.1016/j.patrec.2020.07.042
52	D M Woebbecke, G E Meyer, K Von Bargen, D A Mortensen, D M Woebbecke, G E, Meyer K von Bargen , D A Mortensen (1995). Color indices for weed identification under various soil, residue, and lighting conditions. Trans ASAE, 38(1): 259–269 https://doi.org/10.13031/2013.27838
53	B Wu, J Meng, Q Li, N Yan, X Du, M Zhang (2014). Remote sensing-based global crop monitoring: experiences with China’s CropWatch system. Int J Digit Earth, 7(2): 113–137 https://doi.org/10.1080/17538947.2013.821185
54	M Wu, C Yang, X Song, W C Hoffmann, W Huang, Z Niu, C Wang, W Li, B Yu (2018). Monitoring cotton root rot by synthetic Sentinel-2 NDVI time series using improved spatial and temporal data fusion. Sci Rep, 8(1): 2016 https://doi.org/10.1038/s41598-018-20156-z pmid: 29386526
55	C Yang, J H Everitt, D Murden (2011). Evaluating high resolution SPOT 5 satellite imagery for crop identification. Comput Electron Agric, 75(2): 347–354 https://doi.org/10.1016/j.compag.2010.12.012
56	C Yang, W C Hoffmann (2015). Low-cost single-camera imaging system for aerial applicators. J Appl Remote Sens, 9(1): 096064 https://doi.org/10.1117/1.JRS.9.096064
57	Q Yu, P Gong, N Clinton, G Biging, M Kelly, D Schirokauer (2006). Object-based detailed vegetation classification with airborne high spatial resolution remote sensing imagery. Photogramm Eng Remote Sensing, 72(7): 799–811 https://doi.org/10.14358/PERS.72.7.799
58	X Zhang, P Xiao, X Song, J She (2013). Boundary-constrained multi-scale segmentation method for remote sensing images. ISPRS J Photogramm Remote Sens, 78: 15–25 https://doi.org/10.1016/j.isprsjprs.2013.01.002
59	J Zhang, L Feng, F Yao (2014). Improved maize cultivated area estimation over a large scale combining MODIS–EVI time series data and crop phenological information. ISPRS J Photogramm Remote Sens, 94: 102–113 https://doi.org/10.1016/j.isprsjprs.2014.04.023
60	J Zhang, C Yang, H Song, W Hoffmann, D Zhang, G Zhang (2016). Evaluation of an airborne remote sensing platform consisting of two consumer-grade cameras for crop identification. Remote Sens, 8(3): 257 https://doi.org/10.3390/rs8030257
61	B Zheng, S W Myint, P S Thenkabail, R M Aggarwal (2015). A support vector machine to identify irrigated crop types using time-series Landsat NDVI data. Int J Appl Earth Obs Geoinf, 34: 103–112 https://doi.org/10.1016/j.jag.2014.07.002
62	L Zhong, P Gong, G S Biging (2014). Efficient corn and soybean mapping with temporal extendability: a multi-year experiment using Landsat imagery. Remote Sens Environ, 140: 1–13 https://doi.org/10.1016/j.rse.2013.08.023
63	F Zhou, A Zhang, L Townley-Smith (2013). A data mining approach for evaluation of optimal time-series of MODIS data for land cover mapping at a regional level. ISPRS J Photogramm Remote Sens, 84: 114–129 https://doi.org/10.1016/j.isprsjprs.2013.07.008

[1]	Kevin C. Tse, Hon-Chim Chiu, Man-Yin Tsang, Yiliang Li, Edmund Y. Lam. An unsupervised learning approach to study synchroneity of past events in the South China Sea[J]. Front. Earth Sci., 2019, 13(3): 628-640.
[2]	Kevin C. TSE, Hon-Chim CHIU, Man-Yin TSANG, Yiliang LI, Edmund Y. LAM. Unsupervised learning on scientific ocean drilling datasets from the South China Sea[J]. Front. Earth Sci., 2019, 13(1): 180-190.

Viewed

Full text

Abstract

Cited

Shared

Discussed