A novel spatio-temporal trajectory data-driven development approach for autonomous vehicles

doi:10.1007/s11707-021-0938-1

Front. Earth Sci.

2021, Vol. 15

Issue (3) : 620-630 https://doi.org/10.1007/s11707-021-0938-1

RESEARCH ARTICLE

A novel spatio-temporal trajectory data-driven development approach for autonomous vehicles

Menghan ZHANG^1,^2,³, Mingjun MA^1,^2,³, Jingying ZHANG^1,^2,³, Mingzhuo ZHANG^1,^2,³, Bo LI^1,^2,³, Dehui DU^1,^2,³(

)

¹. School of Software and Engineering, East China Normal University, Shanghai 200062, China
². Shanghai Key Laboratory of Trustworthy Computing, East China Normal University, Shanghai 200062, China
³. MOE International Joint Lab of Trustworthy Software, East China Normal University, Shanghai 200062, China

Download: PDF(1688 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Nowadays, autonomous driving has been attracted widespread attention from academia and industry. As we all know, deep learning is effective and essential for the development of AI components of Autonomous Vehicles (AVs). However, it is challenging to adopt multi-source heterogenous data in deep learning. Therefore, we propose a novel data-driven approach for the delivery of high-quality Spatio-Temporal Trajectory Data (STTD) to AVs, which can be deployed to assist the development of AI components with deep learning. The novelty of our work is that the meta-model of STTD is constructed based on the domain knowledge of autonomous driving. Our approach, including collection, preprocessing, storage and modeling of STTD as well as the training of AI components, helps to process and utilize huge amount of STTD efficiently. To further demonstrate the usability of our approach, a case study of vehicle behavior prediction using Long Short-Term Memory (LSTM) networks is discussed. Experimental results show that our approach facilitates the training process of AI components with the STTD.

Keywords spatio-temporal trajectory data data meta-modeling domain knowledge LSTM vehicle behavior prediction AI component

Corresponding Author(s): Dehui DU

Online First Date: 23 December 2021 Issue Date: 17 January 2022

Cite this article:

Menghan ZHANG,Mingjun MA,Jingying ZHANG, et al. A novel spatio-temporal trajectory data-driven development approach for autonomous vehicles[J]. Front. Earth Sci., 2021, 15(3): 620-630.

URL:

https://academic.hep.com.cn/fesci/EN/10.1007/s11707-021-0938-1
https://academic.hep.com.cn/fesci/EN/Y2021/V15/I3/620

Fig.1 Framework for autonomous driving system driven by STTD.

Fig.2 Spatio-Temporal Trajectory Data meta-model for autonomous driving.

Fig.3 Traffic participant include pedestrians, animals, bicycles, vehicles. Attributes include ID, location, distance, size, speed, driving direction, and legality of behavior.

Fig.4 Passengers are divided into drivers and non-drivers. For non-drivers, it is necessary to consider whether their behavior will affect the driver.

Fig.5 Road types include straight roads, curved roads, junctions, roundabouts, etc. Attributes include ID, road type, road surface condition, lanes, landscaping.

Fig.6 Unknown obstacles are objects that cannot be anticipated, such as rubbish bins, branches, and so on.

Fig.7 Weather data include Sunny, Cloudy, Rainy, Snowy. Attributes include ID, weather type, temperature, humidity.

Fig.8 Traffic infrastructure includes traffic signs, traffic signals, etc.

Fig.9 Spatio-Temporal Trajectory Data preprocess for autonomous driving system.

Fig.10 Vehicle behavior prediction based on STTD.

Sample	Timestamp	Longitude	Latitude	Brightness	Weather	True label
$X 1$	$x 1$	−51.0548	15.16971	Strong light	Sunny	Turn left
	$x 2$	−51.0548	15.16938	Strong light	Sunny
	⋮	⋮	⋮	⋮	⋮
	$x 40$	−36.61	−2.90651	Strong light	Sunny
	⋮	⋮	⋮	⋮	⋮	⋮
$X n$	$x 1$	−67.8006	−6.21437	Sufficient light	Cloudy	Turn right
	$x 2$	−67.8002	−6.21437	Sufficient light	Cloudy
	⋮	⋮	⋮	⋮	⋮
	$x 40$	−53.5013	−19.7462	Sufficient light	Cloudy

Tab.1 Simulation data set based on STTD meta-model

Fig.11 LSTM for training vehicle behavior prediction module.

Tab.2 Hyperparameters of LSTM for vehicle behavior prediction module

Fig.12 Model accuracy and model loss of the training set and the testing set.

Tab.3 Result of LSTM model based on STTD data set

1	X Bai, C Xu, Y Ao, B Chen, D Du (2019). Learning-based Probabilistic Modeling and Verifying Driver Behavior using MDP. In: 2019 International Symposium on Theoretical Aspects of Software Engineering (TASE). IEEE, 152–159
2	K Czarnecki (2018). Automated driving system (ads) task analysis—part 2: structured road maneuvers. Waterloo Intelligent Systems Engineering Lab (WISE) Report, University of Waterloo
3	F Emmert-Streib, Z Yang, H Feng, S Tripathi, M Dehmer, (2020)). An introductory review of deep learning for prediction models with big data. Front Artificial Intell, 3: 4–88
4	G de Vries, M van Someren (2010). Clustering vessel trajectories with alignment kernels under trajectory compression. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin: Springer. 2010: 296–311
5	A Dosovitskiy, G Ros, F Codevilla, A Lopez, V Koltun (2017). CARLA: an open urban driving simulator. In: Conference on robot learning. PMLR, 2017: 1–16
6	D J Fremont, E Kim, T Dreossi, S Ghosh, X Yue, A L Sangiovanni-Vincentelli, S A Seshia (2019). Scenic: a language for scenario specification and scene generation. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, 63–78
7	S F Fung, A Zheng (2018). Effects of data binning techniques on results of analyzing solar wind and geomagnetic indices data. In: AGU Fall Meeting Abstracts, 2018: SM31D–3525
8	J D Li, K W Cheng, S H Wang, F Morstatter, R P Trevino, J L Tang, H Liu (2017). Feature selection: a data perspective. ACM Comput Surv 50(6): 94
9	N Meratnia, R A de By (2004). Spatiotemporal compression techniques for moving point objects. In: Bertino E, Christodoulakis S, Plexousakis, Chriltophides V, Koubarakis M, Böhm K, Ferrari E, eds. Advances in Database Technology-EDBT 2004. Conference Proceedings EDBT 2004. Berlin: Springer
10	O Pannekoucke, P Cébron, N Oger, A Arbogast (2016). From the Kalman filter to the particle filter: a geometrical perspective of the curse of dimensionality. Adv Meteor, 2016: 9372786
11	J P Paardekooper, M Comi, C Grappiolo, R Snijders, W van Vught, R Beekelaar (2021). A hybrid-AI approach for competence assessment of automated driving functions. In: CEUR Workshop Proceedings. CEUR-WS, 2808(2808)
12	S H Park, B D Kim, C M Kang, C C Chung, J W Choi (2018). Sequence-to-sequence prediction of vehicle trajectory via LSTM encoder-decoder architecture. In: 2018 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2018: 1672–1678
13	S A Seshia, A Desai, T Dreossi, D J Fremont, S Ghosh, E Kim, S Shivakumar, M Vazquez-Chanlatte, X Y Yue (2018). Formal specification for deep neural networks. In: International Symposium on Automated Technology for Verification and Analysis. Cham: Springer, 2018: 20–34
14	Y Wang (2020). Modeling, simulation and verification of autonomous driving scenario based on model-driven. Dissertation for the Master’s Degree. Shanghai: East China Normal University (in Chinese)
15	K Xu, X Xiao, J Miao, Q Luo (2020). Data driven prediction architecture for autonomous driving and its application on apollo platform. In: 2020 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2020: 175–181
16	L Zhang, C X Zhang, D H Du, B Liu, B Tian, Q Yuan (2019). Spatial information spatial metadata construction method. Shanghai: CN110532340A, 2019–12–03
17	M H Zhang, D H Du, M Z Zhang, L Zhang, Y Wang, W T Zhou (2021). Spatio-temporal trajectory data-driven autonomous driving scenario meta-modeling approach. J Softw, 32(4): 973–987 (in Chinese)
18	Q Zhang, Y Wang, X Zhang, L Liu, X Wu, W Shi (2018). OpenVDAP: an open vehicular data analytics platform for CAVs. In: 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS). IEEE, 2018: 1310–1320
19	L Zhao, R Ichise, S Mita, Y Sasaki (2015a). Core ontologies for safe autonomous driving. In: International Semantic Web Conference
20	L Zhao, R Ichise, T Yoshikawa, T Naito, T Kakinami, Y Sasaki (2015b). Ontology-based decision making on uncontrolled intersections and narrow roads. In: 2015 IEEE intelligent vehicles symposium (IV). IEEE, 2015: 83–88

Viewed

Full text

Abstract

Cited

Shared

Discussed