Toward autonomous mining: design and development of an unmanned electric shovel via point cloud-based optimal trajectory planning

doi:10.1007/s11465-022-0686-2

Front. Mech. Eng.

2022, Vol. 17

Issue (3) : 30 https://doi.org/10.1007/s11465-022-0686-2

RESEARCH ARTICLE

Toward autonomous mining: design and development of an unmanned electric shovel via point cloud-based optimal trajectory planning

Tianci ZHANG, Tao FU, Yunhao CUI, Xueguan SONG(

)

School of Mechanical Engineering, Dalian University of Technology, Dalian 116024, China

Download: PDF(9251 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

With the proposal of intelligent mines, unmanned mining has become a research hotspot in recent years. In the field of autonomous excavation, environmental perception and excavation trajectory planning are two key issues because they have considerable influences on operation performance. In this study, an unmanned electric shovel (UES) is developed, and key robotization processes consisting of environment modeling and optimal excavation trajectory planning are presented. Initially, the point cloud of the material surface is collected and reconstructed by polynomial response surface (PRS) method. Then, by establishing the dynamical model of the UES, a point to point (PTP) excavation trajectory planning method is developed to improve both the mining efficiency and fill factor and to reduce the energy consumption. Based on optimal trajectory command, the UES performs autonomous excavation. The experimental results show that the proposed surface reconstruction method can accurately represent the material surface. On the basis of reconstructed surface, the PTP trajectory planning method rapidly obtains a reasonable mining trajectory with high fill factor and mining efficiency. Compared with the common excavation trajectory planning approaches, the proposed method tends to be more capable in terms of mining time and energy consumption, ensuring high-performance excavation of the UES in practical mining environment.

Keywords autonomous excavation unmanned electric shovel point cloud excavation trajectory planning

Corresponding Author(s): Xueguan SONG

About author: Tongcan Cui and Yizhe Hou contributed equally to this work.

Just Accepted Date: 28 April 2022 Issue Date: 06 September 2022

Cite this article:

Tianci ZHANG,Tao FU,Yunhao CUI, et al. Toward autonomous mining: design and development of an unmanned electric shovel via point cloud-based optimal trajectory planning[J]. Front. Mech. Eng., 2022, 17(3): 30.

URL:

https://academic.hep.com.cn/fme/EN/10.1007/s11465-022-0686-2
https://academic.hep.com.cn/fme/EN/Y2022/V17/I3/30

Fig.1 Framework of the unmanned electric shovel. GNSS: global navigation satellite system, IMU: inertial measurement unit.

Fig.2 Flowchart of the autonomous excavation.

Fig.3 3D point cloud acquisition device composed of two RS-Ruby Lite 80 laser-beam LiDARs.

Fig.4 (a) Actual material surface and (b) corresponding environmental point cloud.

Fig.5 Excavation trajectory of the unmanned electric shovel.

Fig.6 Generalized coordinates and force analysis in excavation.

Parameters	Value
$L m1$	1.04 m
$L m2$	2.13 m
$r h$	0.18 m
$L h$	2.36 m
$L d$	0.47 m
$H$	1.45 m
$m h$	129.38 kg
$m 0$	364.12 kg
$α b$	45.00°
EF	0.45 m
ED	0.36 m
PF	0.38 m

Tab.1 Values of relevant structural parameters of the UES

Fig.7 Explanatory diagram of unmanned electric shovel’s key structural parameters.

Constraints	Value
$h ε$	0.10 m
$V n$	0.16 m³
$v ha$	0.11 m/s
$F ca$	19.25 kN
$P ca$	3.50 kN
$r a$	2.12 m
$v ra$	0.25 m/s
$F ha$	38.76 kN
$P ha$	8.20 kW

Tab.2 Values of the constraints in trajectory planning

Fig.8 Point cloud of material surface to be reconstructed.

Fig.9 Modeling accuracy and error comparison with different k: (a) polynomial model with k = 2, (b) model absolute residual with k = 2, (c) polynomial model with k = 4, (d) model absolute residual with k = 4, (e) polynomial model with k = 6, (f) model absolute residual with k = 6, (g) polynomial model with k = 8, (h) model absolute residual with k = 8, (i) polynomial model with k = 10, and (j) model absolute residual with k = 10.

Fig.10 Relationship between different k and computational performance.

Constrained boundary	$c y 6$	$c z 6$	$p y (t f)$ /m	$p z (t f)$ /m	$t f$ /s
Lower bound	?5.00	0.00	1.20	1.60	8.00
Upper bound	0.00	5.00	2.25	1.90	16.00

Tab.3 Upper and lower bounds of optimization variables

Constrained boundary	$J 1$ /s	$J 2$	$J 3$ /(kW?h?m^?3)
Lower bound	8.00	0.00	0.00
Upper bound	16.00	2.00	0.03

Tab.4 Upper and lower bounds of sub-objective

Fig.11 Convergence curve of the objective value.

Fig.12 Excavation trajectory and related states: (a) planned excavation trajectory, (b) rope and dipper hand lengths, (c) hoist and crowd velocities, (d) digging forces, and (e) motor powers.

Fig.13 Autonomous excavation process: (a) material scanning, (b) mining initialization, (c) mining process, (d) mining accomplished, (e) rotation process, and (f) unloading process.

Fig.14 Comparison of actual and planned motor speeds: (a) comparison of actual and planned hoist motor speeds, and (b) comparison of actual and planned crowd motor speeds.

Fig.15 Comparison of planned and actual digging forces: (a) comparison of actual and planned hoist forces, and (b) comparison of actual and planned crowd forces.

Method	J	V/m³	t_f/s	E_per/ $(kW ? h ? m ? 3)$
6-D poly	0.6114	0.1452	13.20	0.1061
TVP	0.6581	0.1245	13.70	0.1124
LS	0.6455	0.1441	12.06	0.1058

Tab.5 Performance comparison with three different trajectory planning methods

Fig.16 Comparison of the motor speeds and digging forces: (a) hoist motor speed, (b) crowd motor speed, (c) hoist force provided by hoist motor, (d) crowd force provided by crowd motor. 6-D poly: six-degree polynomial.

Abbreviations
GNSS	Global navigation satellite system
IMU	Inertial measurement unit
LS	Logarithmic spiral
MSoE	Maximum sum of evidence
PID	Proportional integral derivative
PLC	Programmable Logic Controller
PRS	Polynomial response surface
PTP	Point to point
RBF	Radial basis functions
TVP	Trapezoidal velocity profile
UES	Unmanned electric shovel
Variables
a₁, a₂	Acceleration in uniform acceleration and deceleration stages, respectively
a_y	Excavation acceleration in the y direction
a_y1	Acceleration in uniform acceleration stage and uniform deceleration stage in the y direction
a_z	Excavation acceleration in the z direction
a_z1	Acceleration in uniform acceleration stage and uniform deceleration stage in the z direction
c	Polynomial trajectory coefficient
c_y6, c_z6	Six-degree polynomial coefficients in the y andz directions, respectively
C_i	Constraint in trajectory planning
D	Vandermond matrix
D_xy	Projection area in the horizontal direction when the dipper teeth cut the material surface
E_c	Energy consumption of the crowd machinery
E_h	Energy consumption of the hoist machinery
E_per	Energy consumption per volume
f_s	Polynomial function
F_c	Crowd force
F_ca	Maximum allowable value of the crowd force
f_tr(x, y)	Excavation trajectory
$F c max$	Maximum crowd force
$F h$	Hoist force
$F ha$	Maximum allowable value of the hoist force
$F h max$	Maximum hoist force
$F i$	Generalized force
$F n$	Normal excavation resistance
$F t$	Tangential excavation resistance
$g$	Acceleration of gravity
$h b min$	Minimum vertical height of the dipper bottom
$h mf$	Material height corresponding to the final position of the excavation trajectory
$h ε$	Margin height
J	Objective function
$J 1$	Mining efficiency
$J 2$	Mining production
$J 3$	Energy consumption
$J max$	Upper bound
$J min$	Lower bound
$k$	Polynomial order
$L$	Lagrange function
$L d$	Length of the dipper
$L h$	Length of the dipper handle
$L s$	Loss function
$m 0$	Mass of the empty dipper
$m d$	Total mass of the dipper
$m h$	Mass of the dipper handle
$m m$	Mass of the loaded material
$n$	Degree of the polynomial
$N$	Number of points
$p y$	Position of the excavation trajectory in the y direction
$p y (t f)$	Final position of the excavation trajectory in the y direction
$p z$	Position of the excavation trajectory in the z direction
$p z (t f)$	Final position of the excavation trajectory in the z direction
$P$	Point cloud
$P ca$	Maximum allowable value of the crowd power
$P c max$	Maximum crowd power
$P ha$	Maximum allowable value of the hoist power
$P h max$	Maximum hoist power
$q i$	Generalized coordinate
$r$	Stretching length of the dipper handle
$r a$	Maximum allowable value of the stretching length of the dipper handle
$r max$	Maximum stretching length of the dipper handle
$r ˙ (t)$	Velocity of the dipper handle
$r ˙ max$	Maximum velocity of the dipper
$r ˙ min$	Minimum dipper handle velocity
$s$	Degree of freedom
$t$	Time
$t 0$	Initial time in excavation
$t 1$	Switching time between uniform acceleration and uniform stage
$t 2$	Switching time between uniform stage and uniform deceleration stage
$t f$	Final time in excavation
$t y 1$	Switching time between uniform acceleration and uniform stage in the y direction
$t y 2$	Switching time between uniform stage and uniform deceleration stage in the y direction
$t z 1$	Switching time between uniform acceleration and uniform stage in the z direction
$t z 2$	Switching time between uniform stage and uniform deceleration stage in the z direction
$v 0$	Initial velocity
$v ha$	Maximum allowable velocity of the dipper handle
$v ra$	Maximum allowable value of the rope velocity
$v r max$	Maximum velocity of the hoist rope
$v r min$	Minimum rope velocity
$v u$	Velocity in uniform stage
$v y$	Excavation velocity in the y direction
$v z$	Excavation velocity in the z direction
$v r (t)$	Rope velocity
$V$	Loaded volume
$V n$	Nominal load capacity
$x$	Coordinate of the point in the x direction
$x$	Coordinate vector of the point cloud in the x direction
$y$	Coordinate of the point in the y direction
$y$	Coordinate vector of the point cloud in the y direction
$z i$	True value of the sample i
$z ˉ$	Mean value for all samples
$z^$	Z-direction response variable
$z^i$	Prediction value of the sample i
$z$	Coordinate vector of the point cloud in the z direction
$z^$	Prediction value of z
$β$	Coefficient column vector
$β i j$	Coefficient of the polynomial function
$δ$	Cutting angle
$θ$	Angle between the vertical direction and the axis of the dipper handle
$θ p$	Polar angle
$ρ$	Material density
$ρ 0$	Initial polar diameter
$ω 1$ , $ω 2$ , $ω 3$	Weight coefficients
$?$	Angle between the hoist rope and the dipper handle

1	B C Wei , F Gao . A method to calculate working capacity space of multi-DOF manipulator and the application in excavating mechanism. Frontiers of Mechanical Engineering, 2012, 7( 2): 109– 119 https://doi.org/10.1007/s11465-012-0323-6
2	B Wei , F Gao , J Chen , J He , X Zhao . A method for selecting driving system parameters of a new electric shovel’s excavating mechanism with three-DOF. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 2011, 225( 11): 2661– 2672 https://doi.org/10.1177/0954406211407703
3	J Q Wu , G Q Wang , Q S Bi , R Hall . Digging force and power consumption during robotic excavation of cable shovel: experimental study and DEM simulation. International Journal of Mining, Reclamation and Environment, 2021, 35( 1): 12– 33 https://doi.org/10.1080/17480930.2020.1723823
4	K Awuah-Offei , S Frimpong . Numerical simulation of cable shovel resistive forces in oil sands excavation. International Journal of Mining, Reclamation and Environment, 2006, 20( 3): 223– 238 https://doi.org/10.1080/17480930600678289
5	M Stavropoulou , G Xiroudakis , G Exadaktylos . Analytical model for estimation of digging forces and specific energy of cable shovel. Coupled Systems Mechanics, 2013, 2( 1): 23– 51 https://doi.org/10.12989/csm.2013.2.1.023
6	A Rasuli , S Tafazoli , W G Dunford . Dynamic modeling, parameter identification, and payload estimation of mining cable shovels. In: Proceedings of 2014 IEEE Industry Application Society Annual Meeting. Vancouver: IEEE, 2014, 1– 9 https://doi.org/10.1109/IAS.2014.6978451
7	R C Shekhar , J M Maciejowski . Surface excavation with model predictive control. In: Proceedings of the 49th IEEE Conference on Decision and Control (CDC). Atlanta: IEEE, 2010, 5239– 5244 https://doi.org/10.1109/CDC.2010.5717851
8	K Awuah-Offei , S Frimpong . Efficient cable shovel excavation in surface mines. Geotechnical and Geological Engineering, 2011, 29( 1): 19– 26 https://doi.org/10.1007/s10706-010-9366-9
9	S Patnayak , D D Tannant . Performance monitoring of electric cable shovels. International Journal of Surface Mining, Reclamation and Environment, 2005, 19( 4): 276– 294 https://doi.org/10.1080/13895260500327912
10	S Frimpong , Y Li . Stress loading of the cable shovel boom under in-situ digging conditions. Engineering Failure Analysis, 2007, 14( 4): 702– 715 https://doi.org/10.1016/j.engfailanal.2006.02.007
11	Y Li , S Frimpong . Hybrid virtual prototype for analyzing cable shovel component stress. The International Journal of Advanced Manufacturing Technology, 2008, 37( 5–6): 423– 430 https://doi.org/10.1007/s00170-007-0985-0
12	S Frimpong , Y F Hu . Intelligent cable shovel excavation modeling and simulation. International Journal of Geomechanics, 2008, 8( 1): 2– 10 https://doi.org/10.1061/(ASCE)1532-3641(2008)8:1(2
13	X G Song , T C Zhang , Y L Yuan , X B Wang , W Sun . Multidisciplinary co-design optimization of the structure and control systems for large cable shovel considering cross-disciplinary interaction. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 2020, 234( 22): 4353– 4365 https://doi.org/10.1177/0954406220924464
14	T Osa , M Aizawa . Deep reinforcement learning with adversarial training for automated excavation using depth images. IEEE Access: Practical Innovations, Open Solutions, 2022, 10 : 4523– 4535 https://doi.org/10.1109/ACCESS.2022.3140781
15	H Yoshida , T Yoshimoto , D Umino , N Mori . Practical full automation of excavation and loading for hydraulic excavators in indoor environments. In: Proceedings of 2021 IEEE the 17th International Conference on Automation Science and Engineering (CASE). Lyon: IEEE, 2021, 2153– 2160 https://doi.org/10.1109/CASE49439.2021.9551504
16	L J Zhang , J X Zhao , P X Long , L Y Wang , L F Qian , F X Lu , X B Song , D Manocha . An autonomous excavator system for material loading tasks. Science Robotics, 2021, 6( 55): eabc3164 https://doi.org/10.1126/scirobotics.abc3164
17	T G Phillips M E Green P R McAree. Is it what I think it is? Is it where I think it is? Using point-clouds for diagnostic testing of a digging assembly’s form and pose for an autonomous mining shovel. Journal of Field Robotics, 2016, 33(7): 1013– 1033
18	T C Zhang , T Fu , X G Song , F Z Qu . Multi-objective excavation trajectory optimization for unmanned electric shovels based on pseudospectral method. Automation in Construction, 2022, 136 : 104176 https://doi.org/10.1016/j.autcon.2022.104176
19	T Phillips , M Hahn , R McAree . An evaluation of ranging sensor performance for mining automation applications. In: Proceedings of 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics. Wollongong: IEEE, 2013, 1284– 1289 https://doi.org/10.1109/AIM.2013.6584271
20	T G Phillips , N Guenther , P R McAree . When the dust settles: the four behaviors of LiDAR in the presence of fine airborne particulates. Journal of Field Robotics, 2017, 34( 5): 985– 1009 https://doi.org/10.1002/rob.21701
21	M E Green , A N Ridley , P R McAree . Pose verification for autonomous equipment interaction in surface mining. In: Proceedings of 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics. Wollongong: IEEE, 2013, 1199– 1204 https://doi.org/10.1109/AIM.2013.6584257
22	T A D’Adamo , T G Phillips , P R McAree . Registration of three-dimensional scanning LiDAR sensors: an evaluation of model-based and model-free methods. Journal of Field Robotics, 2018, 35( 7): 1182– 1200 https://doi.org/10.1002/rob.21811
23	M Dunbabin , P Corke . Autonomous excavation using a rope shovel. Journal of Field Robotics, 2006, 23( 6–7): 379– 394 https://doi.org/10.1002/rob.20132
24	K Awuah-Offei , S Frimpong . Cable shovel digging optimization for energy efficiency. Mechanism and Machine Theory, 2007, 42( 8): 995– 1006 https://doi.org/10.1016/j.mechmachtheory.2006.07.008
25	Q S Bi , G Q Wang , Y P Wang , Z W Yao , R Hall . Digging trajectory optimization for cable shovel robotic excavation based on a multi-objective genetic algorithm. Energies, 2020, 13( 12): 3118 https://doi.org/10.3390/en13123118
26	D Jud , P Leemann , S Kerscher , M Hutter . Autonomous free-form trenching using a walking excavator. IEEE Robotics and Automation Letters, 2019, 4( 4): 3208– 3215 https://doi.org/10.1109/LRA.2019.2925758
27	B Son , C U Kim , C Kim , D Lee . Expert-emulating excavation trajectory planning for autonomous robotic industrial excavator. In: Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Las Vegas: IEEE, 2020, 2656– 2662 https://doi.org/10.1109/IROS45743.2020.9341036
28	D Lee , I Jang , J Byun , H Seo , H J Kim . Real-time motion planning of a hydraulic excavator using trajectory optimization and model predictive control. In: Proceedings of 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Prague: IEEE, 2021, 2135– 2142 https://doi.org/10.48550/arXiv.2107.02366
29	X B Wang , W Sun , E Li , X G Song . Energy-minimum optimization of the intelligent excavating process for large cable shovel through trajectory planning. Structural and Multidisciplinary Optimization, 2018, 58( 5): 2219– 2237 https://doi.org/10.1007/s00158-018-2011-6
30	X B Wang , X G Song , W Sun . Surrogate based trajectory planning method for an unmanned electric shovel. Mechanism and Machine Theory, 2021, 158 : 104230 https://doi.org/10.1016/j.mechmachtheory.2020.104230
31	A Nurunnabi , G West , D Belton . Robust locally weighted regression techniques for ground surface points filtering in mobile laser scanning three dimensional point cloud data. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54( 4): 2181– 2193 https://doi.org/10.1109/TGRS.2015.2496972
32	D Holz , A E Ichim , F Tombari , R B Rusu , S Behnke . Registration with the point cloud library: a modular framework for aligning in 3-D. IEEE Robotics & Automation Magazine, 2015, 22( 4): 110– 124 https://doi.org/10.1109/MRA.2015.2432331
33	F Durovsky . Point cloud based bin picking: object recognition and pose estimation using region growing segmentation algorithm. Applied Mechanics and Materials, 2015, 791 : 189– 194 https://doi.org/10.4028/www.scientific.net/AMM.791.189
34	K Jain , H S Pannu . Autonomic point cloud-based surface reconstruction using SVR. The Imaging Science Journal, 2018, 66( 1): 59– 67 https://doi.org/10.1080/13682199.2017.1378845
35	J Slabanja , B Meden , P Peer , A Jaklič , F Solina . Segmentation and reconstruction of 3D models from a point cloud with deep neural networks. In: Proceedings of 2018 International Conference on Information and Communication Technology Convergence (ICTC). Jeju: IEEE, 2018, 118– 123 https://doi.org/10.1109/ICTC.2018.8539500
36	R J Pan , V Skala . A two-level approach to implicit surface modeling with compactly supported radial basis functions. Engineering with Computers, 2011, 27( 3): 299– 307 https://doi.org/10.1007/s00366-010-0199-1
37	S Zhao , T F Lu , B Koch , A Hurdsman . Dynamic modelling of 3D stockpile for life-cycle management through sparse range point clouds. International Journal of Mineral Processing, 2013, 125 : 61– 77 https://doi.org/10.1016/j.minpro.2013.09.009
38	M Nazemizadeh , H N Rahimi , K Amini Khoiy . Trajectory planning of mobile robots using indirect solution of optimal control method in generalized point-to-point task. Frontiers of Mechanical Engineering, 2012, 7( 1): 23– 28 https://doi.org/10.1007/s11465-012-0304-9
39	M J D Powell. A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez S, Hennart J P, eds. Advances in Optimization and Numerical Analysis. Mathematics and Its Applications, vol 275. Dordrecht: Springer, 1994
40	B Panda , A Garg , Z Jian , A Heidarzadeh , L Gao . Characterization of the tensile properties of friction stir welded aluminum alloy joints based on axial force, traverse speed, and rotational speed. Frontiers of Mechanical Engineering, 2016, 11( 3): 289– 298 https://doi.org/10.1007/s11465-016-0393-y
41	C Rossi , S Savino . Robot trajectory planning by assigning positions and tangential velocities. Robotics and Computer-Integrated Manufacturing, 2013, 29( 1): 139– 156 https://doi.org/10.1016/j.rcim.2012.04.003

[1]	Yingjun WANG, Liang GAO, Jinping QU, Zhaohui XIA, Xiaowei DENG. Isogeometric analysis based on geometric reconstruction models[J]. Front. Mech. Eng., 2021, 16(4): 782-797.
[2]	Shulei YAO, Xian CAO, Shuang LIU, Congyang GONG, Kaiming ZHANG, Chengcheng ZHANG, Xiancheng ZHANG. Two-sided ultrasonic surface rolling process of aeroengine blades based on on-machine noncontact measurement[J]. Front. Mech. Eng., 2020, 15(2): 240-255.

Viewed

Full text

Abstract

Cited

Shared

Discussed