Footholds optimization for legged robots walking on complex terrain

doi:10.1007/s11465-022-0742-y

Front. Mech. Eng.

2023, Vol. 18

Issue (2) : 26 https://doi.org/10.1007/s11465-022-0742-y

RESEARCH ARTICLE

Footholds optimization for legged robots walking on complex terrain

Yunpeng YIN¹, Yue ZHAO², Yuguang XIAO¹, Feng GAO¹(

)

¹. State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
². AI Institute, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

Download: PDF(5620 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

This paper proposes a novel continuous footholds optimization method for legged robots to expand their walking ability on complex terrains. The algorithm can efficiently run onboard and online by using terrain perception information to protect the robot against slipping or tripping on the edge of obstacles, and to improve its stability and safety when walking on complex terrain. By relying on the depth camera installed on the robot and obtaining the terrain heightmap, the algorithm converts the discrete grid heightmap into a continuous costmap. Then, it constructs an optimization function combined with the robot’s state information to select the next footholds and generate the motion trajectory to control the robot’s locomotion. Compared with most existing footholds selection algorithms that rely on discrete enumeration search, as far as we know, the proposed algorithm is the first to use a continuous optimization method. We successfully implemented the algorithm on a hexapod robot, and verified its feasibility in a walking experiment on a complex terrain.

Keywords footholds optimization legged robot complex terrain adapting hexapod robot locomotion control

Corresponding Author(s): Feng GAO

Just Accepted Date: 15 November 2022 Issue Date: 12 June 2023

Cite this article:

Yunpeng YIN,Yue ZHAO,Yuguang XIAO, et al. Footholds optimization for legged robots walking on complex terrain[J]. Front. Mech. Eng., 2023, 18(2): 26.

URL:

https://academic.hep.com.cn/fme/EN/10.1007/s11465-022-0742-y
https://academic.hep.com.cn/fme/EN/Y2023/V18/I2/26

Fig.1 Photo of the hexapod robot and its coordinate systems. RGBD: red, green, blue, depth.

Fig.2 Electrical system of the robot. IMU: inertial measurement unit, RGBD: red, green, blue, depth.

Fig.3 Control framework of the robot for footholds optimization.

Fig.4 Gait scheduler and tripod-trot gait for the robot.

Fig.5 Example of image convolution in digital image processing.

Fig.6 Convolution interpolation for transforming a pixel into a continuous heightmap.

Fig.7 Pyramid convolution interpolation for transforming a pixel into a continuous heightmap.

Fig.8 NDCI for transforming a pixel into a continuous heightmap, with its gradients. NDCI: normalized derivable convolution interpolation.

Fig.9 Experimental environment and the postprocessing: (a) photograph, (b) point cloud, (c) heightmap, (d) height difference map, and (e) costmap.

Fig.10 Swing trajectory based on cycloid spatial composite trajectory.

Fig.11 Leg virtual model of impedance control.

Fig.12 Experiment setup for footholds optimization with the hexapod robot.

Fig.13 Screenshots of blind walking. Slips are circled in red.

Fig.14 Euler angles during blind walk.

Fig.15 Screenshots of footholds optimization.

Fig.16 Screenshots of point cloud processing.

Fig.17 Step length with footholds optimization.

Fig.18 Euler angles when using foothold optimization.

Tab.1 Contrast between blind walking and footholds optimized walking

Abbreviations
BCS	Body coordinate system
CBC	Centroid balance control
CI	Convolution interpolation
CNN	Convolutional neural network
CoM	Center of mass
GRF	Ground reaction force
IMU	Inertial measurement unit
LCS	Leg coordinate system
NDCI	Normalized derivable convolution interpolation
PCI	Pyramid convolution interpolation
RGBD	Red, green, blue, depth
SLIP	Spring-loaded inverted pendulum
WCS	World coordinate system
Variables
A	Merged matrix for CBC
b, f	Merged vectors for CBC
c_{i, j}	Center of the pixel (i, j) on the map
C	Matrix of friction cone and upper and lower bound constraints for GRF
E	Identity matrix
f_i	GRF of the ith leg
$L f i ∗$	Virtual feedforward force of the ith leg in the LCS
f_lim	Friction cone and upper and lower bound constraints for GRF
$f s t ∗$	Virtual force of the supporting legs
$f s t, i ∗$	Virtual force of the ith supporting leg
$L f s t, i ∗$ , $L f s w, i ∗$	Virtual feedforward forces of the ith supporting leg and swing leg in the LCS, respectivley
g	Local gravitational acceleration
h_z	Step height that needs to be raised in the middle of the swing process
h	Merged vector of h_i
h^*	Horizontal component of optimized footholds
h_i	Horizontal component of p_i
$h i ∗$	Optimized horizontal coordinate of the foothold
h_pre	Merged vector of h_pre,i
h_pre,i	Horizontal component of preplanned foothold
Δh	Intermediate variable of the difference between the optimized footholds and the preplanned footholds
Δh_lim	Maximum amount of adjustment Δp_i allowed
_nΔh	nth element of Δh
i	Index for the item, such as the ith leg, or the (i, j) pixel
^BI	Inertia matrix of the body in the BCS
j	Index for the item, such as the (i, j) pixel
J_i	Jacobian matrix of the ith leg
J (h_i)	Terrain cost at the position of h_i
J (p_h)	Cost function for NDCI
$∇ J (p h)$	Partial derivative of the cost function for NDCI
$K D, j$ , $K P, j$	Joint damping and stiffness matrices, respectively
$K D, p$ , $K P, p$	Damping and stiffness of body position, respectively
$K D, a$ , $K P, a$	Damping and stiffness of body attitude, respectively
$L K D$ , $L K P$	Damping and stiffness of the leg, respectively
^Bl_i	Tip position in the BCS
$L l i$ , $L l ˙ i$	Position and velocity of the ith leg in the LCS, respectively
$L l i ∗$ , $L l ˙ i ∗$	Target position and velocity of the ith leg in the LCS, respectively
m, n	Set numbers of pixels that need to be involved
$p b$	Position of the robot body
$p b ∗$ , $p ˙ b ∗$ , $p ¨ b ∗$	Target position, velocity, and acceleration of the robot’s CoM, respectively
p_h	Horizontal coordinate of the position of the selected point on the whole map
$p i$	Foothold point of the ith leg
$p i, x$ , $p i, y$ , $p i, z$	x, y, and z components of the foothold $p i$ , respectively
$p i ∗$	Optimized footholds
$p i, z ∗$	Height components at the position $p i ∗$ on the heightmap
$p p r e, i$	Preplanning of the target foothold
$p ξ i$	Swing trajectory of the ith leg
$p^b$ , $p ˙^b$	Estimated position and velocity of the robot, respectively
$Δ p b ∗$	Correction amount on the original body trajectory
Δp_i	Foothold adjustment for the ith leg
^Bp_hip,i	Position of the ith leg’s hip joint in the BCS
$q i$ , $q ˙ i$	Current angle and angular velocity read by the ith encoder, respectively
$q i ∗$ , $q ˙ i ∗$	Target angle and angular velocity of the ith joint, respectively
R	Current attitude matrix
$R ∗$	Desired attitude of the body
$R^$	Estimated current attitude of the robot
$R$	Set of real number
S	Selection matrix for CBC
t_sw,i	Time when the ith leg enters the swing phase
T_st, T_sw	Duration of the standing and swing phase, respectively
v	Velocity of the body
v_d	Desired velocity of the body
$Δ v b ∗$	Speed correction
x_h	x component of the horizontal coordinate of the position of the selected point on the heightmap
x_i,j	x component of the center of the pixel (i, j) on the map
y_h	y component of the horizontal coordinate of the position of the selected point on the heightmap
y_i,j	y component of the center of the pixel (i, j) on the map
W, D, Q	Weighting matrices for footholds optimization
α, β, γ,	Weighting factors for footholds optimization
δ	Weighting factor for CBC
σ_x, σ_y	Smoothing factor in x and y directions, respectively
ϕ, θ, ψ	Euler angle roll, pitch, and yaw measured by the IMU
μ	Number of legs in contact
$τ i ∗$	Control torque of the ith joint
ξ_i	Swing phase
$ω^b$	Angular velocity of the body
$ω b ∗$ , $ω ˙ b ∗$	Target angular velocity and acceleration of the robot, respectively
*	Symbol maps from a vector ( $R 3$ ) to an antisymmetric matrix ( $R 3 × 3$ )

1	M H Raibert, E R Tello. Legged robots that balance. IEEE Expert, 1986, 1(4): 89 https://doi.org/10.1109/MEX.1986.4307016
2	G Bledt, M J Powell, B Katz, J Di Carlo, P M Wensing, S Kim. MIT Cheetah 3: design and control of a robust, dynamic quadruped robot. In: Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Madrid: IEEE, 2018
3	P Vernaza, M Likhachev, S Bhattacharya, S Chitta, KushleyevD D A, Lee. Search-based planning for a legged robot over rough terrain. In: Proceedings of 2009 IEEE International Conference on Robotics & Automation. Kobe: IEEE, 2009, 2380–2387
4	M Kalakrishnan, J Buchli, P Pastor, M Mistry, S Schaal. Fast, robust quadruped locomotion over challenging terrain. In: Proceedings of 2010 IEEE International Conference on Robotics & Automation. Anchorage: IEEE, 2010, 2665–2670
5	C Semini, N G Tsagarakis, E Guglielmino, M Focchi, F Cannella, D G Caldwell. Design of HyQ—a hydraulically and electrically actuated quadruped robot. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 2011, 225(6): 831–849 https://doi.org/10.1177/0959651811402275
6	A Winkler, I Havoutis, S Bazeille, J Ortiz, Focchi M, Dillmann R, Caldwell D, C Semini. Path planning with force-based foothold adaptation and virtual model control for torque controlled quadruped robots. In: Proceedings of 2014 IEEE International Conference on Robotics & Automation (ICRA). Hong Kong: IEEE, 2014, 6476–6482
7	M Hutter, C Gehring, D Jud, A Lauber, C D Bellicoso, V Tsounis, J Hwangbo, K Bodie, P Fankhauser, M Bloesch, R Diethelm, S Bachmann, A Melzer, M Hoepflinger. ANYmal—a highly mobile and dynamic quadrupedal robot. In: Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon: IEEE, 2016, 38–44
8	A W Winkler, C D Bellicoso, M Hutter, J Buchli. Gait and trajectory optimization for legged systems through phase-based end-effector parameterization. IEEE Robotics and Automation Letters, 2018, 3(3): 1560–1567 https://doi.org/10.1109/LRA.2018.2798285
9	C Mastalli, M Focchi, I Havoutis, A Radulescu, Calinon S, Buchli J, Caldwell D G, C Semini. Trajectory and foothold optimization using low-dimensional models for rough terrain locomotion. In: Proceedings of 2017 IEEE International Conference on Robotics & Automation (ICRA). Singapore: IEEE, 2017, 1096–1103
10	C Mastalli, I Havoutis, M Focchi, D G Caldwell, C Semini. Motion planning for quadrupedal locomotion: coupled planning, terrain mapping and whole-body control. IEEE Transactions on Robotics, 2020, 36(6): 1635–1648 https://doi.org/10.1109/TRO.2020.3003464
11	C Mastalli, I Havoutis, A W Winkler, D G Caldwell, C Semini. On-line and on-board planning and perception for quadrupedal locomotion. In: Proceedings of 2015 IEEE International Conference on Technologies for Practical Robot Applications (TePRA). Woburn: IEEE, 2019, 1–7
12	P Fankhauser, M Bjelonic, C D Bellicoso, T Miki, M Hutter. Robust rough-terrain locomotion with a quadrupedal robot. In: Proceedings of 2018 IEEE International Conference on Robotics and Automation (ICRA). Brisbane: IEEE, 2018, 5761–5768
13	F Jenelten, T Miki, A E Vijayan, M Bjelonic, M Hutter. Perceptive locomotion in rough terrain—online foothold optimization. IEEE Robotics and Automation Letters, 2020, 5(4): 5370–5376 https://doi.org/10.1109/LRA.2020.3007427
14	X Chai, F Gao, Y L Xu. Perception-based gait planning for a hexapod robot walking on typical structured terrain. In: Zhang X M, Wang N F, Huang Y J, eds. Mechanism and Machine Science. Singapore: Springer, 2017, 169–181
15	D Kim, D Carballo, Di J Carlo, B Katz, Bledt G, Lim B, S Kim. Vision aided dynamic exploration of unstructured terrain with a small-scale quadruped robot. In: Proceedings of 2020 IEEE International Conference on Robotics and Automation (ICRA). Paris: IEEE, 2020, 2464–2470
16	L H Mao, Y Tian, F Gao, Y Zhao. Novel method of gait switching in six-legged robot walking on continuous-nondifferentiable terrain by utilizing stability and interference criteria. Science China Technological Sciences, 2020, 63(12): 2527–2540 https://doi.org/10.1007/s11431-020-1588-5
17	L H Mao, F Gao, Y Tian, Y Zhao. Novel method for preventing shin-collisions in six-legged robots by utilising a robot–terrain interference model. Mechanism and Machine Theory, 2020, 151: 103897 https://doi.org/10.1016/j.mechmachtheory.2020.103897
18	Y Zhao, F Gao, Y P. Yin Obstacle avoidance and terrain identification for a hexapod robot. Research Square, 2020, preprint
19	O A V Magaña, V Barasuol, M Camurri, L Franceschi, M Focchi, M Pontil, D G Caldwell, C Semini. Fast and continuous foothold adaptation for dynamic locomotion through CNNs. IEEE Robotics and Automation Letters, 2019, 4(2): 2140–2147 https://doi.org/10.1109/LRA.2019.2899434
20	T Miki, J Lee, J Hwangbo, L Wellhausen, V Koltun, M Hutter. Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics, 2022, 7(62): eabk2822 https://doi.org/10.1126/scirobotics.abk2822
21	V Tsounis, M Alge, J Lee, F Farshidian, M Hutter. DeepGait: planning and control of quadrupedal gaits using deep reinforcement learning. IEEE Robotics and Automation Letters, 2020, 5(2): 3699–3706 https://doi.org/10.1109/LRA.2020.2979660
22	C Gehring, P Fankhauser, L Isler, R Diethelm, S Bachmann, M Potz, L Gerstenberg, M Hutter. ANYmal in the field: solving industrial inspection of an offshore HVDC platform with a quadrupedal robot. In: Ishigami G, Yoshida K, eds. Field and Service Robotics. Singapore: Springer, 2021, 247–260
23	W Liu, Y Gao, F Gao, S Y Li. Trajectory adaptation and safety control via control barrier functions for legged robots. In: Proceedings of 2021 China Automation Congress (CAC). Beijing: IEEE, 2021, 5571–5576
24	H W Park, P M Wensing, S Kim. Jumping over obstacles with MIT Cheetah 2. Robotics and Autonomous Systems, 2021, 136: 103703 https://doi.org/10.1016/j.robot.2020.103703
25	S H Qi, W C Lin, Z J Hong, H Chen, W Zhang. Perceptive autonomous stair climbing for quadrupedal robots. In: Proceedings of 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Prague: IEEE, 2021, 2313–2320
26	P Gerum. Xenomai—Implementing a RTOS emulation framework on GNU/Linux. White Paper, 2004, 81
27	P Hintjens. ZeroMQ: Messaging for Many Applications. O’Reilly Media, 2013
28	Q Sun, F Gao, X B Chen. Towards dynamic alternating tripod trotting of a pony-sized hexapod robot for disaster rescuing based on multi-modal impedance control. Robotica, 2018, 36(7): 1048–1076 https://doi.org/10.1017/S026357471800022X
29	M P Ekstrom. Digital Image Processing Techniques. 2nd ed. Academic Press, 2012
30	A. Ranganathan. The Levenberg‒Marquardt algorithm. Tutoral on LM Algorithm, 2004, 11(1): 101–110
31	M BlöschM HutterM A HöpflingerS LeuteneggerC GehringC D Remy R Y Siegwart. State estimation for legged robots—consistent fusion of leg kinematics and IMU. In: Proceedings of Robotics: Science and Systems Conference (RSS 2012). Sidney: Robotics: Science and Systems Conference, 2012
32	M Focchi, A Del Prete, I Havoutis, R Featherstone, D G Caldwell, C Semini. High-slope terrain locomotion for torque-controlled quadruped robots. Autonomous Robots, 2017, 41(1): 259–272 https://doi.org/10.1007/s10514-016-9573-1

[1]	Yi ZHENG, Kun XU, Yaobin TIAN, Xilun DING. Different manipulation mode analysis of a radial symmetrical hexapod robot with leg–arm integration[J]. Front. Mech. Eng., 2022, 17(1): 8-.
[2]	Yue ZHAO, Feng GAO, Qiao SUN, Yunpeng YIN. Terrain classification and adaptive locomotion for a hexapod robot Qingzhui[J]. Front. Mech. Eng., 2021, 16(2): 271-284.
[3]	Jimu LIU, Yuan TIAN, Feng GAO. *A novel six-legged walking machine tool for in-situ* operations**[J]. Front. Mech. Eng., 2020, 15(3): 351-364.
[4]	Haitao YU, Haibo GAO, Liang DING, Zongquan DENG. Sagittal SLIP-anchored task space control for a monopode robot traversing irregular terrain[J]. Front. Mech. Eng., 2020, 15(2): 193-208.
[5]	Boxing WANG, Chunlin ZHOU, Ziheng DUAN, Qichao ZHU, Jun WU, Rong XIONG. Untethered quadrupedal hopping and bounding on a trampoline[J]. Front. Mech. Eng., 2020, 15(2): 181-192.
[6]	Kejia LI, Xilun DING, Marco CECCARELL. A total torque index for dynamic performance evaluation of a radial symmetric six-legged robot[J]. Front Mech Eng, 2012, 7(2): 219-230.
[7]	LU Zhen-li, MA Shu-gen, LI Bin, WANG Yue-chao. Design of a cyclic inhibitory CPG controller for the locomotion of a snakelike robot[J]. Front. Mech. Eng., 2006, 1(4): 396-402.

Viewed

Full text

Abstract

Cited

Shared

Discussed