Design of computer vision assisted machine learning based controller for the Stewart platform to track spatial objects

doi:10.1007/s11709-024-1086-y

Front. Struct. Civ. Eng.

2024, Vol. 18

Issue (8) : 1195-1208 https://doi.org/10.1007/s11709-024-1086-y

Design of computer vision assisted machine learning based controller for the Stewart platform to track spatial objects

Dev Kunwar Singh CHAUHAN(

), Pandu R. VUNDAVILLI

Indian Institute of Technology Bhubaneswar, Bhubaneswar 752050, India

Download: PDF(4353 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

The present work aims to develop an object tracking controller for the Stewart platform using a computer vision-assisted machine learning-based approach. This research is divided into two modules. The first module focuses on the design of a motion controller for the Physik Instrumente (PI)-based Stewart platform. In contrast, the second module deals with the development of a machine-learning-based spatial object tracking algorithm by collecting information from the Zed 2 stereo vision system. Presently, simple feed-forward neural networks (NN) are used to predict the orientation of the top table of the platform. While training, the x, y, and z coordinates of the three-dimensional (3D) object, extracted from images, are used as the input to the NN. In contrast, the orientation information of the platform (that is, rotation about the x, y, and z-axes) is considered as the output from the network. The orientation information obtained from the network is fed to the inverse kinematics-based motion controller (module 1) to move the platform while tracking the object. After training, the optimised NN is used to track the continuously moving 3D object. The experimental results show that the developed NN-based controller has successfully tracked the moving spatial object with reasonably good accuracy.

Keywords Stewart platform feed-forward neural networks motion controller inverse kinematics stereo vision

Corresponding Author(s): Dev Kunwar Singh CHAUHAN

Just Accepted Date: 02 July 2024 Online First Date: 26 July 2024 Issue Date: 29 August 2024

Cite this article:

Dev Kunwar Singh CHAUHAN,Pandu R. VUNDAVILLI. Design of computer vision assisted machine learning based controller for the Stewart platform to track spatial objects[J]. Front. Struct. Civ. Eng., 2024, 18(8): 1195-1208.

URL:

https://academic.hep.com.cn/fsce/EN/10.1007/s11709-024-1086-y
https://academic.hep.com.cn/fsce/EN/Y2024/V18/I8/1195

Fig.1 Schematic diagram showing the SPS-type Stewart platform.

Fig.2 Schematic diagram showing the forward object tracking methodology: (a) a targeted point on the surface of the hemisphere for forward object tracking; (b) representation of direction cosines for a forward tracking solution.

Fig.3 Schematic diagram showing the various modules used to track the spatial object.

Fig.4 Data and information integration modules used for tracking the object using the Stewart platform.

Fig.5 The schematic diagram showing: (a) the transformations of the mapped object in 3D space; (b) setup used to collect the data that is used to train the neural network.

Fig.6 The schematic diagram showing the architecture of the machine learning based algorithm.

Fig.7 The schematic diagram showing the ZED stereo camera used in the study.

Fig.8 Images showing the experimental setup for collecting the data with stereo camera facing vertically up.

Fig.9 An experimental setup used for real time tracking of spatially moving objects using the Stewart platform.

Fig.10 Figure showing the convergence plot while training the neural network for the camera facing vertically.

Fig.11 Plot showing the pixel error of the object position and target position in the image frame along x- and y-directions.

Fig.12 Plot showing the pixel error of the object position and target position in the image frame along x- and y-directions when target is out of its normal trackable limit.

Fig.13 Plot showing the pixel error of the object position and target position in the image frame along x- and y-directions when the environment has low brightness lighting conditions.

Fig.14 Plot showing the pixel error of the object position and target position in the image frame along x and y-directions when the target is moving faster than the robot’s maximum speed.

Fig.15 Schematic diagram showing the snap shots of the Stewart platform tracking the object by using the camera facing vertically up: (a) tracking error before the approach of the Stewart platform towards the object; (b) tracking error while the Stewart platform is approaching towards the object; (c) tracking error while the Stewart platform is about to reach closer to the object.

Fig.16 Schematic diagram showing the snap shots of the Stewart platform tracking the object by using the camera facing vertically.

Fig.17 Plot showing the variation of position of six prismatic actuators of Stewart platform while tracking the object in a 3D space.

Fig.18 Plot showing the variation of velocity of six prismatic actuators of the Stewart platform while tracking the object in a 3D space.

Fig.19 Plot showing the variation of acceleration of six prismatic actuators of the Stewart platform while tracking the object in a 3D space.

1	R K MittalI J Nagrath. Robotics and Control. New York, NY: Tata McGraw-Hill, 2003
2	D Stewart. A platform with six degrees of freedom. Proceedings of the Institution of Mechanical Engineers, 1965, 180(1): 371–386
3	K Liu, J M Fitzgerald, F L Lewis, S Member. Kinematic analysis of a Stewart platform manipulator. IEEE Transactions on Industrial Electronics, 1993, 40(2): 282–293 https://doi.org/10.1109/41.222651
4	W Zhou, W Chen, H Liu. A new forward kinematic algorithm for a general Stewart platform. Mechanism and Machine Theory, 2015, 87: 177–190
5	D K S Chauhan, P R Vundavilli. Forward kinematics of the Stewart parallel manipulator using machine learning. International Journal of Computational Methods, 2022, 2142009(8): 1–22 https://doi.org/10.1142/S0219876221420093
6	K Liu, F Lewis, G Lebret, D Taylor. The singularities and dynamics of a Stewart platform manipulator. Journal of Intelligent & Robotic Systems, 1993, 8(3): 287–308 https://doi.org/10.1007/BF01257946
7	G Lebret, K Liu, F L Lewis. Dynamic analysis and control of a Stewart platform manipulator. Journal of Robotic Systems, 1993, 10(5): 629–655 https://doi.org/10.1002/rob.4620100506
8	N Mostashiri, A Akbarzadeh, J Dhupia. A comprehensive inverse dynamics problem of a Stewart platform by means of Lagrangian formulation. In: Proceedings of Dynamic Systems and Control Conference. New York, NY: American Society of Mechanical Engineers, 2017, 58271: V001T30A003
9	Q Meng, T Zhang, J F He, J Y Song, J W Han. Dynamic modeling of a 6-degree-of-freedom Stewart platform driven by a permanent magnet synchronous motor. Frontiers of Information Technology & Electronic Engineering, 2010, 11: 751–761 https://doi.org/10.1631/jzus.C0910714
10	T X Tien, D Cirkl. Solution of inverse dynamics of Stewart−gough platform using substructure technique. Journal of Theoretical and Applied Mechanics, 2021, 59: 329–342 https://doi.org/10.15632/jtam-pl/136057
11	B Dasgupta, T S Mruthyunjaya. A Newton−Euler formulation for the inverse dynamics of the Stewart platform manipulator. Mechanism and Machine Theory, 1998, 33(8): 1135–1152
12	M J Liu, C X Li, C N Li. Dynamics analysis of the Gough−Stewart platform manipulator. IEEE Transactions on Robotics and Automation, 2000, 16(1): 94–98 https://doi.org/10.1109/70.833196
13	A K Jishnu, D K S Chauhan, P R Vundavilli. Design of neural network-based adaptive inverse dynamics controller for motion control of stewart platform. International Journal of Computational Methods, 2022, 19(8): 2142010
14	C M Gosselin, A Hadj-Messaoud. Automatic planning of smooth trajectories for pick-and-place operations. In: Proceedings of International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. New York, NY: American Society of Mechanical Engineers, 1991, 97768: 531–536
15	V E Ömürlü, I Yildiz. A Stewart platform as a FBW flight control unit. Journal of Electrical Engineering, 2011, 62(4): 213–219 https://doi.org/10.2478/v10187-011-0034-y
16	A Koszewnik, K Troc, M Słowik. PID controllers design applied to positioning of ball on the Stewart platform. Acta Mechanicaet Automatica, 2014, 8(4): 214–218 https://doi.org/10.2478/ama-2014-0039
17	I Joseph. Space debris: Understanding the risks to NASA spacecraft. 2023. Available at the website of NASA
18	Wikipedia. Space debris. 2023. Available at the website of Wikipedia
19	R B Ashith Shyam, M Acharya, A Ghosal. A heliostat based on a three degree-of-freedom parallel manipulator. Solar Energy, 2017, 157: 672–686 https://doi.org/10.1016/j.solener.2017.08.073
20	R B Ashith Shyam, A Ghosal. Path planning of a 3-UPU wrist manipulator for sun tracking in central receiver tower systems. Mechanism and Machine Theory, 2018, 119: 130–141 https://doi.org/10.1016/j.mechmachtheory.2017.08.019
21	J K Tharamuttam, A K Ng. Design and development of an automatic solar tracker. Energy Procedia, 2017, 143: 629–634 https://doi.org/10.1016/j.egypro.2017.12.738
22	F Z Baouche, B Abderezzak, A Ladmi, K Arbaoui, G Suciu, T C Mihaltan, M S Raboaca, S V Hudișteanu, F E Țurcanu. Design and simulation of a solar tracking system for PV. Applied Sciences, 2022, 12(19): 9682 https://doi.org/10.3390/app12199682
23	A Cammarata. Optimized design of a large-workspace 2-DOF parallel robot for solar tracking systems. Mechanism and Machine Theory, 2015, 83: 175–186 https://doi.org/10.1016/j.mechmachtheory.2014.09.012
24	X Liu, Y Yang, C Ma, J Li, S Zhang. Real-time visual tracking of moving targets using a low-cost Unmanned Aerial Vehicle with a 3-axis stabilized gimbal system. Applied Sciences, 2020, 10(15): 5064 https://doi.org/10.3390/app10155064
25	A Yilmaz, O Javed, M Shah. Object tracking: A survey. ACM Computing Surveys, 2006, 38(4): 13 https://doi.org/10.1145/1177352.1177355
26	R K Mandava, P R Vundavilli. Near optimal PID controllers for the biped robot while walking on uneven terrains. International Journal of Automation and Computing, 2018, 15(6): 689–706 https://doi.org/10.1007/s11633-018-1121-3
27	M S Khan, R K Mandava. A review on gait generation of the biped robot on various terrains. Robotica, 2023, 41(6): 1888–1930 https://doi.org/10.1017/S0263574723000097
28	E Samaniego, C Anitescu, S Goswami, V M Nguyen-Thanh, H Guo, K Hamdia, X Zhuang, T Rabczuk. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 2020, 362: 112790 https://doi.org/10.1016/j.cma.2019.112790
29	C Anitescu, E Atroshchenko, N Alajlan, T Rabczuk. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials & Continua, 2019, 59(1): 345–359 https://doi.org/10.32604/cmc.2019.06641
30	D J Regner, J D Salazar, P V Buschinelli, M Machado, D Oliveira, J M Santos, C A Marinho, T C Pinto. Object tracking control using a gimbal mechanism. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2021, 43: 189–196 https://doi.org/10.5194/isprs-archives-XLIII-B1-2021-189-2021
31	N Cong Danh. The stability of a two-axis gimbal system for the camera. The Scientific World Journal, 2021, 2021: 2–9 https://doi.org/10.1155/2021/9958848

[1]	Huy Q. LE, Tam T. TRUONG, D. DINH-CONG, T. NGUYEN-THOI. A deep feed-forward neural network for damage detection in functionally graded carbon nanotube-reinforced composite plates using modal kinetic energy[J]. Front. Struct. Civ. Eng., 2021, 15(6): 1453-1479.

Viewed

Full text

Abstract

Cited

Shared

Discussed