Citation: Rubin Wang, Quanli Dou, Qi Zhang, Cheng Zhou. Gesture Recognition Based on MediaPipe for Excavator Teleoperation Control. Journal of Information Technologyin Civil Engineering and Architecture, 2022, 14(4): 9-16. doi: 10.16670/j.cnki.cn11-5823/tu.2022.04.02
2022, 14(4): 9-16. doi: 10.16670/j.cnki.cn11-5823/tu.2022.04.02
Gesture Recognition Based on MediaPipe for Excavator Teleoperation Control
1. | School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology, Wuhan 430074, China |
2. | Weichai Power Co., Ltd., Weifang 261061, China |
Excavators have broad application scenarios, but in certain hazardous working conditions, using teleoperation is a better option. Meanwhile, with the development of machine vision and deep learning in recent years, a number of algorithms and frameworks for gesture recognition are becoming available. In order to explore the application of MediaPipe-based gesture recognition algorithms in excavator teleoperation, this paper corresponds different hand gestures to different movements of the excavator, and proposes a new control method using hand gestures to achieve excavator teleoperation. The kinematic analysis of the experimental excavation robot was carried out in a laboratory environment, and MediaPipe was used to perform real-time static recognition of 15 gesture types, generate commands and realise the control of the excavator using an Arduino microcontroller. The results show that the system has good performance and can be used for teleoperation control of the excavator, which provides a new form of human-computer interaction for remote control of the excavators.
Metrics
- PDF Downloads(68)
- Abstract views(2400)
- HTML views(1250)