Abstract:
Excavators have broad application scenarios, but in certain hazardous working conditions, using teleoperation is a better option. Meanwhile, with the development of machine vision and deep learning in recent years, a number of algorithms and frameworks for gesture recognition are becoming available. In order to explore the application of MediaPipe-based gesture recognition algorithms in excavator teleoperation, this paper corresponds different hand gestures to different movements of the excavator, and proposes a new control method using hand gestures to achieve excavator teleoperation. The kinematic analysis of the experimental excavation robot was carried out in a laboratory environment, and MediaPipe was used to perform real-time static recognition of 15 gesture types, generate commands and realise the control of the excavator using an Arduino microcontroller. The results show that the system has good performance and can be used for teleoperation control of the excavator, which provides a new form of human-computer interaction for remote control of the excavators.