Development of a Gesture Control System for Intuitive Control of a Quadcopter

This paper presents the development and implementation of a quadcopter control method using hand gestures within the ROS2 framework. The approach enables intuitive operator-drone interaction in environments where conventional remote controllers are unavailable or impractical. An Intel RealSense D435i depth camera stream provides precise spatial localization of hand keypoints. High-level control functions — motor arming and disarming, takeoff, and landing — are executed through predefined static gestures. Gesture classification is performed by analyzing the spatial arrangement of hand keypoints in the image plane using a fully connected multilayer perceptron comprising two hidden layers with 168 and 546 neurons, respectively, achieving an optimal trade-off between inference speed and accuracy. Continuous quadcopter control is accomplished by computing roll, pitch, yaw angles, and throttle values derived from three-dimensional hand position analysis, utilizing the camera’s intrinsic calibration parameters and depth map data. To mitigate gimbal lock effects near 90° angles, a deviation range constraint is incorporated into the control pipeline. Control signal smoothing employs a combination of a moving average filter with window size 5 and exponential smoothing with coefficient a = 0.5, ensuring rapid system response while effectively suppressing outliers caused by image processing artifacts. All filtering parameters were determined through systematic experimental evaluation. The system achieves gesture classification accuracy exceeding 99 %, with per-frame processing time below 33 ms (~30 FPS), which eliminates control signal latency and ensures high responsiveness. Test flights conducted in the Gazebo simulator using a DJI Tello quadcopter model validated the proposed method and confirmed its practical potential for interactive unmanned aerial vehicle control applications.

1. Nurutdinov A. A., Yamaletdinova K. Sh., Tagirov I. I. De-tection of leaks at main pipeline facilities using unmanned aerial vehicles, Nauka, obrazovanie, proizvodstvo dlya protivodejstviya tekhnogennym ugrozam i resheniya ekologicheskih problem (Tekhnos-fernaya bezopasnost’-2024), pp. 214—217 (in Russian).

2. Kholodkov K. I., Ivanov S. D., Aleshin I. M. et al. An experience of building digital elevation models with unmanned aerial vehicles, Nauka i tekhnologicheskie razrabotki, 2021, vol. 100, no. 4, pp. 44—56 (in Russian), DOI: 10.21455/std2021.4-4.

3. Zubarev Yu. N., Fomin D. S., Chaschin A. N. et al. Use of unmanned aerial vehicles in agriculture, Vestnik Permskogo federal’nogo issledovatel’skogo centra, 2019, no. 2, pp. 47—51 (in Russian), DOI: 10.7242/2658-705X/2019.2.5.

4. Mallik B., Rahim M. A., Miah A. S. M., Yun K. S., Shin J. Virtual Keyboard: А Real-Time Hand Gesture Recogni-tion-Based Character Input System Using LSTM and Mediapipe Holistic, Computer Systems Science & Engineering, 2024, vol. 48, no. 2, pp. 1—10, DOI: 10.32604/csse.2023.045981.

5. Serpiva V., Fedoseev A., Karaf S., Abdulkarim A. A., Tsetserukou D. OmniRace: 6D Hand Pose Estimation for In-tuitive Guidance of Racing Drone, 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2024, pp. 2508—2513, DOI: 10.1109/IROS58592.2024.10801907.

6. Lugaresi C., Tang J., Nash H., McClanahan C., Ubowe-ja E., Hays M., Zhang F., Chang C.-L., Yong M. G., Lee J., Chang W.-T., Hua W., Georg M., Grundmann M. Mediapipe: А framework for building perception pipelines, arXiv preprint arXiv:1906.08172, 2019, DOI: 10.48550/arXiv.1906.08172.

7. LeCun Y., Bengio Y., Hinton G. Deep learning, Nature, 2015, vol. 521, no. 7553, pp. 436—444, DOI: 10.1038/nature14539.

8. Pukelsheim F. The three sigma rule, The American Statistician, 1994, vol. 48, no. 2, pp. 88—91, DOI: 10.1080/00031305.1994.10476030.

9. Gardner Jr E. S. Exponential smoothing: The state of the art, Journal of forecasting, 1985, vol. 4, no. 1. pp. 1—28, DOI: 10.1002/for.3980040104.

10. Koenig N., Howard А. Design and use paradigms for gazebo, an open-source multi-robot simulator, 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS) (IEEE Cat. No. 04CH37566), IEEE, 2004, vol. 3, pp. 2149—2154, DOI: 10.1109/IROS.2004.1389727.

11. Zlotnikov K. A., Volosyuk A. A., Tan Н. A. Human factors in unmanned aviation and unmanned aircraft system operators training, Chelovecheskij faktor v slozhnyh tekhnicheskih sistemah i sredah, 2016, pp. 231—237 (in Russian).