AI-Based Hand Gesture Recognition Through Camera on Robot

Gergo Csonka, Muhammad Khalid, Husnain Rafiq, Yasir Ali

Research output: Chapter in Book/Report/Conference proceedingConference proceeding (ISBN)peer-review


This paper presents an innovative approach to real-time hand gesture recognition for robot control using Artificial Intelligence (AI). The core of this project is a machine learning model trained on a custom data set of hand gestures, which was meticulously hand-annotated to ensure accuracy. To enhance the model’s performance and generalization, data augmentation techniques were employed. Furthermore, the model leverages the power of transfer learning, with a ResNet backbone serving as the foundation, to efficiently learn from the data set. In addition to the development of the AI model, a custom robot was designed and built using Arduino and Raspberry Pi. This robot is equipped with a camera to capture images of hand gestures, which are then transmitted to the machine learning model for real-time analysis. The hardware of the robot was meticulously optimized to ensure smooth operation and accurate data capture. The resulting system enables real-time hand gesture recognition on the robot, opening up a plethora of applications, from industrial automation to smart home technology. By synergistically combining AI, computer vision, and robotics, this project not only demonstrates the potential for innovative solutions to real-world problems but also significantly enhances the functionality and usability of robots. It paves the way for improved human-computer interaction through the practical implementation of advanced AI and computer vision techniques.
Original languageEnglish
Title of host publication2023 International Conference on Frontiers of Information Technology (FIT)
Number of pages6
ISBN (Electronic)979-8-3503-9578-5
ISBN (Print)979-8-3503-9579-2
Publication statusPublished - 5 Feb 2024

Publication series

NameProceedings - 2023 International Conference on Frontiers of Information Technology, FIT 2023


  • Artificial intelligence (AI)
  • Hand Gesture Recognition
  • Robotics
  • Camera
  • Raspberry Pi
  • Artificial Intelligence
  • Computer Vision
  • Arduino
  • Human-Computer Interaction


Dive into the research topics of 'AI-Based Hand Gesture Recognition Through Camera on Robot'. Together they form a unique fingerprint.

Cite this