Hand gestures are currently considered one of the most accurate ways to communicate in many applications, such as sign language, controlling robots, the virtual world, smart homes, and the field of video games. Several techniques are used to detect and classify hand gestures, for instance using gloves that contain several sensors or depending on computer vision. In this work, computer vision is utilized instead of using gloves to control the robot's movement. That is because gloves need complicated electrical connections that limit user mobility, sensors may be costly to replace, and gloves can spread skin illnesses between users. Based on computer vision, the MediaPipe (MP) method is used. This method is a modern method that is discovered by Google. This method is described by detecting and classifying hand gestures by identifying 21 three-dimensional points on the hand, and by comparing the dimensions of those points. This is how the hand gestures are classified. After detecting and classifying the hand gestures, the system controls the tracked robot through hand gestures in real time, as each hand gesture has a specific movement that the tracked robot performs. In this work, some important paragraphs concluded that the MP method is more accurate and faster in response than the Deep Learning (DL) method, specifically the Convolution Neural Network (CNN). The experimental results shows the accuracy of this method in real time through the effect of environmental elements decreases in some cases when environmental factors change. Environmental elements are such light intensity, distance, and tilt angle (between the hand gesture and camera).The reason for this is that in some cases, the fingers are closed together, and some fingers are not fully closed or opened and the accuracy of the camera used is not good with the changing environmental factors. This leads to the inability of the algorithm used to classify hand gestures correctly (the classification accuracy decrease), and thus response time of the tracked robot's movement increases. That does not present possibility for the system to determine whether the finger is closed or opened.
Abstract
Much attention has been paid for the use of robot arm in various applications. Therefore, the optimal path finding has a significant role to upgrade and guide the arm movement. The essential function of path planning is to create a path that satisfies the aims of motion including, averting obstacles collision, reducing time interval, decreasing the path traveling cost and satisfying the kinematics constraints. In this paper, the free Cartesian space map of 2-DOF arm is constructed to attain the joints variable at each point without collision. The D*algorithm and Euclidean distance are applied to obtain the exact and estimated distances to the goal respectively. The modified Particle Swarm Optimization al
... Show MoreIn this work the effect of choosing tri-circular tube section had been addressed to minimize the end effector’s error, a comparison had been made between the tri-tube section and the traditional square cross section for a robot arm, the study shows that for the same weight of square section and tri-tube section the error may be reduced by about 33%.
A program had been built up by the use of MathCAD software to calculate the minimum weight of a square section robot arm that could with stand a given pay load and gives a minimum deflection. The second part of the program makes an optimization process for the dimension of the cross section and gives the dimensions of the tri-circular tube cross section that have the same weight of
... Show MoreA hand gesture recognition system provides a robust and innovative solution to nonverbal communication through human–computer interaction. Deep learning models have excellent potential for usage in recognition applications. To overcome related issues, most previous studies have proposed new model architectures or have fine-tuned pre-trained models. Furthermore, these studies relied on one standard dataset for both training and testing. Thus, the accuracy of these studies is reasonable. Unlike these works, the current study investigates two deep learning models with intermediate layers to recognize static hand gesture images. Both models were tested on different datasets, adjusted to suit the dataset, and then trained under different m
... Show MoreRecording an Electromyogram (EMG) signal is essential for diagnostic procedures like muscle health assessment and motor neurons control. The EMG signals have been used as a source of control for powered prosthetics to support people to accomplish their activities of daily living (ADLs). This work deals with studying different types of hand grips and finding their relationship with EMG activity. Five subjects carried out four functional movements (fine pinch, tripod grip and grip with the middle and thumb finger, as well as the power grip). Hand dynamometer has been used to record the EMG activity from three muscles namely; Flexor Carpi Radialis (FCR), Flexor Digitorum Superficialis (FDS), and Abductor Pollicis Brevis (ABP) with different
... Show MoreThe robot arm is the most popular robotic form used in industry. Thus, it is crucial to make a system programming which could controlled the movement of each part in the industrial robot to make it works properly. One of the simplest models of the robot arm is EDARM ED-7100 which has a controller to control the movement of the robot arm manually. In this study, the robot controller has been redesigned in order to improve this robot's function. The new controller system used AT89S52 microcontroller which has wire connected to the robot hand. A function has been added with this controller to improve the system of controlling and becomes better than the previous system (only manually). The functions of the new system include three mo
... Show MoreAutonomous motion planning is important area of robotics research. This type of planning relieves human operator from tedious job of motion planning. This reduces the possibility of human error and increase efficiency of whole process.
This research presents a new algorithm to plan path for autonomous mobile robot based on image processing techniques by using wireless camera that provides the desired image for the unknown environment . The proposed algorithm is applied on this image to obtain a optimal path for the robot. It is based on the observation and analysis of the obstacles that lying in the straight path between the start and the goal point by detecting these obstacles, analyzing and studying their shapes, positions and
... Show MoreAerial manipulation of objects has a number of advantages as it is not limited by the morphology of the terrain. One of the main problems of the aerial payload process is the lack of real-time prediction of the interaction between the gripper of the aerial robot and the payload. This paper introduces a digital twin (DT) approach based on impedance control of the aerial payload transmission process. The impedance control technique is implemented to develop the target impedance based on emerging the mass of the payload and the model of the gripper fingers. Tracking the position of the interactional point between the fingers of gripper and payload, inside the impedance control, is achieved using model predictive control (MPD) approach.
... Show MoreIn recent years, the positioning applications of Internet-of-Things (IoT) based systems have grown increasingly popular, and are found to be useful in tracking the daily activities of children, the elderly and vehicle tracking. It can be argued that the data obtained from GPS based systems may contain error, hence taking these factors into account, the proposed method for this study is based on the application of IoT-based positioning and the replacement of using IoT instead of GPS. This cannot, however, be a reason for not using the GPS, and in order to enhance the reliability, a parallel combination of the modern system and traditional methods simultaneously can be applied. Although GPS signals can only be accessed in open spaces, GP
... Show MoreThe control of prostheses and their complexities is one of the greatest challenges limiting wide amputees’ use of upper limb prostheses. The main challenges include the difficulty of extracting signals for controlling the prostheses, limited number of degrees of freedom (DoF), and cost-prohibitive for complex controlling systems. In this study, a real-time hybrid control system, based on electromyography (EMG) and voice commands (VC) is designed to render the prosthesis more dexterous with the ability to accomplish amputee’s daily activities proficiently. The voice and EMG systems were combined in three proposed hybrid strategies, each strategy had different number of movements depending on the combination protocol between voic
... Show More