Artificial intelligence (AI) algorithms depend on different algorithms to track moving objects. The surrounding environment, like lightness, affects detection accuracy. A new algorithm is designed to detect moving objects in real time automatically and calculate the speed of tested objects based on the deep learning algorithm RCNN using MATLAB software. The suggested system consists of a phone camera, four balls in different colors (red, green, blue, and black), and various environmental lights consisting of eight lights. Two luxmeters were used to check the light intensity around the ball and the camera phone. Detecting moving objects has gained a lot of desirability because of its applications like video surveillance, person movement tracking, traffic investigation, and security systems like security systems or surveillance. are four parameters used to evaluate the performance of the algorithms and the system setup: accuracy, average time, detection percentage, and speed. Results show that the quality of detecting and tracking the ball is almost 100%.