Pavement crack and pothole identification are important tasks in transportation maintenance and road safety. This study offers a novel technique for automatic asphalt pavement crack and pothole detection which is based on image processing. Different types of cracks (transverse, longitudinal, alligator-type, and potholes) can be identified with such techniques. The goal of this research is to evaluate road surface damage by extracting cracks and potholes, categorizing them from images and videos, and comparing the manual and the automated methods. The proposed method was tested on 50 images. The results obtained from image processing showed that the proposed method can detect cracks and potholes and identify their severity levels with a medium validity of 76%. There are two kinds of methods, manual and automated, for distress evaluation that are used to assess pavement condition. A committee of three expert engineers in the maintenance department of the Mayoralty of Baghdad did the manual assessment of a highway in Baghdad city by using a Pavement Condition Index (PCI). The automated method was assessed by processing the videos of the road. By comparing the automated with the manual method, the accuracy percentage for this case study was 88.44%. The suggested method proved to be an encouraging solution for identifying cracks and potholes in asphalt pavements and sorting their severity. This technique can replace manual road damage assessment.