A few decades ago, quality assessment was a significant challenge in agriculture, even with tiny amounts of agricultural products. They are subject to diseases, pesticides, and environmental variables that have led to challenges to quality and safety. The quality evaluation of these items serves a crucial function in quality control and increases productivity before consumption. This paper proposes a deep neural network for solving Indian tomato diversity problems based on the assessment of their features. This study was carried out using four different categories of south Indian cherry tomatoes, i.e., spot, BER, calyx, and non-calyx. Thirty (30) relevant features were taken from an RGB-HIS color space: textural color moments, gray level co-occurrence matrix (GLCM), and defective proportion. They were then normalized in MS-EXCEL with a Min-Max function and fed into a multilayer perception deep neural network (MLP-DNN) and a convolutional neural network (CNN), respectively. The MLP-DNN was made with ten (10) hidden layers and was used with a fine-tuned convolutional neural network (CNN) of Alex net architecture with eight (8) learned layers to do grading and detection separately using MATLAB 2018a software. The results of training, testing, and validating both networks showed that they did an excellent job of solving the two-class problem (good and bad), with an overall prediction accuracy of more than 97% and a bit of loss function in both cases. However, the algorithms can be improved and employed to detect defects and sort fruits or vegetables of all kinds in the agro-industry with a larger dataset to reduce overfitting and increase validation accuracy in the future.