The general health of palm trees, encompassing the roots, stems, and leaves, significantly impacts palm oil production, therefore, meticulous attention is needed to achieve optimal yield. One of the challenges encountered in sustaining productive crops is the prevalence of pests and diseases afflicting oil palm plants. These diseases can detrimentally influence growth and development, leading to decreased productivity. Oil palm productivity is closely related to the conditions of its leaves, which play a vital role in photosynthesis. This research employed a comprehensive dataset of 1,230 images, consisting of 410 showing leaves, another 410 depicting bagworm infestations, and an additional 410 displaying caterpillar infestations. Furthermore, the major objective was to formulate a deep learning model for the identification of diseases and pests affecting oil palm leaves, using image analysis techniques to facilitate pest management practices. To address the core problem under investigation, the GoogLeNet deep learning approach was applied, alongside various hyperparameters. The classification experiments were executed across 16 trials, each capped at a computational timeframe of 10 minutes, and the predominant duration spanned from 2 to 7 minutes. The results, particularly derived from the superior performance in Model 4 (M4), showed evaluation accuracy, precision, recall, and F1-score rates of 93.22%, 93.33%, 93.95%, and 93.15%, respectively. These were highly satisfactory, warranting their application in oil palm companies to enhance the management of pest and disease attacks.