Gastrointestinal (GI) diseases are increasing day by day due to the continuous change in people's dietary habits, as these changes play a major role in several intestinal problems. Endoscopy is a medical imaging device used to detect and diagnose gastrointestinal diseases such as esophagitis and benign tumors. Manual diagnosis consumes a long time, so there is an urgent need to use computer techniques for GI disease diagnosis accurate and fast. In this study, a Bayesian optimizer-based pre-trained model architecture (BOPMA) is proposed to improve GI disease detection. The BOPMA is concerned with adapting the Bayesian optimizer in two important aspects: improving the EfficientNetV2s architecture and fine-tuning the hyperparameters of AdamW, which is the latest version currently available in the concept of backpropagation to improve the training process. In the proposed model, the Kvasir dataset of 8000 images, which includes 1,000 images for each of the eight GI disease classes, is used for training. Several transfer learning models, such as ResNet50, InceptionV3, and Xception, are adopted for comparison. To measure the quality of the model, the accuracy measure is used, whereby the accuracy of the BOPMA reached (0.9479), while the accuracy of ResNet50, InceptionV3, and Xception is (0.7995), (0.8568), and (0.9141), respectively. Additionally, the results demonstrated that the proposed model achieved better classification accuracy compared to previous studies that adopted the same dataset (Kvasir dataset). Finally, our study adopts the issuance of a prediction model that works as a doctor’s assistant to improve diagnostic accuracy within the complex domain of gastrointestinal health, opening new avenues for clinical application and contributing to better patient outcomes.