Explainable Artificial Intelligence (XAI) techniques enable transparency and trust in automated visual inspection systems by making black-box machine learning models understandable. While XAI has been widely applied, prior reviews have not addressed the specific demands of industrial and medical inspection tasks. This paper reviews studies applying XAI techniques to visual inspection across industrial and medical domains. A systematic search was conducted in IEEE Xplore, Scopus, PubMed, arXiv, and Web of Science for studies published between 2014 and 2025, with inclusion criteria requiring the application of XAI in inspection tasks using public or domain-specific datasets. From an initial pool of studies, 75 were included and categorized into post-hoc and intrinsic, which were then evaluated with respect to fidelity, robustness, complexity, and localization accuracy. Results show that gradient- and propagation-based methods offer efficient visual explanations suitable for near real-time inspection, though with coarse localization, while perturbation-based and surrogate-model methods provide more detailed attributions at higher computational cost but with reduced robustness. In addition, prototype-based networks and self-attention architectures illustrate trade-offs between interpretability and predictive performance. Selecting the most effective XAI method is not one-size-fits-all; it depends on the dataset, latency, and interpretability needs. This review introduces a unified taxonomy of XAI methods for visual inspection, compares different approaches in both industrial and medical domains using standardized metrics, and proposes a task-based selection workflow to guide practitioners in choosing the optimal method. Compared to prior reviews, this work offers a cross-domain comparative analysis grounded in quantitative benchmarks and outlines directions for standardized evaluation and user-centered validation.