The effective insulation design of the stress grading (SG) system in form-wound stator coils is essential for preventing partial discharges and excessive heat generation under pulse-width modulation excitation. This paper proposes a method to find the optimal insulation design of the SG system aimed at reducing the dielectric and thermal stresses in the machine coil. The non-uniform transmission line model is used to predict the voltage propagation along the overhang, SG, and slot regions considering the variation in the physical properties of the insulation layers. The machine coil parameters for different insulation materials are calculated by using the finite element method. Two optimization algorithms, fmincon and particle swarm optimization (PSO), are applied and compared to find the optimal thickness and material properties of each insulation layer as well as the length and location of the SG system. The results under different rise-time excitation show that the optimized geometry by using PSO can produce a higher reduction in the dielectric and thermal stresses, as well as in the maximum overvoltage along the machine coil than the original geometry and the optimized geometry using fmincon. The machine coil model is validated by means of comparisons with experimental measurements.
