Hydrated lime has been recognized as an effective additive used to improve asphalt concrete properties in pavement applications. However, further work is still needed to quantify the effect of hydrated lime on asphaltic concrete performance under varied weather, temperature, and environmental conditions and in the application of different pavement courses. A research project was conducted using hydrated lime to modify the asphalt concretes used for the applications of wearing (surface), leveling (binder), and base courses. A previous publication reported the experimental study on the resistance to Marshall stability and the volumetric properties, the resilient modulus, and permanent deformation at three different weather temperatures. This paper reports the second phase of the experimental study for material durability, which investigated the effect of hydrated lime content on moisture susceptibility when exposed to a freeze-thaw cycle, and fatigue life. The experimental results showed an improvement in the durability of the modified asphalt concrete mixtures. Optimum hydrated lime contents for different course applications are suggested based on the series experimental studies. Finally, the advantage of using the optimum mixtures for a pavement application is demonstrated.