Highly Modified Asphalt (HiMA) binders have garnered significant attention due to their superior resistance to rutting, fatigue cracking, and thermal distress under heavy traffic loads and extreme environmental conditions. While elastomeric polymers such as Styrene- Butadiene-Styrene (SBS) have been extensively used in HiMA applications, the potential of plastomeric polymers, including Polyethylene (PE) and Ethylene Vinyl Acetate (EVA), remains largely unexplored. This study aims to evaluate the performance of reference binder (RB) modified with plastomeric HiMA asphalt in comparison to SBS-modified binders and determine the optimal polymer dosage for achieving an optimal balance between rutting resistance and fatigue durability. The experimental program involved modifying a base asphalt binder with SBS, PE, and EVA at dosages of 6%, 7%, and 8% by weight of binder. A comprehensive evaluation was conducted, including conventional tests (penetration, softening point, viscosity, mass loss, storage stability, and specific gravity) and rheological characterization using the Dynamic Shear Rheometer (DSR). The Multiple Stress Creep Recovery (MSCR) test was employed to assess high-temperature performance, while the Linear Amplitude Sweep (LAS) test evaluated fatigue behavior. Additionally, an Overall Desirability (OD) analysis was performed to integrate multiple performance criteria and establish a ranking for each modification. The results demonstrated that SBS-modified binders exhibited the most balanced performance, with SBS8 achieving the highest elastic recovery (52.87%) and superior fatigue life, exceeding 1,017,904 cycles at 2.5% strain. PE8 exhibited exceptional rutting resistance, reaching the lowest Jnr 3.2 value (0.0078 kPa−1); however, its limited elasticity (15.7% recovery) indicated reduced flexibility. EVA modifications demonstrated marginal improvements in fatigue resistance but failed to meet the AASHTO M332 criteria for high-traffic applications. The OD analysis ranked SBS8 as the most effective HiMA binder (OD score = 0.715), followed by SBS7 (0.588) and SBS6 (0.509). PE7 (0.354) and PE6 (0.337) demonstrated moderate performance, whereas EVA had the lowest desirability score (0.000). Based on these findings, SBS-modified binders are recommended for applications requiring a balance between fatigue and rutting resistance, whereas PE-based HiMA is more suitable for high-temperature regions where rutting is the primary concern. Further field studies are necessary to validate the long-term durability of plastomeric HiMA binders and optimize their use for specific pavement conditions.