Swelling is recurring challenges, which can lead to formation damage and compromising the wellbore integrity. Synergizing nanomaterials with their distinctive properties, alongside cost-effective materials, can provide a significant approach in enhancing the treatment of wellbore stability. This study employs a green, simple, and economical approach for the biosynthesis of SiO₂/KCl/Xanthan nanocomposites (NCs), aimed at reducing swelling and improving the rheological and filtration properties of Water-Based Drilling Fluids (WBDFs). The impact of SiO₂/KCl/Xanthan nanocomposite on swelling was assessed, followed by a comprehensive investigation into fluid’s filtration and rheological properties, including apparent viscosity, plastic viscosity, yield point, gel strength, and filter cake thickness. The results confirmed that the green-synthesized NCs effectively reduced clay swelling by approximately 22.2%. Furthermore, they significantly improved the mud’s rheological characteristics, decreasing filter cake thickness and fluid loss by approximately 91.29% and 91.7%, respectively, under HPHT conditions. The findings strongly suggest that SiO₂/KCl/Xanthan NC is an effective additive for mitigating clay swelling, enhancing rheology, and reducing filtration challenges in marl rich formation.