The control of an aerial flexible joint robot (FJR) manipulator system with underactuation is a difficult task due to unavoidable factors, including, coupling, underactuation, nonlinearities, unmodeled uncertainties, and unpredictable external disturbances. To mitigate those issues, a new robust fixed-time sliding mode control (FxTSMC) is proposed by using a fixed-time sliding mode observer (FxTSMO) for the trajectory tracking problem of the FJR attached to the drones system. First, the underactuated FJR is comprehensively modeled and converted to a canonical model by employing two state transformations for ease of the control design. Then, based on the availability of the measured states, a cascaded FxTSMO (CFxTSMO) is constructed to estimate the unmeasurable variables and lumped disturbances simultaneously in fixed-time, and to effectively reduce the estimation noise. Finally, the FxTSMC scheme for a high-order underactuated FJR system is designed to guarantee that the system tracking error approaches to zero within a fixed-time that is independent of the initial conditions. The fixed-time stability of the closed-loop system of the FJR dynamics is mathematically proven by the Lyapunov theorem. Simulation investigations and hardware tests are performed to demonstrate the efficiency of the proposed controller scheme. Furthermore, the control technique developed in this research could be implemented to the various underactuated mechanical systems (UMSs), like drones, in a promising way.