To protect digital audio transmissions from unauthorized access and modification, this paper proposes a voice encryption scheme based on chaotic systems. The proposed approach combines multiple chaotic maps into a cryptographic scheme that is applied at the byte level. The input sound is first divided into four equal-length parts. The Rossler system, Lorenz system, Henon map, and Baker map are chaotic systems used to encode each fragment. Four segments are combined, and subsequently, a logistic map is applied to the encoded audio to introduce a flipping step, ensuring a strong spread and confusion throughout the entire signal. During the decoding phase, XOR processes and switching are systematically reversed using saved keys, restoring the original audio. The encoded and retrieved audio is evaluated using several tests to ensure the success of the encoding and retrieval process, such as the signal-to-noise ratio (SNR) test, autocorrelation, mean square error (MSE), histogram analysis, and spectrogram comparison. The results demonstrate that the proposed system is highly secure against attackers and possesses a powerful diffusion and confusion mechanism, enhancing speech communication in the field of communications. Hence, the system is suitable for multimedia applications that require an elevated level of confidentiality, including the transmission of sensitive data, digital rights protection, and military communications.