Modern ciphers are one of the more difficult to break cipher systems because these ciphers high security, high speed, non - propagation error and difficulty in breaking it. One of the most important weaknesses of stream cipher is a matching or correlation between the output key-stream and the output of shift registers.
This work considers new investigation methods for cryptanalysis stream cipher using ciphertext only attack depending on Particle Swarm Optimization (PSO) for the automatic extraction for the key. It also introduces a cryptanalysis system based on PSO with suggestion for enhancement of the performance of PSO, by using Simulated Annealing (SA). Additionally, it presents a comparison for the cryptanalysis system results that were obtained by the proposed technique, which is called Modified PSO (MPSO) with classical PSO and GA. These algorithms can be used for reducing the number of attempts or trials of key space, which are needed to reach to the optimal solution (the exact initial setting of Linear Feedback Shift Register (LFSR)),and increase the speed of the search process to find the best solution. Based on the obtained results, these algorithms were shown to be effective at finding the optimal solution and the MPSO method operates better compared with PSO in the term of time and accuracy. Well known stream cipher systems were attacked by the two soft computing methods as the cases of study, which are Geffe , Brȕer and Shrinking stream cipher systems.
