Most intrusion detection systems are signature based that work similar to anti-virus but they are unable to detect the zero-day attacks. The importance of the anomaly based IDS has raised because of its ability to deal with the unknown attacks. However smart attacks are appeared to compromise the detection ability of the anomaly based IDS. By considering these weak points the proposed
system is developed to overcome them. The proposed system is a development to the well-known payload anomaly detector (PAYL). By
combining two stages with the PAYL detector, it gives good detection ability and acceptable ratio of false positive. The proposed system improve the models recognition ability in the PAYL detector, for a filtered unencrypted HTTP subset traffic of DARPA 1999 data set, from 55.234% in the PAYL system alone to 99.94% in the proposed system; due to the existence of the neural network self-organizing map (SOM). In addition SOM decreases the ratio of false positive from 44.676% in the PAYL system alone to 5.176% in the proposed system. The proposed system provides 80% detection ability of smart worms that are meant to invade the PAYL detector in the PAYL system alone, due to the existence of the randomization stage in the proposed system.
