Recognizing facial expressions and emotions is a basic skill that is learned at an early age and it is important for human social interaction. Facial expressions are one of the most powerful natural and immediate means that humans use to express their feelings and intentions. Therefore, automatic emotion recognition based on facial expressions become an interesting area in research, which had been introduced and applied in many areas such as security, safety health, and human machine interface (HMI). Facial expression recognition transition from controlled environmental conditions and their improvement and succession of recent deep learning approaches from different areas made facial expression representation mostly based on using a deep neural network that is generally divided into two critical issues. These are a variation of expression and overfitting. Expression variations such as identity bias, head pose, illumination, and overfitting formed as a result of a lack of training data. This paper firstly discussed the general background and terminology utilized in facial expression recognition in field of computer vision and image processing. Secondly, we discussed general pipeline of deep learning. After that, for facial expression recognition to classify emotion there should be datasets in order to compare the image with the datasets for classifying the emotion. Besides that we summarized, discussed, and compared illustrated various recent approaches of researchers that have used deep techniques as a base for facial expression recognition, then we briefly presented and highlighted the classification of the deep feature. Finally, we summarized the most critical challenges and issues that are widely present for overcoming, improving, and designing an efficient deep facial expression recognition system.
