Recognizing Human facial expressions and emotions by computer is an interesting and challenging problem. Recently there has been an increasing interest in improving the interaction between humans and computers. The face is a feature that can differentiate from person to person. Emotion is expressed through face, body gestures, and speech. Emotions through faces vary from situation to situation and person to person. The Face Detection and Emotion Recognition System automatically recognizes the faces and emotions of the persons accurately in an image. The convolutional neural network concept applied with machine learning and image processing is used in classifying universal emotions such as Happiness, Sadness, Anger, Disgust, Surprise, Fear, and Neutral. Color images that are showing the human faces are given as input to the detection system. This face emotion recognition system mainly consists of four steps. They are Image Pre-Processing, Face Detection, Facial Feature Extraction, and Emotion Recognition. Image Pre-Processing is a step to change the image in Binary or Grayscale format and resizing the image in 48x48 pixels. Face detection is a method and that is capable of verifying or identifying and capturing the frames of the faces from an image. Feature extraction is a method to identify the characteristics of the person’s face captured in the image and comparing the faces in the image whether the faces of the persons are the same or not. In the final step, Emotion recognition is trying to acquire the various expressions of emotions that a person can make through their faces to communicate with each other. In emotion recognition step it can predict the emotion of the person in any kind of images or videos. This system can detect the faces and recognize the emotion of the person accurately by considering the live feed camera images and pre-existing image and video clues.