A blog post by Shadi Shaheen

With the growth of Internet community, textual data has proven to be the main tool of communication in human-machine and human-human interaction. This communication is constantly evolving towards being as human as possible. One way of humanizing such interaction is to provide a framework that can recognize emotions, thereby enriching user experience. For example, in higher education institutions such as universities, many students are international. Unlike local students, internationals sometimes face unfamiliar cultures, language barriers, discrimination, etc. [1]. These difficulties can lead to poor academic experience. Emotion recognition can be used to detect such difficulties, through recognition of user emotions.

Such a framework is based either on hard or soft sensing methods, which have been traditionally used to recognize user’s emotions. Hard sensing methods relie on cues such as audio, gestures, eye gazes and brain signals [2-5]. Additional sensors may be attached to the user to provide personal physiological cues such as heart rate sensors. Soft sensing methods extract information from software. For example, calendar, email, desktop activity, and social networking interactions can all be analyzed to classify users’ emotions.

Emotion detection from text consists of finding relations between input sentence and emotional content. Intuitively, finding these relations relies first on finding specific terms (emotional keywords, verbs, nouns, etc.) in the input sentence, and, second, on inferences derived from the meaning of the sentence. Once this relation is found, it can be generalized and considered as an “emotion recognition rule”. For example, from the sentence “I received many gifts on Christmas Eve”, we can infer that the user is happy, and generalize the rule: “receiving gifts” reflects the emotion “happiness”.

Several emotional models are used when building emotion recognition system. The emotional model is an important component of any emotional recognition system as it is used in determining emotions representation. The most common models are (1) the Ekman emotion model by Ekman [6] and (2) the dimensional model of emotions using valance and arousal by Russell [7]. Ekman emotion model includes six basic emotions: happiness, sadness, anger, fear, disgust and surprise. Russell’s dimensional model for emotions can be represented as a dimensional space, where valence refers to whether an emotion is positive or negative, and arousal quantifies the energy of the emotion. Every emotion is therefore an area in this space.

Furthermore, four psychological emotion models exist [8]:

(1) Basic Emotion models assume that there are a limited number of biological states (emotions) unique in function, form and cause. Emotions are considered to be universal and caused by dedicated mechanisms.

This model can be applied in computer science by finding emotional keywords in the sentence, where finding the word “happy” in a sentence will make us assume that the affective content of the sentence is happiness.

(2) Appraisal models assume that emotions make sense of the world. In old appraisal models, appraisals are the cause of emotions. However in the new models, appraisals are considered to be constituent of emotions. For example, to be in a state of sadness is to experience loss.

This model can be applied in computer science by event extraction. For example, by extracting the event “losing something” from the input sentence, we can infer the emotional state to be sadness.

Similar to this model there is the James-Lange theory of emotion [9], where an external stimulus leads to a physiological reaction. Your emotional reaction is dependent upon how you interpret those physical reactions. For example, when seeing a grizzly bear in the woods you start trembling and your heart begins to race. From the physical reaction, you infer that you are in a state of fear.

This theory can be mapped into computer science by looking for the cause of the emotional state in the sentence.

(3) Psychological Construction models consider emotions to be a construction of more basic psychological processes. Previous experiences, language and executive functioning (e.g. crying when extremely happy) are such basic processes.

This model can be applied in computer science by considering the personality traits of the person the emotions of whom is being classified. In other words, the emotion recognition approach is personal to each user.

(4) Finally, in Social Construction models, social context plays a big part in generating the emotion. The social world is considered to be constructed by people, rather than nature.

This model can be applied in computer science by considering social events related to the user and the relationships with other people while classifying the emotion.

In our ongoing work [10], we focus on classifying emotions based on the basic emotions models and the appraisals models. This is mainly because we are interested in extracting the emotions present within the sentence, not the emotions of the reader or writer. We propose a framework for emotion classification using Ekman model in English sentences where emotions are treated as generalized concepts extracted from the sentences. We start by generating an intermediate emotional data representation of a given input sentence based on its syntactic and semantic structure. We then generalize this representation using various ontologies such as WordNet and ConceptNet, which results in an emotion seed that we call an emotion recognition rule. Finally, we use a suite of classifiers to compare the generated emotion recognition rule with a set of reference emotion recognition rules extracted from a training set in a similar fashion.

We tested our classifier using different training sets. Our classifier outperformed the state-of-the-art machine learning and rule based classifiers and showed some encouraging results with an average F-Score of 76% in textual emotion classification and with 85% when excluding the emotion “surprise” from the dataset. The results also show the importance of syntactic and semantic analysis in emotion recognition from text.
[1] Chataway, C. J., & Berry, J. W. (1989). Acculturation experiences, appraisal, coping and adaptation: A comparison of Hong Kong Chinese, French, and English students in Canada. Canadian Journal of Behavioural Science, 21, 295–309.

[2] Khalili & M.H. Moradi, “Emotion recognition system using brain and peripheral signals: Using correlation dimension to improve the results of eeg,” in Proc. Int. Joint Conf. on Neural Networks, 2009, pp. 1571–1575.

[3] F. Cohn & G. S. Katz, “Bimodal expression of emotion by face and voice,” Proc. 6th ACM international conference on Multimedia: Face/gesture recognition and their applications, ACM, 1998, pp. 41- 44, doi:10.1145/306668.306683.

[4] C. De Silva & P. C. Ng, “Bimodal emotion recognition,” Proc. 4th IEEE International Conference on Automatic Face and Gesture Recognition, IEEE Computer Society, 2000, pp. 332-335, doi: 10.1109/AFGR.2000.840655.

[5] Yanaru, “An emotion processing system based on fuzzy inference and subjective observations,” Proc. 2nd New Zealand Two-Stream international Conference on Artificial Neural Networks and Expert Systems, IEEE Computer Society, Nov. 1995, pp. 15-20, doi: 0.1109/ANNES.1995.499429.

[6] Ekman, P. (1999) Basic emotions. In T. Dalgleish and T. Power (Eds.) The handbook of cognition and emotion. Pp. 45-60. New York.: John Wiley & Sons.

[7] A. Russell. “A Circumplex Model of Emotions”. Journal of personality and social psychology, 39(6):1161, 1980.

[8] Gross, J. J., & Barrett, L. F. (2011). “Emotion generation and emotion regulation: One or two depends on your point of view”. Emotion Review, 3, 8–16.

[9] Cannon, Walter (December 1927). “The James-Lange Theory of Emotions: A Critical Examination and an Alternative Theory”. The American Journal of Psychology 39: 106–124.

Shadi Shaheen, Wassim El Hajj, Hazem El Hajj & Shady Elbassuoni. “Emotion Recognition from Text Based on Automatically Generated Rules”. 4th edition of Sentiment Elicitation from Natural Text for Information Retrieval and Extraction (SENTIRE), the IEEE ICDM workshop series on opinion mining.

Photo credit: www.lifemoodscoaching.com