Foveal and Extrafoveal Processing of Facial Features for Emotion Recognition

Abstract

This thesis focuses on the differential contributions of foveal and extrafoveal processing of facial features to the identification of facially expressed emotions. Each facial expression of the six basic emotions – happy, sad, fearful, surprised, disgusted and angry – has distinctive features that are informative for their recognition. We investigated whether: (1) foveal processing of emotion-informative features enhances emotion recognition, relative to extrafoveal processing of those features; (2) this is due to high spatial frequency (HSF) processing by the fovea; (3) eye movements target emotion-informative features when not initially fixated; (4) gaze patterns while viewing facial expressions are specific to each expression; and (5) preferential visual sampling of emotion-informative features is linked to better emotion recognition.
Across four experiments, expressions of anger, surprise, fear, sadness and disgust were presented briefly, precluding eye movements, at positions which ensured that either eye, central brow, either cheek or the mouth fell on the fovea. Enforced fixation on the mouth improved recognition accuracy for fearful, surprised and disgusted expressions and reduced misclassification of fear as surprise and disgust as anger (Experiments 1 and 2). Enforced fixation on the brow and mouth led to higher anger recognition accuracy compared to the cheeks (Experiment 2). Intensity of expressions did not modulate the effect of initially fixating on emotion-informative features on emotion recognition. There were fewer neutral responses for anger at the brow region (Experiment 3). There were also fewer neutral responses for fear and surprise at the mouth (Experiment 3). While filtering out the HSF from the fixated emotion-informative features did not affect emotion recognition, occluding them impaired it (Experiment 4). Reflexive first saccades were more often directed upwards from lower features than downwards from upper features, yet more detailed analysis showed that they did not target emotion-informative features as suggested by previous research (Experiments 1, 2 and 3). Finally, when allowed to view faces for 5s, gaze patterns reflected the distribution of emotion-informative features and there was a positive correlation between time spent looking at the mouth and disgust recognition (Experiment 2).
Overall, we found that foveal processing of some emotion-informative features improved emotion recognition, however, this was not due to HSF processing at the fovea. Additionally, instead of being guided by emotion-informative facial features, initial saccades might be guided more strongly by the centre-of-gravity effect or a general top-down knowledge of face configuration. The saccade patterns observed also suggest that observers seek to sample as much of the face space as possible in order to classify its expression.