In 1872, Charles Darwin proposed that facial expressions are evolved behaviors that have a biologically adaptive function  . Nearly one hundred years later, Paul Ekman defined six universal facial expressions that are recognized and expressed cross-culturally―happy, sad, fear, disgust, surprise, and anger  . Facial expressions are indicators of underlying emotional states  , and recognition of the emotion in facial expressions is critical to interpersonal communication, response to imminent threat, and social behavior  . Within a fraction of a second, humans can recognize emotion, sex, relative age, race, and identity, just by viewing a face   . There is substantial evidence that the emotion of facial expressions can even be processed in absence of conscious awareness  . Functional neuroimaging studies have indicated that non-consciously perceived emotional facial expressions can activate subcortical structures   . De Gelder et al. reported on a blind sight patient who could discriminate among facial expressions presented in his blind field  . A study by Esteves et al. also demonstrated that, under conditions that prevent conscious awareness, facial expressions elicit skin conductance responses  . While most studies indicate that emotional facial expressions can induce neurophysiological changes and influence behavior without being consciously perceived, there is accumulating evidence that affective processing does not occur outside of awareness and that any unconscious reaction is dependent on prior semantic processing     . These studies suggest that sematic categorization of a visual stimulus occurs faster than affective evaluation, and that the semantic categorization of a stimulus may be required for affective processing to occur.
Major contributions to the evidence that affective information can be processed without awareness come from affective priming experiments. The affective primacy hypothesis   proposes that affective information can be accessed automatically and precedes conscious cognition. Subliminal affective priming experiments have provided both neural and behavioral evidence that affective stimuli are unconsciously processed     . Subliminally presented affective information has also been shown to influence the evaluation of emotionally ambiguous facial expressions  .
A major effect of priming is that response times are faster when the prime has the same emotional valence as the target stimulus and slower when it is different  . This is referred to as the congruency effect. Many subliminal priming experiments have shown that affectively congruent prime-target pairs facilitate either response time or evaluation of the target stimulus     , however these studies have all employed affective words as their primes. Attempts to replicate the congruency effect using affective images as primes have shown less conclusive results. Using prime images with fear or disgust inducing content, as well as fearful or disgusted facial expressions, Neumann et al. reported faster response time to prime-target pairs with congruent emotions  . On the other hand, Hermans et al. reported the opposite effect on response time (affectively congruent trials were slower than incongruent trials) using pleasant or unpleasant priming images, some of which contained faces  , and Andrews et al. reported lack of congruent affective priming for subliminal trials using happy or angry facial expressions  .
The discrepancy between these studies may be due to the content and type of the prime and target stimuli used. A large number of affective subliminal priming studies employ facial expressions as stimuli   -  due to their innate association with emotions. However, there are significant differences between affective stimuli that contain faces versus those that do not. There is solid evidence that faces are processed separately from non-face stimuli    . Seeing as there are distinct differences between the processing of faces and non-faces, it is possible that the emotion in face and non-face stimuli is also processed differently. However, in most published studies, there is little distinction between emotional primes that contain faces and those that do not. Additionally, the relevance of the priming stimuli to the observer is frequently overlooked. The content of an affective priming image can be directed towards the observer, engaging him/her in the image, or can portray a scene that does not engage the observer at all. Priming images that are more directly relevant to the observer may more effectively capture the attention of the observer over priming images that portray non-relevant scenes. We suggest that the discrepancies between the results of different affective image priming studies may be attributed to the visual characteristics of the priming image stimuli which are often overlooked in the prime image selection criteria.
In this paper, using the same experimental protocol as in our previous study  , we investigated how the affective content of a priming image, the presence of a face or the vantage point of the prime content (image centricity) impact the effect of affective subliminal priming on the identification of target-face expressions (neutral or angry) in young healthy participants.
The experimental design was adapted from that of our previously published paper  . Here our aim was to investigate the effects of different classes of priming images on perceiving facial expressions in young healthy participants.
25 high-school senior and 1st - 2nd year undergraduate students (12 females, mean age = 18.17 years, SD ± 1.49) volunteered for this study. All participants were right handed, had normal or corrected to normal vision, and none had a history of neurologic, psychiatric, or developmental disease. All participants gave written consent before the start of the experimental sessions in accordance with Boston University’s Institutional Review Board Committee on research involving human participants. All data were collected at the Brain and Vision Research Laboratory, Boston University, Boston, Massachusetts.
All tests were written in, responses collected in, and stimuli controlled by the BraviShell toolbox, a MATLAB (MATLAB and Statistics Toolbox Release 2009a, The Math Works, Inc., Natick, Massachusetts, United States) software package developed in the Brain and Vision Research Laboratory (Biomedical Engineering Department, Boston University, Boston, MA, 2005-2015), which is based off the Psychophysics Toolbox extensions    . All tests were administered using a 13-inch Apple MacBook Laptop with Retina Display. Participants were administered three tests in the following order: SAFFIMAP (Subliminal Affective Image Priming), followed by PV (Prime Validation), and then by the FEP (Face Emotion Processing) tests which are described below.
2.3. Test Stimuli
The stimuli in the Subliminal Affective Image Priming (SAFFIMAP) task consisted of priming images, masks, and target-face images. All priming images were selected from the International Affective Picture System (IAPS) database  or from public domain images available on the internet. Indices of the images selected from the IAPS database are as follows: aggressive: 12, 101, 163, 257, 1653, 1790, 1865, 1886, 1999, 2214, 2217, 2227, 2244, 2248, 2258; pleasant: 7, 18, 57, 58, 64, 124, 224, 455, 516, 538, 541, 726, 730, 743, 765, 775, 796, 815, 887, 896, 956, 969, 991, 1190, 1318, 1320, 1527. We chose 120 total images, with 40 images of each three different affective valences: (aggressive, pleasant, or neutral). The priming stimuli were further characterized by two additional attributes: centricity and the presence or absence of a face. Centricity refers to whether the vantage point of the image was egocentric (the content of the image was directed towards the observer) or allocentric (the content of the image was directed away from the observer). Half of the priming images were egocentric, and half were allocentric. The face content category refers to whether the priming image contained a face (60 images each). These subdivisions were spread evenly among the affective categories (10 images per each affectivity/centricity/face group). All images were initially categorized by their affective valence, centricity, and face-content by three independent evaluators, who were age and education matched to the subjects. Only images with agreeing categorizations were used as priming stimuli. Public domain images were selected from the internet where there were insufficient images in the IAPS database to fulfill our specific categories. Examples of each category of priming image are shown in Figure 1(a). The images were cropped and resized to 512 × 512 pixels and then normalized to a constant luminance and contrast using the SHINE toolbox  .
A separate, control, group of five age and education matched participants, reported conscious awareness of the priming image at prime durations longer than 16 ms. Therefore, in the test described here we chose the duration of prime to be 16 ms.
The mask following the priming image was generated by dividing the priming image into 16 × 16 pixels blocks (0.78 × 0.78 degrees), and randomly shuffling these blocks in both the horizontal and vertical positions. The result of this process was a scrambled version of the priming image, where the local image information remained the same, but the meaningful content of the image was destroyed. An example of a priming image mask is shown in Figure 1(a).
The target-face stimuli were pictures of unfamiliar faces portraying either an angry or a neutral facial expression. We selected 28 pictures of each facial expression (angry, happy, and neutral) from the Karolinska Directed Emotional Faces (KDEF) database   . The angry and neutral target-faces were used as the target images for the SAFFIMAP test, and the happy target-faces were included in the FEP task. The face images were cropped and normalized similarly to the prime images. To make the facial expression discrimination task more challenging, we further divided the face pictures into 8 × 8 pixel blocks (23.4 arc minutes in height and width) and replaced 50% of the blocks with a color whose RGB (red, green, blue) components were sampled randomly from a normal distribution with mean and covariance equal to that of the RGB components of the full image. Examples of target-face images are shown in Figure 1(b).
Figure 1. Stimulus details. (a) Example of an aggressive, egocentric, face prime image with its corresponding mask, an example of a neutral, allocentric, face prime, and an example of a pleasant, egocentric, non-face prime. (b) Example of an angry (left) and neutral (right) target face.
Participants were seated in a dimly lit room, 60 cm from the display screen. They were instructed to maintain fixation on a red square (0.3 deg diameter) displayed at the center of the computer screen throughout the duration of the stimulus presentation and to respond via keypresses on the computer keyboard. The participants were instructed to respond as quickly and as accurately as possible. In total, completion of all three tests took approximately 30 minutes. Participants were allowed a brief break between the completion of the SAFFIMAP test and the FEP/PV control tests.
Subliminal Affective Image Priming Task (SAFFIMAP)
The SAFFIMAP assesses the effect of subliminally-presented affective priming images on the perception of neutral or angry target facial expression. A schematic view of an experimental trial is presented in Figure 2(a). Prior to the start of the experiment, participants were informed that the task consisted of a brief flash of a jumbled-up image (the mask) followed by a picture of an angry or neutral face. Participants were told to focus on discriminating the emotion of the face presented at the end of the trial and to ignore the preceding stimulus. Participants were then asked to press any key on the computer keyboard to start a trial when prompted by an exclamation mark (6 degrees in height) displayed in the center of the computer screen. This was followed by a 500 ms blank screen at a neutral gray intensity (42 cd/m2) and the fixation mark which was presented throughout the trial. Next, a priming image, randomly selected from the experimental database, was shown for 16 ms and immediately followed by a mask displayed for 100 ms. Both the prime and the mask subtended 12.5 × 12.5 degrees and were presented in the same spatial location at the center of the computer screen. The target-face was displayed for 500 ms. Participants responded by pressing 1 on the keyboard if they perceived the expression of the target-face as neutral, and by pressing 2 if they perceived it as angry. The next trial began immediately after the participant’s response, or after 4000 ms, if no response was entered. There were in total 120 trials. Each contained a unique prime image and a corresponding mask, while the target-faces were randomly selected from 56 face expressions (28 angry and 28 neutral). After completing the SAFFIMAP, participants reported whether they recognized any of the priming images preceding the face images.
Face Emotion Processing (FEP) Control Test
In a three-alternative forced choice task (3AFC), participants were presented with human faces displaying angry, happy, or neutral expressions for 500 ms. The angry and neutral faces were the same as those used in the SAFFIMAP test. As the FEP test was identical to that used in our previous paper  , happy face expressions were included in the trials. In total, there were 84 unique trials with 28 trials for each of the three expressions. A schematic view of a single trial is shown in Figure 2(b). Each trial was prompted by an exclamation mark, followed by a 500 ms blank screen (at neutral gray, 42 cd/m2). After another 500 ms, the picture of the face expression to be evaluated was presented for 500 ms. Participants reported via keypress whether they perceived the face expression as angry, happy, or neutral (1 if the expression appeared to be angry, 2 if happy, and 3 if neutral). The next trial begins after the participant presses a key or after 4000 ms if no response is entered.
Prime Validation (PV) Control Test
In the PV test, the same experimental design was used to validate the priming images. In place of the target faces, participants were presented with the priming images of aggressive, pleasant, or neutral affectivity. There were 120 unique trials with 40 trials per each affective category. Participants reported via keypress whether they perceived the image as aggressive, pleasant, or neutral (1 if the image appeared to be aggressive, 2 if pleasant, 3 if neutral).
To confirm the validity of the categorization of the priming images used in the SAFFIMAP experiment and the participants’ ability to recognize facial expressions, all participants were administered PV and FEP as control tests.
Figure 2. Timelines of test trials. (a) Timeline of a single trial in the SAFFIMAP test. (b) Timeline of a single trial in the FEP test. The Prime Validation task follows an identical timeline, replacing the target-face with the priming images.
2.5. Statistical Analysis
The dependent measures of interest in the current study were participants’ reaction time (RT) and accuracy for target face categorization. Note that only RTs from correctly answered trials were included in the analysis. A 2 (Prime Face: Absent, Present) × 3 (Prime Affect: Aggressive, Pleasant, Neutral) × 2 (Prime Centricity: Egocentric, Allocentric) × 2 (Target Face: Angry, Neutral) repeated measures analysis of variance (ANOVA) was performed on the RT and accuracy data, respectively. When appropriate, post-hoc pairwise comparisons were performed using Bonferroni correction for multiple comparisons.
After completing the SAFFIMAP test all participants reported verbally that they were unaware of the priming images. No subject exceeded the 4000 ms response time limit. The mean percent correct (n = 29) for the PV and FEP control tests were 90.4% ± 11.1% and 93.6% ± 5.29% respectively. Four participants (3 females) were removed from further analysis due to poor performance (less than 79.0% accuracy) on the PV control test.
3.1. RT Analysis
The ANOVA performed on the RT data showed a significant interaction between Prime Face and Target Face, F(1, 21) = 14.802, MSE = 0.01, p = 0.001, = 0.413. Paired-samples t-tests indicated slower RTs when neutral target faces were preceded by face primes than when they were preceded by non-face primes (p < 0.05, Bonferroni corrected). In contrast, faster RTs were observed when angry target faces were preceded by face compared to non-face primes (p < 0.05, Bonferroni corrected). No other significant main effects or interactions were found (all p-values > 0.1). The mean RTs for the experimental conditions collapsed across Prime Affect and Prime Centricity were shown in Figure 3.
Figure 3. Mean RTs for the combinations of Prime Face and Target Face conditions. Error bars represent standard errors of the mean (SEM). *p < 0.05.
3.2. Accuracy Analysis
The overall average performance (n = 25) for the SAFFIMAP test was 76.1% ± 7.16%. The ANOVA performed on the accuracy data showed a significant main effect of Target Face, F(1, 21) = 5.631, MSE = 0.196, p = 0.027, = 0.211, with more accurate responses to neutral (M = 0.79 , SEM = 0.04) than to angry (M = 0.70, SEM = 0.05) target faces. The interaction between Prime Face and Prime Centricity was also significant, F(1, 21) = 6.665, MSE = 0.039, p = 0.017, = 0.241. Paired-samples t-tests revealed that participants’ categorization performance was worse when a priming image contained an egocentric face compared to when it contained either an allocentric face or an egocentric non-face (both p-values < 0.05, Bonferroni corrected). No other significant main effects or interactions were observed (all p-values > 0.08). Figure 4 shows the accuracy for different experimental conditions collapsed across Prime Affect and Target Face.
4. General Discussion
Our first notable finding is that subliminal priming with a face-containing image significantly quickens the participants’ responses to an angry target-face, compared to priming with a non-face containing image. This effect did not occur when the target-face was neutral, nor did it depend on the affective content of the prime image itself. This suggests that the presence of a face in the priming
Figure 4. Mean accuracy for the combinations of Prime Face and Prime Centricity conditions. Error bars represent SEM. *p < 0.05.
image, independent of its affective content, heightens the observer’s attention to the emotional expression of the target face. Figure 3 shows that when the neutral target-face is primed with a face-containing priming image, the observer’s response is significantly slower than when the priming image does not contain a face. However, when the angry target-face is primed with a face-containing priming image, the observer’s response is significantly quickened compared to when the priming image does not contain a face. The face prime may prepare the observer for an affective target image, which when validated by an angry target face results in a quickened response. In the case of a neutral target-face, we suggest that any face present in the prime image is prepares the viewer for an affective percept, thus when a neutral rather than an affective, angry target-face appears, the response is slowed.
Furthermore, we showed that there was a significant effect of the centricity of the priming image on the evaluation of the target face expression, depending on whether the priming image contained a face. When the priming image contained a face and the content of the prime was directed towards the observer (egocentric), the participant’s performance was significantly poorer on determining the facial expression of the target-face compared to when the priming image did not contain a face, or when the content of the prime was directed away from the observer (allocentric). This suggests that the egocentric face-containing prime engaged the observer and detracted from his/her perception of the target face expression. There may be an inhibitory mechanism involved in the subconscious perception of a relevant stimulus that dampens the observer’s attention to the facial expression of a consciously perceived face. This finding is supported by studies which demonstrate that egocentric faces are able to capture attention preferentially to other non-face stimuli   . There is, however, little information on the ability of allocentric faces to grab attention, as most studies examining the ability of faces to draw attention employ forward-facing, egocentric face images as stimuli. Our findings seem to suggest that allocentric faces are less effective at capturing attention than egocentric faces, and thus when used in a priming image will not detract from the perception of the target face expression.
The affective content of the priming image had no significant effect on performance or RT, rather the face-content and centricity of the prime had the major impacts. Since RTs were quicker regardless of the affective valence of the prime, our results do not support an affective congruency effect. In order to investigate the question of affective congruency, future studies should be more selective of their priming image stimuli. Our results showed a significant effect of the face content of a priming image and the image’s centricity on either response time or accuracy. This finding has important implications for future subliminal affective priming experiments which employ face or scene-containing images as their priming stimuli, as the specific visual properties and the content of a prime image may produce unintended effects which could skew the results of the study. Further studies of what visual information can be processed subconsciously in a masked priming context are necessary to ensure the validity of future experiments. Due to the breadth of the subliminal priming image categories evaluated in this study, there were limited numbers of stimuli per specific affect/face/centricity category in order to keep the duration of the SAFFIMAP test within a reasonable amount of time. This limitation reduced our ability to evaluate more priming and target image categories, which should be included in future studies. In particular, it would be interesting to examine whether more distinct visual characteristics of face-containing prime images, such as gender or race, may subconsciously affect the conscious perception of the emotion displayed in the target face image. Additional image features, such as color, saturation, or content complexity, which were not investigated in this report, may also have significant effects on subliminal affective priming. Our results may also extend beyond the pleasant and aggressive primes used in this study to other frequently employed affective priming stimuli, such as those evoking fear and disgust, which will need to be investigated in future studies alongside additional target face expressions (ex. happy, fearful, disgusted).
The authors thank the subjects for participating in this research.