PSYCH  Vol.10 No.2 , February 2019
The Differences in Semantic Processing of Pictures and Words between Dyslexic and Typical-Reading University Students
Author(s) Shelley Shaul*, Ziva Rom
ABSTRACT
The present study compared the semantic processing of pictures and words by dyslexics to that of typical readers utilizing the electro-physiological (ERP) technique during a semantic categorization decision task. ERPs of 40 university students, 20 typical readers and 20 dyslexic readers were recorded while they participated in a categorization decision task. The subjects were presented with two kinds of stimuli—words and pictures. Results revealed longer reaction times in response to words as compared to pictures in both groups. Electrophysiological measures revealed differences in amplitudes and latencies of ERP components in addition to differences in the mean activity. The results illustrated that the differences between processing words and processing pictures were manifested in timing and in the brain areas involved in the tasks. It also illustrated that although dyslexic and typical readers displayed equal ability to read and understand familiar words, dyslexics could not perform as quickly as typical readers. The semantic domain may be one of the compensatory mechanisms which help compensated dyslexic readers reach and succeed in the higher education system.

1. Introduction

Any visual stimulus, such as words and pictures, is a symbol that represents the meaning of an object, an event or a relationship (Ganis, Kutas, & Sereno, 1996). In everyday life, we constantly encounter pictures and words that represent meaning and unconsciously we use semantic processing to comprehend their meaning. Our brains store a lot of information gathered through life’s experiences and this knowledge is grouped into things with common perceptual and functional features. The information that is quickly retrieved from this storage of knowledge is known as the “semantic memory” (Kutas & Federmeier, 2000). Because semantic memory allows us to understand the meanings of words and pictures, it has an important role in human cognition and communication (Pobric, Jefferies, & Lambon Ralph, 2010; Rogers et al., 2004; Visser, Jefferies, & Lambon Ralph, 2009).

Words and pictures are both physical objects that symbolize the information that they indirectly represent. As such, we can use pictures and words in the same ways and we tend to respond to them in the same manner (Federmeier & Kutas, 2001). There are two main types of models which try to explain how we process words and pictures: 1) the multiple semantic system models which hypothesize that pictures and words are processed in different systems, each having a specialized semantic system. These systems can communicate one with each other but they work independently (Federmeier & Kutas, 2001; Ganis et al., 1996; Shallice, 1988); 2) the common semantic system which assumes that words and pictures have a common semantic storage. Different models allow different degrees of variance between the processing of pictures and words before they meet in the common semantic storage. However, these models assume that semantic analysis happens in a single, amodal system and that all semantic information is stored in a shared format (Bright et al., 2004; Federmeier & Kutas, 2001; Ganis et al., 1996).

The investigation of semantic processing is commonly examined using the semantic classification task, in which a subject must determine whether or not a stimulus belongs to a certain semantic category. Most studies have used verbal stimuli in order to assure that the information is provided by the meaning of words (Khateb et al., 2003; Lebovitz, 2008; Russeler et al., 2007). There are several studies that have compared the semantic classification of pictures and words in order to understand how nonverbal items are represented and classified (Bright et al., 2004; Guenther, Klatzky, & Putnam, 1980; Harmony et al., 2001; Kiefer, 2001). There are basic processes in semantic classification which can be divided into several stages; first there is attention to the perceptual information of the stimulus, especially to the physical features. Once the physical characteristics have been processed, the activation of knowledge from the semantic memory starts. The perceptual information of the stimulus may or may not match the information from the semantic memory. In the final stage of processing, the categorical decision is made (Guenther et al., 1980). Guenther et al. (1980) claimed that the nature of the perceptual information processed is not the same for pictures and words. They investigated whether pictures and words have different conceptual information which plays a role in the classification. They found that pictures were classified faster than words but words were named faster than pictures. When the stimulus and the target were semantically related-negative responses were slowed, and this effect was greater for pictures. Their results supported the hypothesis that pictures and words make contact with different sets of semantic information to make the classification decision. The researchers concluded that the two types of stimuli share a semantic memory that contributes to the decision, but the pictures add a unique pictorial, short term visual memory (Guenther et al., 1980).

In spite of the different models, all different types of visual stimuli (words and pictures) need to begin with the similar process of visual identification. One of the differences between words and objects is that words involve a highly specialized processing mechanisms compared to that of object recognition (Yum, Holcomb, & Grainger, 2011). In addition, recognition of pictures is a superior process among humans; people can accurately recognize numerous pictures that are displayed very briefly (see Intraub, 1981). Visual recognition involves brain systems that are devoted to this purpose while word recognition (reading) involves the activation and integration of different brain parts and highly specialized processing mechanisms (Yum et al., 2011).

Behavioral measures like reaction times and accuracy can be used to examine sematic processing of words and pictures, but they must take into account the visual processing and also the time required for response performance. In contrast, electrophysiological measures such as ERPs can help us examine the different processing stages of visual and semantic processing separately (Kiefer, 2001).

ERPs contain several different components that are connected to different phases of information processing (Friedrich & Friederici, 2004). N400 is a negative ERP component which peaks around 400 msec after stimulus onset; it is a sensitive index of semantic processing in adults and children (Friedrich & Friederici, 2004). There are additional ERP components which are related to language and visual processing: N100―a negative evoked potential with a typical latency of about 100 msec. which is thought to index the initiation of an attention-like process ( Shaul, 2008 in: Breznitz, 2008); P600―has two different interpretations, one associated with memory processes and another related to language (Key et al., 2005).

Based on this, the current study will examine semantic processing by using ERP’s in order to better understand the whole process and the brain activity which underlies semantic processing.

Several studies have examined semantic processing with ERP (see Barrett & Rugg, 1990; Coch, Maron, Wolf, & Holcomb, 2002; Harmony et al., 2001; Khateb et al., 2010; Vandenberghe et al., 1996; Yum et al., 2011). These researches have demonstrated differences in processing pictures and words. The question the current study will try to answer is whether there is a difference in semantic processing among dyslexic as compared to typical readers in processing words and pictures?

About 4% - 10% of children have difficulties in the acquisition of literacy, reading and writing, due to developmental dyslexia (Helenius et al., 1999). “Dyslexia is a specific learning disability which has a neurobiological origin. It is characterized by difficulties with accurate and/or fluent word recognition and by poor spelling and decoding abilities” (Lyon, Shaywitz, & Shaywitz, 2003). For many years, the assumption was that the core deficit of dyslexia is phonological processing. Development dyslexia is a heterogeneous condition and it has been found that dyslexics have a difficulty in other cognitive operations such as visual processing (orthographic processing), lexical or semantic access (Helenius et al., 1999; Russeler et al., 2007), short-term verbal memory and long-term verbal memory problems (Miller-Shaul, 2005) and speed of processing (Breznitz, 2008).

Several studies demonstrated differences between dyslexic and typical readers on the N400 component during semantic tasks (e.g., Brandeis, Vitacco, & Steinhausen, 1994; Lovrich et al., 1997; Neville, Coffey, Holocomb, & Tallal, 1993; Russeler et al., 2007). These experiments proposed that adult dyslexics also have difficulty in the semantic processes. Several studies demonstrated differences between dyslexic and typical readers on other ERPs components; Mayseless (2007) reported differences in P100, N100 and P200 component between dyslexics and typical readers during a semantic task. Several studies demonstrated differences in P600 between dyslexics and typical readers (e.g., Russeler, Probst, Johannes, & Munte, 2003; Shaul, 2005).

There are not many electrophysiological studies on semantic processing among dyslexic readers, and these few studies suggest mixed results. Thus, there is no agreement regarding the difficulty in semantic processing among dyslexics and it certainly has not been examined at different levels. Therefore, the aim of the current study is to investigate the semantic processing linguistic (words) and non-linguistic stimuli (pictures) in order to understand the underlying factors of semantic processing in general and where the breakdown is among the dyslexic readers. Our questions in this study were:

1) Are there differences between processing words and processing pictures in timing and patterns of activation? We hypothesize that there would be different patterns of brain activation in processing words and processing pictures, due to the different brain systems involved in these types of processing.

2) Are there differences in brain activity between dyslexic students and typical readers while processing semantic information of word and pictures? Furthermore, are these differences larger in words? We hypothesize that differences between the dyslexic students and the typical readers will be found while processing pictures and words with a larger gap between the groups in the verbal task due to the word identification difficulties involved in reading.

2. Method

2.1. Participants

40 university students participated in the study―20 typical readers and 20 dyslexic readers, aged 18 - 31, mean age 25.7 years, right handed. All the students were native speakers of Hebrew, matched for age and general ability. The dyslexic readers were recruited from the University of Haifa Student Support Service (“Yahel”); they had been diagnosed as dyslexic in childhood and classified as impaired readers as adults. The criterion of dyslexia was at least one standard deviation below the average in oral reading (speed and accuracy) of words and pseudowords. The typical readers were recruited through advertisements posted throughout the university. All participants were paid for their participation and had no additional neurological deficits such as attention disorders (according to self report and the DSM-VI attention questioner).

2.2. Behavioral Baseline Assessments

In order to validate sample selection and examine the connection between semantic processing and reading related skills, we collected the following measures:

1) General ability―Block design and vocabulary sub-tests from the WAIS-R (Wechsler, 1981).

2) Reading skills.

a) One-minute word and pseudoword reading (Shatil, 1995).

b) Comprehension―Reading comprehension test (Center for Psychometric Tests, 1994).

c) Reading Speed Test-RST (Kornith, 2009 base on Wimmer & Hutzler), this test consists of 77 easy sentences. The test requires marking within a 3 min time constraint as many of these sentences as possible as either correct (e.g. “A cell phone is useful whenever you want to make a call”) or incorrect (e.g. “The moon was colonized by men about 200 years ago”). Reading speed is measured as the number of correctly marked sentences within the 3-minute interval.

2.3. Instrumentation

The EEG (electroencephalogram) was recorded with a BioSemi Active Two system with 64 active electrodes using Active View as recording software. The electrodes of the CMS/DRL circuit were located at occipital regions next to the POz electrode. Three additional external electrodes were used to monitor eye movements. Two of these electrodes were placed on either side of the participants’ eyes to observe horizontal eye movements. The third external electrode was placed below the left eye in order to monitor vertical eye movements and blinks. The analog raw data was digitalized with a 24-bit AD converter at a sampling rate of 2048 Hz without the application of any filter.

2.4. Experimental Task

Two hundred and forty stimuli―forty black and white drawings of animals, forty black and white drawings of non-living objects and forty drawings of different kinds of food, forty corresponding animals’ names, forty corresponding non-living object’s names and forty corresponding food names. Theses stimuli were presented to the subjects in a categorization task. All words were equal in length―3 - 5 letters and were of moderate frequency, and all pictures were the same size. An example of the stimuli is presented in Figure 1.

Subjects sat in front of a computer screen wearing an EEG cap. All stimuli were displayed in the center of the computer screen. The stimulus was displayed for 500 msec and the subject was requested to decide as quickly and as accurately as possible if the target was a member of the animal category or not. In a second experiment, the subject was asked to decide whether stimuli were a non-living object or not. In both experiments, the subject was asked to respond to the target category by pressing one joystick button and another button for the non-target. The SOA between each trial was 2000 msec.

There were two blocks, in the first block. The stimuli were words and in the second block the stimuli were pictures. The blocks were randomly presented to the subjects.

Reaction time and accuracy data were collected from each task, in addition to recording the brain activity.

2.5. ERP Data Analysis

ERPs were analyzed offline using the Brain Vision Analyzer software (Brain-products). The EEG data were filtered (high: 25 Hz and low: 0.1 Hz), and referenced to the common average of all electrodes. Ocular artifacts were corrected for both horizontal and vertical eye movements using the Gratton, Coles, & Donchin (1983) method. Correct responses were divided into epochs of 100 ms pre-stimulus baseline and 1900 ms post-stimulus. Artifacts were rejected, the resulting data were baseline-corrected, and global field power (RMS) was calculated for each segment. All stimuli associated with the same category (words and pictures in accordance with the experimental protocol) were combined for averaging. This procedure was carried out for each subject separately.

2.6. Procedure

The testing procedure was administered to each participant over two sessions in the Edmond, J. Safra Center for the studies of learning disabilities at the University of Haifa. The study was carried out in a sound attenuated room. Each session lasted approximately 1 hour. In the first session, reading level and general ability

Figure 1. Task stimuli.

were tested and only those subjects deemed suitable in accordance with the Israeli definition of dyslexia (Ministry of Education, 2005) were included in the sample. In the second session, the subjects were attached to the EEG instrumentation, and following calibration the subjects performed the different experimental tasks. Data analysis was carried out off-line.

3. Results

3.1. Behavioral Results

3.1.1. Background Measures

In an attempt to verify the subject’s group division, T tests were conducted between the two groups―typical readers and dyslexics―on the baseline measurements.

Typical readers performed better than dyslexics on all of the reading tasks. The dyslexics read less words and nonwords per minute. No significant differences were found on the IQ measures (block design and similarities tasks). The means SD, and T values for each group and are presented in Table 1.

3.1.2. Categorization Task

Reaction time (RT) and accuracy (Acc) of dyslexics and typical readers in the categorization task are presented in Table 2. In order to examine the differences in reaction and accuracy, a 2 × 2 × 2 (group dyslexic/typical readers × stimuli word/picture × target target/nontarget) repeated measures MANOVA was performed. The mean and SD of both groups in the experimental task are presented in Table 2.

Accuracy:

A significant target effect was found (F(1,34) = 11.425, P < 0.01). The identification of target words and pictures was more accurate (average of 38.13) than the non-target stimuli (average of 37.56) among both dyslexics and typical readers.

No significant difference for the group or stimuli type was found nor was there significant interaction.

Table 1. Background behavioral measures.

**P < 0.01; ***P < 0.001 (2 tails).

Table 2. Behavioral measures for the categorization task.

It is important to note that the accuracy was close to ceiling in the task with all participants performing above 90% accuracy.

Reaction time:

A main effect of stimulus type (F(1,34) = 87.951, P < 0.001) was found with longer reaction times registered in response to words (average of 780 msec), as opposed to pictures (average of 680 msec), in both groups. In addition, a significant stimulus type by group interaction (F(1,34) = 8.335, P < 0.01) was found. The difference between RT of words and of pictures was larger among the dyslexics (average of 130 msec) as compared to the typical readers (a gap of 70 msec on the average).

A target effect (F(1,34) = 75.782, P < 0.001) was found among both groups in both―words and in pictures. Target stimuli were identified faster (713 msec) than non-targets words and pictures (746 msec).

3.2. Electrophysiological Findings

In order to examine the differences in latency and amplitude of the different components, a 2 × 2 × 2 (group dyslexic/typical × type word/picture × target target/nontarget) repeated measures MANOVA was performed. Each analysis was performed for the electrodes with the highest amplitude according to the global field power of each condition. The electrodes and the time window of each component are presented in Table 3.

3.2.1. N100 Peak Latency

The latencies of the N100 peak for the different electrode sites are presented in Table 4 and the scalp distribution in Figure 2.

Results revealed a significant main effect of type (F(5,32) = 23.223, P < 0.001) with longer latencies observed for pictures (with mean latency of 103.761 msec) compared to words (with mean latency of 73.773 msec) in both dyslexics and typical readers. These differences were significant at the PO3 (F(5,32) = 41.804, P < 0.001), PO4 (F(5,32) = 52.497, P < 0.001), Pz (F(5,32) = 96.831, P < 0.001), P3 (F(5,32) = 40.547, P < 0.001), P4 (F(5,32) = 53.793, P < 0.001) electrodes locations.

Table 3. Time intervals peaks and the electrodes of each component.

Table 4. The average latency of N100 during processing words and pictures.

Figure 2. Scalp distribution of N100 component.

A significant interaction between target X group was found (F(5,32) = 2.66, P < 0.05). The gap between target and non-target was found to be larger among the typical readers (target yielded mean latency of 92.550 msec while non-target yielded mean latency of 81.563 msec; a gap of approximately 10.98 msec) compared the dyslexics readers (target yielded mean latency of 85.7 msec while non-target yielded mean latency of 87.482 msec; a gap of approximately 1.7 msec). These differences were significant at the P4 electrode location (F(5,32) = 9.060, P < 0.01).

No significant main effects of group or target were found.

3.2.2. P100 Peak Amplitude

The amplitudes of the N100 peak of the different electrodes are presented in Table 5.

There was a significant difference between the groups in amplitude (F(5,32) = 2.603, P < 0.05), with higher amplitudes evoked in dyslexics (with a mean amplitude of −1.859 μV) compare to typical readers (with a mean amplitude of

Table 5. The average amplitude of N100 during processing words and pictures.

−1.160 μV). These differences did not register at a specific electrode location, but were a general difference.

There was a marginal significant main effect of type (F(5,32) = 2.405, P = 0.058), with higher amplitudes evoked in response to pictures (with a mean amplitude of −2.046 μV) than words (with a mean amplitude of −0.973 μV). These differences registered at all electrodes locations: PO3 (F(5,32) = 8.052, P < 0.01), PO4 (F(5,32) = 11.693, P < 0.01), Pz (F(5,32) = 7.303, P < 0.05), P3 (F(5,32) = 8.166, P < 0.01) and P4 (F(5,32) = 9.930, P < 0.01).

3.2.3. N400 Latency

The latencies of the N400 peaks at the different electrode sites are presented in Table 6 and the scalp distribution in Figure 3.

Peak latency:

Results revealed a significant main effect of type (F(5,33) = 30.116, P < 0.001) with longer latencies observed for pictures (with a mean latency of 430.618 ms) compared to words (with a mean latency of 380.209 ms) in both dyslexics and typical readers. These differences were significant for the Cz (F(5,33) = 50.789, P < 0.001), FCz (F(5,33) = 84.937, P < 0.001), AFz (F(5,33) = 93.097, P < 0.001), FC1 (F(5,33) = 97.337, P < 0.001), FC2 (F(5,33) = 68.686, P < 0.001) electrodes.

No significant main effects of group or target were found.

3.2.4. N400 Peak Amplitude

The amplitudes of the N400 peak of the different electrodes are presented in Table 7.

Results revealed a significant main effect of type (F(5,33) = 9.102, P < 0.001) with higher amplitudes evoked in response to words (with a mean amplitude of −4.099 μV) than pictures (with a mean amplitude of −1.766 μV). These differences registered at all electrodes locations: Cz (F(5,33) = 50.789, P < 0.001), FCz (F(5,33) = 84.937, P < 0.001), AFz (F(5,33) = 93.097, P < 0.001), FC1(F(5,33) = 97.337, P < 0.001), FC2 (F(5,33) = 68.686, P < 0.001).

No group or target effect was obtained. A significant interaction between target X type X group (F(5,33), P < 0.05) was found. Words yielded larger amplitudes than pictures especially among the target items (the gap between word and picture

Table 6. The average latency of N400 during processing words and pictures.

Table 7. The average amplitude of N400 during processing words and pictures.

Figure 3. Scalp distribution of N400 component.

among the target items was approximately −3.067 μV, while the gap between word and picture among the non-target items was approximately −1.201). The gap between target and non-target was found to be larger among typical readers (a gap of approximately −1.866 μV) compared to dyslexic readers (a gap of approximately −0.526 V). These differences did not register at any specific electrode location.

3.2.5. P600 Latency

The latencies of the P600 peaks at the different electrode sites are presented in Table 8 and the scalp distribution is shown in Figure 4.

Results revealed a significant main effect of target (F(5,33) = 3.591, P < 0.01) with longer latencies observed among non-target (with a mean latency of 595.565

Table 8. The average latency of P600 during processing words and pictures.

Figure 4. Scalp distribution of P600 component.

ms) than target (with a mean latency of 579.335 ms). These differences registered at Cz (F(5,33) = 16.209, P < 0.001), CPz (F(5,33) = 14.153, P = 0.001), CP2 (F(5,33) = 5.915, P < 0.05).

A significant main effect of type (F(5,33) = 15.093, P < 0.001) with longer latencies observed for words (with a mean latency of 606.990 ms) than pictures (with a mean latency of 567.930 ms). These differences registered at all electrode locations: CZ (F(5,33) = 32.910, P < 0.001), CPz (F(5,33) = 30.335, P < 0.001), CP1 (F(5,33) = 41.540, P < 0.001), CP2 (F(5,33) = 50.944, P < 0.001), P1 (F(5,33) = 52.128, P < 0.001), P2 (F(5,33) = 69.037, P < 0.001).

3.2.6. P600 Peak Amplitude

The amplitudes of the P600 peak of the different electrodes are presented in Table 9.

Results revealed a significant main effect of target (F(5,33) = 3.243, P < 0.05) with higher amplitudes evoked in response to target (with a mean amplitude of 3.121 μV) than non-target (with a mean amplitude of 2.521 μV). These differences registered at all electrode locations: Cz (F(5,33) = 11.866, P = 0.001), CPz (F(5,33) = 18.552, P < 0.001), CP1(F(5,33) = 17.595, P < 0.001), CP2(F(5,33) = 12.631, P = 0.001), P1 (F(5,33) = 8.069, P < 0. 01), P2 (F(5,33) = 5.858, P < 0.05).

A significant main effect of type (F(5,33) = 12.003, P < 0.001) with higher amplitudes evoked in response to pictures (with a mean amplitude of 3.159 μV) than to words (with a mean amplitude of 2.483 μV). These differences registered at all

Table 9. The average amplitude of P600 during processing words and pictures.

electrode locations: Cz (F(5,33) =8.365, P < 0. 01), CPz (F(5,33) = 44.759, P < 0.001), CP1 (F(5,33) = 50.777, P < 0.001), CP2 (F(5,33) = 42.640, P < 0.001), P1 (F(5,33) = 30.794, P < 0.001), P2 (F(5,33) = 39.625, P < 0.001).

4. Discussion

This study used behavioral and electro-physiological (ERP) measures to compare the semantic processing with verbal (words) and nonverbal (pictures) stimuli of dyslexic readers to that of regular readers. The aim of the study was to determine whether there are differences between the processing words and the processing of pictures in both timing and in brain areas and whether there are differences in brain activity between dyslexic students and typical readers while processing meaning from words and from pictures. The results will be discussed accordingly, focusing on two different levels of answers: First on behavioral measures―accuracy percentages of correct answers and reaction times, next on the electrophysiological measures―amplitudes and latencies of ERP components.

First, the differences between processing pictures and words were examined. We assumed that different brain activation patterns would be found for processing words and processing pictures. Although there were no differences between words and pictures in means of accuracy (both groups performed all tasks with very high accuracy), the reaction time differed. Behavioral results were in accordance with this prediction, illustrating longer reaction times in response to words compared to pictures in both groups. These results are in line with previous studies examining the semantic processing of pictures and words (Caramazza, Hillis, Rapp, & Romani, 1990; Guenther et al., 1980; Hogaboam & Pellegrino, 1978; Kieffer, 2001; Snodgrass & McCullogh, 1986; Vandenberghe et al. 1996) and support the fact that there are differences between the processing of words and the processing of pictures. These differences are seen in the faster reaction time in the picture task as compared to the word tasks which strengthens the claim that pictures have privileged access to the semantic system and therefore are processed more quickly than written words. However, some researchers claim that they are stored in a common system that is equally accessible by words and pictures (Caramazza et al., 1990; Snodgrass & McCullogh, 1986). There are some possible explanations for the assumption that pictures are semantically categorized faster than words. One explanation is that pictures receive a wider semantic processing than words thus pictures benefit from deeper and more detailed levels of processing (Intraub & Nicklos, 1985; Nelson, Reed, & McEvoy, 1976). Nelson (1979) claimed that picture superiority effects are due to differences in the degree of perceptual distinctiveness of pictures and words, in which pictures are more distinct from each other; they are encoded with more detail which makes it easier to recall them in the future. Yum et al. (2011) claimed that there are some differences between words and objects―object parts, like animal legs, give us meaning while parts of words, letters, do not. Furthermore, the global shape of the pictures gives us additional information about the object’s identity as compared to words which all look alike (Bar & Neta, 2006; Grainger, 2008). Another explanation for these differences in processing the two types of stimulus is derived from memory performance. Findings propose a better retention for pictures than for words (Paivio & Csapo, 1973; McBride & Dosher, 2002) and this reflects the fact that pictures elicit a verbal code and an image code while written words elicit only verbal coding (Paivio, 1991). These differences can explain the fact that object semantic categorization is faster than word semantic categorization. All the explanations of the picture superiority effect share the hypothesis that memorial representation of pictures is more elaborated, distinctive or meaningful than the representation of words (Hockley, 2008). This hypothesis was strengthened in this study too. These results may imply that children should be taught words together with their picture; this may strengthen their representation in the mental lexicon and help retrieve the words faster, in fact maybe vocabulary should be taught with pictures, this may broaden the child’s vocabulary in a better and easier way.

Regarding the electrophysiological measures, the ERPs elicited by the presentation of the two kinds of stimuli-words and pictures-words were different. Support for this difference can be found in differences in latency and amplitude of the different ERP components which were identified. The results found significant differences in processing the different types of stimuli at all stages of processing―N100, N400―with longer latencies observed in response to pictures compared to words, and with higher amplitudes in response to pictures compared to words at N100, P600.

In the early processing stages (N100) a notable difference in brain activity evoked by pictures and words was observed. A significant difference was found for the different types of stimuli, with pictures eliciting higher amplitudes and longer latencies than words. These main effects were obtained in both right and left occipital electrodes. These results indicate that the differences in brain activity in response to the presentation of words and pictures at this early stage of processing, which usually presents the stages of visual attention and perception ( Shaul, 2008 in: Breznitz, 2008), result mainly from the different visual-spatial appearance of the two forms of stimuli. The semantic classification starts with attention to perceptual information of the stimulus, especially to the physical features (Guenther et al., 1980). N100, which is sensitive to the physical stimuli factor, reflects the fact that visual perceptual processing and its amplitude are enhanced while attention is directed to visual stimuli (Kiefer, 2001; Olofsson, Nordin, Sequeira, & Polich, 2008). A similar early negative effect was demonstrated in previous studies, for example, Bentin, Mouchetant-Rostaing, Giard, Echallier, & Pernier (1999) found that words elicited a sharp early negative peak at 170 msec (140 - 200 msec); others found a shorter latency at about 130 - 170 msec after stimulus onset elicited by written words (Dehaene, 1995; Rossion, Joyce, Cottrell, & Tarr, 2003); Brandeis, Vitacco, & Steinhausen (1994) found the P100, N100, P200 in the ERP elicited by all words in the sentences is typical in response to visual stimuli in children and also in adults. This early effect has also been demonstrated with pictures (for example Mayseless, 2007; Rossion et al., 2003) where it was demonstrated that the N100 component elicited during decoding of meaningful black and white drawings. These results also reinforce the hypothesis that the main processing of pictures happens during the first 300 msec, while the subject perceives the visual features of the figure. This processing may be enough to reach the meaning in pictures, but words need additional processing which happens later (between 400 and 500 mesc) in order to reach the semantic presentation of the word.

In the late processing (N400, P600), a notable difference in brain activity evoked by pictures and words was observed. At N400, a main effect for type of stimulus was evident, with words eliciting higher amplitudes than pictures and pictures eliciting longer latencies than words. These main effects were obtained in both right and left central frontal electrodes. These results indicate differences in brain activity in response to category decision tasks ( Shaul, 2008 in: Breznitz, 2008) between words and pictures. Perhaps this strengthens the idea that pictures are accessed more quickly and easily than words which demand more cognitive effort. The cognitive effort includes activation of knowledge stored in semantic memory for including more abstract information. Then, this information matched, or not matched, with stored information about the target category until the categorical decision is completed (Ganis et al., 1996). After this process, a main effect for type of stimulus was evident at the P600 time window too, with pictures eliciting higher amplitudes than words, and words eliciting longer latencies than pictures. These main effects were obtained in both right and left central frontal electrodes. These results indicate that after the cognitive efforts of semantic process, the processing was faster for words than pictures. It may be that the final stages of verification of our answer are quicker for words than pictured, it is easier to reach the picture presentation but among adults it seems that the meaning of verbal stimuli is stronger. It maybe that during all the years of exposure to pictures and words at adulthood we have a strong semantic representation of both words and pictures, the connection between the verbal and non-verbal information is established and both the word and pictures allow us to reach the semantic information correctly. Years of exposure to print make reading automatic which helps us to reach the semantic information quickly from words too.

When examining the differences in brain activity between dyslexic students and typical readers while processing meaning from words and from pictures, we assumed that differences between the dyslexic students and the typical readers would be found while processing pictures and words with a larger gap between the groups in the verbal task.

The behavioral results of reaction time illustrated differences between RT for words and pictures which was larger among the dyslexics as compared to typical readers, meaning that the dyslexics were slower only in words and not in pictures. In contrast, no significant differences or interaction in accuracy between the groups was found. These results are in line with research that have found that as dyslexic readers get older, many of them succeed to become relatively accurate readers but they remain slow in reading words (Bruck, 1998; Brunswick, McCrory, Price, Frith, & Frith, 1999).

The results obtained from the electrophysiological measures revealed group differences only in N100 component with a higher amplitude among dyslexics compared to typical readers, but not in other components. The higher amplitude reflects an initial perceptual process response to stimuli before a decision making task. Therefore, it may be that the dyslexic readers need a larger amount of activation in the early processes which may then compensate for the later ones.

Contrary to our hypothesis, behavioral and electrophysiological results showed no differences between dyslexic students and typical readers while processing words and pictures, except for slowness of the whole process among dyslexic while processing words. Perhaps dyslexics are slightly slower in every stage and it is not significant but eventually it leads the differences in reaction times.

There are several possible explanations for these results. The dyslexic readers who participated in this study were university students, compensated readers with appropriate accommodation that abled them to enter university. Shaywitz et al. (2003) claimed that the presence of compensatory factors such as stronger cognitive ability allowed minimizing partially the consequences of their phonologic deficit, so that as adults they were identical to normal readers on measures of reading comprehension. The improved accuracy due to compensation processes allowed them to perform the categorization task with accuracy similar to that of typical readers. An additional explanation is that the semantic processing seems to be intact among the compensated dyslexics who participated in our study; it may be a compensatory resource. The impact of dyslexia can be modified by availability of semantic knowledge (Snowling, Bishop, & Stothard, 2000). Furthermore, maybe due the low task difficulty level no differences were found, but a higher difficulty task, like semantic processing of sentences which requires integration, may show differences between the groups. The task was comprised of word stimuli that were short and of moderate frequency; the use of longer words and lower frequency ones may show differences between the groups. Finally, long presentation time enabled dyslexic readers to improve the slowness aspect of their reading and to perform the task at the same accuracy as typical readers. Lebovitz (2008) found that the group differences in accuracy percentages were larger in the shorter presentation time than in the longer.

It can be assumed that different aspects of the subjects and tasks may have led to minor group differences, but it is important to emphasize that all dyslexic students were found to be slower and inaccurate in the basic reading skills as was shown in the verification of the subjects reading ability. So, although the basic skills are impaired, the higher level processing such as semantic is similar to the level of typical adult readers. This maybe one of the compensatory mechanisms which helps adult dyslexics readers reach and succeed in the higher education system.

5. Conclusion

In summary, these results illustrated differences between processing of words and processing pictures in timing and in brain areas. It also illustrated that although dyslexic readers were equal to typical readers in the ability to read and understand familiar stimuli, they could not do it as quickly as typical readers. This slowness existed only in the verbal stimuli. It seems that the semantic domain is not one of the problems underling dyslexia among compensated students. On the contrary, it may be one of the compensating recourses. In addition, it was found that perceptual features played an important role in category identification and here lies the difference between dyslexics and typical readers.

Further research should investigate non-compensated dyslexics, those who are not able to enter university, as well as children who have been identified as dyslexic readers. Examining the full age range may help us to fully understand the semantic process among pure dyslexics. Furthermore, research should investigate the same task with different presentation times (from slow to fast) and with different types of stimuli (longer and harder words, and more complex pictures). These variables may change the level of difficulty of the task and help us to find out whether shorter presentation times and harder tasks will expose the differences between dyslexics and typical readers in the semantic domain. Another perspective which may be interesting to examine is a different task of semantic processing. Perhaps a task which requires more integration of semantic processing such as sentence or reading comprehension, or may be comparing two stimuli each time and deciding if they both belong to the same semantic category will give us additional information regarding semantic processing among dyslexic and typical readers.

The contribution of the present study to the reading research and dyslexia lies on its focus on the semantic processes that help us to comprehend the meaning from written words and objects. The basic semantic domain seems to be intact among the compensated dyslexic students and it is important to strengthen the reliance on it. Berends & Reitsma (2006) suggested that it is important for fluent reading among disabled readers to make them think about the meaning of words (semantics) they are reading and practicing.

Cite this paper
Shaul, S. and Rom, Z. (2019) The Differences in Semantic Processing of Pictures and Words between Dyslexic and Typical-Reading University Students. Psychology, 10, 235-255. doi: 10.4236/psych.2019.102018.
References
[1]   Bar, M., & Neta, M. (2006). Humans Prefer Curved Visual Objects. Psychological Science, 17, 645-648. https://doi.org/10.1111/j.1467-9280.2006.01759.x

[2]   Barrett, S. E., & Rugg, M. D. (1990). Event-Related Potentials and the Semantic Matching of Pictures. Brain and Cognition, 14, 201-212.
https://doi.org/10.1016/0278-2626(90)90029-N

[3]   Bentin, S., Mouchetant-Rostaing, Y., Giard, M. H., Echallier, J. F., & Pernier, J. (1999). ERP Manifestation of Processing Printed Words at Different Psycholinguistic Levels: Time Course and Scalp Distribution. Journal of Cognitive Neuroscience, 11, 235-260.
https://doi.org/10.1162/089892999563373

[4]   Berends, I. E., & Reitsma, P. (2006). Addressing Semantics Promotes the Development of Reading Fluency. Applied Psycholinguistics, 27, 247-265.
https://doi.org/10.1017/S0142716406060279

[5]   Brandeis, D., Vitacco, D., & Steinhausen, H. C. (1994). Mapping Brain Electric Micro-States in Dyslexic Children during Reading. Acta Paedopsychiatrica, 56, 239-247.

[6]   Breznitz, Z. (2008). Literacy Studies: Perspectives from Cognitive Neurosciences, Linguistics, Psychology and Education: Brain Research in Language. New York, London: Springer.

[7]   Bruck, M. (1998). Outcomes of Adults with Childhood Histories of Dyslexia. In C. Hulme, & Malatesha (Eds.), Reading and Spelling: Development and Disorders (pp. 179-200). Mahwah, NJ: Erlbaum.

[8]   Brunswick, N., McCrory, E., Price, C. J., Frith, C. D., & Frith, U. (1999). Explicit and Implicit Processing of Words and Pseudowords by Adult Developmental Dyslexics. Brain, 122, 1901-1917. https://doi.org/10.1093/brain/122.10.1901

[9]   Caramazza, A., Hillis, A. E., Rapp, B. C., & Romani, C. (1990). The Multiple Semantic Hypothesis: Multiple Confusion? Cognitive Neuropsychology, 7, 161-189.
https://doi.org/10.1080/02643299008253441

[10]   Center for Psychometric Tests (1994). Comprehension Test, Israeli Psychometric Scholastic Aptitude Test. Tel Aviv: Center for Psychometric Tests.

[11]   Coch, D., Maron, L., Wolf, M., & Holcomb, P. J. (2002). Word and Picture Processing in Children: An Event-Related Potential Study. Developmental Neuropsychology, 22, 373-406. https://doi.org/10.1207/S15326942dn2201_3

[12]   Dehaene, S. (1995). Electrophysiological Evidence for Category-Specific Word Processing in the Normal Human Brain. Neuroreport, 6, 2153-2157.
https://doi.org/10.1097/00001756-199511000-00014

[13]   Federmeier, K. D., & Kutas, M. (2001). Meaning and Modality: Influences of Context, Semantic Memory Organization, and Perceptual Predictability on Picture Processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 202-224.
https://doi.org/10.1037/0278-7393.27.1.202

[14]   Friedrich, M., & Friederici, A. D. (2004). N400-Like Semantic Incongruity Effect in 19-Month-Old: Processing Known Words in Picture Context. Journal of Cognitive Neuroscience, 16, 1465-1477. https://doi.org/10.1162/0898929042304705

[15]   Ganis, G., Kutas, M., & Sereno, M. I. (1996). The Search for “Common Sense”: An Electrophysiological Study of the Comprehension of Words and Pictures in Reading. Journal of Cognitive Neuroscience, 8, 89-106. https://doi.org/10.1162/jocn.1996.8.2.89

[16]   Grainger, J. (2008). Cracking the Orthographic Code: An Introduction. Language and Cognitive Processes, 23, 1-35. https://doi.org/10.1080/01690960701578013

[17]   Gratton, G., Coles, M. G. H., & Donchin, E. (1983). A New Method for Off-Line Removal of Ocular Artifact. Electroencephalography and Clinical Neurophysiology, 55, 468-484.
https://doi.org/10.1016/0013-4694(83)90135-9

[18]   Guenther, R. K, Klatzky, R. L., & Putnam, W. (1980). Commonalities and Differences in Semantic Decisions about Pictures and Words. Journal of Verbal Learning and Verbal Behavior, 19, 54-74. https://doi.org/10.1016/S0022-5371(80)90524-1

[19]   Harmony, T., Fernandez, T., Fernandez-Bouzas, A., Silva-Pereyra, J., Bosch, J., Diaz-Comas, L., & Galan, L. (2001). EEG Changes during Word and Figure Categorization. Clinical Neurophysiology, 112, 1486-1498.
https://doi.org/10.1016/S1388-2457(01)00582-X

[20]   Helenius, P., Salmelin, R., Service, E., & Connolly, J. F. (1999). Semantic Cortical Activation in Dyslexic. Journal of Cognitive Neuroscience, 11, 535-550.
https://doi.org/10.1162/089892999563599

[21]   Hockley, W. E. (2008). The Picture Superiority Effect in Associative Recognition. Memory and Cognition, 36, 1351-1359. https://doi.org/10.3758/MC.36.7.1351

[22]   Hogaboam, T. W., & Pellegrino, J. W. (1978). Hunting for Individual Differences in Cognitive Processes: Verbal Ability and Semantic Processing of Pictures and Words. Memory & Cognition, 6, 189-193. https://doi.org/10.3758/BF03197445

[23]   Intraub, H. (1981). Rapid Conceptual Identification of Sequentially Presented Pictures. Journal of Experimental Psychology: Human Perception and Performance, 7, 604-610.
https://doi.org/10.1037/0096-1523.7.3.604

[24]   Intraub, H., & Nicklos, S. (1985). Levels of Processing and Picture Memory: The Physical Superiority Effect. Journal of Experimental Psychology: Learning, Memory & Cognition, 11, 284-298. https://doi.org/10.1037/0278-7393.11.2.284

[25]   Key, A. P. F., Dove, G. O., & Maguire, M. J. (2005). Linking Brainwaves to the Brain: An ERP Primer. Developmental Neuropsychology, 27, 183-215.
https://doi.org/10.1207/s15326942dn2702_1

[26]   Khateb, A., Michel, C. M., Pegna, A. J., O’Dochartaigh, S. D., Landis, T., & Annoni, J. M. (2003). Processing of Semantic Categorical and Associative Ralations: An ERP Mapping Study. International Journal of Psychophysiology, 49, 41-55.
https://doi.org/10.1016/S0167-8760(03)00076-X

[27]   Khateb, A., Pegna, A. J., Landis, T., Mouthon, M. S., & Annoni, J. M. (2010). On the Origin of the N400 Effects: An ERP Waveform and Source Localization Analysis in Three Matching Tasks. Brain Topography, 23, 311-320.
https://doi.org/10.1007/s10548-010-0149-7

[28]   Kiefer, M. (2001). Perceptual and Semantic Sources of Category-Specific Effects: Event-Related Potentials during Picture and Word Categorization. Memory & Cognition, 29, 100-116. https://doi.org/10.3758/BF03195745

[29]   Kutas, M., & Federmeier, K. D. (2000). Electrophysiology Reveals Semantic Memory Use in Language Comprehension. Trends in Cognitive Sciences, 4, 463-470.
https://doi.org/10.1016/S1364-6613(00)01560-6

[30]   Lebovitz, L. (2008). Processing Meaning from Printed Words: Do Dyslexic and Regular Readers Follow Similar Patterns of Pre-Semantic Processing? Thesis for the Master degree, Haifa: University of Haifa.

[31]   Lyon, G. R., Shaywitz, S. E., & Shaywitz, B. A. (2003). A Definition of Dyslexia. Ann Dyslexia, 53, 1-14. https://doi.org/10.1007/s11881-003-0001-9

[32]   Mayseless, N. (2007). Brain Pattern Activity of Regular Readers as Compared to Dyslexic Readers during the Processing of Objects and Pseudo Objects. Thesis the Master degree, Haifa: University of Haifa.

[33]   McBride, D. M., & Dosher, B. A. (2002). A Comparison of Conscious and Automatic Memory Processes for Picture and Word Stimuli: A Process Dissociation Analysis. Consciousness and Cognition, 11, 423-460.
https://doi.org/10.1016/S1053-8100(02)00007-7

[34]   Miller-Shaul, S. (2005). The Characteristics of Young and Adult Dyslexics Readers on Reading and Reading Related Cognitive Tasks as Compared to Normal Readers. Dyslexia, 11, 132-151. https://doi.org/10.1002/dys.290

[35]   Nelson, D. L. (1979). Remembering Pictures and Words: Appearance, Significance, and Name. In L. S. Cermak, & F. I. M. Craik (Eds.), Levels of Processing in Human Memory (pp. 45-75). Hillsdale, NJ: Erlbaum.

[36]   Nelson, D. L., Reed, V. S., & McEvoy, C. L. (1976). Learning to Order Pictures and Words: A Model of Sensory and Semantic Encoding. Journal of Experimental Psychology: Human Learning & Memory, 3, 485-497.
https://doi.org/10.1037/0278-7393.3.5.485

[37]   Neville, H. J., Coffey, S. A., Holocomb, P. J., & Tallal, P. (1993). The Neurobiology of Sensory and Language Processing in Language-Impaired Children. Journal of Cognitive Neuroscience, 5, 235-253. https://doi.org/10.1162/jocn.1993.5.2.235

[38]   Olofsson, J. K., Nordin, S., Sequeira, H., & Polich, J. (2008). Affective Picture Processing: An Integrative Review of ERP Findings. Biological Psychology, 77, 247-265.
https://doi.org/10.1016/j.biopsycho.2007.11.006

[39]   Paivio, A. (1991). Dual Coding Theory: Retrospect and Current Status. Canadian Journal of Psychology, 45, 255-287. https://doi.org/10.1037/h0084295

[40]   Paivio, A., & Csapo, K. (1973). Picture Superiority in Free Recall: Imagery or Dual Coding? Cognitive Psychology, 5, 176-206. https://doi.org/10.1016/0010-0285(73)90032-7

[41]   Pobric, G, Jefferies, E., & Lambon Ralph, M. A. (2010). Amodal Semantic Representations Depend on Both Anterior Temporal Lobes: Evidence from Repetitive Transcranial Magnetic Stimulation. Neuropsychologia, 48, 1336-1342.
https://doi.org/10.1016/j.neuropsychologia.2009.12.036

[42]   Rogers, T. T., Lambon Ralph, M. A., Garrard, P., Bozeat, S., McClelland, J. L., & Hodges, J. R. (2004). The Structure and Deterioration of Semantic Memory: A Neuropsychological and Computational Investigation. Psychological Review, 111, 205-235.
https://doi.org/10.1037/0033-295X.111.1.205

[43]   Rossion, B., Joyce, C. A., Cottrell G. W., & Tarr, M. J. (2003). Early Lateralization and Orientation Tuning for Face, Word and Object Processing in the Visual Cortex. Neuroimage, 20, 1609-1624. https://doi.org/10.1016/j.neuroimage.2003.07.010

[44]   Russeler, J., Becker, P., Johannes, S., & Munte, F. (2007). Semantic, Syntactic, and Phonological Processing of Written Words in Adult Developmental Dyslexic Readers: An Event-Related Brain Potential Study. BMC Neuroscience, 8, 1-10.
https://doi.org/10.1186/1471-2202-8-52
http://www.biomedcentral.com/1471-2202/8/52

[45]   Russeler, J., Probst, S., Johannes, S., & Munte, T. (2003). Recognition Memory for High and Low Frequency Words in Adult Normal and Dyslexic Readers: An Event Related Brain Potential Study. Journal of Clinical and Experimental Neuropsychology, 25, 815-829.
https://doi.org/10.1076/jcen.25.6.815.16469

[46]   Shallice, T. (1988). Specialization within the Semantic System. Cognitive Neuropsychology, 5, 133-142. https://doi.org/10.1080/02643298808252929

[47]   Shatil, E. (1995). One-Minute Test for Pseodowords. Unpublished Test, Haifa: University of Haifa.

[48]   Shaul, S. (2005). The Characteristics of Young and Adult Readers on Reading and Reading Related Cognitive Tasks as Compared to Normal Readers. Dyslexia, 11, 132-151.
https://doi.org/10.1002/dys.290

[49]   Shaul, S. (2008). Event-Related Potentials (ERPS) in the Study of Dyslexia. In Z. Breznitz (Ed.), Literacy Studies: Perspectives from Cognitive Neurosciences, Linguistics, Psychology and Education: Brain Research in Language. New York, London: Springer.
https://doi.org/10.1007/978-0-387-74980-8_2

[50]   Shaywitz, S. E., Shaywitz, B. A., Fulbright, R. K., Skudlarski, P., Mencl, W. E., Constable, R. T., Pugh, K. R., Holahan, J. M., Marchione, K. E., Fletcher, J. M., Lyon, G. R., & Gore, J. C. (2003). Neural Systems for Compensation and Persistence: Young Adult Outcome of Childhood Reading Disability. Biological Psychiatry, 54, 25-33.
https://doi.org/10.1016/S0006-3223(02)01836-X

[51]   Snodgrass, J. G., & McCullogh, B. (1986). The Role of Visual Similarity in Picture Categorization. Journal of Experimental Psychology: Learning, Memory and Cognition, 12, 147-154. https://doi.org/10.1037/0278-7393.12.1.147

[52]   Snowling, M., Bishop, D., & Stothard, S. (2000). Is Preschool Language Impairment a Risk Factor for Dyslexia in Adolescence? The Journal of Child Psychology and Psychiatry, 41, 587-600. https://doi.org/10.1111/1469-7610.00651

[53]   Vandenberghe, R., Price, C., Wise, R., Josephs, O., & Frackowiak, R. S. J. (1996). Functional Anatomy of Common Semantic System for Words and Pictures. Nature, 383, 254-256. https://doi.org/10.1038/383254a0

[54]   Visser, M., Jefferies, E., & Lambon Ralph, M. A. (2009). Semantic Processing in the Anterior Temporal Lobes: A Meta-Analysis of the Functional Neuroimaging Literature. Journal of Cognitive Neuroscience, 22, 1083-1094.
https://doi.org/10.1162/jocn.2009.21309

[55]   Wechsler, D. (1981). Wechsler Intelligence Scale for Adults-Revised. Cleveland, OH: Psychological Corp.

[56]   Yum, Y. N., Holcomb, P. J., & Grainger, J. (2011). Words and Pictures: An Electrophysiological Investigation of Domain Specific Processing in Native Chinese and English Speakers. Neuropsychology, 10, 1-13.

 
 
Top