Received 16 November 2015; accepted 17 January 2016; published 20 January 2016
The notion that cognitive functions contribute to functional outcomes has become increasingly apparent in various psychiatric disorders. Impaired functional outcomes are predicted by decreased cognitive performance in schizophrenia   , bipolar disorder  - , attention deficit disorder   , and Alzheimer’s disease  . Cognitive impairment has also been associated with decreased treatment adherence in psychotic disorders  and non-psychiatric illnesses  . Furthermore, cognitive deficits are associated with reductions in work performance  , and daily activities   , as well as with deterioration of self-care in medical illness  . Various health practitioners and clinicians rely on the assessment of cognitive functioning to improve diagnostic accuracy, to assess the evolution of disorders, and, sometimes, to measure the response to medication.
Cognitive functioning is classically assessed via comprehensive neuropsychological testing. This mode of assessment remains the gold standard against which other evaluations of cognition are measured. Unfortunately, access to traditional neuropsychological testing is limited by a number of constraints: the availability of neuropsychologists, the time required for testing, as well as the associated costs. As new data reveals the importance of measuring cognitive functioning in psychiatric disorders, researchers have explored ways in which to screen for putative cognitive deficits, which may guide decision-making as to the necessity for classical methods of assessment of cognition. A plethora of computer assisted neuropsychological batteries has been developed   ; however these require expensive equipment and are usually administered by staff in a dedicated space. As a result, computerized testing has not yet been routinely incorporated in clinical practice.
A major challenge has been the development of brief, easily mastered tests that can be used as screening tools to determine the need for more in-depth neuropsychological assessment, can be administered by clinicians from various backgrounds, are inexpensive, and can be administered in different clinical settings. Examples of such tests currently used in clinical practice are the Mini-Mental State Examination (MME;  ) and the Montreal Cognitive Assessment (MoCA;  ). These tests are used primarily to screen for severe cognitive deficits, and do not have the sensitivity to measure subtle changes found in schizophrenia or mood disorder  . Other screening tests, which have been developed, are the Neurobehavioral Cognitive Status Examination (NCSE;  ), the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS;  ), and the Brief Assessment of Cognition in Schizophrenia (BACS;  ). These screening tools have been developed for specific populations such as the elderly and patients afflicted with schizophrenia and may not be applicable for the general psychiatric population. More importantly, although nominally brief, the RBANS and the BACS require 40 minutes or more to administer.
In 2005, Scott Purdon developed the Screen for Cognitive Impairment in Psychiatry (SCIP), an assessment tool that possessed a number of advantages with respect to other instruments  . This test is intended for administration by a clinician with a minimum of training, and requires, on average, no more than 15 minutes to complete. Because it is a simple paper and pencil test, it is inexpensive and can be used in different settings, including at the patient’s bedside. Three different versions of the SCIP have been developed  , allowing for retesting and thus for following the impact of illness or medication on cognitive function. These features have made this test an instrument of interest for clinicians who wish to screen for potential cognitive deficits and direct their patients to the appropriate resources. German  , Spanish  , and Japanese  versions have already been validated. The Spanish version has been tested in psychosis and schizophrenia   , and in bipolar disorder  .
A cognitive test should possess several qualities  . It should be valid, providing results that correlate with classical neuropsychological testing. It should be reliable, offering similar results for the same subject. The degree of practice effect, that is the improvement of result in the same individual because of prior exposure, should be minimal. The original English version of the SCIP possesses these characteristics. It shows no main effect of sex; although a difference was found on one of the five subtests in one of the versions of the SCIP. An effect of practice is observed on the working memory, verbal fluency and visuomotor subtests in the original English version. Globally, the SCIP does not replace a complete neuropsychological battery, but can be a useful tool for screening cognitive impairment in psychiatric populations and following therapeutic effects.
The need for a screen of cognitive function in French has led us to develop a validated French version of the SCIP. We report in this paper the results of this validation.
2. Material and Methods
This study was approved by the Institutional Review Board of the Institut de Santé Mentale de Montréal (IUSMM) following established guidelines. All participants provided written informed consent after study procedures were explained. The work described was carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans.
Seventy-two individuals with a college level education who were fluent in French participated in this study. Inclusion criteria included the capacity to give informed consent and the absence of psychiatric, neurological illness, or other conditions which could be expected to affect cognitive function. Participants were recruited through posters displayed in a local university, the IUSMM and the website of the IUSMM. Participants who could not attend two testing sessions of the SCIP were also excluded.
2.2.1. Screen for Cognitive Impairment in Psychiatry
The SCIP is composed of five subscales that assess short and long term verbal learning memory, working memory, verbal fluency, and psychomotor velocity  . Similar subscales exist in validated neuropsychological tests. Three alternative versions of the SCIP have been developed.
The verbal learning and delayed recall subtest (Verbal Learning Test, VLT) consists of a list of ten words which is read three times to the participant, who is asked to repeat the words in any order following each reading (Verbal Learning Test-Immediate Recall, VLT-I). After five minutes, the subject is once again asked to recall as many of the words in the list as possible (Verbal Learning Test-Delayed Recall, VLT-D) without it being read beforehand. The three different versions of this subtest include different series of words. The score of the VLT-I and VLT-D is equivalent to the number of words remembered during every trial. The verbal learning and delayed recall subscale is based on the Rey Auditory Verbal Learning Test (RAVLT;  ).
The working memory subscale using triads of consonants is similar to the Brown-Peterson Consonant Trigram test (CTT;  ). It consists of a sequence of 3 consonants that are read to the participant, followed by a given number. Before repeating the three consonants, the participant must count backwards from the given number for a specific period of time of 0, 3, 9, or 18 seconds. This subscale assesses working memory using an interference task, and is scored based on the number of consonants remembered. Twenty-four randomly distributed triads of consonants have been included in the SCIP, 8 for every version of the SCIP. The first 2 triads have no delay or interference (the subject has to repeat the sequence of consonants immediately and without performing any interfering task), while the remaining 6 have been divided into pairs, each of which having a differing delay of 3, 9, or 18 seconds.
The Verbal Fluency Test (VFT) consists of generating as many words as possible that start with a given letter, during a 30 seconds lapse of time. Each version of the SCIP has 2 specific letters and the score is based on the total number of words produced. This task is similar to the Controlled Oral Word Association Test (COWAT;  ). The Information Processing Speed (Visuomotor Tracking Test, VMT) subtest requires the participant to write the Morse code version of a series of letters within 1 minute. This task requires copying the Morse characters in blank spaces as fast as possible. The score is based on the number of accurate codes generated.
2.2.2. Psychometric Assessment
To estimate intellectual functioning level, participants completed the four subtests (Blocs, Matrix, Similarities, and Vocabulary) of the Wechsler Abbreviated Scale for Intelligence (WASI;  ). Some subtests from the Wechsler Adult Intelligence Scale  were administered including those contributing to the Speed Processing Index (SPI) (Digit-Symbol Coding and Symbol Search) and of the Working Memory Index (WMI) (Digit Span and Letter-Number Sequencing) in order to examine their relationship with the VMT and WMT subtests of the SCIP, respectively. The Verbal Fluency subtest from the Delis-Kaplan Executive Functions System (D-KEFS;  ), which is very similar to the VFT in its administration, was used. Finally, participants completed the California Verbal Learning Test (CVLT;  ), a test assessing verbal learning and memory in which a wordlist is read five times and participants have to recall immediately after each reading as many words as they can. They must repeat the list in a free recall then a cued recall, right after the five readings and 20 to 30 minutes afterwards.
2.2.3. French Version of the Screen for Cognitive Impairment in Psychiatry
The SCIP was translated to French following the inverse translation design. The scale and instructions were translated by a bilingual psychiatrist (S. V. Tourjman) from Quebec, Canada, whose mother tongue is English and by a neuropsychologist (Josée Gagné) whose mother tongue is French. A first back translation was carried out by a bilingual research assistant (Mara Du Bow) whose mother tongue is English. Discrepancies were scarce and addressed by consensus. A second back translation was carried out by a bilingual psychiatrist from Quebec (René Desautels), whose mother tongue is French. Both individuals involved in the back translation were blind to the original English version. The final French translation of the 3 versions of the SCIP was compared to the original English versions, and no differences in the meaning of the different items or the subtests instructions have been found. All the words employed for VLT were literally translated. For the VFT the letter “W” was replaced by the letter V, since the letter W is not employed as often in French as in English. The number of words generated from the letters in each of the French versions was comparable to the other French versions as well as to the English version.
2.3. Testing Procedure
Participants completed questionnaires in order to capture demographic data. A traditional neuropsychological battery was administered. The neuropsychological tests used were selected by two of the authors (MB, MNC) who are neuropsychologists and researchers in the field of cognitive development. Participants were tested with the SCIP on two occasions separated by at least a one-week interval. The order of testing was random and included all six possible permutations of the different versions of the SCIP (version 1:2, 1:3, 2:1, 2:3, 3:1, 3:2). Evaluations were administered by a psychology undergraduate student and a clinical assistant with training as a physician after observing several evaluations administered by a neuropsychologist and then being supervised until judged autonomous.
2.4. Statistical Analyses
Data analyses were performed using the Statistical Package for the Social Sciences Version 9.3 (SAS Institute Inc., Cary, NC). The three alternative forms of the SCIP were analysed with regards to consistency, practice effects, and comparability with results on the classical neuropsychological battery. The three versions of the SCIP were compared considering only those that were passed during the first assessment. Gender differences were analyzed with multivariate mixed analyses of variance. Univariate analyses of variance were performed when main effects emerged as being significant. Cronbach’s alpha coefficient was calculated in order to determine the internal consistency between the subtests. Test-retest reliability was assessed calculating the intra-class correlation coefficient (ICC) between the first and the second administration of equivalent SCIP versions. To verify the internal validity of the SCIP, we performed a factor analysis, using the principal component analysis method with a varimax rotation. Finally, Pearson’s correlation coefficient between the SCIP total score and the global z score from the neuropsychological battery was calculated. For all analyses, the statistical threshold was set at p < 0.05.
All participants were able to complete the SCIP. Demographic data are presented in Table 1. The average age was 26 and spanned from 18 to 48 years of age. Most of the sample had a college or more advanced diploma and
Table 1. Demographics.
the majority were students.
3.1. Screen for Cognitive Impairment in Psychiatry Versions
No significant differences in the subjects’global performance were found between the three versions of the SCIP (p > 0.05).
3.2. Sex Differences
Results yielded interactions between sex and SCIP version (p < 0.05), showing a significant difference between ratings for the version 1, where women scored higher than men. No differences were found for the versions 2 and 3 of the SCIP between men and women. Considering sex as separate groups, men scored lower on the version 1 compared to versions 2 and 3 of the SCIP, which were equivalent. Women showed no significant differences between the 3 versions of the SCIP.
When gender and subtests were evaluated, it was found that for the subtest VLT-I, men scored similarly for the versions 2 and 3, but lower for the version 1. Men also scored lower than women for VLT-I. For VLT-D, men scored lower for version 3 compared to versions 1 and 2, where they scored similarly. Men also scored lower than women on this subtest. WMT showed no effect of gender or version, so this subtest can be considered as equivalent between the 3 versions and for both genders. For VFT, an effect of version was observed where participants of both genders scored higher in the version 2, but no differences between the sexes were observed. VMT was higher for women in all the different versions, but no effect of version was detected.
3.3. Practice Effect
No effect of practice for the subtests VLT-I (F(1,59) = 0.02, p = 0.9), VLT-D (F(1,6) = 1.01, p = 0.3), and VFT (F(1,6) = 1.44, p = 0.2) were observed. By contrast, significant differences for WMT (F(1,6) = 14.50, p < 0.01) and VMT (F(1,6) = 9.54, p < 0.01) were found, where higher scores were observed in favour of the versions administered at time 2.
3.4. Test-Retest Reliability
Test retest reliability between the first and the second administration of the SCIP was assessed with the calculation of the ICCs core, which yielded a high score. The ICCs ranged from 0 to 6 for the WMT subscale to 0 to 8 to the VMT subscale. The sum of the subscale scores achieved an ICC of 0.9. The ICCs also showed high scores for VLT-I (ICC = 0.7), WMT and VLT-D (ICC = 0.8), VMT (ICC = 0.8) and VFT (ICC = 0.7).
3.5. Internal Coherence
Using a varimax rotation analysis of the main components, 2 factors were obtained for version 1 and 2, with 4 (VLT-I, VLT-D, VMT, WMT) and 3 (VLT-I, VLT-D, VMT) subtests respectively loading heavily on factor 1. Only one factor comprising of all SCIP subtests was found for version 3. Taking all versions as a whole all subtests load onto one factor suggesting coherence between them.
3.6. Internal Consistency
Results yielded Cronbach’s alpha coefficients of 0.44 for VLT-I, 0.46 for VLT-D, 0.50 for VMT, 0.71 for VFT, and 0.50 for WMT. A weak correlation between the total scores and VFT Cronbach’s alpha coefficient was found, and the rejection of this subtest, increased the Cronbach’s alpha coefficient to 0.73. Normalised alpha’s yielded a score of 0.66.
3.7. Standardized Scores
Z scores were calculated by dividing the difference between the obtained scores and the expected score by the standard deviation. Mean Z scores for the SCIP were not different from the expected score of zero implying that the sample tested was comparable to the original normative data for the SCIP (Table 2).
3.8. Correlations with Neuropsychological Testing
The SCIP was correlated with scores on traditional neuropsychological tests. As expected, the three versions of the SCIP correlated with most of the neuropsychological tests administered, such as subtests of the Wechsler Adult Intelligence Scale (WAIS-III), the California Verbal Learning Test and Verbal fluency of the Delis-Kap- lan Executive Functions System, (see Table 3), since the SCIP score consists of the sum of the same functions tested by these tests. Total score for the SCIP version 1 was significantly correlated to all but the following tests of the neuropsychological battery: sequences, symbols, list b. The SCIP version 2 total score was related to all but the recognition false positives score of the neuropsychological battery. Finally, the total score of the SCIP 3 was related to all but the arithmetic and similarities subtest and to the working memory subscale.
Next, the correlations of interest are reported in Table 4 showing how each score of the SCIP correlated with existing neuropsychological tests that measure the same abilities. As expected, in all three versions of the SCIP, VLT-I correlated with Recall 5 and Immediate Free Recall of the CVLT and VLT-D correlated with Delayed Free Recall and Delayed Cues Recall assessing learning abilities of a list of words and its retention after a delayed recall.
WMT is supposed to be a measure of working memory and indeed all versions correlated with Digit Span of the WAIS-III but surprisingly version 1 did not correlate with Letter-Number Sequence, while versions 2 and 3 did. However, all 3 versions correlated with the Working Memory Index of the WAIS-III. VFT is supposed to assess verbal fluency abilities. Indeed, it correlated with all three version of the Verbal Fluency test from the D-KEFS. Finally, all three versions of VMT, which is supposed to assess visuo-motor processing speed, correlated with Digit Symbol Coding, and with PSI, though only version 3 of VLT correlated with Symbol Search subtest. Not other asks of the SCIP correlated with some of the neuropsychological subtests, which is not surprising since some of these functions have relations together.
Table 2. Standardized scores for the French validation of the screen for cognitive impairment in psychiatry (SCIP-F).
*Z score is thus not different from expected score of Z = 0.
Table 3. Correlations between French validation of the screen for cognitive impairment in psychiatry (SCIP-F) standardized scores and neuropsychological battery results.
Note: †p < 0.10; *p < 0.05; **p < 0.01; ***p < 0.00.
The present study successfully validated a French version of the Screen for Cognitive Impairment in Psychiatry (SCIP-F). The Z scores of the SCIP-F are comparable to the scores obtained in the original English versions of the SCIP despite a greater heterogeneity in the sample confirming comparability of the French and English versions of the SCIP. The main advantages of the SCIP compared to traditional neuropsychological batteries are the ease and rapidity of administration of the test (around 15 minutes), and the requirement of inexpensive materials (pencil, paper and clock) that can be used in different settings without special requirements. The feasibility of using the SCIP-F in clinical settings is currently being evaluated in several clinical populations.
The validity of the SCIP-F is confirmed by the correlation of the subtests of the SCIP-F with related tasks in the neuropsychological battery. Thus the verbal learning tests are correlated with the CVLT, which is a validated test to assess verbal learning and memory. The WMT which tests working memory is correlated with the working memory index of the WAIS-III and the subtest of Digit Span, but version 1 did not correlate with Letter- Number Sequence though version 2 and 3 did. The verbal fluency task of the SCIP-F correlates with the verbal fluency subtest of the D-KEFS battery. Finally the visuomotor task is correlated with the Coding subtest of the WAIS-III which tests the same function, that is mainly visuo-motor processing speed. This task is also correlated with the speed processing index on all versions, and with the symbol search task on version 3. Therefore,
Table 4. Correlations matrix for the French validation of the screen for cognitive impairment in psychiatry (SCIP-F) and the neuropsychological battery.
Note: †p < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001; Abbreviations: VLT: verbal learning task, WMT: nworking memory task, VFT: verbal fluency task, VMT: visuomotor task.
the SCIP-F yields good validity since all abilities that are supposed to be assessed by it correlate with well validated neuropsychological tests in their French version.
Internal consistency showed good correlation between total scores and the subtests VLT-I, VLT-D, WMT and VFT, and a weaker correlation to VFT. Test-retest reliability showed a good correlation between the first and the second administrations of the SCIP-F. No practice effect was observed for VLT1, VLT2 and VFT subtests, but an improvement in the scores was obtained during the second administration for WMT and VMT, similarly to that seen in the Spanish version. Such practice effects are commonly reported in neuropsychological testing and must be taken into account in the design of studies focusing on cognition and on repeated testing in a clinical context.
4.1. Limitations and Future Directions
The SCIP does not cover important cognitive domains that are frequently affected in mental disorders. Moreover, the SCIP-F was administered by a non-psychiatrist physician rather than by psychiatrists for the English and the Spanish versions. Despite this limitation, results were comparable to the English version underscoring the ease of use of this instrument. Further research should address the relationship of performance on the SCIP to functional outcomes, thus further elucidating the clinical utility of this instrument.
The SCIP-F has the potential to be a very useful test in screening psychiatric patients for cognitive deficits, assisting clinicians in deciding who will benefit from a full neuropsychological assessment and in establishing an individualized treatment plan. It does not constitute nor can replace a complete and thorough neuropsychological exam. This study is performed by non psychiatrist professionals, so it provides information regarding the feasibility of the administration by other professionals. In conclusion, the SCIP constitutes a fast and inexpensive cognitive screening tool and current evidence supports the use of alternative forms such as our successfully validated SCIP-F.
Conflict of Interest
This study was possible as a result of a grant from +Prend soins de toi+.
BACS = Brief Assessment of Cognition in Schizophrenia;
COWAT = Controlled Oral Word Association Test;
CTT = Brown-Peterson Consonant Trigram Test;
CVLT = California Verbal Learning Test;
D-KEFS = Delis-Kaplan Executive Functions System;
ICC = Intraclass Correlation Coefficient
IUSMM = Institut de Santé Mentale de Montréal;
MME = Mini-Mental State Examination;
MoCA = Montreal Cognitive Assessment;
NCSE = Neurobehavioral Cognitive Status Examination;
RAVLT = Rey Auditory Verbal Learning Test
RBANS = Repeatable Battery for the Assessment of Neuropsychological Status;
SCIP = Screen for Cognitive Impairment in Psychiatry;
SCIP-F = Screen for Cognitive Impairment in Psychiatry-French Validation;
SPI = Speed Processing Index;
VFT = Verbal Fluency Test;
VLT = Verbal Learning Test;
VLT-I = Verbal Learning Test-Immediate Recall;
VLT-D = Verbal Learning Test-Delayed Recall;
VMT = Visuomotor Tracking Test;
WASI = Wechsler Abbreviated Scale for Intelligence;
WAIS-III = Wechsler Adult Intelligence Scale;
WMI = Working Memory Index