The United States Center for Disease Control reports that approximately 4% of the American population is living with disability resulting from Cerebral Vascular Accidents (CVA) and Traumatic Brain Injury (TBI)  , with TBI demonstrating a 3:1 higher incidence rate   . Survival rates continue to rise for both groups with improved medical management of hypertension mitigating the impact of stroke and advances in emergency medical technology saving lives following TBI   . As the survival rates have increased so to have the number of persons living with long-term disability, which is currently estimated to be 11.5 million people in the United States  . Although the mechanism of injury differs for CVA and TBI, both types of injuries often result in impaired functioning in communication, mobility, vision, memory, information processing and behavioral control.
Concomitant with the increase in the number of persons living with disability has been the growth of post-hospital neurological rehabilitation programs  . These programs may be either residential or outpatient and are designed to treat persons with acquired brain injuries such as TBI, CVA, brain tumors and anoxic/hypoxic injuries. Treatment typically involves 45 to 60 minutes each of physical, occupational, and speech therapies per day, up to five days per week  . Medical management of medications and psychological counseling are included as indicated. Current clinical practice involves individual members from a multi-disciplinary rehabilitation treatment team conducting assessments, identifying deficits, and developing goals to improve performance in the identified problem areas  . This approach, however, is accomplished without empirical knowledge of which deficits have the greatest impact on overall outcome. Rather, the approach considers deviations from non-impaired performance to targeted goals for remediation. A more focused approach would involve targeting those deficits that have the greatest effect on functional outcome and determining optimal treatment strategies by discipline to lessen their impact. A statistical approach utilizing Rasch modeling techniques offers an efficient means to accomplish the first step in this process: identifying the deficits most relevant to outcome.
Rasch analysis, most commonly associated with Item Response Theory, is used to improve the accuracy and reliability of tests or questionnaires comprised of items with multiple response options. Rasch uses a logistical model of probability to identify a finite number of human traits that comprise a construct (e.g. “disability”). The model accounts for a response to a specific item in relationship to the probability of a specific response to other items on the measure  . This enables the calculation of the metric distance between items and supports reliable differentiation among persons tested on the measure. As a result, Rasch reveals an item hierarchy based on test items least likely to be endorsed to those most likely to be endorsed. Further, by establishing the metric distance between test items, Rasch enables the conversion of ordinal Likert-type measures into interval level measurement appropriate for use with parametric statistics.
Rasch analysis was employed by Malec and his colleagues  in the development of the Mayo Portland Adaptability Inventory Version-4 (MPAI-4). The MPAI-4 is a functional assessment rating scale designed to evaluate performance in post-hospital rehabilitation following neurologic injury. Using Rasch, the authors identified 29 functional areas that best illustrate the range of limitations experienced following neurologic injury. These 29 items were further organized into 3 subscales representing different domains of functioning. In a recent study, Lewis and Horn  extended that application of Rasch analysis to include identification of MPAI-4 items with the greatest probability of presenting with severe disability following a TBI. The results of this study demonstrated that persons in different stages of recovery presented with distinct and different disability profiles.
The purpose of the present study was to extend this line of research by applying Rasch analysis of MPAI-4 data to examine differences in disability profiles for clinical groups. The current study identified TBI and CVA survivors treated in community-residential, post-hospital brain injury rehabilitation programs for analysis. Additionally, this study evaluated the effectiveness of these treatment programs in reducing disability from admission to discharge.
The study sample was selected from 2990 neurologically impaired individuals with consecutive admissions from 36 post-hospital residential rehabilitation programs in 16 states from 2011 to 2017. From the population of 2990, a sample of 874 individuals met study inclusion criteria: diagnosed with a TBI (n = 687) or CVA (n = 187), age 18 or older, minimum length of stay of 1 month, maximum chronicity (onset of injury to admission interval) of one year and admitted and discharged from residential neurorehabilitation. Table 1 presents demographic data for the two groups. This data reveals that the two groups were comparable in level of disability as measured by the MPAI-4 Total T-score at admission (independent samples T-test, t(872) = 0.47, p < 0.64, n.s.). The CVA group was older (t(872) = 9.3, p < 0.001) and on average experienced shorter length of stays than the TBI group (t(872) = 3.3, p < 0.001).
Table 1. Demographics and injury related variables for TBI and CVA samples.
Participants’ were assessed with the MPAI-4 at the time of admission and time of discharge from the treatment facilities involved in the study. Specifically, the MPAI-4 consists of 29 items rated from 0 to 4 on a 5-point scale, where 0 represents no limitations and 4 represents a severe problem interfering with activity more than 75% of the time. Raw scores on the 29 items are converted to T-scores within three subscales: Abilities Index (physical, communication, and cognitive skills), Adjustment Index (emotional, behavioral, and social skills), and Participation Index (instrumental activities of daily living skills). T-scores have a mean of 50 and a standard deviation of 10. Higher T-scores indicate greater disability. The MPAI-4 has undergone rigorous psychometric testing and has proven reliability and validity as determined through Rasch analysis, Item Cluster, Principle Component Analyses (PCA), and measures of concurrent and predictive validity  .
2.3. Rehabilitation Treatment
The over-arching goal of the programs involved in the study was to maximize participants’ functional independence for return to home and family. With this goal, each participant received physical therapy, occupational therapy, speech therapy, recreation and community integration, counseling (based on need) and medical management provided by nursing and physicians specializing in physical medicine and rehabilitation. Additionally, they received an average of 5 to 6 hours a day of life skills acquisition training including community integration.
2.4. Data Collection
Each participant was evaluated within approximately two weeks of admission using the MPAI-4 by treatment team consensus. Discharge MPAI-4s were completed in a similar fashion within the final week of the participant’s stay. The results of all evaluations with demographic data were compiled into a national database for analysis.
3. Statistical Analysis
Rasch analysis was performed to determine reliability of MPAI-4 admission and discharge assessments and item difficulty profiles for the TBI and CVA samples. A repeated measures multivariate analysis of co-variance (RM MANCOVA) was provided to evaluate change scores on Abilities, Adjustment, and Participation Indices from admission to discharge and to evaluate differences between groups at admission and discharge. Analyses were performed using SPSS version 25 for the RM MANCOVA and follow-up tests while Winsteps version 3.81 was used to conduct Rasch Analyses.
3.1. Rasch Item Difficulty Statistics
Rasch analysis orders items by identifying the probability of an item receiving a particular rating along the measurement scale (i.e. no limitation to severe limitation). For example, mean item difficulty is the point at which the highest and lowest categories have an equal probability of being observed  . In the case of the MPAI-4, the more difficult items would be those that have a higher probability of a moderate to severe limitation being observed than no limitation. Difficulty measures are presented as logits with two decimal points. For a given population this measure is useful for ordering items from least to most likely to be impaired.
3.2. Construct Validity
Construct validity refers to the extent to which an evaluation tool measures the underlying construct that it is intended to measure. Rasch fit statistics accomplish this by evaluating expected values for an item to the actual value obtain from the data set. Fit statistics also provide an estimate of the distinct contribution for each item in describing the underlying construct and the extent to which they differentiate among people along the continuum of that construct  . As applied to the MPAI-4, Rasch Infit and Outfit statistics illustrate the fit of each item representing unique contribution to a person’s level of disability (latent construct). Fit values that are nearest to 1.0 indicate minimal distortion. Values falling below 1 indicate that persons are answering incorrectly when they are expected to answer correctly (Guttman error). Low fit values on the MPAI-4 suggest that high levels of limitation are observed when low levels would be expected for that person on those items. Values greater than 1 indicate that there is more random variation on an item than would be expected. Therefore, Fit values falling between 0.5 and 1.5 are considered productive for measurement use  . Items that fall outside those parameters may not reliably represent the latent construct being measured.
Reliability refers to the consistency of a measure or the extent to which a measure produces similar results from one testing occasion to another. Key statistics provided by Rasch analysis to evaluate measurement consistency are Person and Item Reliability and Person and Item Separation. Specifically, Person Reliability indicates how well items comprising a measure distinguish among individuals (e.g. those possessing a lot or a little of the construct measured) while Item Reliability refers to whether test items relate to each other in a consistent way in describing a disparate group of individuals. A coefficient of 0.80 or greater is considered acceptable for Person Reliability, while a coefficient of at least 0.90 is optimal for Item Reliability  .
Separation values reveal how well items distinguish among people along a performance continuum (Person Separation) and the unique contribution of items to the construct being measured. Person Separation values indicate the number of performance levels detected by a measure. For example, a Person Separation index of 2.00 means that two levels of performance can be reliably identified.
Item Separation refers to the extent to which items on a test are consistently ranked from least difficult to most difficult. Low Item Separation (<3.00) implies that the item difficulty hierarchy is not reliable, whereas magnitudes exceeding 3.00 indicate greater consistency of item hierarchy.
4.1. Construct Validity
Table 2 presents Rasch Infit and Outfit statistics by diagnostic group that fell outside the 1.0 ± 0.5 parameter established for acceptable fit.
Table 2. MPAI-4 items infit and outfit values by program type outside acceptable parameters.
indicates value within acceptable level.
Each of the misfit items presented in the Table 2 exceeded 1.5, revealing significant unexplained variation in observations and a tendency for outlier responding. Not surprisingly, the CVA group, presented infit and outfit values greater than 1.5 for “paid work” at each assessment. For this group the combination of advanced age and disability resulted in return to work being rarely endorsed on the evaluations and contributing to the instability of the item. Paid work values were outside the criterion at admission (e.g., no one was working at the time of admission due to the impact of an acute injury), but not discharge for TBIs. For the TBI group, audition was the most unstable item, with infit and outfit values exceeding criteria at admission and discharge. Within this group, audition was rarely endorsed as a limitation and was thus more susceptible to outlier responding. “Unpaid work” (e.g., home making, volunteering, school) at admission was the next most unstable item due to individuals not being involved in these activities. The remainder of misfit items were marginally above the 1.50 upper limit. Overall, for both groups, the majority of MPAI-4 items accurately contributed to measuring disability after brain injury supporting a high level of construct validity for this instrument.
4.2. Person and Item Reliability
Rasch person reliability coefficients for the MPAI-4 at admission were 0.91 for the TBI group and 0.88 for the CVA group. Admission MPAI-4 item reliability coefficients were 0.99 for both groups. At discharge person reliability was 0.95 and 0.93 respectively for TBI and CVA. Again, MPAI-4 item reliability was 0.99 for both groups at discharge. These findings indicate that MPAI-4 assessments effectively distinguished persons along the disability continuum (person reliability) and there was a consistent level of agreement within groups identifying easy through difficult items (item reliability).
4.3. Person and Item Separation
Rasch person separation values for admission MPAI-4 assessments were 3.10 for TBI and 2.67 for CVA. At discharge, the values were 4.23 and 3.60 respectively for TBI and CVA groups. These values indicate the existence of at least three performance strata within each group at admission and discharge. Item separation values ranged from 8.44 (CVA admission) to 17.10 (TBI discharge). These values reveal a consistent item hierarchy from least difficult to most difficult for admission and discharge assessments within both diagnostic groups.
With acceptable levels of reliability and validity established, further analyses were conducted to determine item difficulty profiles and performance differences admission to discharge.
4.4. Item Difficulty
Figure 1 through Figure 3 illustrate item difficulty comparisons for the TBI and CVA groups on MPAI-4 Abilities, Adjustment, and Participation Indices. Again, difficulty values of zero indicate the there is an equal probability of observing high or low levels of disability. Items with values less than zero have a higher probability of receiving a severe disability rating. Positive values reflect a greater likelihood of receiving a rating of mild to no disability. The strength of the probability is reflected in the absolute value of the logit for a given item. See figures for the item analysis with difficulty values.
Figure 1 shows the difficulty values for Abilities Items. CVA participants were most likely to experience significant disability on Novel Problem Solving (−0.44), Memory (−0.32), Mobility (−0.38), Use of Hands (−0.17), and Visuospatial (−0.14). For the TBI group, the magnitude of disability was most probable in the areas of Novel Problem Solving (−0.55), Memory (−0.62), and Attention and Concentration (−0.43).
Figure 2 shows difficulty values for Adjustment Items. For both groups the impact of items on the Adjustment scale was less severe than those on the Abilities scale. The TBI group, however, experienced greater disability with emotional adjustment than the CVA group; TBI participants had higher disability ratings in 7 of 8 Adjustment items. Disability was most pronounced on Impaired Awareness (−0.43) for TBI and Fatigue (−0.18) for CVA.
Figure 3 shows the Participation Items with the level of disability experienced in the home and community (e.g., application of skills outside of a facility). Figure 3 clearly illustrates that these items presented the greatest difficulty for both groups.
The CVA group received negative difficulty values on each of the 8 items on the scale. Difficulty values were negative on 7 of 8 items for the TBI group. Transportation, Residence (home skills), money management presented the greatest difficulty (highest level of disability) for both groups.
4.5. Difficulty Admission and Discharge
Table 3 shows the top 5 most disabling items for both groups at admission and discharge. The data shows that at discharge the magnitude of difficulty was
Figure 1. Item difficulty values for TBI and CVA on the Abilities Index.
Figure 2. Item difficulty values for TBI and CVA on the Adjustment Index.
reduced for each of the most disabling functional areas with the exception of transportation.
With few exceptions (memory and impaired awareness) the most difficult items were application skills from the Participation Index. For all participants, transportation, home skills, and money management presented the greatest difficulty at admission and discharge. Items in the top five remained the same for the CVA group from admission to discharge, with only slight changes in the
Figure 3. Item difficulty values for TBI and CVA on the Participation Index.
Table 3. Top five most difficult items at admission and discharge for TBI and CVA.
DV = difficulty value. Items in italics changed from admission to discharge.
order of the first three items. For the TBI group leisure skills and memory were replaced by productivity (engagement in meaningful activity paid or unpaid) and impaired awareness in the top five. Transportation remained unchanged and presented the greatest magnitude of disability.
4.6. Change Admission to Discharge
With age entered as a covariate, a RM MANCOVA revealed a significant main effect for pre-post testing, F(1, 871) = 128.97 p = 0.0005, Wilks Lambda = 0.87, partial eta2 = 0.13, power to detect = 1.00. Follow-up paired sample t-tests found that MPAI-4 T-Scores were significantly lower (less disability) at discharge for both the TBI and CVA groups. Table 4 presents the paired sample T-values, significance levels, and Cohen’s d effect sizes for each pre-post comparison on the Abilities, Adjustment, and Participation measures.
Table 4. Mean MPAI-4 T-scores at Admission and Discharge by diagnostic group.
*p < 0.001 for each comparison.
Results of the RM MANCOVA also showed a significant two-way measure by diagnostic group interaction, F (2, 870) = 10.89, p = 0.005, Wilks Lambda = 0.87, partial eta2 = 0.024. To interpret this interaction, independent group t-tests were performed on MPAI-4 admission and discharge T-scores. This analysis revealed that the TBI group had significantly higher T-scores (greater impairment) on the Adjustment Index at admission [t(872) = 3.25, p < 0.001)] and at discharge [t(872) = 2.40, p < 0.01]. No other comparisons reached statistical significance.
After discharge from acute hospitalization, persons who have suffered a TBI or CVA often face a lifetime of significant disability. Post-hospital brain injury rehabilitation programs provide comprehensive multidisciplinary treatment with the goal of reducing disability and restoring functional independence. While research has demonstrated the effectiveness of these programs    , the great variability in symptoms resulting from neurologic insult presents challenges to the development of best practice protocols for this specialized rehabilitation niche. The present study applied Rasch analysis to rehabilitation outcomes. Specifically, this evidenced-based method statistically targeted improving current practice by producing a hierarchical impairment model that is diagnostically specific.
The first step toward development of best practice requires application of psychometrically sound measures capable of reliably detecting change in performance. Consistent with previous research    , Rasch analysis demonstrated that the MPAI-4 met established criteria for reliability and validity. Given that result, multivariate statistics were used to examine effectiveness of current practices for reducing disability following a TBI or CVA. Completion of a RM MANCOVA found that both groups showed a statistically significant reduction in disability from admission to discharge. The only group difference was found on the Adjustment Index. The TBI group was significantly more impaired than the CVA group on this measure of neurobehavioral adjustment and control. In relation to the additional findings within the Abilities Index (reduced attention, memory and problem solving), this finding is not surprising given that the frontal lobes (prefrontal cortex) and temporal lobes (limbic system) have a strong influence on both cognitive and behavioral control. These areas are at greater risk from diffuse rotational injury associated with TBI than with focal injuries more common with CVA. Within the diagnosis of CVA, it is more common to impact a smaller region or isolated portion of the brain where the blood supply is disrupted.
Change from admission to discharge for each of the MPAI-4 Indices yielded moderate to large effect sizes for both groups (Range = 0.79 - 1.40). While this is a positive finding, examination of mean T-scores at discharge across measures indicates that many participants in both groups were discharged with moderate levels of disability. Therefore, additional emphasis may need to be placed on other possible interventions to further reduce disability. One possibility would be to extend the time in program with the assumption that the addition of time in therapies that have been demonstrated to be effective would yield even greater improvement. While this is a logical assumption, in most cases time in program is not determined by the treatment team but by funding sources. These decisions are often based on short-term cost considerations rather than using an evidenced based model to determine appropriate length of stay to maximize disability reduction. Given the impact of potential funding limitation, treatment teams may be able to achieve greater disability reduction by using a prescriptive model in their rehabilitation treatments. Prescriptive modeling can target deficits in an established order thereby producing a greater impact on disability reduction. In addition this prescriptive modeling may also impact how and when remediation and compensatory strategies are used throughout the recovery process.
Rasch analysis assists in the meaningful targeting of treatment by identifying skills that have the highest probability of severe disability. The present study demonstrated that the CVA and TBI groups presented with different disability profiles at admission. The CVA group had a greater likelihood of experiencing disability in skills such as use of hands, mobility, visuospatial abilities, and novel problem solving. This pattern of disability is characteristic of focal lesions often seen in CVA. The TBI group was more likely to exhibit more diffuse disability including novel problem solving, memory, attention/concentration, impaired awareness and initiation. This constellation of cognitive and neurobehavioral symptoms is the hallmark of frontal and temporal lobe disruption associated with TBI.
Both groups experienced the greatest change with Abilities and Adjustment items, but the greatest challenge was within the applied skills of the Participation Index (e.g., instrumental activities of daily living). Rehabilitation within the first year of recovery tends to show the greatest gains with physical, cognitive, and communication skills along with moderate behavioral stability. However, application of skills into real-world settings and situations requires extensive learning and insight development that is often not evident until much later in recovery. Limitations experienced in these skills for the current study were the result of different patterns of disability with regard to the physical, cognitive, and emotional/behavioral functions that were related to the neuropathology and mechanism of injury type. Application of skills tends to be the greatest limiting factor in recovery from neurological injury.
Although both groups saw improvement on participation skills at discharge, greater reduction in disability may have been achieved by targeting the high impact deficits identified at admission with longer and more frequent therapies. Thus, this study provides an example of evidence-based hierarchical modeling with Rasch analysis to provide improved targeted treatment that is independent of time in recovery. The use of Rasch seems to be a promising application for the development of more hierarchical prescriptive treatment for persons recovering from TBI or CVA.
Conflicts of Interest
The authors declare no conflicts of interest regarding the publication of this paper.