The purpose of this study was to demonstrate an application of Rasch analysis to identify differences in disability profiles resulting from traumatic brain injury (TBI) and cerebral vascular accident (CVA) and to examine outcome differences between the two groups following post-hospital residential rehabilitation. Participant data w ere collected from 32 facilities in 16 states. From 2990 neurologically impaired individuals with consecutive admissions from 2011 through 2017, 874 met inclusion criteria: TBI (n = 687) or CVA (n = 187), 18 years or older, minimum length of stay of one month, and maximum chronicity of 1 year. Participants were evaluated at admission and discharge on the Mayo Portland Adaptability Inventory - Version 4 (MPAI-4). Rasch analysis was performed to establish item reliability, construct validity and item difficulty. A Repeated Measures Multivariate Analysis of Covariance (RM MANCOVA) determined group differences and improvement from admission and discharge. Rasch Analysis demonstrated satisfactory construct validity and internal consistency (Person reliability > 0 .90, Item reliability > 0 .98 for admission and discharge MPAI-4s). Both groups showed significant improvement on the MPAI-4 (p < 0 .0005). The TBI group was more impaired on the adjustment scale at both admission and discharge (p < 0 .001). Rasch analysis identified two distinct impairment patterns. CVA participants exhibited deficits characteristic of focal impairment while the TBI group presented with deficits reflective of diffuse impairment. Rehabilitation was shown to be beneficial in reducing disability following neurologic injury in both groups. Importantly, Rasch Analysis accurately produced unique disability profiles that differentiated the treatment groups. This unique statistical technique offers a promising prescriptive hierarchical model for guiding neurological rehabilitation treatment.
The United States Center for Disease Control reports that approximately 4% of the American population is living with disability resulting from Cerebral Vascular Accidents (CVA) and Traumatic Brain Injury (TBI) [
Concomitant with the increase in the number of persons living with disability has been the growth of post-hospital neurological rehabilitation programs [
Rasch analysis, most commonly associated with Item Response Theory, is used to improve the accuracy and reliability of tests or questionnaires comprised of items with multiple response options. Rasch uses a logistical model of probability to identify a finite number of human traits that comprise a construct (e.g. “disability”). The model accounts for a response to a specific item in relationship to the probability of a specific response to other items on the measure [
Rasch analysis was employed by Malec and his colleagues [
The purpose of the present study was to extend this line of research by applying Rasch analysis of MPAI-4 data to examine differences in disability profiles for clinical groups. The current study identified TBI and CVA survivors treated in community-residential, post-hospital brain injury rehabilitation programs for analysis. Additionally, this study evaluated the effectiveness of these treatment programs in reducing disability from admission to discharge.
The study sample was selected from 2990 neurologically impaired individuals with consecutive admissions from 36 post-hospital residential rehabilitation programs in 16 states from 2011 to 2017. From the population of 2990, a sample of 874 individuals met study inclusion criteria: diagnosed with a TBI (n = 687) or CVA (n = 187), age 18 or older, minimum length of stay of 1 month, maximum chronicity (onset of injury to admission interval) of one year and admitted and discharged from residential neurorehabilitation.
Gender | TBI (n = 687) | CVA (n = 187) |
---|---|---|
Male | 82% | 62% |
Female | 18% | 38% |
Age (years) | ||
Mean | 46.0 | 57.2 |
SD | 15.4 | 11.2 |
Range | 19 - 87 | 20 - 85 |
Time Since Injury (months) | ||
Mean | 3.9 | 3.1 |
SD | 2.8 | 2.7 |
Range | 1 - 12.0 | 1.0 - 12.0 |
Length of Stay (months) | ||
Mean | 5.5 | 3.5 |
SD | 7.3 | 4.1 |
Range | 1 - 65.5 | 1 - 46.1 |
Race | ||
African American | 10% | 13% |
Asian/Pacific | 1% | 0% |
Caucasian | 74% | 72% |
Hispanic | 11% | 12% |
Multi-racial | 1% | 3% |
Middle Eastern | 1% | 0% |
Other | 2% | 0% |
Severity of Disability (MPAI-4 Total T-Score) | (Admission Total T-Score) | (Admission Total T-Score) |
Mild (<40) Mild-moderate (40 - 49) | 2.0% 20.0% | 2.0% 13.0% |
Moderate (50 - 59) | 33.0% | 42.0% |
Severe (60+) | 45.0% | 43.0% |
Participants’ were assessed with the MPAI-4 at the time of admission and time of discharge from the treatment facilities involved in the study. Specifically, the MPAI-4 consists of 29 items rated from 0 to 4 on a 5-point scale, where 0 represents no limitations and 4 represents a severe problem interfering with activity more than 75% of the time. Raw scores on the 29 items are converted to T-scores within three subscales: Abilities Index (physical, communication, and cognitive skills), Adjustment Index (emotional, behavioral, and social skills), and Participation Index (instrumental activities of daily living skills). T-scores have a mean of 50 and a standard deviation of 10. Higher T-scores indicate greater disability. The MPAI-4 has undergone rigorous psychometric testing and has proven reliability and validity as determined through Rasch analysis, Item Cluster, Principle Component Analyses (PCA), and measures of concurrent and predictive validity [
The over-arching goal of the programs involved in the study was to maximize participants’ functional independence for return to home and family. With this goal, each participant received physical therapy, occupational therapy, speech therapy, recreation and community integration, counseling (based on need) and medical management provided by nursing and physicians specializing in physical medicine and rehabilitation. Additionally, they received an average of 5 to 6 hours a day of life skills acquisition training including community integration.
Each participant was evaluated within approximately two weeks of admission using the MPAI-4 by treatment team consensus. Discharge MPAI-4s were completed in a similar fashion within the final week of the participant’s stay. The results of all evaluations with demographic data were compiled into a national database for analysis.
Rasch analysis was performed to determine reliability of MPAI-4 admission and discharge assessments and item difficulty profiles for the TBI and CVA samples. A repeated measures multivariate analysis of co-variance (RM MANCOVA) was provided to evaluate change scores on Abilities, Adjustment, and Participation Indices from admission to discharge and to evaluate differences between groups at admission and discharge. Analyses were performed using SPSS version 25 for the RM MANCOVA and follow-up tests while Winsteps version 3.81 was used to conduct Rasch Analyses.
Rasch analysis orders items by identifying the probability of an item receiving a particular rating along the measurement scale (i.e. no limitation to severe limitation). For example, mean item difficulty is the point at which the highest and lowest categories have an equal probability of being observed [
Construct validity refers to the extent to which an evaluation tool measures the underlying construct that it is intended to measure. Rasch fit statistics accomplish this by evaluating expected values for an item to the actual value obtain from the data set. Fit statistics also provide an estimate of the distinct contribution for each item in describing the underlying construct and the extent to which they differentiate among people along the continuum of that construct [
Reliability refers to the consistency of a measure or the extent to which a measure produces similar results from one testing occasion to another. Key statistics provided by Rasch analysis to evaluate measurement consistency are Person and Item Reliability and Person and Item Separation. Specifically, Person Reliability indicates how well items comprising a measure distinguish among individuals (e.g. those possessing a lot or a little of the construct measured) while Item Reliability refers to whether test items relate to each other in a consistent way in describing a disparate group of individuals. A coefficient of 0.80 or greater is considered acceptable for Person Reliability, while a coefficient of at least 0.90 is optimal for Item Reliability [
Separation values reveal how well items distinguish among people along a performance continuum (Person Separation) and the unique contribution of items to the construct being measured. Person Separation values indicate the number of performance levels detected by a measure. For example, a Person Separation index of 2.00 means that two levels of performance can be reliably identified.
Item Separation refers to the extent to which items on a test are consistently ranked from least difficult to most difficult. Low Item Separation (<3.00) implies that the item difficulty hierarchy is not reliable, whereas magnitudes exceeding 3.00 indicate greater consistency of item hierarchy.
TBI | TBI | CVA | CVA | |||||
---|---|---|---|---|---|---|---|---|
MPAI-4 Items | Admission | Discharge | Admission | Discharge | ||||
Infit | Outfit | Infit | Outfit | Infit | Outfit | Infit | Outfit | |
Paid work | 2.07 | 1.99 | - | - | 2.05 | 2.32 | 1.75 | 1.62 |
Unpaid work | 1.81 | 2.22 | - | - | 2.11 | - | - | - |
Audition | 1.56 | 2.06 | 2.13 | 2.71 | - | - | - | - |
Use of hands | - | 1.58 | - | - | - | - | 1.58 | 1.55 |
Motor Speech | - | - | - | - | - | - | 1.55 | 1.55 |
indicates value within acceptable level.
Each of the misfit items presented in the
Rasch person reliability coefficients for the MPAI-4 at admission were 0.91 for the TBI group and 0.88 for the CVA group. Admission MPAI-4 item reliability coefficients were 0.99 for both groups. At discharge person reliability was 0.95 and 0.93 respectively for TBI and CVA. Again, MPAI-4 item reliability was 0.99 for both groups at discharge. These findings indicate that MPAI-4 assessments effectively distinguished persons along the disability continuum (person reliability) and there was a consistent level of agreement within groups identifying easy through difficult items (item reliability).
Rasch person separation values for admission MPAI-4 assessments were 3.10 for TBI and 2.67 for CVA. At discharge, the values were 4.23 and 3.60 respectively for TBI and CVA groups. These values indicate the existence of at least three performance strata within each group at admission and discharge. Item separation values ranged from 8.44 (CVA admission) to 17.10 (TBI discharge). These values reveal a consistent item hierarchy from least difficult to most difficult for admission and discharge assessments within both diagnostic groups.
With acceptable levels of reliability and validity established, further analyses were conducted to determine item difficulty profiles and performance differences admission to discharge.
The CVA group received negative difficulty values on each of the 8 items on the scale. Difficulty values were negative on 7 of 8 items for the TBI group. Transportation, Residence (home skills), money management presented the greatest difficulty (highest level of disability) for both groups.
reduced for each of the most disabling functional areas with the exception of transportation.
With few exceptions (memory and impaired awareness) the most difficult items were application skills from the Participation Index. For all participants, transportation, home skills, and money management presented the greatest difficulty at admission and discharge. Items in the top five remained the same for the CVA group from admission to discharge, with only slight changes in the
TBI | CVA | ||||||
---|---|---|---|---|---|---|---|
Admission | Discharge | Admission | Discharge | ||||
MPAI-4 Item | DV | MPAI-4 Items | DV | MPAI-4 Item | DV | MPAI-4 Items | DV |
Transportation | −1.47 | Transportation | −1.47 | Home Skills | −1.71 | Transportation | −1.30 |
Home Skills | −1.15 | Money Manage | −1.02 | Transportation | −1.38 | Home Skills | −1.11 |
Money Manage | −0.97 | Home Skills | −0.92 | Money Manage | −1.13 | Money Manage | −1.07 |
Leisure Skills | −0.70 | Productivity | −0.70 | Leisure Skills | −0.92 | Leisure Skills | −0.61 |
Memory | −0.62 | Impaired Awareness | −0.52 | Social Contact | −0.52 | Social Contact | −0.45 |
DV = difficulty value. Items in italics changed from admission to discharge.
order of the first three items. For the TBI group leisure skills and memory were replaced by productivity (engagement in meaningful activity paid or unpaid) and impaired awareness in the top five. Transportation remained unchanged and presented the greatest magnitude of disability.
With age entered as a covariate, a RM MANCOVA revealed a significant main effect for pre-post testing, F(1, 871) = 128.97 p = 0.0005, Wilks Lambda = 0.87, partial eta2 = 0.13, power to detect = 1.00. Follow-up paired sample t-tests found that MPAI-4 T-Scores were significantly lower (less disability) at discharge for both the TBI and CVA groups.
MPAI-4 Indices | TBI | CVA | ||||
---|---|---|---|---|---|---|
Admission | Discharge | Cohen’s d | Admission | Discharge | Cohen’s d | |
Abilities T-score* | 57.8 | 48.9 | 0.84 | 58.9 | 49.9 | 1.3 |
Adjustment T-score* | 58.0 | 49.5 | 0.91 | 55.4 | 47.2 | 1.4 |
Participation T-score* | 56.7 | 48.3 | 0.79 | 57.7 | 49.0 | 1.4 |
*p < 0.001 for each comparison.
Results of the RM MANCOVA also showed a significant two-way measure by diagnostic group interaction, F (2, 870) = 10.89, p = 0.005, Wilks Lambda = 0.87, partial eta2 = 0.024. To interpret this interaction, independent group t-tests were performed on MPAI-4 admission and discharge T-scores. This analysis revealed that the TBI group had significantly higher T-scores (greater impairment) on the Adjustment Index at admission [t(872) = 3.25, p < 0.001)] and at discharge [t(872) = 2.40, p < 0.01]. No other comparisons reached statistical significance.
After discharge from acute hospitalization, persons who have suffered a TBI or CVA often face a lifetime of significant disability. Post-hospital brain injury rehabilitation programs provide comprehensive multidisciplinary treatment with the goal of reducing disability and restoring functional independence. While research has demonstrated the effectiveness of these programs [
The first step toward development of best practice requires application of psychometrically sound measures capable of reliably detecting change in performance. Consistent with previous research [
Change from admission to discharge for each of the MPAI-4 Indices yielded moderate to large effect sizes for both groups (Range = 0.79 - 1.40). While this is a positive finding, examination of mean T-scores at discharge across measures indicates that many participants in both groups were discharged with moderate levels of disability. Therefore, additional emphasis may need to be placed on other possible interventions to further reduce disability. One possibility would be to extend the time in program with the assumption that the addition of time in therapies that have been demonstrated to be effective would yield even greater improvement. While this is a logical assumption, in most cases time in program is not determined by the treatment team but by funding sources. These decisions are often based on short-term cost considerations rather than using an evidenced based model to determine appropriate length of stay to maximize disability reduction. Given the impact of potential funding limitation, treatment teams may be able to achieve greater disability reduction by using a prescriptive model in their rehabilitation treatments. Prescriptive modeling can target deficits in an established order thereby producing a greater impact on disability reduction. In addition this prescriptive modeling may also impact how and when remediation and compensatory strategies are used throughout the recovery process.
Rasch analysis assists in the meaningful targeting of treatment by identifying skills that have the highest probability of severe disability. The present study demonstrated that the CVA and TBI groups presented with different disability profiles at admission. The CVA group had a greater likelihood of experiencing disability in skills such as use of hands, mobility, visuospatial abilities, and novel problem solving. This pattern of disability is characteristic of focal lesions often seen in CVA. The TBI group was more likely to exhibit more diffuse disability including novel problem solving, memory, attention/concentration, impaired awareness and initiation. This constellation of cognitive and neurobehavioral symptoms is the hallmark of frontal and temporal lobe disruption associated with TBI.
Both groups experienced the greatest change with Abilities and Adjustment items, but the greatest challenge was within the applied skills of the Participation Index (e.g., instrumental activities of daily living). Rehabilitation within the first year of recovery tends to show the greatest gains with physical, cognitive, and communication skills along with moderate behavioral stability. However, application of skills into real-world settings and situations requires extensive learning and insight development that is often not evident until much later in recovery. Limitations experienced in these skills for the current study were the result of different patterns of disability with regard to the physical, cognitive, and emotional/behavioral functions that were related to the neuropathology and mechanism of injury type. Application of skills tends to be the greatest limiting factor in recovery from neurological injury.
Although both groups saw improvement on participation skills at discharge, greater reduction in disability may have been achieved by targeting the high impact deficits identified at admission with longer and more frequent therapies. Thus, this study provides an example of evidence-based hierarchical modeling with Rasch analysis to provide improved targeted treatment that is independent of time in recovery. The use of Rasch seems to be a promising application for the development of more hierarchical prescriptive treatment for persons recovering from TBI or CVA.
The authors declare no conflicts of interest regarding the publication of this paper.
Lewis, F.D. and Horn, G.J. (2018) Traumatic Brain Injury and Cerebral Vascular Accident: Application of Rasch Analysis to Examine Differences in Disability and Outcome in Post-Hospital Rehabilitation. Open Journal of Statistics, 8, 670-683. https://doi.org/10.4236/ojs.2018.84044