ABSTRACT

A growing literature indicates that learning potential (LP) measures, which examine performance changes following training on a task, may be important for understanding the role of cognition in functional outcome among people with schizophrenia and other serious mental illnesses. Because much of what is known about LP in this population has been demonstrated using the Wisconsin Card Sorting Test, the present study sought to extend this work by administering the Rey Osterrieth Complex Figure Test (ROCFT) in an LP format. 81 adults with schizophrenia or schizoaffective disorder were tested on the ROCFT using a test-train-test LP protocol. Results indicated significant performance improvements following training on the ROCFT. Further, the LP protocol differentiated subgroups of learners, non-learners, and high scorers, consistent with other LP work. These findings support the feasibility of adapting existing neurocognitive measures to examine learning potential. Further development of the LP literature is needed in order to examine the extent to which LP is test-dependent or is a more generalized construct.

1. INTRODUCTION

There is an extensive literature on cognitive impairments associated with schizophrenia; these deficits have been observed across a variety of domains such as attention, memory and executive functions [1-3]. Further, it has been reported that 20% - 60% of the variance in outcomes for people with schizophrenia may be attributed to cognition [4]. In recent years, there have been efforts to examine more complex models of the cognition-outcome relationship [5-7]. In this regard, learning potential (LP) has been proposed as an important variable in models predicting functional outcomes [4,8].

Learning potential refers to one’s ability to benefit from instruction and attain/utilize new skills and knowledge. LP typically is measured using dynamic assessment methods, which employ repeated test administrations and a training component. Improvement following training is taken as an index of LP. Dynamic assessment can be contrasted to traditional testing methods, which are static in nature and designed to capture a snapshot of current performance in a particular domain. The concept of dynamic assessment is often credited to the work of Vygotsky [9] and has been applied most extensively in educational research. More recently, dynamic assessment and LP have been extended to areas such as traumatic brain injury [10] and studies of schizophrenia [11].

Wiedl and colleagues [11,12] applied LP to schizophrenia research when they introduced a learner status typology based on dynamic assessment with the Wisconsin Card Sorting Test (WCST), a measure of executive functioning. Using a pre-test/training/post-test paradigm, Wiedl [11] identified three learner subtypes based on changes in performance. Learners were those who performed poorly at pre-test, but showed significant improvements at post-test, following training. Non-retainers were individuals who performed poorly at pre-testanddid not have significant improvement at post-test. High-scorers were those who performed well across both preand post-test.

Subsequent to this original work, several studies have examined LP in persons with schizophrenia and other serious mental illnesses and found that it is associated with various dimensions of functioning. LP has been associated with attention and memory [12,13-15], as well as distinct patterns of cerebral metabolism [16,17]. In terms of functional outcomes, LP has been associated with work skills [18], functional status [13], and a measure of rehabilitation readiness [19]. Further, some studies have found positive associations between LP and intervention response [8,11,20], although it should be noted that Tenhula et al. [21] and Wooning et al. [22,23] reported negative findings in studies examining intervention response. In sum, there has been a growing literature on the potential value of LP in schizophrenia research and there is need for further development in the LP literature, including expansion of the concept beyond the limited measures that have been used.

Most of the schizophrenia-LP studies reviewed above involved the WCST. There have been limited efforts to extend LP research to other measures, including the California Verbal Learning Test-II [13,14,19] and the Auditory Verbal Learning Test [12]. The present study extends the LP literature by using a novel measure, the Rey Osterrieth Complex Figure Test (ROCFT) [24-26]. Although the ROCFT has been previously administered in an LP-type format with children [27], we are not aware of published studies of LP and schizophrenia using the ROCFT.

The ROCFT is a commonly used measure of visual memory, requiring test-takers to copy and later recall a complex, two-dimensional design containing abstract elements. Visual memory impairments on the ROCFT have been well-documented among people with schizophrenia, and are influenced by the use of poor organizational strategies [28-30]. Thus, the ROCFT is an excellent candidate for adaptation as an LP measure, in which test-takers are provided with the opportunity to learn an organizational strategy.

The purpose of present study was to examine the feasibility of using the ROCFT as a learning potential assessment in people with schizophrenia. Specifically, we had two aims: 1) to examine whether our training improved participant performance on the ROCFT and 2) to examine whether this ROCFT measure would distinguish three learner groups identified in prior LP studies: higher scorers, learners and non-learners.

2. METHOD

2.1. Participants

Eighty-one participants with schizophrenia or schizoaffective disorder were recruited from three community mental health centers using flyers, announcements, and a peer recruiter at each site. Participants received compensation of $25. All participants provided informed consent and all study procedures were reviewed and approved by the relevant institutional review boards. The Structured Clinical Interview for DSM-IV [31] was administered and the chart was reviewed to insure that participants met eligibility criteria for schizophrenia (n = 45) or schizoaffective disorder (n = 36). Exclusionary criteria included: substance abuse/dependence in past 30 days, and known developmental or neurological disability. Medication data were available for 73 of 81 participants. Of these, 4 were not taking antipsychotic medication and 69 were prescribed antipsychotics (62 were prescribed second generation antipsychotics and 7 were prescribed only conventional antipsychotics). Average age of participants was 41.7 (SD = 8.6) years; 40 were female and 41 were male. Table 1 includes additional demographic information.

The Scale for Assessment of Negative Symptoms (SANS) [32] and the Scale for the Assessment of Positive Symptoms (SAPS) [33] were administered as indices of symptom severity. The sums of all positive and negative symptoms (excluding global ratings) were calculated for the SANS and SAPS. Participants had a mean SANS total score of 21.25 (SD = 11.85) and a mean SAPS total score of 24.35 (SD = 15.06).

2.2. Learning Potential Assessment

The LP dynamic assessment protocol involved three consecutive administrations of the ROCFT: 1) The first administration (pre-test) followed standard procedure for that test: i.e. a figure copy trial followed by a standard recall trial in which the participant was asked to recreate the figure from memory; 2) The second administration of the task involved the training/learning phase, in which the participant received additional coaching and cues. Specifically, the training highlighted and reinforced the organization and structure of the figure. The participant was taught to construct the design in three sequential steps, beginning with larger structural elements and then filling in smaller details. If participants made a mistake, they were corrected before continuing. At the completion of their drawing, the participants were directed to notice the components and organizational features of the complex figure. The organizational strategy was similar to methods described by Hadas-Lidor & Katz [34]. Similarly, during recall, participants were cued as needed for the design elements and organizational sequence; 3) The post-test administration of the ROCFT followed standard copy and recall procedures. ROCFT drawings were scored according to Meyers & Meyers [26], yielding a scoring range from 0 to 36 for the copy and the recall trials.

2.3. Statistical Analyses

In order to examine our first aim, paired samples t tests were conducted to examine changes in mean performance from pre-testing to the training and post-testing trials. As a second aim, we classified participants into three learner groups on the basis of the algorithm previously reported e.g. [35] and used Analysis of Covariance (ANCOVA) to compare ROCFT recall performance across the groups, while controlling for positive symptoms.

3. Results

3.1. ROCFT Performance across Trials

Means and standard deviations for the three ROCFT trials are presented in Table 2. Paired samples t tests were conducted to evaluate whether there were significant

performance improvements from the pre-test to the training/post-test trials. In terms of copy performance, the results indicate a significant improvement from pre-test to both the training [t(80) = −11.26, p < 0.01) and the post-test [ t(80) = −6.79, p < 0.01]. Similarly, on the recall trials, there was a significant improvement from the pre-test to the training [t(80) = −14.18, p < 0.01] as well asbetween the pre-test and the post-test [t(80) = −9.15, p < 0.01].

3.2. Learner Status Categorization

Classification into high-scorer, learner and non-learner categories used the algorithm developed by Schottke et al. [35] and used in other studies (e.g. [15]) of LP and schizophrenia. The algorithm uses reliability of the measure and the standard deviation of the test to compute a standard error of prediction that is used to determine a confidence interval. Scores outside the confidence interval are classified as a real change or difference. The advantage of this method of classification is that it avoids problems associated with unreliable change scores.

A cutoff of 9 points was estimated to indicate significant change in recall performance. Initially this cutoff score was applied to the total possible score of 36 on the ROCFT, meaning that a score of 27 or above would classify the individual as a high scorer; however, no participants in the study scored in this range. Therefore the upper range of the test was revised and a score of 26 was selected as this represents the upper band of the 95% confidence interval for normative data derived from a meta-analysis of CFT studies [36]. This seemed reasonable as an upper range instead of the total score of 36 since only 5% of individuals in the normative sample scored above a 26. With an upper range of 26 and a cutoff of 9 points, high scorers would have to achieve a score of 17 or higher and learners would have to improve their score by 9 points or greater from pre-test to posttest.

In our sample there were 11 high-scorers (13.6%), 37 learners (45.7%), and 33 non-learners (40.7%). There were no significant differences among the learner groups in terms of age (F = 1.07, p = 0.35) or SANS ratings (F = 0.08, p = 0.92). There was a significant group difference on the SAPS (F = 3.58, p = 0.03). Follow-up t-tests for unequal variances were conducted and revealed that non-learners had significantly higher ratings on the SAPS (M = 29.63, SD = 16.98) than high scorers (M = 19.18, SD = 10.74; t(27.92) = −2.37, p = 0.03) and learners (M = 21.32, SD = 13.21, t(58.23) = −2.24, p = 0.03), which did not differ from one another. Therefore, SAPS scores were entered as a covariate in the analysis of covariance (ANCOVA) reported below.

3.3. Learner Status and ROCFT Performance

Recall performance on the ROCFT at the three testing points (pre-test, training, and post-test) for each of the learner groups are displayed in Figure 1. As discussed by Vaskinn et al. [23], examining performance improvements of each of the three learner groups can serve as a validity check for learner group classification, as it is assumed that learners should exhibit greater performance gains than the other two groups. An ANCOVA was conducted with learner group status as the independent variable and performance improvements from pre-test to post-test on ROCFT recall as the dependent variable. SAPS scores were entered as the covariate. Preliminary

analysis indicated that the homogeneity-of-slopes assumption was met [37]. The ANCOVA was significant, F(2, 76) = 42.57, p < 0.001, and follow-up tests based on the LSD procedure evaluated pairwise differences among the adjusted means. As expected, the adjusted means for the learners (M = 14.19, SE = 0.60) differed significantly from the adjusted means for both the high-scorers (M = 8.29, SE = 1.11) and non-learners (M = 6.03, SE = 0.66), indicating that the learner group demonstrated significantly greater improvements from pre-test to post-test than the other two groups. Adjusted means for the high-scorers and non-learners did not differ significantly from each other.

4. DISCUSSION

This study examined a novel neurocognitive assessment, the ROCFT, as a learning potential measure in persons with schizophrenia spectrum disorders. This is the first study, to our knowledge, to explore the ROCFT as a measure of LP in people with schizophrenia. With regard to our first aim, we found that in the full sample of participants, there were significant performance improvements from pre-testing to the training and the post-testing trials for both recall and copy portions of the ROCFT. Given the research that persons with schizophrenia exhibit difficulties with this complex figure task, it is noteworthy that the present sample showed significant performance improvements following training with a simple organizational strategy.

Further, our results provide preliminary support for the use of the ROCFT specifically as a learning potential measure. This adapted ROCFT protocol was able to differentiate among the three learner subtypes established in prior research e.g. [11,23]. As expected, the group designated as learners exhibited significantly greater improvement from preto post-testing on the ROCFT than the other two groups, further supporting the validity of this categorization.

One associated limitation of the present study is the possibility of simple practice effects that might result from the repeated testing in learning potential designs. Thus, it may be that some participants showed performance improvements not because of directly benefitting from the organizational strategy provided, but due to repeated exposure to the figure across three trials. Certainly this is a possible limitation in LP designs and further research is needed to address it. However, it should be noted that not all participants appear to benefit from the training or the repeated exposure (i.e. non-learners). In fact, one of the benefits of an LP approach is that the learner status categorization places emphasis on intra-individual change and therefore allows for the identification of subgroups with these very different performance patterns.

It is important to point out that because this is the first study to examine LP in people with schizophrenia using the ROCFT, further research is needed to confirm the findings. As stated previously, most of the prior LP research in this population has been conducted using the WCST, so it has been unclear whether the extant findings have reflected generalized learning ability or are specific to the particular task used. The present study extends this literature by demonstrating that a novel cognitive task can successfully be adapted as a measure of LP. The ROCFT, in particular, may be a useful LP measure as there has been relatively little research in the field of schizophrenia on the potential role in functional outcomes of tasks with high visual-spatial demands. There is some face validity to the idea that many aspects of real-world functioning, such as transportation, shopping, or work skills, require complex visual-spatial processing. Further research with the ROCFT in terms of functional outcomes and learning potential are therefore warranted.

5. ACKNOWLEDGEMENTS

This study was supported by grant #5R03MH68601 from the National Institute of Mental Health (NIMH).

REFERENCES

NOTES

^*Corresponding author.