ABSTRACT

Objective: This study presents the psychometric properties of the Charleston Psychiatric Outpatient Satisfaction Scale—Hebrew Version adapted for use in Israel. Methods: Two sub-versions were constructed: one for adult clients—Mental Health Client Satisfaction Scale-Hebrew (MHCSS-H) and one for parents (or other caregivers) accompanying children with mental health problems—Mental Health Parent Satisfaction Scale-Hebrew (MHPSS-H). The scales were administered to representative samples of 453 adult outpatients with severe mental disorders and 255 parents, respectively. Results: Internal consistency was excellent for both scales (Cronbach’s alpha coefficients 0.94 and 0.88, respectively). For both subvertsions, there were moderate to strong correlations between satisfaction with five service domains (Availability/Accessibility, Quality of care, Explanation/ Participation, Staff’s attitude, and Facilities conditions) and anchor items (“Overall quality of the care provided” and “Would you recommend this clinic to a friend or relative should they need treatment?”) scores. Clients with more time in treatment were more satisfied with all the service domains, except for the staff's attitude domain. Conclusions: The results suggest that both Hebrew subversions are appropriate for routine satisfaction surveys in mental health outpatient settings and for research purposes in Israel.

1. INTRODUCTION

Assessing clients’ satisfaction with services is an important component of quality assurance procedures in the health field. Such quality assurance programs have only recently been developed and implemented in the mental health service system in Israel. As part of these efforts, the need for a psychiatric outpatient satisfaction scale was identified. Steps were then taken to review existing scales and examine the possibility of developing a new one.

The assumption that surveys using self-administered satisfaction questionnaires contribute to the quality of services was investigated in relation to inpatient [1-5] as well as outpatient [6-9] mental health services. A number of factors contributing to clients’ satisfaction were noted: the professional level of therapists; their attitude and level of empathy; continuity of care and provision of information and explanations about the client condition and treatment process. Yet, some researchers [10] continue to doubt the methodological soundness of surveys based upon self-administered questionnaires. The main concern is the clients’ ability to provide unbiased and reliable views of the services, as well as the phenomenon of “hallo effect”, i.e. the tendency to give very high counterintuitive ratings with little variability [5,6]. These doubts about the validity of self-administered satisfaction questionnaires were seriously considered while setting out to develop quality assurance procedures for the Israeli mental health outpatient service system. A number of considerations guided the authors, in their respective administrative and research capacities, through the process of choosing a course of action. Most importantly, it was thought that offering providers, in the Israeli mental health outpatient service system, with face-valid satisfaction scales would be a timely organizational step. This step would agree with prevailing views about the need to empower clients and to increase their participation in the feedback loop of quality assurance processes [11]. Once the use of satisfaction scales becomes a standard in the service system, improvements in questionnaires construction and advances in survey’s technology can be introduced. Next it was necessary to choose an appropriate satisfaction scale. The review of the relevant literature revealed the Charleston Psychiatric Satisfaction Scale (CPSS), a brief questionnaire designed for use in outpatient mental health settings [7]. The questionnaire fits the Client Satisfaction Questionnaire model [12] that focuses on evaluating satisfaction with such aspects of the service as accessibility, availability, amount and quality of information given and attitudes toward the client rather than on measuring satisfaction with the treating agent (psychiatrist or psychotherapist) or specific outcome variables. After the CPSS was chosen, the authors translated the instrument into Hebrew with few necessary modifications. This paper reports the validation study of the two Hebrew subversions of the CPSS adapted for use in Israel.

2. METHOD

2.1. The Questionnaire

The Charleston Psychiatric Satisfaction Scale is a selfreport questionnaire designed to measure satisfaction with mental health services among clients visiting outpatient mental health clinics. Its 15 items describe diverse service domains of satisfaction, including two anchor items (“Overall quality of the care provided” and “Would you recommend this clinic to a friend or relative?”). Responses are rated on a 5-point Likert scale ranged from 1 (“very dissatisfied”) to 5 (“very satisfied”), with higher score indicating more satisfaction.

Prior to administration the instrument was translated into Hebrew taking into account cultural aspects of the questions. Then, the questionnaire was slightly modified: two items from the original questionnaire concerning payment (items 2 and 14) were deleted as irrelevant for the Israeli clinics, where the treatment is free of charges. Instead, 3 new items were added as they were deemed important in the Israeli settings: 1) waiting time for a response when calling the office, 2) attitude of treatment staff, and 3) flexibility in scheduling sessions. Thus, the modified instrument comprised of 16 items.

Two subversions of the Israeli instrument were prepared and validated. One for adult clients entitled the Mental Health Client Satisfaction Scale-Hebrew (MHCSSH), and one for parents or other caregivers accompanying children with mental health problems—Mental Health Parent Satisfaction Scale-Hebrew (MHPSS-H). In the latter, the contents of the items remained unchanged but they were rephrased to reflect the fact that it was a proxy evaluation of the service received by the child. A bilingual colleague not involved in the project, translated the scales back to English. The back-translations were compared to the original questionnaire, both translators discussed the few discrepancies revealed, and a third bilingual colleague was consulted. The items in question were then presented as a pilot test to the first 10 subjects recruited to the study. Evaluation of the responses confirmed face validity of the translation.

2.2. Data Collection

Research setting for this study comprised 27 government-owned community mental health clinics (CMHC) located across Israel. In 15 of the clinics the client’s version of the instrument was administrated (MHCSS-H), and in 12 others the parent version (MHPSS-H) was used. A total of 453 clients and 254 parents or other caregivers completed the questionnaire. In order to encourage clients to fill out the questionnaire, anonymity was assured.

Anonymity was provided by the following procedure. Self-administered questionnaires were placed in waiting area in an accessible spot. By use of notices, placed on visible spots, the clients visiting the clinic were notified of the survey and encouraged to approach the receptionist for instructions. All receptionists were instructed to encourage the clients to participate in the survey without exerting any pressure on them. To each questionnaire was attached information sheet where the aim of the study (validation of the Hebrew version of the instrument) was explained. Questions on time elapsed in treatment and number of therapeutic sessions received were included. All completed questionnaires were dropped into available ballot-like boxes.

2.3. Ethics

The Institutional Review Board in each participating center approved the survey and waved the requirement for informed consent because of the voluntary and anonymous nature of the survey.

2.4. Data Analysis

All analyses were performed using the SPSS-17 software package (SPSS Inc., Chicago, Illinois). Mean scores with standard deviations and median scores (if need) were calculated for each item and for the total instrument. Internal consistency of both Hebrew subversions of the instrument was assessed with Cronbach’s coefficient α. To examine convergent validity, Pearson product moment correlation coefficients were computed among all items scores, between each item and anchor items scores, and, finally, between the five instrument domains and the anchor item scores. Wilcoxon sum-rank tests were performed to examine an association between duration of treatment (or number of sessions received) and client’s satisfaction with distinct service domains.

3. RESULTS

3.1. Psychometric Properties

The internal consistency of both Hebrew subversions was excellent (Cronbach’s α was 0.94 for the client’s version and 0.88 for the parent version). Preliminary convergent validity of the MHCSS-H was supported by the finding that scores for all items were significantly correlated with the scores for the two anchor items (r = 0.32 to 0.79 for anchor item 9, “satisfaction with overall quality of care”, and r = 0.32 to 0.74 for anchor item 16, “likelihood of recommending the clinic to others in need”). The corresponding figures for the parent version of instrument were similar for anchor item 9 (r = 0.20 to 0.70) and were somewhat lower for anchor item 16 (r = 0.21 to 0.58) (Table 1). Correlations between the two anchor items were r = 0.74 for the client’s version of the instrument and r = 0.56 for its parent version.

3.2. Service Satisfaction Domains

Correlational analysis of both subversions of the instrument showed that their item scores were strongly intercorrelated, except for items 10, 14, and 15, where intercorrelations also were significant but moderate in magnitude (Table 2). Based on consistently high correlations between distinct but logically connected items, we grouped them in five distinct domains of service-related satisfaction: availability/accessibility of care (items 1, 3, 12 to 15), quality of received care (items 9 and 16), explanation/participation in care (item 4 throughout 7), staff’s attitude to patients (items 2 and 8), and facilities’ conditions (items 10 and 11).

As can be seen, the MHCSS-H and MHPSS-H domains had moderate to strong correlations with the anchor items demonstrating their good convergent validity (Table 3). Importantly, in both subversions of the instrument, the quality of care domain had the highest correlations with the anchor items, whereas the facilities’ conditions domain had lowest correlations.

3.3. Time in Treatment

For 409 adult clients mean time in treatment was 18.1 months (SD = 8.5), and median time was 25 months. Only nine patients (2.2%) reported that they were only one month in the current treatment. Mean number of treatment sessions was 19.7 (SD = 7.7), and median number of the sessions was 25. Only 23 patients (5.6%) reported that they had one or two treatment sessions. Thus, the vast majority of the clients had enough time to be well familiar with the service in order to be able to assess the extent of their satisfaction.

To examine an association between satisfaction with service and time elapsed in treatment, we splitted the entire sample into two subgroups: chronic (n = 197) and

non-chronic (n = 212) clients. Chronic clients were operationally defined as those who were in treatment 25 months and over, and had at least 20 treatment sessions. We found that the chronic clients scored higher on satisfaction with all domains (all p < 0.05) than their nonchronic counterparts, except for the staffs’ attitude

domain where no between-group differences were noted (Table 4). The analogous comparison for the parent subversion of the instrument showed no differences between parents of chronic (n = 181) and non-chronic (n = 59) clients with regard to their satisfaction with all the service domains.

4. DISCUSSION

We investigated the psychometric properties of the Charleston Psychiatric Outpatient Satisfaction Scale— Hebrew Version in the two representative samples of adult outpatient mental health clinics and of parents or other caregivers accompanying children with mental health problems to child and adolescent facilities. We demonstrated excellent internal consistency of both subversions of the instrument and their good convergent validity. We also were able to show a direct association between duration of treatment and satisfaction with the service domains. The obtained results suggest that both subversions of the instrument are appropriate for use in outpatient mental health facilities in Israel for both quality assurance and research purposes.

The main difference between the Hebrew version of the scale from the original instrument, is grouping items into five distinct service domains: Availability/Accessibility, Quality of care, Explanation/Participation, Staff’s attitude, and Facilities’ conditions. These domains allowed a differential assessment of satisfaction with diverse components of services and comparisons between them. This comparative analysis has demonstrated (what was intuitively expected) that satisfaction with the quality of care is the substantially more important characteristic of the mental health service than its other characteristics. More specifically, the results show that satisfaction with the overall quality of care seems to be the most important factor in determining whether or not a client would choose to recommend the clinic to a friend. Other factors, such as flexibility in scheduling sessions, attitude of treating staff, and the physical conditions (convenient location, comfortable parking, etc.) seem less important.

Another interesting finding was that duration of treatment is associated with the client’s satisfaction with almost each service domain, except the Staff’s Attitude domain. This finding may suggest that over time clients tend to become accustomed and satisfied with various aspects of the treatment setting but not necessarily with the persons rendering the treatment. Further, we may note that as reported by others (Barak, 2001), one of the key factors determining satisfaction appears to be the extent to which clients received information or explanations about their treatment. The findings of our validation study appear to support this (also intuitively anticipated) observation. We found that the correlation between the Explanation/Participation domain and anchor items (9 and 16) scores was 0.83 and 0.74, respectively. This indicates a strong association between the general level of the client’s satisfaction and the degree of client’s involvement in their own treatment, brought about by information and by being given a chance to voice their preferences.

Owing to the fact that study’s survey was anonymous, we have no information about diagnoses of those filling out the questionnaires. From utilization studies of the government-owned service system in Israel we know that clients represent the wide range of diagnoses and that about 50% of the clients visiting the clinics at any given time have a diagnosis of severe mental illness [13]. Therefore, we may assume that our validated instrument is appropriate for use across a wide range of diagnoses. Finally, we suggest that further studies should aim at investigating possible correlation between patterns of satisfaction and type of diagnosis. Such studies may contribute to attempts at improving overall quality and satisfaction with services.

5. ACKNOWLEDGEMENTS

We are grateful to staff members of all participating Community Mental Health Clinics for their assistance with data collections. Dr. A. M. Ponizovsky was supported in part by the Ministry of Immigrant Absorption.

REFERENCES

NOTES

^*Corresponding author.