ABSTRACT

The aim of the present study was to evaluate the psychometric properties and dimensionality of the instrument Quality in Psychiatric Care-Forensic In-Patient Staff (QPC-FIPS) and to describe the perceived quality of psychiatric care among forensic inpatient service staff. A sample of 348 forensic inpatient staff from 18 forensic wards in Sweden participated in the study. A confirmatory factor analysis revealed a seven-factor structure with item loadings > 0.50 on expected factors, indicating adequate psychometric properties. The staff’s ratings of quality of care were high, 94% being positive. The highest ratings were found for the secluded-environment dimension and the lowest for the secure-environment dimension. Several factors influenced the ratings of quality of care, for instance, staff’s time to perform their duties and staff’s age. It is concluded that the QPC-FIPS can give valuable information about staff’s perceptions of the quality of care provided at inpatient forensic psychiatric care services, which can be used to identify areas for quality improvement. Use of the QPC-FIPS is an easy and inexpensive way to evaluate quality in forensic inpatient care, preferably in conjunction with the QPC-FIP instrument developed for forensic inpatients and covering the same items and dimensions.

1. INTRODUCTION

Quality of care in psychiatric care is considered to be a multifaceted concept [1] consisting of many aspects [2]. In contrast to patient satisfaction, quality of care includes the perspectives of all interested parties [3,4]. The staff’s experience of the care they provide can be seen as one aspect of quality of care and could be used as an additional indicator of quality of care [5]. It has been shown, however, that the different professions in general psychiatric care have dissimilar views on what characterizes quality of care [6,7]. In spite of this discrepancy, staff are expected to cooperate to give the patient good quality of care [8].

Evaluations of health care are increasingly seen as an important outcome in the aim to improve care [9]. Patients’ assessments of quality of care in psychiatric services are often used as an important outcome [10], while staff’s assessment of quality of care rarely is used in psychiatric services [7,11].

Previous studies regarding staff’s assessment in forensic psychiatric care have focused on, for instance, risk assessment [12], burnout, and job satisfaction [13], what nurses perceived as important in forensic nursing [14], and male caregivers’ experiences of giving care to patients [15]. Obviously, little attention has been given to the investigation of staff’s assessment of the quality of care in forensic psychiatric services. One reason could be that there is a lack of instruments for measuring quality of care in forensic psychiatric care.

There seems to be need of such an instrument, as Jung et al. [16] are of the opinion that staff’s assessment of the quality of care they provide would be important information in the work to improve the care, which subsequently could lead to changes in organization of care or in staff’s performance.

A measuring instrument should be designed for the specific services and settings in which quality of care is to be evaluated [10], as quality of care or satisfaction is considered to be context specific [17,18]. The instrument Quality in Psychiatric Care (QPC) is based on a definition of quality of care from the patients’ perspective in psychiatric care [19]. The definition has been developed from a phenomenographic interview study with inand outpatients [20]. The instrument was tested for face validity in a pilot study and also empirically tested [19]. The instrument QPC is a family of instruments developed for specific psychiatric settings, with a common core and context-dependent modules. The QPC family currently consists of four instruments, one psychometrically tested for in-patients (QPC-IP) [21], one for outpatients (QPC-OP), [22], one for forensic in-patients (QPC-FIP) [23], and one for forensic in-patient staff (QPC-FIPS), which was used in this study.

Measures of quality of care have traditionally and up till now concentrated only on the patients’ views in forensic inpatient care. From the best of the authors’ knowledge, there is no existing instrument to measure staff’s perception of the quality of forensic psychiatric inpatient care provided, based on what patients perceived as high quality of care. Often there have been the opposite; instruments measuring patients’ perceptions of quality of care have been based on professionals’ views [24, 25]. When a new measurement instrument is developed, it is necessary to develop, refine, and establish the instrument’s psychometric properties [24,26]. Therefore, the aim of this study was to evaluate the psychometric properties and dimensionality of the instrument Quality in Psychiatric Care-Forensic In-Patient Staff (QPC-FIPS) and to describe the perceived quality of psychiatric forensic care among forensic inpatient service staff.

2. METHODS

2.1. Participants and Procedure

The sample consisted of all staff permanently employed at 18 inpatient forensic wards at two regional forensic psychiatric clinics and two county departments that conduct highly specialized care in Sweden. The data were collected during six weeks in October and November 2011.

The staff eligible for participation were informed orally and in writing about the aim of the study, their right to withdraw from the study at any time, and that participation was voluntary. The staff who gave their consent completed the QPC-FIPS questionnaire anonymously and returned it in a sealed envelope in a letterbox at the department or gave it to a contact person. During the data collection period, a total of 508 staff were permanently employed at the forensic inpatient wards. Six did not receive the QPC-FIPS due to staff-related reasons and 148 declined to participate. Six of the questionnaires collected had 30% or more missing items and were excluded from the analysis. Hence, the study group consisted of 348 staff. Characteristics of the study group are presented in Table 1. The study was approved by a Swedish regional research ethics committee.

2.2. The QPC-FIPS Instrument

The staff’s perceptions of the quality of care were obtained using the QPC-FIPS. The QPC-FIPS consists of 34 items and is based on the QPC-FIP instrument by Schröder et al. [23], with the items reworded to fit the context of forensic inpatient staff (Table 2). QPC-FIPS is a self-administered instrument designed to measure seven dimensions: encounter (8 items), participation (8 items), discharge (3 items), support (4 items), secluded environment (2 items), secure environment (3 items), and forensic specific (6 items). Each item was related to the statement “I feel that…” The responses were made on 4-point Likert scales, ranging from 1 (totally disagree) to 4 (totally agree). For each item there was also the possibility of answering “not applicable”. In addition, the questionnaire included a number of background questions covering demography and general clinical characteristics, and an open question at the end inviting comments from the responder. To access the instrument, please contact the first author.

2.3. Data Analysis

Descriptive statistics were conducted using SPSS 17. Cronbach’s alpha [27] was used to assess the scales. The 0.70 criterion was used for adequate homogeneity [28]. Confirmatory factor analysis (CFA) was performed using LISREL 8.8 [29]. Generally Weighted Least Squares estimation was used on the asymptotic covariance matrices and the polychoric and polyserial correlation matrix, which were obtained by the PRELIS program [30]. The parameters were estimated by the Weighted Least Squares (WLS) method using the asymptotic covariance matrix, which is recommended by Jöreskog and Sörbom [29] when the measurement variables are ordinal.

Imputation was performed before analysis by replacing missing data points with the mean of that item. To test the tenability of the a priori proposed factor structure model based on the QPC-FIP [23], a CFA was performed, specified with the following model: Items 7, 10, 11, 12,

15, 17, 19, and 24 were assumed to represent encounter; items 1, 5, 6, 13, 14, 25, 27, and 28 represented participation; items 8, 16, and 20 represented discharge; items 18, 21, 22, and 23 represented support; items 3 and 26 represented secluded environment; items 2, 4, and 9 represented secure environment; and items 29 to 34 represented the forensic department situation.

The adequacy of the models was evaluated using the Satorra-Bentler scaled chi-squared test in addition to the Comparative Fit Index (CFI), the Standardized Root Mean Square Residual (SRMR), and the Root Mean Square of Approximation (RMSEA). Values ≥ 0.90 and 0.95 for the CFI, and £0.10 and 0.08 for the SRMR, and <0.08 and 0.05 for the RMSEA were considered to constitute an adequate and excellent level of goodness of fit, respectively [31-33]. These criteria are arbitrary, and it is unclear what the absolute fit reference standard should be [34-36].

3. RESULTS

3.1. Psychometric Evaluation of the QPC-FIPS

Confirmatory factor analysis was performed on the model that represents the factor structure based on the QPCFIP factor structure. The CFA showed a significant chi square (SBχ² = 1011.75, df = 506, p < 0.001), a CFI = 0.99, an RMSEA = 0.054 (CI = 0.049 - 0.058), p-value for test of close fit RMSEA (RMSEA < 0.05) = 0.10, and an SRMR = 0.071, indicating an excellent goodness of fit. Summary statistics are given in Table 2.

As seen in Table 3, six (29%) of the correlation coefficients among the seven QPC-FIPS dimensions were strong (>0.60), and six coefficients were moderate (0.40 - 0.60) in size, implying the possibility of a common second-order factor. A test of a common second-order factor showed, however, a slightly worse fit (SBχ² = 1040.4, df = 520) and was discarded. Therefore, the a priori seven first-order factor model with 34 items was regarded as a satisfactory representation of the factor structure of the QPC-FIPS and was considered the final model.

3.2. Descriptions of the Staff’s Perception of the Quality of Forensic In-Patient Care

Mean and standard deviations of the seven QPC-FIPS dimensions are given in Table 4. As seen in Figure 1, the staff perceived the quality of the secluded-environment dimension as the highest, followed by the support dimension, the forensic-specific dimension, the encounter dimension, the discharge dimension, and the participation dimension; and finally, they perceived the secure dimension as the worst one. Bonferroni-corrected t-tests showed that the perceived quality of any QPC-FIPS dimension was significantly larger than the one following it in rank (all p’s < 0.05), except for the forensic-specific dimension (third in rank) and the encounter dimension (fourth in rank), which did not differ significantly from each other (p > 0.10).

3.3. Demography and Quality of Care

There were almost as many men as women working in the departments (Table 1). A vast majority (97%) were Swedish, and approximately half of the staff were 50 years or older. Half of the staff had worked for 10 years or more in their current profession, and about 12% had worked for more than 40 years.

The univariate regression analyses of the QPC-FIPS total and dimension scores showed significant relationships with several of the background questions (Table 5). The QPC-FIPS total score was related to all background variables except gender and nationality. For the QPCFIPS dimensions there were no gender differences in regard to perceived quality of care. Staff’s age and year in occupation were significantly related to perceived quality of care in some of the QPC-FIPS dimensions. Swedish staff had significantly higher ratings on the secure environment than non-Swedish staff. The greater the degree to which the staff felt they had sufficient time to perform their duties, the greater they rated the quality of encounter, participation, and secure environment. The more the staff felt their work to be stimulating and the better they perceived the psychosocial working situation, the greater they perceived the quality of care in all QPCFIPS dimensions, except secluded environment. The more the staff were open to developing quality assurance work at the department, the greater their perception of the quality of care in encounter, participation, discharge, and forensic-specific dimensions. Finally, assuming that they did not work there themselves, the more the staff agreed that if they had a relative who was in need of forensic psychiatric care they would want him/her to be cared for at that specific department, the higher the ratings on any QPC-FIPS dimension.

In addition, one-way ANOVA on working time (days, nights, or days and nights) showed it to be related to perceived quality of discharge only (F(2,339) = 6.21, p = 0.002). Staff working day shifts perceived the quality of discharge to be significantly higher (M = 3.19 SD = 0.57)

Note: Two male participants and one female participant had missing data on age. Additionally, six participants had missing data on gender and age.

in post hoc Tukey HSD tests (α = 0.05) than those working days and nights (M = 2.99, SD = 0.62) or those working night shifts only (M = 2.88 SD = 0.62), respectively. The two latter groups did not differ in perceived quality of discharge. Furthermore, one-way ANOVA analyses on staff’s perception of quality of care related to type of staff’s profession (see Table 1 for type of professions) revealed significant effects in the encounter (F(8,331) = 4.78, p < 0.001), the participation (F(8,331) = 2.77, p = 0.006), the support (F(8,331) = 4.98, p < 0.001), the secluded environment (F(8,331) = 2.23, p = 0.025), and the forensic-specific dimensions (F(8,331) = 2.79, p = 0.005). However, post hoc Tukey HSD tests revealed inappreciable differences in encounter, supportand secluded-environment dimensions; psychologists perceived the quality as significantly lower than did psychotherapists. It should be noted that these professions comprises about 2% of the staff.

3.4. Multiple Regression Analysis

Follow-up stepwise multiple regression analyses (Table 6) revealed that the four variables, “relative in care at department”, “work stimulating”, “year in profession”, and “nationality” significantly predicted the QPC total score. The variable “relative in care at department” was the strongest predictor for all QPC dimension scores, and the only significant one for encounter, participation,

support, and secluded environment. In addition, age was a significant predictor when it came to the forensicspecific dimension. The older the staff, the more favorably they perceived the quality of the environment for the patients at the clinics to be. Finally, the better the psychosocial working situation, the better the staff perceived the security on the department to be.

4. DISCUSSION

The instrument Quality in Psychiatric Care-Forensic InPatient Staff (QPC-FIPS) was developed to capture staff’s experiences of the quality of forensic inpatient care provided. The focus of forensic inpatient staff’s ratings of quality of care have not previously been investigated in this area. However, staff ratings of the quality of care they provide can be a good complement to patients’ ratings of their experiences of the quality of care received, as both perspective are needed in the improvement of the care.

It is said that studies regarding quality of care that lack validity and reliability cast doubt on the credibility of the results [37]. In the present study the newly developed instrument QPC-FIPS has good evidence for reliability and validity. That is, the a priori model of the QPC-FIPS factor structure, based on the patients’ view of quality of care (i.e., QPC-FIP), received excellent goodness of fit. This result makes a strong argument for the reliability and validity of the QPC-FIPS instrument. Moreover, the results show that staff’s views of the concept quality in forensic psychiatric care are highly similar to patients’ views of the concept, which does not necessary imply that they agree on the level of quality of care provided.

However, the equality of the concept in staff and patients opens up for such comparisons. Although the full instrument had excellent psychometric quality, some dimensions were below adequate homogeneity, particularly the secluded-environment dimension. Given that it consists of two items, lower internal consistency is to be expected. However, since the loadings that represent item validity were moderate to high and accorded with the loadings of the other dimensions, the lower homogeneity is of less concern [38].

Somewhat surprisingly, almost all of the staff (94%) perceived the quality as high, that is, having a mean score greater then 2.5 (which is the center of the scale in QPC). Previous research on general psychiatric inpatient care has shown that staff have the highest ratings of quality in items addressing the staff-patient relationship, such as how they related to the patients, interest in solving patients’ problems [39], and emotional aspects of caring [40]. Consistent with that, the support dimension was among the highest rated dimensions in the present study. However, the highest rating of quality of care was found for the secluded-environment dimension. This is not an unexpected finding, as the secluded environment has been identified as an important aspect for quality of psychiatric care from the staff’s perspective [41,42]. In a previous interview study, the staff described good quality in psychiatric care as the patient having access to a personal space where he or she can be left in peace [42]. In addition, it can be argued that in the psychiatric care context, it is important that patients are satisfied with the secluded aspect of the environment, as dissatisfaction with the physical environment can trigger aggressive behavior [43]. This can lead to a stressful work environment for staff [44], and it has been shown that stress can have an impact on staff’s goodwill in providing good care [45].

In a previous study nurses judged the maintenance of security as one of the most important issues regarding staff’s competence in forensic care [14], as it is a highly specialized field with much focus on safety; the patients being cared for have committed crimes [15] and can be aggressive and difficult to work with [44]. Consequently, it seems central in forensic care that staff provide security [46]. Therefore, there is a notable and serious finding in this study that the lowest ratings were reported in the secure-environment dimension, as it means that about half of the staff don’t perceive that the ward atmosphere is secure for the patients. This finding emphasizes previous studies in which staff describe security in the psychiatric ward environment as important [42,47] to enable patients to feel secure in the ward environment [42].

This study shows various characteristics being related to staff’s perception of quality of care. In regard to the overall QPC-FIPS score, older staff perceived the quality of care as higher than younger staff did. One possible explanation is that younger staff make greater demands on quality of care, based on results showing that younger people in general are less satisfied with the quality of care than older people [48]. This is in line with previous studies on psychiatric patients showing that older patients have higher quality of care ratings than younger patients [21,23,49]. Whether this finding is related to a general age/generation difference or is specific to psychiatry staff needs to be examined in future research.

The more the staff felt their work to be stimulating, and the better they perceived the psychosocial working situation to be, the higher they perceived the quality of care, except in the secluded environment. Previous studies have shown that quality of care is related to staff’s well-being [50] and job satisfaction [51,52] The staff that indicated they had greater time to perform their duties also perceived the quality in the secure-environment dimension as higher in this study. This finding is in line with Bowers et al. [53], who found that time available for giving care has an impact on quality of care. In addition, Räikkönen et al. [51] argue that staff with sufficient time performed work of better quality than those with insufficient time.

Despite a relatively large number of participants, the gender, professional, and nationality differences of quality of care ratings were few. The only nationality difference observed was that Swedish staff were more satisfied with the secure environment on the ward than were nonSwedish staff.

The QPC-FIPS instrument has its merits. 1) It has been developed from a definition of quality of care derived from interviews from the patients’ perspective. From the validity point of view, many earlier scales have been criticized for their lack of a theoretical foundation [54]. 2) It has been validated by patients in both general psychiatric care and forensic inpatient care, according to advice by Rat et al. [55]. 3) The QPC-FIPS combines a general instrument with the addition of a context-specific module, as Boyer et al. [17] recommended. 4) The QPC-FIPS is psychometrically adequate for use among staff in forensic inpatient care and can be used to identify dimensions with low or high quality of care. 5) The QPC-FIPS can be used together with the QPC-FIP instrument [23] developed for forensic inpatients and covering the same dimensions and items.

To our knowledge, this is the only instrument measuring staff’s perceptions of the quality of care in forensic inpatient settings where the items were generated from the patient’s perspective and represent the patients’ views on high quality of care. Research is warranted on crosscultural examinations of quality of care. Therefore, translations of the QPC-FIPS into other languages are necessary.

One possible limitation worthy of mention is that we have not examines staffing factors that have been found to have an impact on quality of care, such as number of staff on duty [53] and staff turnover [56]. However, this was not the aim of the study; future research is required to better understand factors that can have an impact on quality of care in psychiatric settings.

5. CONCLUSION

The results of this study show that the QPC-FIPS has good psychometrical properties and is adequate to be used as a self-report instrument with the aim to improve forensic inpatient services. The staff’s ratings of quality of care were generally high, and several factors influenced these ratings. Taking into account staff’s perceptions of the quality of care provided can augment the identification of areas for improvement in forensic psychiatric care. The QPC-FIPS will contribute to development of theory.

Clinical Implications

The knowledge about staff’s ratings of low quality of care in the dimension secure environment can be used in the improvement of quality of care and in professional education aimed at understanding the needs of secure care. It is also important for management and decisionmakers in the planning of forensic care. QPC-FIPS is an inexpensive, simple, user-friendly tool that can be recommended for use in the daily routine work of evaluating staff’s experiences of the quality of care provided. It can be used, preferably together with the QPC-FIP [23] for patients, to compare both perspectives of quality of care at the same ward in order to enhance the quality of care and patient security. It can also be used in research studies regarding quality of forensic inpatient care.

REFERENCES

NOTES

^*These authors contributed equally to this work.

^#Corresponding author.