Open Journal of Psychiatry
Vol.07 No.03(2017), Article ID:77997,15 pages

How do Patients with Schizophrenia Perceive Relatives’ Levels of Expressed Emotions? A Brief Patient Rated Scale

Heidi Bjørge1,2*, Tor Helge Notland3, Turid Birgitte Boye4,5, Ingun Dina Ulstein2,6, Nina Aarhus Smeby7, Ulrik Fredrik Malt2,8, Håvard Bentsen9

1Oslo and Akershus University College of Applied science, Faculty of Health, Institute of Nursing and Health Promotion, Oslo, Norway

2Institute of Clinical Medicine, Faculty of Medicine, University of Oslo, Oslo, Norway

3Blakstad Hospital, Asker, Norway

4Section of Psycho-oncology, Oslo University Hospital, Oslo, Norway

5Institute of Basic Medical Sciences, Faculty of Medicine, University of Oslo, Oslo, Norway

6Department of Psychiatry of Old Age; Oslo University Hospital, Oslo, Norway

7Department of Research, Division of Mental Health and Dependence, Oslo University Hospital, Oslo, Norway

8Department of Research and Education, Division of Surgery and Clinical Neuroscience, Oslo University Hospital, Oslo, Norway

9Center for Psychopharmacology, Diakonhjemmet Hospital, Oslo, Norway

Copyright © 2017 by authors and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY 4.0).

Received: January 8, 2017; Accepted: July 25, 2017; Published: July 28, 2017


Background: Patients’ perspective on relatives’ attitude and behaviour towards them (Expressed emotion―EE) may be an important addition to the current focus on relatives’ perspective only, as measured by Camberwell Family Interview (CFI) or other methods. Based on the theory of EE, we have designed a brief, three-item questionnaire completed by patients, named Felt Expressed Emotion Rating Scale (FEERS). FEERS measures the patient’s experience of criticism (Cri) and emotional over involvement (i.e. worry (Wo), and control (Con). Aims: To investigate the test-retest reliability of the FEERS and associations between the FEERS and the CFI and to which extent FEERS scores were modified by severity of psychotic symptoms, cognitive function, patient mood and amount of face-to-face contact with relatives. Methods: Forty-five patients with schizophrenia and related psychoses admitted to a psychiatric hospital and 67 relatives were included. Assessments included FEERS, CFI and Positive and Negative Syndrome Scale (PANSS). Results: FEERS-Cri test-retest intra-class correlation (ICC1,1) was 0.71 among patients with low total PANSS scores, low cognitive impairment (0.59) and depression (0.63). For low levels of cognitive impairment, the ICCs of the FEERS-Wo and the FEERS-Con were 0.62 and 0.83, respectively. The FEERS-Cri and FEERS- Wo correlated significantly with CFI-CC and CFI-positive comments, respectively. Among the relatives that the patient deemed “not at all critical” (low FEERS-Cri scores), 94% had low CFI-CC levels. Conclusions: The FEERS may be a brief, time-saving alternative for identifying relatives with low levels of criticism. However, illness severity, cognitive function and mood influence FEERS test-retest reliability and link to CFI.


Expressed Emotion, Perceived Criticism, Schizophrenia, Validity

1. Introduction

A substantial number of patients with schizophrenia do not respond to traditional treatment [1] . One explanation has been increased level of expressed emotion (EE) in the family of patients [2] [3] . EE refers to attitudes and behaviours towards the patient that reflect criticism, hostility, and emotional over-involvement. Currently, interventions based on reducing high EE are considered integral to the psychosocial component of treatments for schizophrenia [4] . The gold standard method for assessing EE is the Camberwell Family Interview (CFI) [5] . A major limitation of CFI is the training required to rate its reliability and the time required to administer and score the interview. In addition, the CFI addresses the relatives’ points of view, as interpreted by an investigator. Several studies have suggested that patient perceptions of the emotional climate in the family may differ from the perceptions revealed by interviewing parents or other relatives [6] [7] .

CFI-EE was developed to predict relapse and pave the way for family intervention to prevent relapse. Despite the importance of this aspects as well, emphasis of patient perspective on relative’s attitudes and behaviour is of great interest, not the least when it comes to psychological treatment. Nevertheless, only a few questionnaires have been published on this issue: Level of Expressed Emotion scale (LEE), Family Attitude Scale (FAS) and the Perceived Criticism Scale (PCS). However, the intention of those scales was to predict long-term outcome, not to measure how patients perceive relatives attitudes and behaviour per se. Level of Expressed Emotion scale (LEE) and the Family Attitude Scale (FAS) are scales that measure both relative’s and patient’s perspectives, but both are rather lengthy. The 60 items LEE is based on the EE construct and provides four subscales (Intrusiveness, Emotional Response, Attitude Toward Illness, Tolerance and Expectations) and a score for the level of EE overall [8] . However, the three core dimensions of EE (CC, H, EOI) are not explicitly measured. The 30 items FAS assesses a respondent's attitudes and behavior towards another person [9] . However, only a few studies have used the patient’s version [10] and no data are available on the test-retest reliability or how relatives CFI-scores relate to FAS scores, when completed by patients.

Among the three core dimensions of EE, CFI-CC has by many authors been viewed as the most important individual dimension. Hostility is most often highly correlated with CC. Hooley and Teasdale [11] constructed the Perceived Criticism Scale (PCS). On a 10-point Likert-type scale, the patient should rate “How critical is your spouse of you?”. An important limitation of PCS is the mere focus on critical comments. Emotional over involvement, not only CC, is an important factor in understanding the effects of family behaviours and attitudes on patients. PCS did not aim to measure this dimension. When EOI is scored applying Camberwell Family Interview, both level of worry and control is taken into consideration. An excessively high level of worry might imply EOI, which can be stressful for the patient. However, an absence of worry may be perceived by the patient as not caring, which also can be distressing. High levels of control may also be perceived as over-involvement. For example, Hooley and Campbell (2002) found that relatives’ high level of behaviour control significantly predicted relapse in patients with schizophrenia. Thus, a brief questionnaire that addresses the patient’s experience of criticism and the perceived levels of worry and control may provide important clinical information about EE compared to LEE, FAS or PCS.

How patients perceive interaction with significant others, e.g. spouse or family, may be influenced by the severity of psychotic disorder; impairment of cognitive function and mood. For example, high EE has been related to better cognitive functioning of the patient [12] [13] . Furthermore, some studies on EE research have shown that the amount of face-to-face contact with significant others may influence the perceived level of distress [14] . Despite this, such aspects have not been studied when applying patient based versions of LEE, FAS or PCS [15] .

Considering the lack of brief patients based questionnaires to measure how patients with schizophrenia perceive their interaction with relatives with emphasis on both critical comments and emotional over involvement, i.e. degree of worry and control, we developed a brief three-item questionnaire, the Felt Expressed Emotion Rating Scale (FEERS).

Aims and Hypotheses

The aims of this study were to investigate a) the test-retest reliability of the FEERS and b) the association between the FEERS and the CFI based on interview with relatives. We also examined c) the influence of severity of psychosis, cognitive function and mood influenced on FEERS responses and d) whether the amount of face-to-face contact influenced the association between CFI-derived EE dimensions and FEERS. Our hypotheses suggested that both test-retest reliability and severity of the psychosis (PANSS), impaired cognitive function and high mood scores should weaken the association between patients’ FEERS score and relatives CFI-based EE dimensions.

2. Materials and Methods

Inclusion required that cognitive abilities of both the patient and the relatives had to be good enough to provide written informed consent. Fifty-eight consecutive patients (age 18 - 39 years) that had been admitted to two psychiatric hospitals, due to an acute psychotic episode during the previous week were eligible for the study. Nine of the 58 patients or their relatives refused to participate. One patient committed suicide before interview; one patients was discharged early from the hospital and two patients did not fill in FEERS. This resulted in samples of 45 patients and 67 relatives. To allow for comparison with CFI-based studies, we diagnosed our patients according to DSM-III-R. However, we obtained additional information about symptoms, behaviour and length of illness that have subsequently been included in DSM-IV, DSM-5 and ICD-10 research criteria for schizophrenia. In this paper we report diagnosis according to DSM-5. All our patients fulfilled the diagnostic criteria for all those definitions: schizophrenia (n = 50), schizoaffective disorders (n = 6) and schizophreniform disorders (n = 2).

One or two key relatives per patient were invited to participate. All patients had face-to-face contact with relatives before admission. The amount of face-to-face contact between the patient and family members during a typical week was assessed. When possible, we interviewed the patients at baseline within one week after admission to the hospital. Patients that stayed more than 3 months in the hospital were excluded from the study.

2.1. Instruments

The FEERS, conceptualized by last author, is a questionnaire derived from the theory of EE [5] . The FEERS was translated from Norwegian to English and back into Norwegian to ensure an optimal translation. At baseline, patients completed the FEERS to rate their perceived attitudes of each relative that they had seen regularly (face-to-face contact) for the last month. Each question was rated on a six-point scale (0 - not at all; 5- extremely): 1) How critical do you feel he /she has been to you the last month? (FEERS-Cri); 2) How much do you feel he / she has worried about you the last month? (FEERS-Wo); 3) How much have you felt controlled by him / her the last month? (FEERS-Con). FEERS was filled in by the patients within a few days of the point in time when CFI was administered to the relatives.

CFI measures five dimensions [5] [16] : critical comments (CC), hostility (H), emotional over-involvement (EOI), warmth (W), and positive comments (Pos). The first three dimensions (CC, H, and EOI) are used to assess the level of EE (low, high). The relatives were interviewed with the CFI within three weeks after the patient’s admission. All interviews were taped and scored by eight raters who had been trained and found reliable raters by Christine Vaughn [5] . The EE index was rated high when the CC was high (six or more critical comments), when H was present (1 - 3, on a scale of 0 - 3), or when EOI was high (3 - 5, on a scale of 0 - 5). Our research group has previously demonstrated acceptable inter-rater reliability on CC, H, and EOI [intra-class correlation coefficient (ICC1,1) 0.61 - 0.69] [17] . Neither warmth (CFI-W) nor positive comments (CFI-POS) are included in the EE index. However, our clinical experience suggests that both CFI-W and CFI-POS are important dimensions. Thus, we included these dimensions in the present study.

Severity of psychosis, cognitive impairment and mood was assessed by PANSS [18] . It is a semi-structured interview designed to assess symptom levels in patients with schizophrenia and other psychotic disorders. The total PANSS consists of 30 items. The symptoms were grouped into components identified in a principal-component analysis which included 28 items [19] . These components were: positive symptoms (P1, P3, P5, G9, G12), negative symptoms (N1, N2, N3, N4, N6, G5, G7, G11, G13, G16), cognitive symptoms (P2, N5, G10), depressive symptoms (G1, G2, G3, G6, G15), and hostility symptoms (P4, P7, G4, G8, G14). For each patient, we calculated a mean score for each component and the total PANSS (i.e., the sum of scores/the number of items). The inter-rater reliability of raters (n = 6) was very good (ICC1,1 0.93, 0.79, 0.88, 0.79, 0.80 for the five components and 0.92 for the total PANSS score, respectively [20] .

2.2. Ethical Aspects

Informed consent was obtained from all study participants. The study protocol was approved by the Regional Ethics Committee on Medical Research and the Norwegian Data Inspectorate.

2.3. Statistical Analyses

A consecutive sub-sample of the patients (n = 18) completed the FEERS a second time, three days after the first assessment, and ICC1,1 was calculated. The association between the ordinal scales of CFI dimensions and FEERS was studied with Kendall’s tau-b. To estimate the sensitivity, specificity, and positive predictive value of FEERS-Cri in relation to CFI-CC, we dichotomized the responses per number of critical comments (low CFI-CC: <6 critical comments; high CFI-CC: ≥6 critical comments). An EOI score 0 - 3 refers to low EOI and 4 - 5 high EOI.

To estimate the effect of patient symptoms (PANSS scores) on the association between FEERS and CFI, we performed logistic regression analyses, with the dichotomized CFI dimension as the dependent variable and the FEERS as the independent variable. Each PANSS component was dichotomized at its median score. For each FEERS item and each PANSS component, we introduced the product of the FEERS score * PANSS score as an interaction term. Analyses were centred at groups of low and high symptom levels, consecutively. Then, we estimated the sensitivity and specificity of FEERS for the binary values of CFI-CC and CFI-H.

The effect of patient symptoms on the stability of the FEERS ratings was evaluated with multiple linear regression analyses. Each retest FEERS score was a dependent variable, while the test FEERS score, a continuous PANSS component score, and their interactions were independent variables. All statistical analyses were carried out with SPSS (versions 14.0 - 20). The required level of significance was p ≤ 0.05 (two-sided).

We estimated the influence of face-to-face contact (time) on the associations between the FEERS and the CFI, between the FEERS test and retest, and between the FEERS and PANSS at follow-up, by testing the interactional terms (contact time * other predictor). Relatives were assumed to be independent of each other, although 44 of the 67 relatives (76%) were living as a couple. This assumption was tested. Mixed model analyses were conducted, but the intercept mixed models did not converge. Thus, patients with two relatives were included twice in the statistical analyses, i.e., once for each questionnaire completed.

3. Results

3.1. Demographic and Clinical Data

Patient demographic and clinical data are summarized in Table 1. Most patients

Table 1. Demographic and clinical characteristics of patients.

§SD = Standard deviation; #IQR = Interquartile range; &For each person, the mean PANSS scores were calculated for each component (scores/number of items); then, the distribution of these average scores was assessed across the sample (non-parametric test).

(84%) had never married or cohabitated with a partner. Among the 67 relatives (30 men and 37 women), 34 (52%) were mothers (1 stepmother), 27 (40%) were fathers, 3 (5%) were spouses/cohabitants, and 3 (5%) were siblings. The relatives had an average age of 56 (range 26 - 82) years.

3.2. Test-Retest Reliability

Of the 18 patients filling in FEERS twice, eight had two relatives, which yielded 26 questionnaires. The test-retest ICC1,1 were 0.46 for FEERS-Cri (p = 0.007), 0.22 for FEERS-Wo (p = 0.15), and 0.37 for FEERS-Con (p = 0.03). We performed linear regression analyses to assess whether the PANSS components (dichotomized) modified the correlations between the test and retest FEERS scores (Table 2). We found that the test-retest reliability (ICC1,1) of the FEERS- Cri was higher when the Total PANSS (0.71), PANSS cognitive (0.59), PANSS hostility (0.62) and PANSS depressive components were low (0.63) (Table 2). Test-retest in patients with The PANSS cognitive scale also significantly modified the test-retest ICC1,1 stability of the FEERS-Wo (p = 0.04) and FEERS-Con (p = 0.01). Thus, analyses centred at low levels of the cognitive component showed a strong relationship between the test and retest results of the FEERS- Con scores (t = 3.46, p = 0.003). In contrast, there was no significant relationship when the analyses were centred at high cognitive component levels (p = 0.66). Among patients with low PANSS cognitive component scores, the test-retest reliability of FEERS-Con was 0.83. Analogously, the test FEERS-Wo scores agreed with the retest scores at low levels of cognitive component (t = 2.06, p = 0.05), but not at high cognitive component levels (p = 0.31). FEERS-Wo scores were more stable at low depression levels (p = 0.08) than at high depression levels (p = 0.31). However, no test-retest comparisons of the FEERS-Wo reached an ICC1,1 of 0.70.

3.3. Comparison of FEERS with CFI

The means and standard deviations (SD) of patient ratings on the FEERS scales were 2.0 (1.7) for FEERS-Cri; 1.9 (1.6) for FEERS-Wo; and 1.8 (1.6) for FEERS- Con. The number of relatives’ critical comments, measured on the CFI (CFI- CC), ranged from 0 to 22 comments, with a median of 2 (IQR = 5) comments.

Table 2. Test-retest reliability (intra-class correlations) of the FEERS (18 patients; 26 questionnaires, see text) per high or low patient symptom levels (PANSS).

Numbers represent ICC1,1 between the test and the 3-day retest results of the three FEERS scales, among patients dichotomized by low/high (compared to the median) PANSS component scores. Intra-class correlations were derived from multivariate linear regressions. Dependent variable: FEERS retest; independent variables: FEERS test, PANSS continuous scores, and interaction between the FEERS item test and PANSS continuous scores. Interaction effect: *p ≤ 0.05; (*) 0.05 < p ≤ 0.10.

The mean (SD) CFI-EOI ratings of the sample was 2.2 (1.4), on a scale of 0-5. Seventeen (25%) relatives had high scores (≥6) on the CFI-CC. CFI-H was present in 11 (16%) relatives. Twenty-nine (43%) relatives had a high score (≥3) on the CFI-EOI, and 37 (55%) were rated as high EE (CFI-CC ≥ 6, CFI-H present, and/or CFI-EOI ≥ 3).

Among patients with two relatives (n = 22), patient perceptions of the two relatives’ attitudes were not correlated significantly for the FEERS-Cri (r = 0.24; p = 0.26) or the FEERS-Wo (r = 0.05; p = 0.80). However, there was a positive correlation for the FEERS-Con (r = 0.44; p = 0.04); thus, the low variance of this variable reduced the statistical power of tests involving the FEERS-Con.

Patient scores on the FEERS-Cri correlated positively with relatives’ CFI scores on the CFI-CC and CFI-H, but they were not significantly associated to other CFI measures (EOI, W, or POS) (Table 3). A FEERS-Cri score of 0, versus 1 - 5, had a sensitivity of 94%, a specificity of 34%, a positive predictive power of 34%, and a negative predictive power of 94% for agreement if CFI-CC was high (cut-off 5/6 comments). Higher FEERS-Cri cut-off scores resulted in lower sensitivity and negative predictive power. There was also a positive correlation between FEERS-WO and CFI-derived positive comments (Table 2). No other significant correlations were found.

The link (Odd ratio―OR) between FEERS-Co and CFI Hostility was modified (p = 0.08) by total PANSS (low/high). Thus, lower total PANSS indicates a significant positive link (OR = 2.2, 1.0 - 4.9, p = 0.04), while at higher PANSS scores the link was non-significant (p = 0.79). Neither FEERS-worry nor control was modified by PANSS. However, FEERs worry was modified by high face-to- face contact time (higher worry when more face-to-face contact). FEERS-control score was modified by total PANSS score and PANSS negative component. PANSS total score was negatively modified by worry or CC (Table 3). Neither applying alternative cut-off for CFI-EOI (e.g. 0-2/3-5 or 0-4/5) nor weighting CFI-EOI or FEERS-Co (e.g. double the score), did not change the results.

Table 3. Correlations (Kendall’s tau-b) between the FEERS and CFI scales (N = 67 questionnaires, except for CFI-POS, where n = 66). CFI-CC, CFI-H, and CFI-EOI scales are dichotomized as absent (0) or present (1); see methods.

#Several of the correlations are modified by PANSS depressive scores and face-to-face contact (patient ? relatives). *0.01 < p ≤ 0.05, **p ≤ 0.01. #Modified by PANSS depressive component (low/high; see text). &Modified by Total PANSS score (low/high; see text). §Modified by PANSS Negative component (low/high; p = 0.02, see text). %Modified by face-to-face contact time (low/high; see text).

In contrast to total PANSS, PANSS cognitive component did not modify any link between FEERS-scores and CFI. However, the link between FEERS-Cri and CFI-CC (low/high) was modified (p = 0.08) by the PANSS depressive component (low/high) (see Table 3). Thus, when the depressive component was low, there was a significant positive link (OR = 2.1, 95% CI 1.2 - 3.6, p = 0.006); i.e., for each step on the FEERS-Cri scale, the likelihood of a high CFI-CC was doubled. In contrast, when the depressive component was high, the link was not significant (p = 0.68).

Face-to-face contact modified the link between FEERS-Cri and CFI-H (p = 0.06). More time spent together indicated a significant positive link (OR=2.7, 1.1 - 6.6, p = 0.03), whereas there was no link when less time has been spent together (p = 1.0). The link between FEERS-Con and CFI-CC was modified (p = 0.01) by face-to-face contact (low/high) (Table 3). Thus, when more than median time was spent together, there was a significant positive link (OR = 2.8, 1.3 - 6.1, p = 0.01) (i.e., for each step on the FEERS-Con scale, the likelihood of CFI-derived high criticism almost tripled), while there was no link when less time is spent together (p = 0.50). Further, the link between FEERS-Wo and CFI-EOI was also modified (p = 0.09) by face-to-face contact. More time spent together indicated a significant negative link (OR = 0.56, 0.33 - 0.95, p = 0.03), whereas there was no link when less time has been spent together (p = 0.83). The link between FEERS- EOI and CFI-EOI was non-significantly negative (p = 0.13; n = 67), and suggested that high CFI-EOI was not perceived as harmful by the patient, in contrast to CFI-CC.

4. Discussion

The main aim of this study was to test the test-retest reliability of a brief, patient-rated questionnaire (the FEERS) for assessing how patients perceive relatives’ attitudes and behavior, that is criticism and emotional over involvement, i.e. control, and worry. We also studied the link between FEERS and CFI-scores obtained from relatives. Finally, we studied how FEERS scores was influenced by PANSS total and component scores and the amount of face-to-face contact with relatives. FEERS-Cri had 94% negative predictive power of CFI-derived CC. I.e. if FEERS-Cri score is 0, it is very unlikely that relatives are critical. Low levels of CFI-CC are taken to suggest that psycho-education, aimed at reducing critical comments, is not indicated. Furthermore, low parental critical attitudes have been associated with higher patient awareness of symptoms, which may increase compliance with treatment [21] . Thus, the FEERS-Cri may be a brief, cost-effective way of identifying relatives that do not require an intervention aimed at reducing levels of critical comments. However, high FEERS-Cri scores were less useful. In those situations, additional interviews with the relatives will be required to identify relatives with high levels of criticism.

However, the test-retest reliability of FEERS-criticism, worry and control was low in the total group of patients (ICC1,1 = 0.46; 0.22 and 0.37 respectively), suggesting that how patients perceive relatives attitude and behaviour may vary over days or is dependent of the clinical state of the patient. The latter interpretation is supported by the finding that test-retest reliability of FEERS-criticism was acceptable if the patient’s psychosis was less severe with little cognitive impairment and depressive symptoms. Although not significant, a trend towards better test-retest reliability of FEERS-criticism was also associated with higher positive relative to negative symptomatology and low hostility measure by PANSS. Test-retest reliability of FEERS-worry and control was also higher if cognition measured by PANSS was little impaired. However, test-retest reliability of FEERS-control also had higher test-retest reliability when the positive component of PANSS was low, i.e. less severe psychotic symptoms. The test-retest reliability for FEERS-Wo was acceptable (0.62) among patients with better cognitive functioning (i.e., a low score on the PANSS cognitive component).

The lower test-retest reliability of the FEERS among patients with more severe psychosis (high total PANSS scores) or more cognitive impairment (high PANSS cognitive domain scores) is consistent with clinical experience and research, which has shown that self-rating instruments are of questionable value among these patients. Thus, FEERS should be applied only when the acute symptomatology is mild to moderate. We also found that a higher level of depressive symptoms (PANSS depressive component) was associated with lower test-retest reliability. This finding was also consistent with clinical experience and research. Depressed patients have a negative bias; i.e., they are more prone to misinterpret interactions with others as criticism. However, in a study on students [22] , the data suggested that it was unlikely that perceived criticism simply reflected the negatively biased perceptions of distressed individuals.

Interestingly, there was only modest correlation, although statistical significant, between FEERS measured criticism and relatives’ critical comments and hostility measured by CFI (r = 0.29 and 0.27 respectively). However, a higher level of depression was associated with a lower correlation between the FEERS- Cri and CFI-CC. This suggest that how patients perceive relatives attitude and behaviour is rather different from how relatives report their attitude and behaviour towards the patient. This discrepancy has also been observed between patient with schizophrenia experiences of side-effects of drugs and the treating physicians’ opinion [23] . It is likely that acute psychotic patients’ clinical response will be influenced by how they perceive their relatives attitude and behaviour. Thus, our study demonstrates the importance of assessing patients’ perspective as well. For this purpose, FEERS is suitable. It should be added, however, that a small study of patients with schizophrenia (n = 33), interviews about perceived EE among patients did correlate with relatives’ FMSS-rated EE [24] . However, not using CFI and the fact that only a few relatives had high EE in this study, the validity of this study may be challenged.

FEERS-Wo score correlated positively with the number of CFI-POSs. Thus, relatives with high affection for their loved ones may be experienced by the patient that their relatives worry about their mental state. High levels of worry are within EE theory often considered to reflect high emotional over involvement and thus harmful. However, it is also possible that some patients may perceive this as a positive response, i.e. that they relatives care about them. In contrast, FEERS-Con seem more related to CFI criticism or hostility, but only among patients with milder symptoms and more face-to-face contact. Accordingly, the test-retest reliability for FEERS-Con was high (ICC1,1 = 0.83) among patients with better cognitive functioning. Finally, perceived control was positively linked to hostility, but only for patients with less symptoms.

Finding that any of the FEERS-Wo and CFI-EOI did not correlate should come to no surprise. The concept of EOI is problematic and several researchers have questioned the validity of this dimension as important for the course of psychosis. FEERS-Co seems to reflect critic-warmth dimension, not CFI-EOI. Thus, the non-significant association between CFI-EOI and FEERS-derived estimation of EOI suggests that high CFI-EOI is not perceived as harmful by the patients, in contrast to CFI-CC.

4.1. Comparison of FEERS with Alternative Studies of Patient Perceived EE

Only one study of 15 schizophrenic patients and 22 of their relatives has looked at LEE filled in by patients [25] . The total score on the patient version of the LEE showed a 0.32 correlation with the number of critical comments reported on the CFI critical comments (CFI-CC). This is like the correlation between FEERS and CFI -CC when we do not correct for modifiers. Neither test-retest reliability nor correlations with LEE versus other EE dimensions, or the effect of possible modifiers such as PANSS scores or face-to-face contact was reported. It should also be added that data from the LEE are difficult to evaluate because of the changes in the instrument (e.g. 38 items, 33 items) that have occurred over time [26] . So far FAS applied to patient has not revealed data that allows comparison with our findings.

The correlation between the PCS and the CFI-CC is like that between the FEERS and the CFI-CC. One study of outpatients with schizophrenia demonstrated a clear relationship between level of the patient’s perception of being criticized by the family and patient symptomatology such as anxiety and psychosis [27] . However, CFI-data from relatives was not reported. In a longitudinal study, 27 relatively stable outpatients with schizophrenia or schizoaffective disorder filled in PCS and the single “most influential other” (MIO) person was interviewed with CFI [28] . Patient ratings on the perceived criticism item were significantly related to the high vs low CFI criticism ratings (p = 0.03). However, neither test-retest reliability, sensitivity and specificity of PCS in relation to CFI, nor correlation between CFI-CC and PCS score was reported. To the best of our knowledge, cognitive status, severity of depression and amount of face-to-face contact as effect modifiers of the patients’ PCS rating have not been studied. Furthermore, PCS does not include assessment of EOI aspects such as worry or control.

In fact, this study underlines that the validity of patients’ score when filling in questionnaires is dependent of the severity of the mental disorder at the time of filling in the questionnaire, and this our setting, the time spent with relatives. One may argue that FEERS requires rather well functioning patients. However, the same limitation is true for PCS. Furthermore, in contrast to our study, no previous study on the validity of questionnaires assessing how patients perceive of relatives’ behaviour, have investigated the effect of such modifiers.

4.2. Limitations

Although studies applying LEE or FAS to measure how patients perceive their relatives attitude and behaviour have included smaller sample than our study, our sample is nevertheless not big. Furthermore, patients judged to be severely cognitively impaired or more treatment-resistant (defined as requiring more than three months of inpatient stay) were not included in the study. Finally, both patients and relatives had to be willing to participate. Thus, our inclusion criteria may have selected a patient group with relatively low family dysfunction and low psychopathology severity, despite the high total PANSS scores among some of our patients and the fact that 55% of the relatives were rated as high EE. Thus, our findings require confirmation in studies with larger samples and ideally, more diverse group of patients including out-patients. In addition, future studies should apply more accurate measurements of cognitive impairment (i.e., standardized neuropsychological tests) to validate our finding that cognitive impairment could modify the self-reported perception of a relative’s attitude during acute psychosis.

4.3. Implications for Further Research

Studies addressing how patients perceive relatives’ attitudes and behaviour towards them are surprisingly few. Thus, we do not know to which extent taking patients’ perspective on EE into account in addition to relatives’ perspective, may require modified psychoeducational or psychotherapeutic approaches in the treatment of schizophrenia and thus possibly improve outcome. However, in a recent study, levels of parental criticism was not as predictive of symptom trajectories as patients’ perceptions of criticism [29] , suggesting that how patients’ perceive expressed emotions is important. Prospective studies comparing the predictive value of short- and long-term outcome of FEERS versus relatives CFI or shorter methods (e.g. five minutes’ speech sample) are clearly needed to address this important question. Because high EE also seems to be of importance for the short- and long-term outcome of other mental disorders as well, the feasibility and validity of FEERS should also be tested in such populations.

5. Conclusion

The FEERS may be an efficient screening instrument for identifying how patients perceive relatives attitude and behaviour, especially in patients with less severe psychosis, less depression and less impaired cognitive function. FEERS may also be a brief and valid instrument to rule out high levels of EE in families. Those capacities may have important treatment implications.


This research was supported by grants from Josef and Haldis Andresen’s Legacy.

Cite this paper

Bjørge, H., Notland, T.H. Boye, T.B., Ulstein, I.D., Smeby, N.A., Malt, U.F. and Bentsen, H. (2017) How do Patients with Schizophrenia Perceive Relatives’ Levels of Expressed Emotions? A Brief Patient Rated Scale. Open Journal of Psychiatry, 7, 220-234.


  1. 1. Austin, S.F., et al. (2015) Long-Term Trajectories of Positive and Negative Symptoms in First Episode Psychosis: A 10 Year Follow-Up Study in the OPUS Cohort. Schizophrenia Research, 168, 84-91.

  2. 2. Wearden, A.J., et al. (2000) A Review of Expressed Emotion Research in Health Care. Clinical Psychology Review, 20, 633-666.

  3. 3. Xia, J., Merinder, L.B. and Belgamwar, M.R. (2011) Psychoeducation for Schizophrenia. Cochrane Database of Systematic Reviews, 6, 1-166.

  4. 4. Nordentoft, M., et al. (2015) From Research to Practice: How OPUS Treatment Was Accepted and Implemented throughout Denmark. Early Intervention in Psychiatry,. 9, 156-162.

  5. 5. Vaughn, C. and Leff, J. (1976) The Measurement of Expressed Emotion in the Families of Psychiatric Patients. British Journal of Social & Clinical Psychology, 15, 157-165.

  6. 6. Bachmann, S., et al. (2006) Perceived Criticism in Schizophrenia: A Comparison of Instruments for the Assessment of the Patient's Perspective and Its Relation to Relatives' Expressed Emotion. Psychiatry Research, 142, 167-175.

  7. 7. Onwumere, J., et al. (2009) Patient Perceptions of Caregiver Criticism in Psychosis: Links with Patient and Caregiver Functioning. Journal of Nervous and Mental Disease, 197, 85-91.

  8. 8. Cole, J.D. and Kazarian, S.S. (1988) The Level of Expressed Emotion Scale: A New Measure of Expressed Emotion. Journal of Clinical Psychology, 44, 392-397.<392::AID-JCLP2270440313>3.0.CO;2-3

  9. 9. Kavanagh, D.J., et al. (2008) Predictive Validity of the Family Attitude Scale in People with Psychosis. Psychiatry Research,. 160, 356-363.

  10. 10. Vandeleur, C.L., et al. (2013) French Version of the Family Attitude Scale: Psychometric Properties and Relation of Attitudes to the Respondent's Psychiatric Status. Psychiatry Research, 210, 641-646.

  11. 11. Hooley, J.M. and Teasdale, J.D. (1989) Predictors of Relapse in Unipolar Depressives: Expressed Emotion, Marital Distress, and Perceived Criticism. Journal of Abnormal Psychology, 98, 229-235.

  12. 12. Bentsen, H., et al. (1998) Criticism and Hostility in Relatives of Patients with Schizophrenia or Related Psychoses: Demographic and Clinical Predictors. Acta Psychiatrica Scandinavica, 97, 76-85.

  13. 13. Heikkila, J., et al. (2006) Cognitive Functioning and Expressed Emotion among Patients with First-Episode Severe Psychiatric Disorders. Comprehensive Psychiatry, 47, 152-158.

  14. 14. Renshaw, K.D. (2008) The Predictive, Convergent, and Discriminant Validity of Perceived Criticism: A Review. Clinical Psychology Review, 28, 521-34.

  15. 15. Masland, S.R. and Hooley, J.M. (2015) Perceived Criticism: A Research Update for Clinical Practitioners. Clinical Psychology: Science and Practice, 22, 211-222.

  16. 16. Van Humbeeck, G., et al. (2002) Expressed Emotion: A Review of Assessment Instruments. Clinical Psychology Review, 22, 321-341.

  17. 17. Bentsen, H., et al. (1996) Inter-Rater Reliability of Expressed Emotion Ratings Based on the Camberwell Family Interview. Psychological Medicine, 26, 821-828.

  18. 18. Kay, S.R., Flszbein, A. and Opfer, L.A. (1987) The Positive and Negative Syndrome Scale (PANSS) for Schizophrenia. Schizophrenia Bulletin, 13, 261-276.

  19. 19. Kay, S.R. and Sevy, S. (1990) Pyramidical Model of Schizophrenia. Schizophrenia Bulletin, 16, 537-545.

  20. 20. Bentsen, H., et al. (1996) The Interrater Reliability of the Positive and Negative Syndrome Scale (PANSS). International Journal of Methods in Psychiatric Research, 6, 227-235.

  21. 21. Macgregor, A., et al. (2015) Insight of Patients and Their Parents into Schizo-phrenia: Exploring Agreement and the Influence of Parental Factors. Psychiatry Research, 228, 879-886.

  22. 22. Gerlsma, C., van Duijn, M.A.J., Hale III, W.W. and van Hout, W.J.P.J. (2009) Perceived Criticism: Associations with Perceiver Characteristics and Interpersonal Behaviour. Psychiatry Research, 170, 234-240.

  23. 23. Lindstrom, E., et al. (2001) Patient-Rated versus Clinician-Rated Side Effects of Drug Treatment in Schizophrenia. Clinical Validation of a Self-Rating Version of the UKU Side Effect Rating Scale (UKU-SERS-Pat). Nordic Journal of Psychiatry, 55, 5-69.

  24. 24. Tompson, M.C., et al. (1995) Schizophrenic Patients’ Perceptions of Their Relatives’ Attitudes. Psychiatry Research, 57, 155-167.

  25. 25. Kazarian, S.S., Malla, A.K., Baker, B. and Cole, J.D. (1990) Comparisons of Two Expressed Emotion Scales with the Camberwell Family Interview. Journal of Clinical Psychology, 46, 306-309.<306::AID-JCLP2270460310>3.0.CO;2-2

  26. 26. Hooley, J.M. and Parker, H.A. (2006) Measuring Expressed Emotion: An Evaluation of the Shortcuts. Journal of Family Psychology, 20, 386-396.

  27. 27. Guada, J., Hoe, M., Floyd, R., Barbour, J. and Brekke, J.S. (2011) The Importance of Consumer Perceived Criticism on Clinical Outcomes for Outpatient African Americans with Schizophrenia. Community Mental Health Journal, 47, 637-645.

  28. 28. Docherty, N.M., et al. (2011) Anxiety Interacts with Expressed Emotion Criticism in the Prediction of Psychotic Symptom Exacerbation. Schizophrenia Bulletin, 37, 611-618.

  29. 29. O’Brien, M.P., Miklowitz, D.J. and Cannon, T.D. (2015) Decreases in Perceived Maternal Criticism Predict Improvement in Subthreshold Psychotic Symptoms in a Randomized Trial of Family-Focused Therapy for Individuals at Clinical High Risk for Psychosis. Journal of Family Psychology, 29, 945-951.