Vol.08 No.07(2017), Article ID:76304,23 pages

Only One Burnout Estimator Is Consistently Associated with Health Care Providers’ Perceptions of Job Demand and Resource Problems

Jan Beckstrand, Nancy Yanchus*, Katerine Osatuke

Health Administration National Center for Organizational Development, Cincinnati, OH, USA

Copyright © 2017 by authors and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY 4.0).

Received: March 6, 2017; Accepted: May 19, 2017; Published: May 22, 2017


Five “high” burnout estimators are in common use. Each is based on a different subset of the three aspects of “Burnout Syndrome”, so each gives a different estimate of the “high” burnout prevalence in a population. Managers often don’t know these prevalences are incomparable. Managers also have little specific information on how their institution’s burnout measure is associated with their employees’ perceptions of problems in job demands and resources in their work environment, as regularly queried in organizational health surveys. In the current study, we demonstrated the differences in the prevalences of “high” burnout obtained using each of five “high” burnout estimators. We also evaluated and compared the five burnout estimators’ associations with employee-perceived problems in each of 31 areas of job demands and resources in human-systems functioning. We measured these associations by how much more the aspects of burnout queried in the estimator were reported by those who perceived a problem than by those who did not (the positive likelihood ratio, LR+). We examined five types of physicians (6599), nurse practitioners (2158) and physician assistants (786). We found that four of the “high” burnout estimators showed few associations with employee perceptions of problems in job demands or resources, but one estimator―the trivariate joint occurrence of “high” (i.e. frequent) emotional exhaustion, “high” depersonalization and “low” sense of personal accomplishment (measured by well-validated single-item surrogates for the three Maslach Burnout Inventory subscales)―was clinically significantly associated with 97% (30) of the problems in job demands and resources studied, in at least one of the health provider groups. Our results challenge the current preference for “high” emotional exhaustion dominated “high” burnout estimators.


Burnout, Job Demands, Health Care Providers, Physicians, Nurse Practitioners

1. Introduction

Burnout is important in health care for at least four reasons. It is a symptom of serious organizational human-systems problems that affect the quality of care (de Oliveira et al., 2013; Hakanen, Schaufeli, & Ahola, 2008; Leiter et al., 2013) . It is potentially detrimental to employees themselves and to their capacity to provide care (Leiter et al., 2013; Shanafelt et al., 2010; West, Dyrbye, Satele, Sloan, & Shanafelt, 2012) . It is induced by the work environment (Bakker & Demerouti, 2006; Bakker, Demerouti, & Euwema, 2005; Demerouti, Bakker, de Jonge, Janssen, & Schaufeli, 2001; Schaufeli & Bakker, 2004) and is possibly reducible by organizational-level interventions (Maslach, Jackson, & Leiter, 1996; Zis, Anagnostopoulos, & Sykioti, 2014) . However, there are problems both with the current understanding of the burnout phenomenon and with its measurement so that what constitutes an accurate and useful measure of “high” burnout in an employee and its prevalence in a health care organization is unsettled (Campbell, Prochazka, & Gopal, 2011; Dyrbye & Shanafelt, 2015; Hansen & Pit, 2016; Prins et al., 2007) . In the current study we compared the prevalences of “high” burnout obtained from the different estimators of “high” burnout commonly used in health care organizations. We also evaluated the different “high” burnout estimators’ associations with health care providers’ perceptions of problems in their job demands and resources in 31 areas of human-systems functioning.

1.1. Basic Problem in Estimating Rates of “High” Burnout

Problems in estimating “high” burnout prevalence in organizations arise because of a fundamental disparity between how burnout is conceptualized and how it is measured. Burnout is considered a univariate continuous phenomenon of “experienced feeling”, but a person’s level of “experienced feeling” is indicated in a non-additive non-linear way by a person’s joint levels on three aspects of the phenomena. These aspects, or what are probably better understood as symptoms, are one’s joint levels of emotional exhaustion, depersonalization, and sense of personal accomplishment (Maslach et al., 1996; Maslach, Schaufeli, & Leiter, 2001) . The highest state of “experienced feelings of burnout” is known as “Burnout Syndrome”, identifiable by the trivariate joint occurrence of “high” frequency of emotional exhaustion, “high” frequency of depersonalization, and “infrequent” sense of personal accomplishment in relation to one’s work (Maslach et al., 1996) . One can also identify low “experienced feeling of burnout” at work-when a person jointly experiences “infrequent” emotional exhaustion, “infrequent” depersonalization, and a “high” frequency of a sense of personal accomplishment. The measurement problem arises in deciding how to gauge the level of “experienced feeling of burnout” in people who report one of the other six possible trivariate configurations of “high” and “low” frequencies on the three “aspect” variables; as for example when one has both a frequent sense of personal accomplishment and infrequent emotional exhaustion, but also has a “high” frequency of depersonalization. What these signify about a person’s position on the univariate “experienced feelings of burnout” continuum remains uncertain (Boersma & Lindblom, 2009; Demerouti, Verbeke, & Bakker, 2005; Maslach et al., 1996) .

These uncertainties have led investigators to consider five different estimators of “high” burnout. Each is distinguished by which of the six “uncertain” joint configurations they classify as indicating “high” levels of burnout. In essence each “high” burnout estimator embodies a different hypothesis about what the six “uncertain” trivariate joint configurations imply about the person’s level of “experienced feelings of burnout”. Figure 1 shows the five “high” burnout estimators and how each maps (collapses) the trivariate configurations of “high” and “low” emotional exhaustion, depersonalization and sense of accomplishment into a dichotomous “univariate experience of burnout” scale. The joint configurations indicating “high” burnout in the respective five estimators are: 1) any configurations with “high” frequency of emotional exhaustion, 2) any with either “high” frequency of emotional exhaustion or “high” frequency of depersonalization, 3) any with “high” frequency of emotional exhaustion plus either “high” frequency of depersonalization or “low” frequency of personal accomplishment (a.k.a. emotional exhaustion plus one), 4) any with both “high” frequency of emotional exhaustion and depersonalization, and 5) only those with jointly “high” frequency of emotional exhaustion and depersonalization and “low” frequency of sense of personal accomplishment.

1.2. The Burnout Measurement Literature Does Not Indicate Which Estimator to Use

There is evidence within the burnout measurement literature to support the use of each of the estimators. This evidence includes: the intercorrelations among the three Maslach Burnout Inventory (MBI, Maslach et al., 1996 ) subscale scores, the MBI subscales’ scores’ correlations with job demands or resources (Lee & Ashforth, 1996; Schaufeli & Salanova, 2007; West et al., 2006) , and the MBI factor and bifactor loadings (Gil-Monte, 2005; Loera, Converso, & Viotti, 2014; Meszaros, Adam, Szabo, Szigeti, & Urban, 2014; Schaufeli, Bakker, Hoogduin, Schaap, & Kladler, 2001) . A few studies have examined some estimators’ discriminant validity or their utility in predictive models (Brenninkmeijer & Van Yperen, 2003; Schaufeli et al., 2001; West et al., 2012) . The use of the two most popular estimators, “‘high’ emotional exhaustion alone” or “either ‘high’ emotional exhaustion or depersonalization”, is supported by their associations with undesirable sequelae of “high” burnout, such as perceived medical errors, suicidality and alcoholism (Pedersen, Sorensen, Bruun, Christensen, & Vedsted, 2016; Oreskovich et al., 2012; Shanafelt et al., 2010; West et al., 2012) .

Figure 1. (a) How the eight cells in the three-way contingency table obtained from dichotomizing measures of emotional exhaustion (E), depersonalization (D) and sense of personal accomplishment (P) are partitioned as “Burnout Syndrome” and as each of these three “aspects” of “Burnout Syndrome” and (b) how eight cells are mapped by each of the five estimators into “high” or “low” “experienced feelings of burnout”.

There are also weaknesses in the evidence and in its use. For example, the sequelae studies demonstrate only that some of the aspects of “Burnout Syndrome” are, individually, associated with potentially serious consequences (Ores- kovich et al., 2012; Shanafelt et al., 2010) . It is also unclear in many of the factor and bifactor studies that the authors have adequately considered the potential for spurious correlations due to skewedness and to extreme points and outliers in the tridimensional data. In addition, many studies do not adequately consider the potential effects of the non-additive nonlinear structure within the tridimensional configuration of the three variables with respect to burnout experience, as has been pointed out by Demerouti et al. (2005) . Also few studies have treated the sets of joint aspects that are queried by the five estimators as symptoms and examined their associations as markers of employee- perceived problems in job demands and resources in human-systems functioning.

A final problem is that evidence from some older studies, like those of Rafferty et al.’s (1986) and Schaufeli et al.’s (2001) , continue to be frequently cited as strong support for the use of some popular estimators even though the strength of the evidence in the studies is weak (Bianchi, Schonfeld, & Laurent, 2014, 2015a, 2015b; Kleijweg, Verbraak, & Van Dijk, 2013; Rafferty, Lemkau, Purdy, & Rudisill, 1986; Thomas, 2004; Van Dam, Keijsers, Verbraak, Eling, & Becker, 2015) . For example, Rafferty et al.’s (1986) used an older version of the MBI and found 67 residents’ emotional exhaustion frequency scores and intensity scores and their depersonalization intensity scores were statistically significantly correlated with their responses to a single burnout item (scaled as 1 (not at all burned out) to 9 (very burned out)). In spite of the small sample size, this result is often used to justify the claim that the sense of personal accomplishment aspect of burnout is less important than the other two aspects. The study does not justify the conclusion because the sense of personal accomplishment aspect of burnout was inadequately described in their single burnout item’s stem, thus biasing the results. In addition, the study also found that the residency-directors’ assessments of the residents’ burnout, were associated with the residents’ MBI sense of accomplishment frequencies but not with their depersonalization scores. Schaufeli et al.’s (2001) finding that the MBI emotional exhaustion and depersonalization subscales had discriminant validity for work-related neurasthenia is also often cited as strong support for the sole use of these two aspects as burnout estimators, even though both the neurasthenia disorder and the measurement of it used in the study have since lost their scientific support.

1.3. The Prevalence of “High” Burnout Depends on the Estimator Used

The concern over which of the five estimators to use is important because the prevalence of “high” burnout depends on which estimator is used. Some investigators argue strongly that some of the five estimators overestimate the prevalence of “high” burnout (Blanchard, Rodrigues, & Colombat, 2012; Kleijweg et al., 2013) . Others argue that restricting the estimator of “high” burnout to the occurrence of “Burnout Syndrome” alone (the joint prevalence of “high” emotional exhaustion and depersonalization and “low” sense of personal accomplishment) underestimates it (Campbell et al., 2011; Dyrbye & Shanafelt, 2015; Shanafelt, Dyrbye, & West, 2013) . Figure 1 shows mathematically that four of the five estimators will necessarily yield prevalences of “high” burnout that are greater than the prevalence of “Burnout Syndrome” alone. This is due to the fact that the occurrence of “Burnout Syndrome” can be no more prevalent than the least prevalent of any one of its aspects alone or in combinations. In addition, large differences in the “high” burnout prevalence will be observed when assessed in a single population at the same time using the five different estimators. This problem is easy to see in the literature, which we next describe.

In the studies we examined where the joint occurrence of all three aspects of Burnout Syndrome was used as the estimator of “high” burnout, prevalence ranged only from 2% to 13% (Al-Dubai & Rampal, 2010; Contag et al., 2010; Gabbe et al., 2008; Golub, Johns, Weiss, Ramesh, & Ossoff, 2008; Golub, Weiss, Ramesh, Ossoff, & Johns, 2007; Gorter, Albrecht, Hoogstraten, & Eijkman, 1999) . When data on this estimator were not directly available, we used the least prevalent aspect reported in the study as the upper bound on the possible joint prevalence of all three aspects and found the upper bounds ranged from 12.4% to 28% (Campbell, Prochazka, Yamashita, & Gopal, 2010; Dyrbye et al., 2008; Grassi & Magnani, 2000; Klimo Jr. et al., 2013; Shanafelt et al., 2012; Shanafelt et al., 2015) . The “high” burnout prevalences based on the bivariate joint occurrence of only two aspects―“both ‘high’ emotional exhaustion and depersonalization”- were similarly relatively low (for example, 17% in Gabbe et al., 2008 ), as were “high” burnout prevalences based on “‘high’ emotional exhaustion plus one other aspect” (for example, 18% to 29.9% in (Brenninkmeijer & Van Yperen, 2003; Dyrbye & Shanafelt, 2015; Schaufeli & Van Dierendonck, 2000; van der Wal, Bucx, Hendriks, Scheffer, & Prins, 2016) . In contrast, using the “‘high’ emotional exhaustion alone” estimator yielded alarmingly high prevalences of “high” burnout; from 27.5% to 63.2% (Al-Dubai & Rampal, 2010; Bressi et al., 2009; Campbell et al., 2010; Dyrbye et al., 2008; Grassi & Magnani, 2000; Grunfeld et al., 2000; Klersy et al., 2007; Shanafelt et al., 2012; Shanafelt et al., 2015) . This range was similar to the “18% to 76%” range for “high” burnout often cited in the organizational literature (Campbell et al., 2010; Prins et al., 2007) . The prevalence using the “either ‘high’ emotional exhaustion or ‘high’ depersonalization” estimator was similarly high (45.4% to 67%) (Campbell et al., 2011; Campbell et al., 2010; Dyrbye et al., 2015; Dyrbye & Shanafelt, 2015; Dyrbye et al., 2008; Shanafelt et al., 2012; Shanafelt et al., 2015) . Also, in studies where cost-efficient single-item “indicators” or “markers” (Indrayan, 2013) of the MBI subscale scores were substituted for the full MBI subscales, the relative magnitudes of the prevalences obtained from the five estimators were consistent with those obtained using the full MBI scales (Dolan et al., 2015; West et al., 2012; West, Dyrbye, Sloan, & Shanafelt, 2009) .

1.4. Importance of the Current Study

The differences in the prevalences of “high” burnout obtained from the five estimators poses a serious practical problem (Dyrbye, West, & Shanafelt, 2009) in health care organizations. One consequence is comparisons of the prevalences of “high” burnout across some institutions may be spurious. Another is that institutions are vulnerable to 1) spurious data on the prevalence of “high” burnout and burnout vulnerability in their employees, 2) spurious inferences about the effects of employee-reported human-systems problems on “high” burnout prevalence, and 3) spurious assessments of the need for and effects and effectiveness of attempted interventions.

Currently, managers have no research-based information on the association of the aspects of “high” burnout used in the estimators and the specific problems in job demands and resources in human-systems functioning their employees report in annual organizational health surveys. Knowledge of these associations could help to gauge the usefulness of the different “high” burnout estimators in practice and help managers to identify problems that may be affecting burnout. In the current study, we investigated the association (measured as the positive likelihood ratio (LR+)) between each of the subsets of aspects of “Burnout Syndrome” measured in the five burnout estimators and employee-perceptions of problems in 31 types of job demands and resources in human-systems functioning, to determine whether any of the subsets of aspects measured in the specific “high” burnout estimators might act as potential symptoms (or markers) of the employee-perceived problems. The specific research question was: How much more often were the respective three aspects of “Burnout Syndrome” (and their subsets used as estimators of “high” burnout) self-reported by persons who perceived and reported a specific problem in their workplace than in persons who did not (the positive likelihood ratio (LR+)) (McGee, 2002; Scott, Greenberg, & Poole, 2008) .

2. Method

2.1. Participants

We used data from a cross-sectional organizational health census-survey conducted annually in the Veterans Health Administration (VHA) (N(all employees) = 284,634). The estimated overall response rate was 56.3% and complete data were given by 97% of respondents. The survey was voluntary; advertised daily; and available in electronic, paper or telephone format during the last 3 weeks in September 2013. Both participation-confidentiality and response-anonymity were ensured by the data collection, storage and reporting procedures (Osatuke et al., 2012) . Use of the aggregated survey data for research and publication was approved by the Cincinnati (Ohio) Veterans Health Administration Facility IRB (signed documentation available on request).

Here we analyze the data from the 9543 respondents in seven health care provider groups (primary care physicians, surgeons, psychiatrists, anesthesiologists, medicine specialists, nurse practitioners, and physicians’ assistants) (N = 22,721) from among the 87,358 direct-care providers in 16 occupational groups (N(direct care providers) = 160,124) who responded to the survey. Their estimated overall response rate in the direct care providers was 53%, and the complete data rate was 97%. Table 1 shows the specific sample sizes, estimated response rates, and demographic characteristics in each of the seven groups in the current study (nmin = 340 anesthesiologists, nmax = 2165 physician in all medical specialties, nMdn = 1265 psychiatrists; response ratemin = 34% in primary care physicians, response ratemax = 56% in psychiatrists, and response rateMdn = 41.5%).

2.2. Procedure and Measures

All items used in this study were administered on the same survey as a standard

Table 1. Health care providers: sample sizes and demographic percentages.

Note. The groups in this table are respectively: Primary Care Physicians, Surgeons, Psychiatrists, All Medicine-Specialists, Anesthesiologists, Nurse Practitioners (NP), Physicians Assistants (PA), and All Health Care Providers. aBlack or African American. bAmerican Indian or Alaskan Native. cNative Hawaiian or other Pacific Islander.

part of the VHA annual survey.

2.2.1. Burnout Syndrome Aspects

We used single-items from the respective MBI subscales chosen based on the literature and also in consultation with the authors of the MBI (Maslach et al., 1996) to measure the three aspects of “Burnout Syndrome”. As in previous studies using MBI single-item surrogates, employees rated each on the MBI’s ordinal 7-category frequency scale: 0 to 6 (Never, A few times a year, Once a month, A few times a month, Once a week, A few times a week, Every day). For emotional exhaustion and depersonalization we considered ratings of “Once a week” or more (scores of 4 or higher), to represent a “high” frequency of these aspects. We considered sense of personal accomplishment ratings of “A few times a month” or less (3 or less), to represent “low” frequency of a sense personal accomplishment.

We used responses to the well-validated single-item “I feel burned out from my work” (item 8) as a surrogate for the MBI-HSS: EE subscale score (West et al., 2012; West et al., 2009; West et al., 2006) . The well-validated depersonalization item: “I’ve become more callous toward people since I took this job.” (West et al., 2012; West et al., 2009) lacked face-validity for this construct across our wide-range of employees. Based on validity studies, which are described in the next paragraph, we instead used the MBI item: “I worry that this job is hardening me emotionally” (MBI-HSS: DP item 11). We found no previously-used well-validated surrogate item in the literature for the MBI-HSS sense of personal accomplishment subscale score, so based on our validity studies, we used: “I have accomplished many worthwhile things in this job” (MBI-HSS: PA item 19).

We examined the psychometric properties of all three items using data from N = 7081 VHA employees who took the full MBI within the context of their jobs from 2012 to 2016. These showed all three of our items perform as surrogates for their respective MBI scales at the same levels on the same psychometric tests as those examined by West et al. (2009) and Dolan et al. (2015) . The Spearman correlations with deleted-item MBI subscale composite scores were 0.65, 0.75 and 0.84 for the sense of personal accomplishment item, the depersonalization item and the emotional exhaustion items, respectively. The respective Areas Under the Curves (AUCs) in our Receiver Operating Curve Analyses were .82, .94, and .95; and the positive likelihood ratios (LR + s) were 25.0, 40.1, and 31.8, for the responses in the targeted subscale ranges. We used the same methods as used by West et al. (2009) , and found that supposing the base prevalence of 20% for self-reported infrequent sense of personal accomplishment and our stratified likelihood ratios, a response of “A few times a year or less” to our single item measure of sense of personal accomplishment indicated a 70.8% probability of “low” scores on the full MBI-HSS: PA subscale. Supposing base prevalences of 25%, and 30%, respectively, for high depersonalization and for high emotional exhaustion and the stratified likelihood ratios for these domains, we found that responding “A few times a week” on our depersonalization item or on the emotional exhaustion item indicated a probability of “high” scores on these respective MBI subscales of 90.1% and 95.2%. These results support the validity of the single items we used as surrogates for their respective MBI subscales.

2.2.2. Human-Systems Problems

We used the 31 additional single items (shown in Table 2) to query employees

Table 2. Items querying problems in job demands and resources in human-systems functioning and their abbreviations.

Note. Items were scored on a 5-point ordinal-category scale (1 (Strongly Disagree) to 5 (Strongly agree)). Scores of 2 or less were considered to indicate an employee “perceived problem”. aThese items were reverse scored.

perceptions of problems in job demands or resources in human-systems functioning (e.g. the workplace civility; job control; supportiveness of the supervisor and workgroup; physical, psychological and professional safety; and relationships with the unit supervisor, workgroup, organization, and upper level management). Each item was scored on a 5-point ordinal-category scale (1 (Strongly Disagree) to 5 (Strongly Agree)). Scores of 2 or less were considered to indicate an employee “perceived problem”. Psychometric properties were discussed in Osatuke et al. (2012) .

2.2.3. Occupation

Employees self-identified their occupations from a list.

2.3. Analysis Strategy

To measure associations, in each health care provider group, we calculated how much more the “high” burnout estimator’s aspects occurred among persons who perceived a workplace problem than in persons who did not (the positive likelihood ratio (LR+)). This statistic is frequently used in clinical medicine and epidemiology. We used the epiR program (Stevenson, 2015) in R (R Development Core Team, 2014) . To simplify interpretation, we translated the LR+ statistics into “how much a finding of the presence of the burnout estimator’s aspects increased the prevalence of the perception of the problem beyond the baseline prevalence in the group’ (the “added prevalence”). We used McGee’s (2002) formula: “added prevalence” = [100 × (0.19 × ln(LR+)). Like LR+, the “added prevalence” captures both linear and nonlinear associations in ordered categorical data, and is independent of the baseline prevalence of the problem in the population (Indrayan, 2013; Scott et al., 2008) . Our large sample sizes ensured statistical significance, so we judged “added prevalence” to be clinically significant if the lower bound on its 99% confidence interval was greater than 20% (McGee, 2002) .

3. Results

Table 3 showcases the differences in the prevalences of “high” burnout produced by the five estimators. It also shows within each health care provider groups the prevalence of each of the eight possible trivariate joint configurations of “high and “low” emotional exhaustion, depersonalization, and sense of accomplishment and how these prevalences are mapped by each of the five “high” burnout estimators into the prevalence of “high” “experienced feelings of burnout”. Table 4 shows the clinically significant associations between each of the 31 problems studied and each of the five “high” burnout estimators in each health provider group.

3.1. The “High” Burnout Prevalence Depended on the Estimator Used

We found that, within each group, the five “high” burnout estimators produced large differences in the estimated prevalence of “high” burnout, as shown in the

Table 3. Percent prevalence of trivariate outcomes of dichotomized measures of the three burnout aspects and of each of the five “high” burnout estimators in five groups of health-care providers.

Note. For both Emotional Exhaustion (E) and Depersonalization (D), we defined frequencies as “High” (H) if the occurrence was “once a week” or more (4 or higher). For Sense of Personal Accomplishment (PA), we defined frequencies as “Low” (L) if the occurrence was “a few times a month” or less (3 or less). Groups in this table are respectively: Primary Care Physicians, Surgeons, Psychiatrists, All Medicine-Specialists, Anesthesiologists, Physicians Assistants, Nurse Practitioners, and All Health Care Providers. aThe 99% Confidence Intervals on the percent prevalence estimates were each smaller than ±0.02%. b“Burnout Syndrome”. cNo Aspects of “Burnout Syndrome”.

lower part of Table 3. For psychiatrists the estimated prevalence of “high” burnout using “‘high’ emotional exhaustion or depersonalization” (36.8%) was six times higher than the prevalence estimated using the trivariate joint occurrence of all three aspects of “Burnout Syndrome” (6.0%). The prevalence of “high” burnout was lowest in all groups when estimated as “Burnout Syndrome”. It ranged from 5.6% in all medical specialties and anesthesiologists to 13.4% in primary care physicians. The estimated prevalence of “high” burnout was highest in all groups when estimated by the combined occurrence of “either ‘high’

Table 4. Base and Clinically-Significant Added Percent Prevalence (≥20%) of Perceived Problems in Job Demands and Resources in Human-Systems Functioning Associated with the Subsets of “High” Emotional Exhaustion (EE), “High” Depersonalization (DP) and “Low” Sense of Personal Accomplishment (PA) Used as “High” Burnout Estimators within Health Care Providers (99% CIs in Brackets).

Note. The groups in this table are: Primary Care Physicians, Surgeons, Psychiatrists, All Medicine-Specialists, Anesthesiologists, Nurse Practitioners and Physicians Assistants. EE = “High” Emotional Exhaustion: Emotional Exhaustion frequencies of “once a week” or more (4 or higher). DP = “High” Depersonalization: Depersonalization frequencies of “once a week” or more (4 or higher). PA =”Low” Sense of Personal Accomplishment: Sense of Personal Accomplishment frequencies of “a few times a month” or less (3 or less). The five “high” burnout estimators and the single aspects of Burnout Syndrome for which clinically significant results were obtained are indicated as follows: EE ≡ “High” Emotional Exhaustion alone; DP ≡ “High” Depersonalization alone; EE OR DP ≡ High” Emotional Exhaustion or “High” Depersonalization; EE & (DP OR PA) ≡ High” Emotional Exhaustion and either (“High” Depersonalization or “Low” Sense of Personal Accomplishment); EE & DP ≡ High” Emotional Exhaustion and “High” Depersonalization; and EE & DP & PA ≡ the trivariate joint occurrence of High” Emotional Exhaustion and “High” Depersonalization and “Low” Sense of Personal Accomplishment. The “problem- base” rows provide the percent prevalence of the problem in the group. “Add, If” indicates that rows following it give the “Added Percent Prevalence” if the estimator occurs.. To calculate the “the percent prevalence of the human- systems problem, given the presence of the aspects of “Burnout Syndrome” queried by the estimator”, sum: the Base % Prevalence + Added % Prevalence. For example, Among Medicine-Specialists who self-report EE & DP & PA, the risk of perceived problems in “safety in disagreeing” is: 11% (base prevalence in our institution) + 34% = 45%. To calculate the positive likelihood ratio from the data use:. aThe full item queries related to the abbreviated problem description are listed in Table 1. bIf the base prevalence < 10%, the “Added Percent Prevalence” formula we used underestimates the change in probability. The exact value can be calculated from the information in the table by first calculating the posterior odds as. Then calculating the “exact value for added prevalence” = (posteriorodds/(1+ posteriorodds)) − base. cThe “Added Percent Prevalence” given in this section is for only for EE&DP&PA. The values for EE&DP are lower.

emotional exhaustion or ‘high’ depersonalization”. It ranged from 24.2% in anesthesiologists to 50.2% in primary care physicians.

We noted that reporting the prevalence of all eight trivariate configurations in a group, like those shown in the first part of Table 3, may reveal vulnerabilities within the group to large increases in burnout prevalence. For example, primary care physicians appeared vulnerable to an increase in the prevalence of “Burnout Syndrome” from 13.4% to as high as 33% if the 19.6% of them who were jointly “high” both on emotional exhaustion and depersonalization, but also currently “high” on sense of personal accomplishment (H, H, H) were to lose the factors in their work environment sustaining their “high” sense of personal accomplishment.

3.2. Job Problems Were Consistently Associated only with “Burnout Syndrome”

As Table 4 shows, when “high” burnout was estimated as the trivariate joint occurrence of “‘high’ emotional exhaustion, ‘high’ depersonalization and ‘low’ sense of personal accomplishment”, it was clinically-significantly associated with 97% (30) of the 31 problems studied in at least one of the health care provider groups.

In contrast, we found the two most commonly used “high” burnout estimators (“‘high’ emotional exhaustion alone”, and “either ‘high’ emotional exhaustion or ‘high’ depersonalization”) were associated only with six of the human- systems problems studied, but only within four provider groups. Table 4 shows both of the most commonly used estimators were associated with workload within surgeons and medicine specialists. “High” emotional exhaustion alone was associated with workload in psychiatrists and nurse practitioners. Within anesthesiologists, 1) both estimators were associated with cooperation among workgroups; 2) “‘high’ emotional exhaustion alone’ was associated with skills development and supervisor relationship; and 3) “either ‘high’ emotional exhaustion or ‘high’ depersonalization was associated with problems with respect at work. Turnover intentions were also associated with both commonly used estimators in several groups.

Although not shown in Table 4, “high” depersonalization alone was associated with more perceived human-systems problems (including job control, safety in problem solving, and work/family balance) than (a) the “‘high’ emotional exhaustion alone” estimator and (b) the “either ‘high’ emotional exhaustion or ‘high’ depersonalization” estimator.

Table 4 also shows the three more rarely used estimators were associated with many more of the human-systems problems we studied.

4. Discussion

Our results indicated that “Burnout Syndrome” was the preferred estimator of “high” burnout, if one wished to measure the prevalence of “high” burnout and interpret it as a potential symptom or marker of specific potential problems in job demands and resources in human-systems functioning perceived by health care providers.

Only a few associations were detected between the employee perceived problems in job demands and resources and the two most commonly used “high” burnout estimators. This finding was not unexpected, given the pervasiveness of emotional exhaustion in direct health care providers, where it is thought to arise from multiple factors (Van Dam et al., 2015) . Its pervasiveness explains why “‘high’ emotional exhaustion” treated as a symptom was often not more likely to occur among those who perceived a specific problem than among those who did not. In contrast, in studies that treat “‘high’ emotional exhaustion’ as a problem associated with potential sequelae such as suicidality, medical errors, or alcoholism, it is reasonable to expect that concerns such as perceived medical errors are more likely to occur among those who have “high” emotional exhaustion than among those who do not. Our results show that even though this aspect of burnout has unfavorable consequences, it does not help managers to locate the problems in the work environment that may be affecting the occurrence of “high” burnout.

Finally, the variation across the health care provider groups in how perceived problems were associated with the joint trivariate occurrence of “‘high’ emotional exhaustion, ‘high’ depersonalization and ‘low’ sense of personal accomplishment” suggests that “high” burnout may be related not only to the levels of general factors within the structure of work but also to the occurrence of specific human-systems problems in job demands and resources within that environment. Our results indicate that specific kinds of problems in job demands and resources may also interact with the specific characteristics of work within occupations (and the concomitant occupation-specific micro environment) in ways that affect group-specific burnout vulnerability.

5. Limitations

This study used cross-sectional data. Participation was voluntary, therefore employees with the highest levels of burnout may have been either under or over represented. The presence of problems in the 31 job demands and resources studied was based solely on employee perceptions; they have no reference standards (i.e. no “gold standard”) (Indrayan, 2013) . We did not investigate the associations of the “high” burnout estimators with the joint occurrence of any subsets of the perceived problems. This type of study may provide additional insight into problems in demands and resources associated with “high” burnout. We controlled only partially for chance associations across the problems studied-if we had set the CIs at 99.8% instead of 99% we would have provided an overall alpha less than 0.05. We opted for the less conservative 99% intervals because the study was exploratory. Making this change would have increased the width of the CIs in Table 4 by one percentage point.

In small groups, the number of employees experiencing the joint occurrence of all three aspects of “Burnout Syndrome” may be too small to show clinically-significant associations. In these cases, our data indicate that the joint experience of “‘high’ depersonalization and ‘low’ sense of personal accomplishment” or the experience of “‘high’ depersonalization alone” may still exhibit clinically significant associations with some perceived problems.

Future research is needed to show the sensitivity of “high” burnout prevalence to change when the perceived problem(s) associated with it are reduced. Variation in these sensitivities to change, across occupational groups, is also of interest. The “virulence of perceived problems” is also important: “Which perceived problems produce the greatest increase in ‘high’ burnout prevalence within and across specific occupations, and within and across institutions”. Two other important questions are “How quickly do the various associated perceived prob- lems produce a “high” burnout response?” and “what in the job demands and resources environment may protect those, currently experiencing two of the three joint aspects of ‘high’ burnout, from experiencing the third?”

Cite this paper

Beckstrand, J., Yanchus, N., & Osatuke, K. (2017). Only One Burnout Estimator Is Consistently Associated with Health Care Providers’ Perceptions of Job Demand and Resource Problems. Psychology, 8, 1019-1041.


  1. 1. Al-Dubai, S. A. R., & Rampal, K. G. (2010). Prevalence and Associated Factors of Burnout among Doctors in Yemen. Journal of Occupational Health, 52, 58-65. [Paper reference 2]

  2. 2. Bakker, A. B., & Demerouti, E. (2006). The Job Demands-Resources Model: State of the Art. Journal of Managerial Psychology, 22, 309-328. [Paper reference 1]

  3. 3. Bakker, A. B., Demerouti, E., & Euwema, M. C. (2005). Job Resources Buffer the Impact of Job Demands on Burnout. Journal of Occupational Health Psychology, 10, 170-180. [Paper reference 1]

  4. 4. Bianchi, R., Schonfeld, I. S., & Laurent, E. (2014). Is Burnout a Depressive Disorder? A Reexamination with Special Focus on Atypical Depression. International Journal of Stress Management, 21, 307-324. [Paper reference 1]

  5. 5. Bianchi, R., Schonfeld, I. S., & Laurent, E. (2015a). Burnout-Depression Overlap: A Review. Clinical Psychology Review, 36, 28-41. [Paper reference null]

  6. 6. Bianchi, R., Schonfeld, I. S., & Laurent, E. (2015b). Is Burnout Separable from Depression in Cluster Analysis? A Longitudinal Study. Social Psychiatry and Psychiatric Epidemiology, 50, 1005-1011. [Paper reference null]

  7. 7. Blanchard, P., Rodrigues, M., & Colombat, P. (2012). On the Prevalence and Causes of Oncologist Burnout. Journal of Clinical Oncology, 30, 3029-3030. [Paper reference 1]

  8. 8. Boersma, K., & Lindblom, K. (2009). Stability and Change in Burnout Profiles over Time: A Prospective Study in the Working Population. Work and Stress, 23, 264-283. [Paper reference 1]

  9. 9. Brenninkmeijer, V., & Van Yperen, N. (2003). How to Conduct Research on Burnout: Advantages and Disadvantages of a Unidimensional Approach in Burnout Research. Occupational and Environmental Medicine, 60, 16-20. [Paper reference 2]

  10. 10. Bressi, C., Porcellana, M., Gambini, O., Madia, L., Muffatti, R., Peirone, A., Zanini, S., Erlicher, A., Scarone, S., & Altamura, A. C. (2009). Burnout among Psychiatrists in Milan: A Multicenter Survey. Psychiatric Services, 60, 985-988. [Paper reference 1]

  11. 11. Campbell, J., Prochazka, A. V., Yamashita, T., & Gopal, R. (2010). Predictors of Persistent Burnout in Internal Medicine Residents: A Prospective Cohort Study. Academic Medicine, 85, 1630-1634. [Paper reference 4]

  12. 12. Campbell, J., Prochazka, A., & Gopal, R. (2011). The Need for a Uniform Use of the Construct of Burnout Reply. Academic Medicine, 86, 661-661. [Paper reference 3]

  13. 13. Contag, S. P., Golub, J. S., Teknos, T. N., Nussenbaum, B., Stack Jr., B. C., Arnold, D. J., & Johns III, M. M. (2010). Professional Burnout among Microvascular and Reconstructive Free-Flap Head and Neck Surgeons in the United States. Archives of Otolaryngology-Head & Neck Surgery, 136, 950-956. [Paper reference 1]

  14. 14. de Oliveira, G. S., Chang, R., Fitzgerald, P. C., Almeida, M. D., Castro-Alves, L. S., Ahmad, S., & McCarthy, R. J. (2013). The Prevalence of Burnout and Depression and Their Association with Adherence to Safety and Practice Standards: A Survey of United States Anesthesiology Trainees. Anesthesia and Analgesia, 117, 182-193. [Paper reference 1]

  15. 15. Demerouti, E., Bakker, A. B., de Jonge, J., Janssen, P. P. M., & Schaufeli, W. B. (2001). Burnout and Engagement at Work as a Function of Demands and Control. Scandinavian Journal of Work Environment & Health, 27, 279-286. [Paper reference 1]

  16. 16. Demerouti, E., Verbeke, W., & Bakker, A. B. (2005). Exploring the Relationship between a Multidimensional and Multifaceted Burnout Concept and Self-Rated Performance. Journal of Management, 31, 186-209. [Paper reference 2]

  17. 17. Dolan, E. D., Mohr, D., Lempa, M., Joos, S., Fihn, S. D., Nelson, K. M., & Helfrich, C. D. (2015). Using a Single Item to Measure Burnout in Primary Care Staff: A Psychometric Evaluation. Journal of General Internal Medicine, 30, 582-587. [Paper reference 2]

  18. 18. Dyrbye, L. N., & Shanafelt, T. D. (2015). Using Liberal Criteria to Identify Burnout Poses the Risk of Pathologizing Normal Adaptive States Reply. Academic Medicine, 90, 1584-1584. [Paper reference 4]

  19. 19. Dyrbye, L. N., Eacker, A., Durning, S. J., Brazeau, C., Moutier, C., Massie, F. S., Satele, D., Sloan, J. A., & Shanafelt, T. D. (2015). The Impact of Stigma and Personal Experiences on the Help-Seeking Behaviors of Medical Students with Burnout. Academic Medicine, 90, 961-969. [Paper reference 1]

  20. 20. Dyrbye, L. N., Thomas, M. R., Massie, F. S., Power, D. V., Eacker, A., Harper, W., Durning, S., Moutier, C., Szydlo, D. W., Novotny, P. J., Sloan, J. A., & Shanafelt, T. D. (2008). Burnout and Suicidal Ideation among US Medical Students. Annals of Internal Medicine, 149, 334-W370. [Paper reference 3]

  21. 21. Dyrbye, L. N., West, C. P., & Shanafelt, T. D. (2009). Defining Burnout as a Dichotomous Variable. Journal of General Internal Medicine, 24, 440. [Paper reference 1]

  22. 22. Gabbe, S. G., Webb, L. E., Moore, D. E., Harrell Jr., F. E., Spickard Jr., W. A., & Powell Jr., R. (2008). Burnout in Medical School Deans: An Uncommon Problem. Academic Medicine: Journal of the Association of American Medical Colleges, 83, 476-482. [Paper reference 2]

  23. 23. Gil-Monte, P. R. (2005). Factorial Validity of the Maslach Burnout Inventory (MBI-HSS) among Spanish Professionals. Revista de Saúde Publica, 39, 1-8. [Paper reference 1]

  24. 24. Golub, J. S., Johns III, M. M., Weiss, P. S., Ramesh, A. K., & Ossoff, R. H. (2008). Burnout in Academic Faculty of Otolaryngology—Head and Neck Surgery. Laryngoscope, 118, 1951-1956. [Paper reference 1]

  25. 25. Golub, J. S., Weiss, P. S., Ramesh, A. K., Ossoff, R. H., & Johns III, M. M. (2007). Burnout in Residents of Otolaryngology-Head and Neck Surgery: A National Inquiry into the Health of Residency Training. Academic Medicine, 82, 596-601. [Paper reference 1]

  26. 26. Gorter, R. C., Albrecht, G., Hoogstraten, J., & Eijkman, M. A. J. (1999). Professional Burnout among Dutch Dentists. Community Dentistry and Oral Epidemiology, 27, 109-116. [Paper reference 1]

  27. 27. Grassi, L., & Magnani, K. (2000). Psychiatric Morbidity and Burnout in the Medical Profession: An Italian Study of General Practitioners and Hospital Physicians. Psychotherapy and Psychosomatics, 69, 329-334. [Paper reference 2]

  28. 28. Grunfeld, E., Whelan, T. J., Zitzelsberger, L., Willan, A. R., Montesanto, B., & Evans, W. K. (2000). Cancer Care Workers in Ontario: Prevalence of Burnout, Job Stress and Job Satisfaction. Canadian Medical Association Journal, 163, 166-169. [Paper reference 1]

  29. 29. Hakanen, J. J., Schaufeli, W. B., & Ahola, K. (2008). The Job Demands-Resources Model: A Three-Year Cross-Lagged Study of Burnout, Depression, Commitment, and Work Engagement. Work and Stress, 22, 224-241. [Paper reference 1]

  30. 30. Hansen, V., & Pit, S. (2016). The Single Item Burnout Measure Is a Psychometrically Sound Screening Tool for Occupation Burnout. Health Scope, 5, e32164. [Paper reference 1]

  31. 31. Indrayan, A. (2013). Medical Biostatistics (3rd ed.). Boca Raton, FL: CRC Press. [Paper reference 3]

  32. 32. Kleijweg, J. H. M., Verbraak, M., & Van Dijk, M. K. (2013). The Clinical Utility of the Maslach Burnout Inventory in a Clinical Population. Psychological Assessment, 25, 435-441. [Paper reference 2]

  33. 33. Klersy, C., Callegari, A., Martinelli, V., Vizzardi, V., Navino, C., Malberti, F., Tarchini, R., Montagna, G., Guastoni, C., Bellazzi, R., Rarnpino, T., David, S., Barbieri, C., Dal Canton, A., Politi, P., & Working Grp Burnout, D. (2007). Burnout in Health Care Providers of Dialysis Service in Northern Italy—A Multicentre Study. Nephrology Dialysis Transplantation, 22, 2283-2290. [Paper reference 1]

  34. 34. Klimo Jr., P., DeCuypere, M., Ragel, B. T., McCartney, S., Couldwell, W. T., & Boop, F. A. (2013). Career Satisfaction and Burnout among US Neurosurgeons: A Feasibility and Pilot Study. World Neurosurgery, 80, e59-e68. [Paper reference 1]

  35. 35. Lee, R. T., & Ashforth, B. E. (1996). A Meta-Analytic Examination of the Correlates of the Three Dimensions of Job Burnout. Journal of Applied Psychology, 81, 123-133. [Paper reference 1]

  36. 36. Leiter, M. P., Hakanen, J. J., Ahola, K., Toppinen-Tanner, S., Koskinen, A., & Vaananen, A. (2013). Organizational Predictors and Health Consequences of Changes in Burnout: A 12-Year Cohort Study. Journal of Organizational Behavior, 34, 959-973. [Paper reference 2]

  37. 37. Loera, B., Converso, D., & Viotti, S. (2014). Evaluating the Psychometric Properties of the Maslach Burnout Inventory-Human Services Survey (MBI-HSS) among Italian Nurses: How Many Factors Must a Researcher Consider? PLoS ONE, 9, e114987. [Paper reference 1]

  38. 38. Maslach, C., Jackson, S. E., & Leiter, M. P. (1996). Maslach Burnout Inventory Manual (3rd ed.). Palo Alto, CA: Mind Garden. [Paper reference 6]

  39. 39. Maslach, C., Schaufeli, W. B., & Leiter, M. P. (2001). Job Burnout. Annual Review of Psychology, 52, 397-422. [Paper reference 1]

  40. 40. McGee, S. (2002). Simplifying Likelihood Ratios. Journal of General Internal Medicine, 17, 647-650. [Paper reference 3]

  41. 41. Meszaros, V., Adam, S., Szabo, M., Szigeti, R., & Urban, R. (2014). The Bifactor Model of the Maslach Burnout Inventory-Human Services Survey (MBI-HSS)—An Alternative Measurement Model of Burnout. Stress and Health, 30, 82-88. [Paper reference 1]

  42. 42. Oreskovich, M. R., Kaups, K. L., Balch, C. M., Hanks, J. B., Satele, D., Sloan, J., Meredith, C., Buhl, A., Dyrbye, L. N., & Shanafelt, T. D. (2012). Prevalence of Alcohol Use Disorders among American Surgeons. Archives of Surgery, 147, 168-174. [Paper reference 2]

  43. 43. Osatuke, K., Draime, J., Moore, S. C., Ramsel, D., Meyer, A., Barnes, S., Belton, S., & Dyrenforth, S. R. (2012). Organization Development in the Department of Veterans Affairs. In T. Miller (Ed.), The Praeger Handbook of Veterans Health: History, Challenges, Issues and Developments, Volume IV: Future Directions in Veterans Healthcare (pp. 21-76). Santa Barbara, CA: Praeger. [Paper reference 2]

  44. 44. Pedersen, A. F., Sorensen, J. K., Bruun, N. H., Christensen, B., & Vedsted, P. (2016). Risky Alcohol Use in Danish Physicians: Associated with Alexithymia and Burnout? Drug and Alcohol Dependence, 160, 119-126. [Paper reference 1]

  45. 45. Prins, J. T., Gazendam-Donofrio, S. M., Tubben, B. J., van der Heijden, F., de Wiel, H., & Hoekstra-Weebers, J. (2007). Burnout in Medical Residents: A Review. Medical Education, 41, 788-800. [Paper reference 2]

  46. 46. R Development Core Team (2014). R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. [Paper reference 1]

  47. 47. Rafferty, J. P., Lemkau, J. P., Purdy, R. R., & Rudisill, J. R. (1986). Validity of the Maslach Burnout Inventory for Family Practice Physicians. Journal of Clinical Psychology, 42, 488-492.
    3.0.CO;2-S><488::AID-JCLP2270420315>3.0.CO;2-S [Paper reference 3]

  48. 48. Schaufeli, W. B., & Bakker, A. B. (2004). Job Demands, Job Resources, and Their Relationship with Burnout and Engagement: A Multi-Sample Study. Journal of Organizational Behavior, 25, 293-315. [Paper reference 1]

  49. 49. Schaufeli, W. B., & Salanova, M. (2007). Efficacy or Inefficacy, That’s the Question: Burnout and Work Engagement, and Their Relationships with Efficacy Beliefs. Anxiety, Stress, and Coping, 20, 177-196. [Paper reference 1]

  50. 50. Schaufeli, W. B., & Van Dierendonck, D. (2000). Utrechtse Burnout Schaal (UBOS), Handleiding [Utrecht Burnout Scale, Manual]. Lisse, The Netherlands: Swets & Zeitlinger. [Paper reference 1]

  51. 51. Schaufeli, W. B., Bakker, A. B., Hoogduin, K., Schaap, C., & Kladler, A. (2001). On the Clinical Validity of the Maslach Burnout Inventory and the Burnout Measure. Psychology & Health, 16, 565-582. [Paper reference 4]

  52. 52. Scott, I. A., Greenberg, P. B., & Poole, P. J. (2008). Cautionary Tales in the Clinical Interpretation of Studies of Diagnostic Tests. Internal Medicine Journal, 38, 120-129. [Paper reference 2]

  53. 53. Shanafelt, T. D., Balch, C. M., Bechamps, G., Russell, T., Dyrbye, L., Satele, D., Collicott, P., Novotny, P. J., Sloan, J., & Freischlag, J. (2010). Burnout and Medical Errors among American Surgeons. Annals of Surgery, 251, 995-1000. [Paper reference 3]

  54. 54. Shanafelt, T. D., Boone, S., Tan, L., Dyrbye, L. N., Sotile, W., Satele, D., West, C. P., Sloan, J., & Oreskovich, M. R. (2012). Burnout and Satisfaction with Work-Life Balance among US Physicians Relative to the General US Population. Archives of Internal Medicine, 172, 1377-1385. [Paper reference 3]

  55. 55. Shanafelt, T. D., Dyrbye, L. N., & West, C. P. (2013). Physician Burnout: An Urgent Call for Early Intervention—Reply. JAMA Internal Medicine, 173, 710-711. [Paper reference 1]

  56. 56. Shanafelt, T. D., Hasan, O., Dyrbye, L. N., Sinsky, C., Satele, D., Sloan, J., & West, C. P. (2015). Changes in Burnout and Satisfaction with Work-Life Balance in Physicians and the General US Working Population between 2011 and 2014. Mayo Clinic Proceedings, 90, 1600-1613. [Paper reference 3]

  57. 57. Stevenson, M. (2015). epiR: Tools for the Analysis of Epidemiological Data. [Paper reference 1]

  58. 58. Thomas, N. K. (2004). Resident Burnout. JAMA-Journal of the American Medical Association, 292, 2880-2889. [Paper reference 1]

  59. 59. Van Dam, A., Keijsers, G., Verbraak, M., Eling, P., & Becker, E. (2015). Level and Appraisal of Fatigue Are Not Specific in Burnout. Clinical Psychology & Psychotherapy, 22, 133-141. [Paper reference 2]

  60. 60. van der Wal, R. A. B., Bucx, M. J. L., Hendriks, J. C. M., Scheffer, G.-J., & Prins, J. B. (2016). Psychological Distress, Burnout and Personality Traits in Dutch Anaesthesiologists: A Survey. European Journal of Anaesthesiology, 33, 179-186. [Paper reference 1]

  61. 61. West, C. P., Dyrbye, L. N., Satele, D. V., Sloan, J. A., & Shanafelt, T. D. (2012). Concurrent Validity of Single-Item Measures of Emotional Exhaustion and Depersonalization in Burnout Assessment. Journal of General Internal Medicine, 27, 1445-1452. [Paper reference 6]

  62. 62. West, C. P., Dyrbye, L. N., Sloan, J. A., & Shanafelt, T. D. (2009). Single Item Measures of Emotional Exhaustion and Depersonalization Are Useful for Assessing Burnout in Medical Professionals. Journal of General Internal Medicine, 24, 1318-1321. [Paper reference 5]

  63. 63. West, C. P., Huschka, M. M., Novotny, P. J., Sloan, J. A., Kolars, J. C., Habermann, T. M., & Shanafelt, T. D. (2006). Association of Perceived Medical Errors with Resident Distress and Empathy—A Prospective Longitudinal Study. JAMA-Journal of the American Medical Association, 296, 1071-1078. [Paper reference 2]

  64. 64. Zis, P., Anagnostopoulos, F., & Sykioti, P. (2014). Burnout in Medical Residents: A Study Based on the Job Demands-Resources Model. The Scientific World Journal, 2014, Article ID: 673279. [Paper reference 1]