Open Journal of Radiology
Vol.4 No.2(2014), Article ID:44337,10 pages DOI:10.4236/ojrad.2014.42019

Improving Ordering Practices and Radiology Workflow in the Emergency Department through Multidisciplinary Didactic Series: Survey Based Evaluation

Geraldine Abbey-Mensah, Andrew Kesselman, David Areman

Department of Radiology, SUNY Downstate Medical Center, Brooklyn, USA

Email: Andrew.Kesselman@downstate.edu

Copyright © 2014 by authors and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY).

http://creativecommons.org/licenses/by/4.0/

Received 10 February 2014; revised 10 March 2014; accepted 17 March 2014

ABSTRACT

The purpose of our Quality Improvement (QI) project was to evaluate the effectiveness of radiology-led didactics on efficiency and ordering practices in the emergency department. Residents of the emergency medicine (EM) program at Downstate Medical Center/Kings County Hospital (postgraduate years 1 - 4) participated in a didactic lecture series during the first two months of the 2013 academic year. Two fifteen-minute lectures with a question and answer session were designed to focus on radiation risk, American College of Radiology (ACR) appropriateness criteria and the department of radiology’s policies. A short survey questionnaire that included questions on ACR appropriateness criteria, radiology experiences and opinions/attitudes regarding radiology was distributed and analyzed before and after the didactic series. The average score for the knowledge-based technical questions on ACR appropriateness criteria was 60.5% initially and 76.4% post-lecture with an overall improvement of 25%, which was statistically significant (p < 0.0001). Following a short didactic lecture series, EM residents showed significant improvement of their knowledge of appropriate image utilization on the post-lecture survey. This highlights the need for dedicated radiology-led lecture series for EM residents. This type of program could be implemented yearly and expanded to other departments to promote interdepartmental communication, increases radiology awareness, improves ordering practices and encourages appropriate imaging utilization.

Keywords: ACR Appropriateness Criteria; Quality Improvement; Emergency Department; Referring Physicians

1. Introduction

Diagnostic imaging plays a crucial role in the decision-making process and triage of patients in the emergency department [1] . Recent technological advances in CT and greater accessibility have led to a rapid increase in utilization of CT, particularly in the emergent setting. However, this increase has also led to growing concern about radiation risk and the increasing cost of healthcare. In this new healthcare paradigm, ordering the appropriate study is of utmost importance. It is in the best interest of patients, referring physicians, radiologists and healthcare payers that appropriate imaging is utilized [2] .

The ACR appropriateness criteria, an evidence-based guideline for clinicians was developed in 1993 to aid in the process of ordering the appropriate study for given clinical situations thereby ensuring cost-effective, high quality patient care. However, studies have shown low utilization of these guidelines by referring clinicians when ordering studies for their patients. Furthermore, radiologists are often not consulted due to time constraints in the ER and difficulty with getting a hold of a radiologist after hours, leading to unnecessary redundancies and errors that can cause delays in-patient care [3] . This, combined with the fact that up to 70% of emergency physicians admit to ordering more diagnostic tests than medically indicated due to the current era of defensive medicine, leads to increased radiation exposure and rising healthcare costs [4] . It was, therefore our purpose to increase radiology awareness and educate emergency medicine residents on radiation risk and the ACR appropriateness criteria through radiology-led didactics, and evaluate its effectiveness on efficiency and ordering practices in the emergency department.

2. Materials and Methods

The study was approved by our institutional review board for exemption. The target population was all 85 residents in the emergency medicine (EM) and combined emergency medicine/internal medicine (EM/IM) residency program at SUNY Downstate/Kings County Medical Center. A link to the survey (described below) was sent to the chief residents, who then forwarded the link to their residents.

The survey was created with Google forms and distributed to EM and EM/IM residents in July 2013. The survey consisted of five questions regarding current ordering practices and fourteen knowledge-based technical questions to determine emergency medicine residents’ knowledge of radiation dose/risk and their ability to select the most appropriate study for a given clinical indication (Appendix 1). Respondents answered anonymously but were required to enter their post-graduate year level. This survey was followed by two 15-minute didactic lectures in August 2013 by two radiology residents under the supervision of an Emergency Radiology Attending. One lecture focused on radiation risk, while the other focused on general information/policies about our radiology department and appropriate ordering practices based on ACR Appropriateness Criteria. A post-lecture survey was administered two weeks later, again created with Google forms and distributed to all EM and EM/IM residents. This 19-question survey consisted of five questions related to EM residents’ perception of the lectures and their impact on future ordering practices. The remaining 14 questions were the same questions as the initial survey to see if there was an improvement in scores, post-lecture (Appendix 2).

The surveys were scored, analyzed and compared based on post-graduate year. Additionally, the answers to the initial five questions on current ordering practices were analyzed. Statistical significance was defined as p < 0.05 and Mann Whitney tests were used to obtain p values.

3. Results

A total of 33 pre-lecture surveys were completed prior to the lecture series. Of these responses, 4 (12%) were PGY-1s, 10 (30%) were PGY-2s, 12 (36%) were PGY-3s and 7 (22%) were PGY-4s. The average score for the knowledge based technical questions was 60.5%, with PGY breakdown of 55.8% for PGY-1s, 58.5% for PGY-2s, 62.8% for PGY-3s and 64.8% for PGY-4s. Of the questions related to perceptions of our radiology department, most emergency physicians (55%) polled were somewhat comfortable contacting the radiology department for protocoling and ordering examinations. Also, 69% of emergency physicians polled indicated they had ordered CTs in place of MRI or US on at least a few occasions secondary to time constraints or inability to reach a radiology provider. Finally, 85% of emergency physicians reported they were unfamiliar with the concept of “as low as reasonably achievable” (ALARA).

A total of 20 post-lecture surveys were completed after the lecture series, scored and analyzed. Six (30%) of the respondees were PGY-1s, 3 (15%) were PGY-2s, 6 (30%) were PGY-3s and 5 (25%) were PGY-4s. The average score for the knowledge-based technical questions was 76.4%, with PGY breakdown of 69.2% for PGY-1s, 84.6% for PGY-2s, 73.1% for PGY-3s and 78.5% for PGY-4s (Figure 1). Eighty five percent of emergency physicians polled reported that the lecture series helped with questions and concerns related to ordering and radiation risk and that the experience would change their ordering practices. Seventy percent of the emergency physicians polled reported that the lecture series should be offered each year and 55% indicated that their perception of our radiology department had changed for the better. Finally, 50% of emergency physicians polled reported they were now familiar with the concept of ALARA.

4. Discussion

Technological advances, greater accessibility and increased speed of cross-sectional imaging have caused a rapid increase in the use of such technology, particularly in the ER, where rapid diagnosis and triage is critical. Such crucial decisions are made based on imaging findings. However, because of the inherently fast-paced nature and other challenges of emergency medical practice, radiologic examinations are often ordered without direct consultation with a radiologist [5] . This often leads to unnecessary studies, excessive radiation exposure, and delays in diagnosis and treatment. With the upcoming changes with the Affordable Care Act and a new government proposal that would deny Medicare payment if the ordering provider did not consult appropriateness criteria, evidence-based medicine is an increasingly important determinant in decision-making. The ACR appropriateness criteria provides evidence-based guidelines to assist referring clinicians in selecting the most appropriate imaging exam based on a specific clinical situation, however, several studies have shown low utilization of these guidelines criteria by referring physicians due to lack of awareness on its existence [3] [6] -[8] .

In this study, we sought to improve radiology awareness and educate referring physicians on utilizing ACR appropriateness criteria in the emergent and non-emergent settings. Dym et al., like several other studies showed lack of improvement in ordering practices with advancing level of training [5] . However, these studies did not attempt to educate referring clinicians about ACR appropriateness criteria after learning of these results. Our study demonstrated that with radiology-led didactic lectures, knowledge and ordering practices could improve among emergency resident physicians.

Typical online survey response rates are 30%. With 30 of 85 residents (35%) responding to the initial survey and 20 of 85 residents (24%) of residents responding to the subsequent survey, we felt our sample size was adequate.

The overall average score for the technical questions on the initial survey was 60.5%, which was lower than the results in Dym et al., which had an average score of 71% [5] . The more senior EM and EM/IM residents (PGY-3 and PGY-4), scored slightly higher than their junior counterparts (PGY-1 and PGY-2), which is expected. The post-lecture survey showed overall improvement of 25% above the initial scores across all PGY levels, which was statistically significant (p < 0.0001). The greatest improvement was seen in more junior residents (PGY-1 and PGY-2), with their scores almost matching that of their seniors. There was an average improvement of 29% above the initial scores for junior residents as opposed to an average of 20% for more senior residents (PGY-3 and PGY-4) (Figure 2). This may be due to senior residents ordering studies based on personal experience rather than utilizing radiology guidelines. Also, the question that was most incorrectly answered

Figure 1. Technical questions, summary of scores.

Figure 2. Technical questions, average score, junior vs. senior residents.

was about whether or not intravenous contrast was recommended for suspected small bowel obstruction (SBO). According to ACR appropriateness criteria, CT abdomen and pelvis with intravenous contrast is recommended for SBO, however, 85% of EM and EM/IM residents answered incorrectly, stating intravenous contrast was not needed. There was slight improvement, post-lecture with almost 50% of residents getting the answer correct.

In addition, the program helped improve emergency department and radiology department relations. The initial survey demonstrated that there was some confusion over the role of the radiologist in the emergent setting, while the follow-up survey demonstrated that the EM and EM/IM residents had a better understanding of radiology’s role and necessity as a consultant to advise on the most appropriate study thereby improving overall workflow in both departments. We were also able to identify areas of improvement in our department such as accessibility to protocoling resources; we now have a direct link to ACR appropriateness criteria on every hospital workstation.

This type of program should be implemented yearly as part of emergency physician training and expanded to other departments, such as medicine, family medicine, pediatrics and surgery to foster interdepartmental communication, increase radiology awareness and improve ordering practices. Further research with larger series across multiple departments may be helpful to substantiate these results.

5. Conclusion

We observed significant improvement amongst the emergency physician residents in scores related to knowledge based technical questions regarding radiation risk and ACR appropriateness criteria following radiologyled didactic series. The greatest improvement in scores was seen in the junior EM residents (PGY-1 and PGY-2). The lectures were viewed as helpful and influential to ordering practices in the future by EM an EM/IM resident physicians. This emphasizes the need to educate referring clinicians on radiation risk and increase awareness, acceptance and use of ACR appropriateness criteria.

References

  1. Eng, J., Mysko, W.K., Weller, G.E.R., et al. (2000) Interpretation of Emergency Department Radiographs: A Comparison of Emergency Medicine Physicians with Radiologist, Residents with Faculty, and Film with Digital Display. American Journal of Roentgenology (AJR), 175, 1233-1238. http://dx.doi.org/10.2214/ajr.175.5.1751233
  2. Gunderman, R.B., Bettmann, M. and Davis, L.P. (2007) Promoting Educational Innovation: Lessons from the Request for Proposals for ACR Appropriateness Criteria Usage in Medical Education. Journal of the American College of Radiology, 4, 919-924. http://dx.doi.org/10.1016/j.jacr.2007.07.003
  3. Bautista, A.B., Burgos, A., Nickel, B.J., et al. (2009) Do Clinicians Use the American College of Radiology Appropriateness Criteria in the Management of Their Patients. American Journal of Roentgenology (AJR), 192, 1581-1585. http://dx.doi.org/10.2214/AJR.08.1622
  4. Studdert, D.M., Mello, M.M., Sage, W.M., et al. (2005) Defensive Medicine among High-Risk Specialist Physicians in a Volatile Malpractice Environment. The Journal of the American Medical Association (JAMA), 293, 2609-2617. http://dx.doi.org/10.1001/jama.293.21.2609
  5. Dym, R.J., Burns, J. and Taragin, B.H. (2013) Appropriateness of Imaging Studies Ordered by Emergency Medicine Residents: Results of an Online Survey. American Journal of Roentgenology (AJR), 201, W619-W625. http://dx.doi.org/10.2214/AJR.12.10487
  6. Griffey, R. and Sodickson, A. (2009) Cumulative Radiation Exposure and Cancer Risk Estimates in Emergency Department Patients Undergoing Repeat or Multiple CT. American Journal of Roentgenology (AJR), 192, 887-892. http://dx.doi.org/10.2214/AJR.08.1351
  7. News Releases (2013) SGR Replacement Policy Draft Endorses Use of Appropriateness Criteria in Medical Imaging Exam Ordering.
  8. Taragin, B.H., Feng, L. and Ruzal-Shapiro, C. (2003) Online Radiology Appropriateness Survey: Results and Conclusions from an Academic Internal Medicine Residency. Academic Radiology, 10, 781-785. http://dx.doi.org/10.1016/S1076-6332(03)80123-X

Appendices

Appendix 1

Department of Emergency MedicineThank you for your time and participation in our Quality Improvement (QI) project. The goal of our project is to improve ordering practices and the efficiency in which exams are performed for our ED staff. The first part of our QI project uses a preliminary survey to gauge how our radiology department is viewed, possibly highlight some areas for improvement and ends with technical questions which we will answer in a two part lecture series. This will be followed by two-15 minute lectures on ordering exams and CT radiation risk, which will be given in August. The final part of our project will involve a follow-up survey. We hope this project will improve communication between the Radiology and the Emergency Medicine staff, as well as improve patient care. We greatly appreciate your help and look forward to answering any questions you may have.

Thank you for participating in our survey. We look forward to working with you!

1) How often do you consider a patient’s prior radiation exposure (ex. prior scans) before ordering new scans when presenting with history of trauma?

a        Never

b        Sometimes

c        Most of the time

d        Always

2 How often do you consider a patient’s prior radiation exposure (ex. prior scans) before ordering new scans when presenting with non-traumatic history?

a        Never

b        Sometimes

c        Most of the time

d        Always

3) How comfortable do you feel contacting the radiology department for questions concerning protocol and ordering of examinations?

a        Not comfortable

b        Somewhat comfortable

c        Very comfortable

4) How often do you have questions about when and what type of contrast is necessary for an examination? a Never

b        Sometimes

c        Most of the time

d        Always

5) Have you ever ordered a CT exam in place of an US or MR secondary to time constraints or inability to reach the radiology provider?

a        No

b        Yes on a single occasion

c        Yes on a few occasions

d        Yes on many occasions

6) Are you familiar with the concept of ALARA?

a        Yes

b        No

7) The radiation dose of a CT of the abdomen and pelvis is equal to approximately how many single view chest x-rays?

a        10

b        100

c        1000

d        >10,000

8) How much of annual cancers are thought to be radiation induced? a        <1%

b        2%

c        10%

d        25%

9) The lowest annual radiation dose at which there is clearly an increased risk of cancer is approximately:

a        10 mSv

b        100 mSv

c        500 mSv

d        1000 mSv

10) True or False. Pre and Post-contrast scans are necessary for routine CT of the abdomen and pelvis for abdominal pain.

a        True

b        False

11) How long before a CT of the abdomen and pelvis should oral contrast be administered to adequately opacify the small bowel and proximal large bowel?

a        Night before

b        60 - 90 minutes prior

c        90 - 120 minutes prior

d        4 - 6 hours prior

e        At the time of the study

12) Which of the following is a recommended indication for a CT scan?

a        Uncomplicated headache

b        Follow up for adnexal cysts

c        Low pretest probability for PE

d        Focal abdominal pain

e        All of the above

f        None of the above

13) Which of the following cutoffs should be used for the administration of IV contrast when calculating GFR?

a        0

b        <30

c        <60

d        <90

14) True or False. IV contrast is necessary to rule out small bowel obstruction.

a        True

b        False

15) Which of the following is not an indication for an emergent pelvic ultrasound?

a        Pelvic pain

b        Postmenopausal bleeding

c        First trimester vaginal bleeding with Beta HCG >2000

d        Pregnant woman in first trimester with falling Beta HCG

16) Which of the following is the most likely cause of bilateral lower extremity swelling?

a        Bilateral DVTs

b        Venous Insufficiency

c        CHF exacerbation

d        Peripheral vascular disease

17) Which study requires at least a 20 gauge peripheral IV access?

a        Head CT

b        CT of the abdomen and pelvis with contrast

c        Pulmonary CTA

d        CT of the chest with contrast

18) Which test does not require IV contrast?

a        Bone mass

b        Liver mass

c        Small bowel obstruction

d        Lymphadenopathy

e        Pulmonary nodule

19) If a patient had an allergic reaction to IV contrast, can you give them oral contrast?

a        Yes

b        No

20) What is your current PGY status?

a        PGY 1

b        PGY 2

c        PGY 3

d        PGY 4

e        Other

Appendix 2

Department of Emergency MedicineThank you for your time and participation in our Quality Improvement (QI) project. The goal of our project is to improve ordering practices and the efficiency in which exams are performed for our ED staff. We have already given an initial survey, had two tailored lecture series and hopefully answered any questions you may have had. The final step of our project is this short post experience survey. We hope this project will improve communication between the Radiology and the Emergency Medicine staff, as well as improve patient care. We greatly appreciate your help and look forward to answering any questions you may have.

Thank you again for participating in our survey. We look forward to working with you!

1) Has this lecture series helped with any questions or concerns you may have in relation to ordering studies and radiation risk?

a        Yes

b        No

c        Undecided

2) How do you feel that your ordering practices may change secondary to this experience?

a        Not at all

b        Marginally

c        Substantially

3) Do you feel that this lecture series was helpful and should be offered each year?

a        Yes

b        No

4) Did this experience change your perception of our radiology department?

a        Yes for the better

b        Yes for the worse

c        No

d        Undecided

5) Do you feel more confident in ordering practices and deciding on contrast necessity after this lecture series?

a        Yes

b        No

c        Undecided

6) Are you familiar with the concept of ALARA?

a        Yes

b        No

7) The radiation dose of a CT of the abdomen and pelvis is equal to approximately how many single view chest x-rays?

a        10

b        100

c        1000

d        >10,000

8) How much of annual cancers are thought to be radiation induced?

a        <1%

b        2%

c        10%

d        25%

9) The lowest annual radiation dose at which there is clearly an increased risk of cancer is approximately:

a        10 mSv

b        100 mSv

c        500 mSv

d        1000 mSv

10) True or False. Pre and Post-contrast scans are necessary for routine CT of the abdomen and pelvis for abdominal pain.

a        True

b        False

11) How long before a CT of the abdomen and pelvis should oral contrast be administered to adequately opacify the small bowel and proximal large bowel?

a        Night before

b        60 - 90 minutes prior

c        90 - 120 minutes prior

d        4 - 6 hours prior

e        At the time of the study

12) Which of the following is a recommended indication for a CT scan?

a        Uncomplicated headache

b        Follow up for adnexal cysts

c        Low pretest probability for PE

d        Focal abdominal pain

e        All of the above

f        None of the above

13) Which of the following cutoffs should be used for the administration of IV contrast when calculating GFR?

a        0

b        <30

c        <60

d        <90

14) True or False. IV contrast is necessary to rule out small bowel obstruction.

a        True

b        False

15) Which of the following is not an indication for an emergent pelvic ultrasound?

a        Pelvic pain

b        Postmenopausal bleeding

c        First trimester vaginal bleeding with Beta HCG >2000

d        Pregnant woman in first trimester with falling Beta HCG

16) Which of the following is the most likely cause of bilateral lower extremity swelling?

a        Bilateral DVTs

b        Venous Insufficiency

c        CHF exacerbation

d        Peripheral vascular disease

17) Which study requires at least a 20 gauge peripheral IV access?

a        Head CT

b        CT of the abdomen and pelvis with contrast

c        Pulmonary CTA

d        CT of the chest with contrast

18) Which test does not require IV contrast?

a        Bone mass

b        Liver mass

c        Small bowel obstruction

d        Lymphadenopathy

e        Pulmonary nodule

19) If a patient had an allergic reaction to IV contrast, can you give them oral contrast?

a        Yes

b        No

20) What is your current PGY status?

a        PGY 1

b        PGY 2

c        PGY 3

d        PGY 4

e        Other