BRIEF REPORT

Emergency Radiology “Boot Camp”:

Educating Emergency Medicine Residents
Using E-learning Radiology Modules
Shlomo Minkowitz, MD, Kristen Leeman, MD, Ashley E. Giambrone, PhD, MS,
Jennifer F. Kherani, MD, Lily M. Belfi, MD, and Roger J. Bartolotta, MD

ABSTRACT
Objectives: There is an overall paucity of literature on the radiologic education of emergency medicine (EM)
clinicians. Given the fact that many EM clinicians preliminarily review images for their patients, we hypothesized
that a brief imaging curriculum could be efficacious in teaching basic and relevant radiologic interpretation.

Methods: We designed a 4-hour “radiology boot camp” for a group of 20 EM residents (from all years of
training) covering several subject specific e-learning modules. They completed precourse and postcourse quizzes
to evaluate the efficacy of these modules. These modules included interactive PowerPoint-based tutorials, games,
and imaging decision support simulators. Matched results from the pre- and posttests were analyzed using
paired t test. An additional questionnaire was administered to the EM residents to evaluate their perception of the
educational experience.

Results: The precourse and postcourse quizzes demonstrated a statistically significant level of improved
knowledge due to the educational modules (p < 0.0001). In addition, all of the participants believed the modules
were a useful learning experience (100%) and a vast majority described them as a valuable resource for future
reference (95%).

Conclusion: We demonstrate a model for providing an easy and effective means of educating EM residents on
basic imaging interpretation and utilization, using e-learning modules.

I maging plays a crucial role in the clinical work of

the emergency medicine (EM) physician. Patients
who visit the emergency department (ED) will often
radiologists may not be available at all times in all
institutions, and immediate interpretation by a
radiologist may not be readily available.2 Given
undergo some form of diagnostic imaging, ranging the acuity of many clinical scenarios in the ED, it
from chest radiography to brain magnetic resonance often benefits the EM physician to be able to
imaging to ultrasound of the lower extremities. For interpret basic imaging studies (and/or key urgent
example, a recent study demonstrated that com- findings on more complex imaging studies) to
puted tomography (CT) scans are ordered in approx- guide urgent clinical decisions.
imately 16.7% of adult patients visiting the ED.1 Our review of the literature shows that EM
In most hospital settings, a board-certified diag- physicians may have difficulty with certain radio-
nostic radiologist is relied upon for the formal logic interpretations, such as chest x-ray and head
interpretation of these imaging studies. However, CT, as discussed below under Discussion. At

From the Department of Radiology, New York-Presbyterian Hospital-Weill Cornell Medicine (SM, KL, LMB, RJB), the Department of Emergency
Medicine, New York-Presbyterian Hospital-Weill Cornell Medicine (JFK), and the Department of Biostatistics and Epidemiology, Weill Cornell Medi-
cine (AEG), Brooklyn, NY.
Received August 2, 2016; revision received September 20, 2016; accepted September 23, 2016.
The authors have no relevant financial information or potential conflicts to disclose.
Supervising Editor: Teresa Chan, MD, MHPE.
Address for correspondence and reprints: Shlomo Minkowitz, MD; e-mail: shm2024@[Link]
AEM EDUCATION AND TRAINING 2017;1:43–47.

© 2017 by the Society for Academic Emergency Medicine

doi: 10.1002/aet2.10002 ISSN 2472-5390 43
44 Minkowitz et al. • EMERGENCY RADIOLOGY “BOOT CAMP”

present, EM residents in our institution receive no RadGame on basic chest radiographic findings and a
formal radiology training within their curriculum, RadTorial focusing on the radiographic appearance
and there is an overall paucity of literature on the of tubes and lines. Each group of EM residents
topic of radiologists providing education to EM rotated through all three of the module sets and
physicians. We hypothesize that a dedicated radio- completed precourse and postcourse examinations
logy curriculum for the EM residents at our institu- (as described below).
tion would improve their diagnostic skills when Each module set was facilitated by a radiologist,
comparing precourse and postcourse assessments. with a ratio of approximately five to seven EM res-
We developed educational modules to instruct EM idents per one radiologist. During each module set,
residents to make basic and relevant clinical find- the EM residents worked independently, with the
ings in the acute setting, as well as to provide a radiologist answering questions and offering teach-
resource for them to utilize going forward. ing points throughout the session.

Data Collection
METHODS
After institutional review board approval, the EM
residents completed precourse and postcourse
Course Description
knowledge assessments as well as a perception survey
As a platform for our educational modules used in this of their educational experience. The knowledge
course and other projects, faculty and residents from assessment of 15 multiple-choice questions was
our Department of Radiology participated in the cre- administered immediately prior to and subsequent
ation of a variety of e-learning modules in the form of to the course. The assessment was created by our
RadTorials, RadGames, and an interactive imaging radiology faculty specifically for the course and
decision support simulator (“ICARUS”; Interactive included material from all elements of the course.
Clinical Anatomy and Radiology Utilization Simula- The test consisted of five questions in relation to
tor). These resources are available at [Link]-rad. each of the three modules. For example, one of the
com, and the various modules, described in detail else- neuroradiology questions asked: “What is the key
where,3 will be briefly explained below. CT finding that differentiates chronic infarction
The RadTorial modules review imaging evalua- from acute infarction?” (answer choices: vasogenic
tion of various clinical conditions or patient popu- edema, hemorrhage, density, or volume loss). Addi-
lations. The RadGames modules are interactive tional sample questions are included in Appendix 1.
learning modules in question/answer format (Game The residents used anonymous numeric identifiers
of Unknowns, Ace the Case), each with a specific allowing us to track their pretest and posttest
subspecialty focus. The Interactive Clinical Anat- responses. Additionally, after the session, the resi-
omy and Radiology Utilization Simulator dents responded anonymously to three perception
(ICARUS) is a program that uses a simulation questions asking whether they found the modules to
platform as a means of teaching basic radiologic be useful, whether they would recommend them to
anatomy, imaging appropriateness, and basic imag- other EM residents, and whether they would refer to
ing review. There are approximately 40 fully devel- them in the future as a resource. For these three
oped modules at the time of writing this article. questions, they were given the option of choosing:
The current course was designed as a one-time “strongly agree,” “agree,” “neutral,” “disagree,” or
course that was 4 hours in length. The course partic- “strongly disagree.”
ipants consisted of 20 EM residents who were cho-
sen randomly and equally spread between all years Data Analysis
of residency. They were split into smaller groups of 5 Continuous variables are represented as mean  stan-
to 7 that rotated to each module station. We chose dard deviation and categorical variables as number
three sets of modules for our course: module set 1 (percentage). Precourse and postcourse knowledge
consisted of RadTorials on stroke and cervical spine was analyzed by comparing the mean test scores via
injury; module set 2 included a RadTorial on female paired t test. All p-values are two sided with signifi-
pelvic ultrasound and an ICARUS module on small cance evaluated at the 0.05 alpha level. Data were
bowel obstruction; and module set 3 consisted of a analyzed with SAS, version 9.3 (SAS Institute).
AEM EDUCATION AND TRAINING • January 2017, Vol. 1, No. 1 • [Link] 45

RESULTS may have difficulty with radiologic interpretations.

For example, in one study examining performance-
With respect to the precourse and postcourse exami-
based clinical skill assessment of EM residents, the
nations administered to the EM residents, our results
EM residents were scored on their ability to do
demonstrated a statistically significant level of
multiple clinical and diagnostic tasks. The diagnos-
improved knowledge between the pre- and posttests.
tic tasks included the interpretation of chest radio-
The mean scores of the precourse and postcourse
graphs, which are one of the most commonly
examinations were 38 and 71%, respectively, with a
performed radiologic examinations in the ED set-
paired t test p-value of <0.001. In addition, with
ting. Of all the clinical and diagnostic tasks that
respect to the perception questions, the vast major-
they were required to perform, they scored the
ity of participants believed that the modules were
lowest on chest x-ray interpretation.4
not only a useful learning experience (100%) but
Additionally, there have been several studies
also a valuable resource for future reference (95%;
that evaluate the accuracy of imaging interpreta-
Figure 1). The three radiologists who served as on-
tion by EM physicians vis-a-vis that of the diag-
site facilitators for these sessions found the sessions
nostic radiologist, many of which demonstrate that
engaging as well and appreciated the opportunity to
EM physicians have decreased accuracy in inter-
interact with residents from another department and
pretation of radiologic studies as compared with
hear their perspectives. All involved radiologists
diagnostic radiologists. For example, one study
indicated that they would happily participate in a
demonstrated that EM physicians interpreted head
similar educational experience in the future.
CTs with a significant number of clinically impor-
tant false-positive and false-negative interpreta-
tions (sensitivity of 88% and specificity of 80%)
DISCUSSION
compared with neuroradiologists.5 Similarly,
The current project demonstrates an innovative another investigation concluded that a significant
curriculum developed by our radiology department number of findings were missed on chest radiogra-
that was implemented in a 4-hour course for the phy when interpreted by the EM physician (with a
EM residents at our institution, for whom radiology reported sensitivity of 20%–65%) compared with
education is not currently a part of their curricu- the diagnostic radiologist.6
lum. The efficacy of the course was then demon- Our hypothesis therefore was that implementa-
strated using precourse and postcourse assessment. tion of a radiology curriculum may be efficacious,
Of additional secondary benefit was the collabora- and we have indeed found that our modules pro-
tion between the residents of the radiology and vided an effective way of teaching radiology to the
EM departments. Both radiology residents and EM EM residents. When searching to see whether any-
residents indicated that they found the sessions thing similar had been done in the past, we found
favorable and would like to be involved in some- an overall paucity of literature on the topic of radi-
thing similar in the future. ologists providing education to nonradiology clini-
Our review of the literature demonstrates that cians of any specialty, including EM. The
there are indeed areas in which EM physicians literature does describe EM departments that

Figure 1. Perception questions.

46 Minkowitz et al. • EMERGENCY RADIOLOGY “BOOT CAMP”

provide their own radiology education to their trai- online educational tool ([Link]) as
nees, with one publication describing a course an effective way to educate nonradiologists (such
designed to educate EM residents on assessment of as emergency medicine residents) through various
critical findings on head CTs.7 That study demon- online modules and educational tools.
strated that EM residents had some deficiencies in At the present, emergency radiology education is
their interpretation of head CTs (prior to interven- not a part of the formal emergency medicine resi-
tion) but that an educational course was beneficial. dent curriculum in our institution. We created an
Similar paradigms do exist in the education of effective model for presenting educational radiology
medical students, for whom radiologists (and other modules to nonradiology residents, which could be
physicians) have thoroughly evaluated how we pro- expanded to meet the needs of other departments
vide effective education. For example, a recently residents as well as other clinical scenarios.
published study described a unique educational
method for educating medical students, where stu-
References
dent preclass review of material was combined
with dedicated class time for interactive knowledge 1. Kirsch TD, Hsieh YH, Horana L, Holtzclaw SG, Silver-
application.3 man M, Chanmugam A. Computed tomography scan
utilization in emergency departments: a multi-state
analysis. J Emerg Med 2011;41:302–9.
LIMITATIONS 2. DeFlorio R, Coughlin B, Coughlin R, Li H, Santoro J,
Akey B, Favreau M. Process modification and emer-
Our study has several limitations. Our sample size
gency department radiology service. Emerg Radiol
is small and only included EM residents, due to
2008;15:405–12.
the ease of convening such a group together, and 3. Belfi LM, Bartolotta RJ, Giambrone AE, Davi C, Min
because from an education viewpoint, they are RJ. “Flipping” the introductory clerkship in radiology:
more of a controlled group relative to attendings. impact on medical student performance and percep-
A more important limitation is that we evaluated tions. Acad Radiol 2015;22:794–801.
only immediate postcourse knowledge and did not 4. Burdick WP, Ben-David MF, Swisher L, et al. Reliabil-
assess long-term effect on diagnostic interpretation ity of performance-based clinical skill assessment of
by the EM residents, nor long-term change in clin- emergency medicine residents. Acad Emerg Med
ical management. Furthermore, our postcourse 1996;3:1119–23.
assessment was not piloted beforehand and was 5. Boyle A, Staniciu D, Lewis S, et al. Can middle grade
composed of a multiple-choice quiz, which and consultant emergency physicians accurately inter-
although provides a standardized way of assessment pret computed tomography scans performed for head
trauma? Cross-sectional study. Emerg Med J
does not simulate the challenges of diagnostic
2009;26:583–5.
interpretation as accurately as would an assessment
6. Gatt ME, Spectre G, Paltiel O, Hiller N, Stalnikowicz
of their actual interpretive skills. A follow-up study R. Chest radiographs in the emergency department: is
assessing whether this cohort of EM residents the radiologist really necessary? Postgrad Med J
demonstrate improved diagnostic skills would be 2003;79:214–7.
helpful to evaluate the longitudinal effect of our 7. Perron AD, Huff JS, Ullrich CG, Heafner MD, Kline
intervention. JA. A multicenter study to improve emergency medi-
cine residents’ recognition of intracranial emergencies
on computed tomography. Ann Emerg Med 1998;
CONCLUSION 32:554–62.
In conclusion, our 4-hour “boot camp” was well
received by the emergency medicine residents and Appendix 1
demonstrated that a single interactive educational
session could have significant positive impact on ID number: ______________________________
emergency medicine residents’ knowledge of emer- 1. A 36 year-old female with right upper quadrant
gency radiology. Additionally, the results of our abdominal pain and fever presents to ED. Her
analysis provide evidence for the validity of our WBC count is elevated. You suspect acute
AEM EDUCATION AND TRAINING • January 2017, Vol. 1, No. 1 • [Link] 47

cholecystitis. Which of the following is the best 4. Which of the following ovarian abnormalities
initial imaging test? can demonstrate the string of pearls sign (en-
a. HIDA scan larged ovary with peripheralized follicles)?
b. CT scan of the abdomen a. Corpus luteum
c. Right upper quadrant ultrasound b. Hemorrhagic cyst
d. Upper GI series c. Ovarian torsion
2. Which of the following is NOT a relative con- d. Endometrioma
traindication to intravenous contrast for CT? e. Dermoid
a. Asthma 5. Which of the following is the most reliable dif-
b. Multiple myeloma ferentiator between retained products of con-
c. Shellfish allergy ception and gestational trophoblastic disease?
d. Sickle cell disease a. Beta-HCG levels
e. Collagen vascular disease b. Color Doppler flow
3. Which of the following values is closest to the c. Echogenicity
effective dose of radiation for CT abdomen/pel- d. Free fluid
vis? e. Internal septations
a. 1 mSv
b. 10 mSv
c. 20 mSv
d. 30 mSv
e. 50 mSv