Creative Education
2012. Vol.3, Special Issue, 1115-112 1
Published Online October 2012 in SciRes (http://www.SciRP.org/journal/ce) http://dx.doi.org/10.4236/ce.2012.326167
Medical Students Can Help Avoid the Expert Bias in Medicine
Franz Porzsolt1*, Peter Braubach1, Petra Inge Fl urschütz1, Alex Göller1,
Maria Barbara Sailer1, Manfred Weiss2, Peter Wyer3
1Clinical Economics at the Institute of History, Philosophy and Ethics in Medicine,
University of Ulm, Ulm, Germany
2Department of Anesthesiology, University Hospital, Ulm, Germany
3Associate Clinical Professor of Medicine, Columbia University College of Physicians and Surgeons,
New York, USA
[e-mail], [e-mail], [e-mail], [e-mail]
Received August 23rd, 2012; revised September 21st, 2012; accepted October 3rd, 2012
Background: Applying the principles of Evidence Based Health Care (EBHC) in an academic environ-
ment we became aware of important dif ferences betw een medical stud ents and th e users of clinica l rese a r c h.
The latter may be clinicians, educators, guideline developers, or industry managers. These users are
adapted to the system and have some kind of conflict of interest: they are either b ias ed by patients’ demands,
by main stream thinking, by medical standards and/or by economic interests. All are under time pressure,
want to avoid conflicts with their employer and the analysis of scientific reports may not really be their
main focus. Medical students were not exposed to these confounders. Methods: Medical students under
professional supervision completed critical assessments of more than 100 published studies. Their analy-
ses were limited to identification of mistakes, bias and errors using a check list of potential weaknesses in
design and conduct but included the feedback to the academic supervisors. Results: Medical students
trained in Evidence Based Medicine are capable of identifying problems in clinical trials by the system-
atic application of an assessment checklist. Conclusion: In our approach we demonstrate that students can
assist health care professionals and academic teachers with the assessment of clinical evidence. The
premise of the approach is that the final appraisals, which involve consideration of clinical, practical and
value issues, necessarily reside with the academic teachers, writers of guidelines or industry managers
who constitute the active users of research.
Keywords: Evidence Based Health Care; Medical Education; Validity of Scientific Evidence
Introduction
Critical appraisal of scientific literature is an essential step
when applying the principles of evidence-based medicine (EBM)
(Guyatt et al., 1992). EBM may have different meanings to
different people starting at a decision for a single patient and
extending up to the definition of a complete guideline. If the
principles of scientific evidence are scrupulously applied, a
critical appraisal is time consuming. Especially the analysis of
studies published in recognized journals often leads to discus-
sions lasting several hours (Porzsolt et al., 2009).
The time and logistical demands of rigorous critical appraisal
reflects on multiple levels of application. Many, if not most,
protocols used in the development of practice guidelines for
wide spread use restrict the evaluation of evidence supporting
the recommendations to categorization of study designs. Only
recently have systems emerged that take into account also the
quality of individual studies (Schünemann et al., 2006) as
elaborated in the PRISMA statement (Liberati et al., 2009). It
will be difficult to detect bias if the critical review of evidence
in guidelines is restricted to the confirmation of the levels of
evidence (Kopp et al., 2008). Conscientious readers of such
guidelines can tell whether the described validity of a study or
whether the level of evidence merely describes the kind of in-
formation source (e.g., meta-analysis, cohort study, expert
opinion).
The feasibility of practicing physicians performing viable
critical appraisal on their own has been questioned in the litera-
ture (Guyatt et al., 2000). Practitioners aspiring to evidence-
based care may be best advised to rely on publications such as
ACP Journal Club, in which professionally done critical ap-
praisals of leading clinical studies are performed and super-
vised by epidemiologists trained in the principles of EBHC but
not involved in the clinical practice areas relevant to the in-
cluded studies. There is abundant literature on teaching princi-
ples of critical appraisal to undergraduates (Norman et al.,
1998). Although such instruction has been shown to increase
knowledge and skills, it has not tended to reflect in changes in
clinical behavior (Coomarasamy et al., 2004). Survey informa-
tion indicates that medical undergraduates perceive that their
limited ability to actively apply principles of EBHC impedes
acceptance of their importance (Ilic et al., 2010). We perceived
an opportunity to develop a unique framework for application
of medical students critical appraisal and EBHC skills by turn-
ing their relatively undeveloped clinical skills into an asset, at
the same time providing them with an important role to play
within the ‘food chain’ that leads from the production of clini-
cal research to its consumption within the various venues of
clinical policy and practice.
*Corresponding author.

F. PORZSOLT ET AL.
The aim of our study was to propose a method which in-
creases the chance to detect mistakes and biases in scientific
publications. The method should be easy to perform, inexpen-
sive, fast and acceptable to the users of EBM.
Methods
About three years ago we started to invite medical students
who knew already the medical discipline of their future spe-
cialization to complete a medical thesis on the “usability of
scientific publications (USP)”. These students should ask a
mentor of this discipline to suggest a topic which is discussed
controversially. The students were trained in an EBM weekend
course and received a structured questionnaire (Table 1) whic h
they had to use for preparation of their assessments of the se-
lected topic. The completed assessments and the assessed pub-
lications were distributed to the participants of the USP discus-
sion group. In an average session of this group, 2 - 3 papers
were discussed. The average group consisted out of two students
and two supervisors. All members of the group had to read the
paper and to prepare their comments. One of the students had to
present the paper and his critical comments which was dis-
cussed and supplemented by the other three members of the
group. One of the supervisors was a clinical expert in the area
of the discussed paper, the second supervisor was also clinician
with experience in clinical epidemiology. The students were
motivated to participate as they could either complete a doc-
toral thesis (which is in Germany usually completed during the
last years at medical school) or received payment which was
slightly higher than the payment as service in pubs or academic
student tutorship. The duration of a session was usually two
hours. Some of the papers had to be discussed in more than one
session because additional background information was needed.
The two examples presented in this paper (Beckett et al.,
2008; Sprung et al., 2008) were derived from projects of our
students (AG, PIF, MBS) who did critical appraisals on the
Table 1.
Standardized questionnaire used to assess the validity of medical pub-
lications.
1 Study question clearly stated?
2 Design appropriate to answer study question?
3 Can the study design be used in a confirmatory study?
4 Study cond ucted prope rly?
5 Appropriate endpoints selected to answ er study question?
6 Risk profiles of the study populat ions similar?
7 Allocation to study groups concealed?
8 Doctors and patients continuously blinded?
9 Follow-up long enough to detec t endpoints?
10 All patients included in reported results?
11 Adequate statistics applied?
12 Results possibly influenced by conflicting interests?
13 Validity of report acceptable?
14 Described effect clinically relevant?
validity of about 100 publications on lipolysis (AG), on recom-
mendations in intensive care units (PIF) and in trauma surgery
(MBS). The results were used to discuss the validity of the
analyzed studies and finally a consensus statement was created.
This assessment is only the first step of a complete appraisal.
The second step, the appraisal itself, is left to the decision
maker. We identified four conditions that must be fulfilled to
complete a reliable assessment of the validity of a scientific
publication.
 Persons must be available who can perform an appraisal;
 The effort required to train and supervise these persons
should be as little as possible;
 The persons who are to perform the critical appraisal must
be motivated to do it;
 The professional authorities for whom these critical apprai-
sals are compl ete d must not be challenged.
These four prerequisites are met by medical students if they
are offered introductory seminars, regular supervision and ap-
propriate motivation. The seminars focus on the basics of EBM
and practical experience applying its specific assessment methods
to published papers. The papers selected for training contain
frequently seen errors, such as study design which is inappro-
priate to answer the study question, inadequate descriptions of
risk factors, and differences between the number of partici-
pating patients and those reported in the final evaluations.
Most of these errors are not difficult to detect but the detection
is time consuming and time is a permanent problem to any
clinic ian.
The students had to prepare a written version of their as-
sessments using a standardized questionnaire (Table 1). These
assessments were compared, discussed, and a consensus proto-
col was edited during the group meetings.
The assessment starts by identifying the scientific question
investigated in the study. The appropriate study question is
usually extracted from the end of the introduction section. The
students have then to construct an “ideal” and rather simple
study design which could be used to answer the study question.
This “ideal” study design must be compared with the study de-
sign used in the publication analyzed.
The students then assess whether the baseline criteria of the
included study populations are comparable and if: all obvious
risk factors have been accounted for, blinding was maintained,
follow-up was long enough, and all patients who were included
in the study were also included in the calculation of the results.
If questions arise which cannot be solved by the participants of
the discussion groups, additional experts (statisticians, clinical
experts) are consulted. The final report containing a short
summary can be used as a brief and lucid statement on the
method used in the trial.
Results
The questions presented in Table 1 were slightly modified
several times. In Question #3 it should be stated whether or not
the same study design will be in a second study in which the
results of the first study had to be confirmed. Question #4 was
phrased in a rather broad meaning as there were too many pos-
sibilities for inadequate conduct. Question #8 addressed the
point that some of the studies were published as “blinded” al-
though the results strongly suggested that doctors and/or pa-
tients could identify rather fast the allocation. Question #11
turned out to be less important as the use of questionable statis-
1116

F. PORZSOLT ET AL.
tical methods was usually associated with many other flaws. On
the other hand we called a statistician in the rare cases where
we were concerned about the statistics despite in an otherwise
carefully designed and completed study. By answering the last
questions the students should express their impression about a
possible conflict of interest (Question #12) sponsoring, author-
ship and message of the publication, whether or not the study
really demonstrated (Question #13) what it claimed to demon-
strate and whether or not the demonstrated effect was considered
clinically important (Question #14).
The evaluations of the two studies by Sprung et al. (2008)
and Beckett et al. (2008) are shown as table which represents
the final results of our assessment (Table 2). The abstract of the
paper by Sprung et al. (2007) was used as reference in a guide-
line for treatment of patients with severe sepsis and septic
shock (Dellinger et al., 2008). Our second example (Beckett et
al., 2008) was used for teaching purposes and was listed among
significant publications of cardiovascular literature in 2008.
The problems we identified in the assessed studies were re-
lated to the inclusion of patients who met the exclusion criteria
as described in the methods section of the publication (Sprung
et al., 2008), to imbalanced risk profiles of the included patients
and to inclusion of fewer patients than requested according to
the power calculation of the study. In the second example
(Beckett et al., 2008) the study question was not clearly stated,
the study design was not appropriate, there were too many con-
founders, the study was not conducted properly, the risk pro-
files in the compared groups were not well balanced neither-
were all patients included in the reported results (Table 2).
The assessment of the study by Beckett et al. (2008) was supple-
mented by a detailed description of the study method and study
conduct.
 It was not clear to the assessors how many patients were
recruited for the run-in-phase;
 The criteria for stratification into the 4 study groups
changed during the study. Between 2000 and 2003 a dia-
stolic blood pressure of 90 - 109 mmHg was required. After
2003 a diastolic blood pressure of <110 and systolic blood
pressure (standing) of >139 was introduced into the study
protocol;
 The investigator could administer a second treatment to pa-
tients that did not reach an acceptable blood pressure with
only the study medication. This intervention was again placebo
controlled.
 Patients were withdrawn from the study if the additional
treatment failed.
 Data from more than 300 patients was censored form the
study some because the local investigator died or retired.
 Some data was analyzed according to the intention to treat
principle while other was analyzed per protocol. The assessors
did not find a proper reasoning for this kind of data analysis.
The assessment was concluded with a statement that the
Table 2.
Results of the assessment of two studie s10,11 as presented in our project.
# Sprung et al. 11 Becke tt et al.10
1 Study question clearly stated? Yes. (…), evaluated the efficacy and safety of low-dose hydrocortisone thera py in a broad popula-
tion of patients with septic shock (…). Unclear, see page 1888/para 3 left/lines 8 - 12
2 Design appropriate to answer study
question? Yes No. See Appendix 1
3 Can the study design be used in a
confirm atory study? Yes No. Too many confounders
4 Study conducted properly? No. Patients with exclusion c rit eria were included.
Study was underpowered. No. See Appendix 1
5 Appropriate endpoints selected to answ er
study question? Yes Yes
6 Risk profiles of the study populations
similar? No. In the placebo arm there were more patients with severe diseases. No. Risk marginally higher in six risk indicators in the placebo group.
7 Allocation to study groups concealed? Yes Unclear
8 Doctors an d p atients continuously
blinded? Yes No. See protocol
9 Follow-up long enou gh to detec t
endpoints? Yes. 28 days follow-up is acceptable in ICU studies.Yes
10 All patients included in reported results? Yes No. See Figure 1, page 1891.
11 Adequate statistics applied? Yes Yes
12 Results influenced by conflicting inter-
ests? No Possibly
13 Validity of report acceptable? No. Study i ncluded le ss th an the recommende d
number of 8 00 p atients. No. Too many confounders
14 Described effect clinically relevant? No. Due to lack of validity No. Due to lack of validity

F. PORZSOLT ET AL.
complex study protocol may have given way to many possible
confounders. Overall there are many problems concerning the
validity of the study severely limiting the possibility to derive
meaningful conclusions for clinical practice.
Discussion
It is generally recommended to check carefully the quality of
scientific information especially if used not only for individual
patient decisions but rather to support students’ teaching, the
writing of guidelines or the approval of new drugs (Ford et al.,
2010). Both of the papers analyzed here were critically re-
viewed in the ACP journal club and were at the end accepted by
clinical commentators as relevant to clinical decision making
(Hung et al., 2008; Lamontagne et al., 2008). Our reviews may
appear to have come to different conclusions. None of these
two papers were considered to be devoid of potentially impor-
tant flaws. It has to be considered that there are four essential
differences between the ACP reviews and our reviews.
 We address specified validity criteria and intent to be as
critical as possible to avoid mistakes, bias, and errors but
not providing an assessment such as those provided by the
clinical commentators of ACP Journal Club synopses.
 The appraisal is left to the persons who are responsible for
the use of the published information. The ACP reviews
provide assessments which are accompanied by an appraisal
and this appraisal can directly be used by the reader of the
ACP review.
 Our assessments are usually not prepared for publication.
The publication of the two above assessments is necessary
to provide examples. If this rigorous approach is published
it will create a serious problem because economic competi-
tors, i.e. other pharmaceutical companies may use the crude
information for marketing purposes. The emerging ethical
conflict in the cooperation of researchers from industry and
academy was recently addressed in a comment (Fangerau,
2009).
 Finally, our assessments include two feed-back modules
which are essential in teaching (Hattie, 2009). One is the
didactic feed-back. The second is an epistemological feed-
back between students and supervisors—and this is some-
time rather time-consuming—because discussions about
truth, believe, justification based on different degrees of
skepticism cannot be avoided. This complete circle of
feed-back is a basic requirement for “reframing” i.e. for
generating consensus in a situation where the discussants do
not agree.
Of the two papers illustrated in the example, one of them was
used as a reference in a guideline (Sprung et al., 2008), and the
other paper was used for teaching purposes and was considered
the leading article in a list of significant publications in the
cardiovascular literature in 2009 (Becket et al., 2008).
Our results indicate that medical students can be quickly
trained in the required attitudes, skills, and knowledge of EBM
to complete valuable critical appraisals (Finkel et al., 2003).
Students can be motivated to do this if they can generate inco mes
comparable to other student jobs. Finally, acrimony among
professional authorities can be avoided if the process of critical
appraisal is divided into two parts. The students exclusively
perform assessment and present their completed assessments
with their evaluative statement only to the client who requested
the assessment. The appraisal, including acceptance of the final
result of the entire process, has to be made by the clients, i.e.
professional academic teacher, the professional researcher, or
the writer of guidelines. The final decisions are left to the
health-care professionals. Students lack clinical and research
experience as well as measures of appropriateness. In addition,
experts and students have different expectations in the validity
of scientific information which in fact is a matter of judgment
(Rawlins, 2008; Schulz et al., 2010).
We are convinced that most of the 40 available instruments
for critical appraisals (Darmoni et al., 2001; Forestier et al.,
2005) can be used for such a project. The result of the project
will probably depend on the depth of the critical discussion.
It is essential that the students who use the questionnaire for
validity assessment are aware of the most frequent mistakes and
biases and know the questions that have to be asked for a pro-
found assessment of the validity. In Table 3 we demonstrate
that our questionnaire and the instrument recommended by CA S P,
the Birmingham Critical Appraisal Skills Programme (CASPb,
2012) is similar, except that the CASP instrument includes a
question about randomization and the assessment of the study
results. The CASP instrument places less emphasis on the
details of the study design, on the follow-up time, and on a
possible conflict of interest. Our instrument neither addresses
the question of whether the study results would apply to the
local population nor investigates the precision of the data. Our
results indicate that even the concise version of a questionnaire
is sufficient to detect mistakes and bias if its users are trained
to recognize the problems that may be hidden in a study with
brilliant results.
Another instrument for a systematic appraisal is the GRADE
system (Guyatt et al., 2008) recommended for the preparation
of clinical guidelines. Although different versions of GRADE
are not absolutely identical (Table 4) they cover almost the
same questions as our instrument and the CASP instrument as
far as these questions are related to RCTs.
We believe that students can make significant contributions
at different stages in the generation and application of scientific
evidence (Figure 1) and may therefore have a direct effect on
the experts’ judgment which is an essential ingredient of deci-
sion making (Rawlins, 2008).
This method may also be useful as a teaching tool for EBM
as requested by Murad et al. (2009). The detailed discussion of
the study methodology mediates the understanding of EBM
(Plint et al., 2006) without giving students the impression of
attending a theoretical seminar without significance to their
future job. The students’ appraisal cannot have the function of a
final scientific appraisal but may induce in the client considera-
tions about the acceptability of the evidence. In addition, stu-
dents who are not native English speakers are supported to
overcome their well known language problems (Letelier et al.,
2007). This expected benefit of our approach could meanwhile
be confirmed in cooperation with an Italian Group (Rosati et al.,
2009; Rosati et al., 2012).
In summary, the students provide a critical appraisal of in-
formation which has been accepted by experts and is published.
The information is provided within a short time at low expense.
It is checked by an academic supervisor, but leaves the ap-
praisal of the evidence and, therefore, the final decision about
acceptance or rejection of the evidence to the person who re-
quested the assessment. Finally, the students and the institution
which organized their supervision will benefit from this pro-
gram, as the students acquire valuable experience they can use
1118

F. PORZSOLT ET AL.
Table 3.
Comparison of the CA SP que st ion nai re and the questionnaire used in this proje ct.
# CASP questionnaire #Our questi onnaire
A1
Did the trial address a clearly-focused issue?
An issue can be “focused” in terms of
- the population studied
- the interv ention given
- the outcomes considered
1
2Study question clearly stated? Design appropriate to answer study question?
A2 Was the assign ment of patients to treatments randomized?
A3 Were all of t h e patients who entered the trial properly accounted f or at its co n clusion?10All patients included in reported results?
7Allocation to study groups concealed?
B4
Were patients, health workers, and study personnel “ b lind” to treatment?
- were the patients
- were the health workers
- were the study per sonnel
8Doctors an d p atients continuously blinded?
B5 Were the groups similar at the start of the trial ? In terms of o ther facto r s that might effect the outcome, such as age, sex,
social class 6Risk profiles of the study popula tions similar?
B6 Aside from the experimental intervention, were the groups treated equally? 2
4Design appropriate to answer study question? Study conducted properly?
B7 How large was the treatment effect?
What outcomes were measured? 11Described effect clinically relevant?
B8 How precise was the estimate of the treatment effect?
What are its conf idence intervals limits?
C9 Can the re s u lt s b e applie d t o the loca l po pulation? Do you think that the patients covered by the trial are similar enough to
your population? 3Can the study design be used in a confirmatory study?
C10 Were all clinically important ou tcomes considered?
If not, does this affect the decision? 5Appropriate endpo ints selected to answer study question?
C11 Do the benefits outw eigh the harms and costs?
This was probably not addresse d by the trial, but what do you think? 11Described effect clinica lly relevant?
Table 4.
Quality criteria incorporated into GRADE system.
Randomize d Trials (ref 26) Allocation concealment
Lack of blinding, including outcome adjudicators-more important when outcome assessment potentially
subjective and subject to bias
Lack of intention to treat analysis
Stopping early for benefit
Failure to repo rt important outcomes
Randomize d Trials (ref 25) Use of unvalidated pa tient reported outcomes
Carry over effects in cross over trials
Recruitment bias in cluster randomized trials
Observational Studies (ref 26) Removal of all biases would increase effect size
Observational Studies (ref 25) Under or over-matching of case-control studies
Selection of exposed and unexposed in cohor t studies from different populations
Differences in measurement of exposure (e.g. re call bias in case-con t r o l studies)
Differential surveil lance f o r o u t come in exposed and unexposed in cohor t s tudies
Failure of accurate measurement of all known prognostic factors
Failure to match for prognostic factors and/or adjustment in st atistical ana lysis
General Depending on the context and study type, there can be additional limitations than those list ed above .
Guideline panels an d authors of systematic reviews s hould consider all possible limitations.

F. PORZSOLT ET AL.
Acad.
teaching
and CME
Real world
outcomes
Clinical
guidelines
Health care
expenditure
Clinical
practice
Approval of
innovation
Clinical
research
Publication
of the results
Possible students’ contributions on the way from clinical research to real world outcomes.
little influence
considerable influence
indirect influence
Figure 1.
Possible students’ contributions on the way from clinical research to
real world outcomes. little influence, considerable influence,
indirect influence.
in their later professional life. Institutions that can continuously
guarantee a high-quality output will contribute to an urgently
need change: we will have to become be much more critical in
what we accept if our primary goal is to maintain the best pos-
sible but also affordable health care system.
REFERENCES
Beckett, N. S., Peters, R., Fletcher, A. E. et al. (2008). Treatment of
hypertension in patients 80 years of age or older. Journal of Medi-
cine, 358, 1887-1898. doi:10.1056/NEJMoa0801369
CASPb Working Group (2012) Crib sheet for an RTC: 11 questions to
help you make sense of a trial. URL (last checked 29 January 2012).
http://medweb4.bham.ac.uk/websites/caspb/cribsheets
Coomarasamy, A., & Khan, K. S. (2004). What is the evidence that
postgraduate teaching in evidence based medicine changes anything?
A systematic review. British Medical Journal, 329, 1017.
doi:10.1136/bmj.329.7473.1017
Darmoni, S. J., Haugh, M. C., Lukacs, B., & Boissel, J. P. (2001).
Quality of health information about depression on internet. Level of
evidence should be gold standard. Brit i s h Medical Journal, 322, 1367.
Dellinger, R. P., Levy, M. M., Carlet, J. M. et al. (2008). Surviving
sepsis campaign: International guidelines for management of severe
sepsis and septic shock: 2008. Intensive C ar e M ed ic in e, 3 4, 17-60.
doi:10.1007/s00134-007-0934-2
Fangerau, H. (2009). Research ethics, publication ethics and the dialec-
tics of scientists trying not to behave badly: A comment on the ad-
vantages and limitations of twin assessment of clinical trials (TACT).
Journal of Public Health, 17, 433-434.
doi:10.1007/s10389-009-0284-3
Finkel, M. L., Brown, H. A., Gerber, L. M., & Supino, P. G. (2003).
Teaching evidence-based medicine to medical students. Medical
Teacher, 25, 202-204. doi:10.1080/0142159031000092634
Ford, A. C., Guyatt, G. H., Talley, N. J., & Moayyedi, P. (2010). Errors
in the conduct of systematic reviews of pharmacological interventions
for irritable bowel syndrome. American Journal of Gastroenterol, 105,
280-288. doi:10.1038/ajg.2009.658
Forestier, R., Françon, A., & Graber-Duvernay, B. (2005). Les pa-
ramètres de validité d'un essai thérapeutique et leur influence sur l'é-
laboration d'une médecine fondée sur les preuves: revue de la littéra-
ture. Annales de Réadaptation et de Médecin e Physique, 48, 250-258.
doi:10.1016/j.annrmp.2005.02.004
GRADEpro Software Package (2012). URL (last checked 29 January
2012). http://ims.cochrane.org/revman.
Guyatt, G., Cairns, J., Churchill, D. et al. (1992). Evidence-based medi-
cine: A new approach to teaching the practice of medicine. Journal of
the American Medical Association, 268, 2420-2425.
doi:10.1001/jama.1992.03490170092032
Guyatt, G., Meade, M., Jaeschke, R., Cook, D., & Haynes R. B. (2000).
Practitioners of evidence based care. British Medical Journal, 320,
954-955. doi:10.1136/bmj.320.7240.954
Guyatt, G. H., Ox man, A. D., Kunz, R., Vist , G. E., Falck-Ytter, Y., &
Schunemann, H. J . (2008). What is “quality of evidence” and why is it
important to clinicians? British Medical Journal, 336, 995-998.
doi:10.1136/bmj.39490.551019.BE
Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses
relating to achievement. London: Routledge.
Hung, W. W., & Leipzig, R. M. (2008). Antihypertensive therapy with
indapamide and perindopril reduced mortality in patients greater than
or equal to 80 ye a r s. ACP Journal Club, 149, 10.
Ilic, D., & Forbes, K. (2010). Undergraduate medical student percep-
tions and use of Evidence Based Medicine: A qualitative study. BMC,
Medical Education, 10, 58. doi:10.1186/1472-6920-10-58
Kopp, I., Thole, H. T. L., Selbmann, H., & Ollenschläger, G. (2008).
Deutsches Instrument zur methodischen Leitlinien-Bewertung (DELBI):
Fassung 2005/2006. ZaeFQ, 99, 468-519.
Lamontagne, F., & Meade, M. O. (2008). Low-dose hydrocortisone did
not improve survival in patients with septic shock but reversed shock
earlier. ACP Journal Club, 148, 6.
Letelier, L. M., Zamarin, N., Andrade, M. et al. (2007). Exploring
language barriers to Evidence-based Health Care (EBHC) in post-
graduate medical students: A randomised trial. Education Health, 20,
82.
Liberati, A., Altman, D., Tetzlaff, J. et al. (2009). The PRISMA state-
ment for reporting systematic reviews and meta-analyses of studies
that evaluate healthcare interventions: Explanation and elaboration.
British Medical Journal, 339, b2700. doi:10.1136/bmj.b2700.
Murad, M., Montori, V., Kunz, R. et al. (2009). How to teach evi-
dence-based medicine to teachers: Reflections from a workshop ex-
perience. Journal of Evaluation Clini c a l Practice, 15, 1205-1207.
doi:10.1111/j.1365-2753.2009.01344.x
Norman, G. R., & Shannon, S. I. (1998). Effectiveness of instruction in
critical appraisal (evidence-based medicine) skills: A critical appraisal.
Canadian Medical Association Journal, 158, 177.
Plint, A. C., Moher, D., Morrison, A. et al. (2006). Does the CON-
SORT checklist improve the quality of reports of randomised con-
trolled trials? A systematic review. The Medical Journal of Australia,
185, 263.
Porzsolt, F., Bonotto, de O., Costa, I. C., & Thomaz, T. G. (2009).
Advantages and limitations of Twin Assessement of Clinical Trials
(TACT). Journal of Public Health, 17, 425-435.
doi:10.1007/s10389-009-0283-4
Rawlins, M. (2008). De testimonio: On the e vi de nc e f or decisions about
the use of therapeutic interventions. Clinical Medicine, Journal of the
Royal College of Physicians, 8, 579-588.
Rosati, P., Ciampalini, P., Grossi, A., Giovannelli, L., Giustini, F.,
Inglese, R., Fiscarelli, E., Castellano, C., Mazziotta, M. R. M., Gentile,
S., Giampaolo, R., & Porzsolt, F. (2009). An alternate evaluation of
evidence: Results from a CASP workshop. In P. J. Sharek, D. Berg-
man, & F. M. Ducharme (Eds.), Beclomethasone for asthma in chil-
dren: Effects on linear growth (pp. 21-22). London: Cochrane Air-
ways Group. doi:10.1002/14651858.CD001282
Rosati, P., & Porzsolt, F. (2012). A practical educational tool for teach-
ing child-care hospital professionals attending evidence-based prac-
tice courses for continuing medical education to appraise internal va-
lidity in systematic reviews. Journal of Evaluation Clinical Practice.
Schünemann, H. J., Fretheim, A., & Oxman, A. D. (2006). Improving
the use of research evidence in guideline development: 9 grading evi-
dence and recommendations. Health Research Policy and Systems, 4,
21. doi:10.1186/1478-4505-4-21
Schulz, K., Altman, D., & Moher, D. (2010). CONSORT 2010 state-
1120

F. PORZSOLT ET AL.
ment: Updated guidelines for reporting parallel group randomised tri-
als. BMC Medicine, 8, 18. doi:10.1186/1741-7015-8-18
Sprung, C. L., Annane, D., & Briegel, J. (2007). Corticosteroid therapy
of septic shock (CORTICUS). Abstract American Journal of Respi-
ratory and Critical Care M ed i c i n e , 175, A507.
Sprung, C. L., Annane, D., Keh, D. et al. (2008). Hydrocortisone ther-
apy for patients with septic shock. The New England Journal of
Medicine, 358, 111-124. doi:10.1056/NEJMoa071366