Levels of Evidence – How to Rate the Quality of a Study
USPSTF Methodology to Evaluate Study Quality
Professional bodies assess the quality of relevant studies when developing standards and guidelines. ACOG, for example, uses the following methodology (USPSTF 2001, see ‘Learn More-Primary Sources’) and one can find the specific level of evidence after each reference.
I Evidence obtained from at least one properly designed randomized controlled trial
II–1 Evidence obtained from well–designed controlled trials without randomization
II–2 Evidence obtained from well–designed cohort or case–control analytic studies, preferably from more than one center or research group
II–3 Evidence obtained from multiple time series with or without the intervention. Dramatic results in uncontrolled experiments also could be regarded as this type of evidence
III Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees
Using the highest level of evidence following data review, ACOG will provided graded recommendations Level A Recommendations are based on good and consistent scientific evidence Level B Recommendations are based on limited or inconsistent scientific evidence Level C Recommendations are based primarily on consensus and expert opinion
Assessing the Quality of a Study
The concept of ‘levels of evidence’ follows the NAM standard that quality of evidence should be an integral part of medical guideline development. Note that the NAM does not prescribe a single rating system. Different professional bodies may use different scoring systems. What is important is that ‘levels of evidence’ be readily apparent to the guideline reader. The same approach can be used when reviewing a paper.
Assess the quality of a study by asking 3 basic questions:
Is the study Interventional: do the investigators ‘intervene’ in the management of the study subject? Or does nature take its course?
If yes to the above, is the study randomized?
Did the authors include a section describing how they determined that they included enough study subjects to draw conclusions?
Level I Evidence – interventional study – randomized
Randomized control trial (RCT) – study subjects are placed into an experimental versus a control group in a random fashion, beyond the control of the investigators
Even unknown or unanticipated effects will hopefully balance out between the two groups
The ‘non-exposed’ group can receive either a placebo or alternate drug, test, or management plan
Level II-1 Evidence – interventional study – not randomized
Allocation to the experimental versus control group is left to the investigators and therefore bias is more likely than Level I
Level II-2 Evidence – observational study – no intervention by the researchers
(a) Cohort study – these are studies where the starting point is a particular exposure
Study subjects are classified as ‘exposed’ or ‘not exposed’ and followed for a period of time
Timing can be prospective (following study subjects for a prescribed amount of time) or retrospective – looking back through charts
The rate of outcome can be compared between the exposed and unexposed group and relative risks can be determined
(b) Case control studies – these are ‘retrospective’ studies – which means that the events or interventions being studied have already taken place in the past and the outcome of interest drives the design, rather than the exposure (see cohort study, above)
A group who had the outcome of interest (for example cesarean section) is matched to a group that did not
The intervention of interest is then assessed in both groups (to continue the example – exposure to an epidural during labor)
The odds of undergoing a cesarean section following exposure to an epidural can then be compared to the odds of vaginal delivery following exposure to an epidural
These studies can determine association but not causation
Level II-3 Evidence – a ‘snapshot’ in time
Cross sectional studies – At any given time, what is the exposure and outcome?
Useful if looking at prevalence of a disorder as well as potential variables
For example, one could look at prevalence of uterine cancer in a particular population, based on age or exposure to hormonal therapy
These studies will often assess other factors (e.g., socioeconomic level) to make sure the variable of interest is truly an independent factor
Level III Evidence – other studies with essentially minimal design, ‘descriptive’ and lowest level of evidence
Case study – description of clinical features related to a particular topic of interest
Expert opinion – individual expert opinion was previously used frequently for recommendations but now used in the absence of better study design
Used in the context of expert panels and committees and may be important when data is lacking or research is ongoing
“Level of evidence” is a standardized way to determine the quality of a research project, which is based on study design. Clinical trials are ‘interventional’, if researchers intervene and are the highest level of evidence, followed by lower levels of evidence where observational approaches are used and ‘nature is allowed to take its course’. Ultimately, the goal of any such rating system is to help clinicians provide evidenced-based care to patients and aid in public health policy.
Certain topics of interest are not always amenable to RCTs and may require other approaches.
In the case of rare outcomes, case-control studies may be more practical
Because an RCT has not been done does not mean that something is not true or valid
An intervention that has been proven not to work via an RCT is not the same as an intervention that has shown promise using other study designs but has not yet been studied using an RCT
Blinding investigators is another important way of limiting bias, but is not always practical or realistic – for example, some surgical interventions
Always check to see if there is a section describing how the investigators determined the appropriate number of study subjects. If a study is too small, an important difference may exist but statistical significance will not be achieved
OBG Project CME requires a modern web browser (Internet Explorer 10+, Mozilla Firefox, Apple Safari, Google Chrome, Microsoft Edge). Certain educational activities may require additional software to view multimedia, presentation, or printable versions of their content. These activities will be marked as such and will provide links to the required software. That software may be: Adobe Flash, Apple QuickTime, Adobe Acrobat, Microsoft PowerPoint, Windows Media Player, or Real Networks Real One Player.
Disclosure of Unlabeled Use
This educational activity may contain discussion of published and/or investigational uses of agents that are not indicated by the FDA. The planners of this activity do not recommend the use of any agent outside of the labeled indications.
The opinions expressed in the educational activity are those of the faculty and do not necessarily represent the views of the planners. Please refer to the official prescribing information for each product for discussion of approved indications, contraindications, and warnings.
Participants have an implied responsibility to use the newly acquired information to enhance patient outcomes and their own professional development. The information
presented in this activity is not meant to serve as a guideline for patient management. Any procedures, medications, or other courses of diagnosis or treatment discussed or suggested in this activity should not be used by clinicians without evaluation of their patient’s conditions and possible contraindications and/or dangers in use, review of any applicable manufacturer’s product information, and comparison with recommendations of other authorities.
Jointly provided by
NOT ENOUGH CME HOURS
It appears you don't have enough CME Hours to take this Post-Test. Feel free to buy additional CME hours or upgrade your current CME subscription plan