Elsevier

World Neurosurgery

Volume 112, April 2018, Pages 233-242
World Neurosurgery

Literature Review
Misclassification of Case-Control Studies in Neurosurgery and Proposed Solutions

https://doi.org/10.1016/j.wneu.2018.01.171Get rights and content

Objective

Case-control studies (CCS) and cohort studies (CS) are common research designs in neurosurgery. But the term case-control study is frequently misused in the neurosurgical literature, with many articles reported as CCS, even although their methodology does not respect the basic components of a CCS. We sought to estimate the extent of these discrepancies in neurosurgical literature, explore factors contributing to mislabeling, and shed some light on study design reporting.

Methods

We identified 31 top-ranking pure neurosurgical journals and searched them for articles reported as CCS, either in the title or in the abstract. The articles were read to determine if they really were CCS according to STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines. Article assessment was conducted in duplicate (agreement [κ statistics] = 99.82%).

Results

Two hundred and twenty-four articles met our inclusion criteria, 133 of which (59.38%) correctly labeled the case-control design, whereas 91 (40.62%) misclassified this study design. Cohort studies (CS) were the most common design mislabeled as case-control studies in 76 articles (33.93%), 57 of which (25.45%) were retrospective CS. The mislabeling of CCS impairs the appropriate indexing, classification, and sorting of evidence. Mislabeling CS for CCS leads to a downgrading of evidence as CS represent the highest level of evidence for observational studies. Odds ratios instead of relative risk are reported for these studies, resulting in a distortion of the measurement of the effect size, compounded when these are summarized in systematic reviews and pooled in meta-analyses.

Conclusions

Many studies reported as CCS are not true CCS. Reporting guidelines should include items that ensure that studies are labeled correctly. STROBE guidelines should be implemented in assessment of observational studies. Researchers in neurosurgery need better training in research methods and terminology. We also recommend accrued vigilance from reviewers and editors.

Introduction

Case-control studies (CCS) and cohort studies (CS) are well-known research designs in contemporary use in neurosurgery. They are analytic (comparative) and observational (the investigator does not allocate exposure) studies. However, several studies have been reported using the term “case-control study” in the title or methodology to describe this study design. On closer examination, these studies turn out to be nothing of the kind: at least, not in the sense in which the term is understood by epidemiologists. Instead of comparing the distribution of exposures among diseased and nondiseased subjects to make inferences regarding causation, these studies typically compare the clinical outcomes of groups of patients treated with different interventions to make inferences about effectiveness.1 Most are retrospective CS and are notoriously prone to inappropriately conclude that one treatment is superior to another.1 This confusion about CCS is because the terms cases and controls, used to refer to the participants in the study, are confused with the design of the study.

To illustrate this point, according to a recent article entitled “Efficacy and safety of a porcine collagen sponge for cranial neurosurgery: a prospective case-control study,” the authors reported the design to be “a prospective case-control nonrandomized study” carried out between November 2009 and 2010 of 50 patients in whom collagen matrix dural graft was used comparing with a “control” group of 50 patients who were treated with autologous duraplasty material. The follow-up period was 3 months and the primary objective was to study the incidence of postoperative cerebrospinal fluid leak, including fistula and pseudomeningocele, and postoperative infection by comparing autologous material and a new collagen graft.2

According to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines3 and epidemiologic textbooks,4, 5, 6, 7 the study is a “(retrospective) cohort study” because the purported “cases” were actually the exposed and “controls” the nonexposed (the use of autologous material is the gold standard for dural repairs). The patients were followed over time (3 months), starting from a well-described inception point (dural repairs) and a well-defined outcome assessed (incidence of cerebrospinal fluid leak).The study is retrospective rather than prospective because at the time the study was performed, the exposure and outcome had already occurred.

A CS is a research design in which patients are sampled on the basis of exposure (intervention) and are followed over time, and the occurrence of outcomes (e.g., complications and recurrence) is assessed.4, 5, 8

A CCS is a research design in which participants are selected on the basis of whether they do (cases) or do not (controls) have a particular disease (outcome) under study. Cases and controls are then compared with respect to existing or past attributes or exposures (intervention) or characteristics under study believed to be relevant to the development of the condition or disease under study.4, 5, 9

Classifying a CS as a CCS is not without consequences. The mislabeling impairs the appropriate indexing, classification, and sorting of evidence. It leads to a downgrading of evidence because CS represent the highest level of evidence for observational studies. Odds ratios (ORs) instead of relative risk are reported for these studies, resulting in a distortion of the measurement of the effect size, compounded when these are summarized in systematic reviews and pooled in meta-analyses.

Other examples of confusing descriptions, such as a “prospective cross-sectional case-control study,”10 “case-control cohort study,”11 “retrospective case-control matched cohort,”12 “repeated measures case-control study,”13 or “prospective double-blind randomized case-control study”14 represent the tip of the iceberg and underscore a dire need to clarify and differentiate CCS not only from cohort designs but also from cross-sectional studies and other research designs. Although an assessment of CCS in the Neurosurgery Publishing Group15 showed a 52% misclassification, mislabeling of studies is not peculiar to neurosurgery, because the proportions of studies mislabeled as CCS are as high as 30%–97% in other disciplines.1, 9, 16

In this succinct review, we estimate the extent to which the label “case-control study” is misused or mislabeled in the neurosurgical literature worldwide, analyze factors contributing to mislabeling, and highlight the consequences of this mislabeling on their level in the evidence-based neurosurgery hierarchical ladder. We also provide a succinct description of CS and CCS in the context of neurosurgery, elucidating their anatomy, pros and cons, differences, and uses.

Section snippets

Identification of Studies

Thirty-one of the highest-ranked pure neurosurgical journals17 were searched through the PubMed/Medline portal to identify articles reported as CCS in the titles or abstracts. This search included studies published from January 1, 1944 until December 31, 2013, a period of 69 years.

The journals screened included Acta Neurochirurgica, Asian Journal Of Neurosurgery, British Journal Of Neurosurgery, Central European Neurosurgery, Child's Nervous System, European Spine Journal, Journal Of Korean

Results

Of the 36 journals identified, 31 were screened and 17 fulfilled our inclusion criteria. Fourteen of the screened journals (45.2%) had no “case control” study in the title or abstract and were thus excluded, whereas the remaining 17 journals (54.8%) had at least 1 article with the case-control design. The mean impact factor (±standard deviation) was 2.2281 ± 0.6042 (range, 0.423–4.295). Two hundred and twenty-four articles were found and included after reading the abstracts/articles (Figure 1).

Discussion

Mislabeling of research methods remains a generic problem and the confusion of CS as CCS is a chronic problem in published neurosurgical literature, although this issue has been raised in other domains.1, 9, 16, 20, 21 For more than 7 decades, more than 40% (Table 1) of the studies in neurosurgery claiming to be case-control designs have been a misclassification. Nesvick et al.15 even reported higher rates (52%) from the 5 North American journals.

Most of the mislabeled studies were

Recommendations and Conclusions

Our findings suggest a persistent lack of appreciation of research methods in the neurosurgical community, despite recent efforts to promote evidence-based neurosurgery. Retrospective CS are the most common design mislabeled as CCS. This mislabeling misleads readers as to what was done and impairs appropriate indexing, classification, and sorting of evidence, because CCS belong to a lower level of evidence than do CS.

Our review outlines and provides a framework for researchers, journal editors,

References (23)

  • N.E. Mayo et al.

    When is a case-control study not a case-control study?

    J Rehabil Med

    (2009)
  • Cited by (0)

    Conflict of interest statement: The authors declare that the article content was composed in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

    View full text