Negative Trials: Long-Term Bang

In results published in the latest issue of JAMA Oncology, the researchers found that first reports of the results of positive Phase III cancer trials were twice as likely to be cited in scientific journals. But over time, when all articles associated with the trials were considered, the scientific impact of negative trials and positive trials was about the same. The study was funded by the National Cancer Institute (NCI), and conducted by members of SWOG, one of five cancer research groups in the NCI’s National Clinical Trials Network.

“Negative trials aren’t scientific failures,” said Joseph Unger, PhD, a SWOG statistician based at the Fred Hutchinson Cancer Research Center and the lead author of the study. “We found that they have a positive, lasting impact on cancer research.”

Unger and the SWOG team analyzed every randomized, Phase III cancer trial completed by SWOG over a 30-year period stretching from 1984 to 2014. That’s 94 studies involving 46,424 patients. Of those 94 studies, 26 were positive, meaning that the treatment they tested performed measurably better than the standard treatment at the time. Positive trials often result in new drugs or drug combinations that get used in the clinic – which get them lots of attention in the medical community and in the press.

The SWOG team found that, indeed, primary manuscripts first announcing these encouraging results were published in journals with higher impact factors and cited twice as often as negative trials. However, they found that the number of citations from all primary and secondary manuscripts did not differ between positive and negative trials.

The findings are important for a few reasons. First, federally funded cancer clinical trials represent a major investment of U.S. tax dollars. In fiscal year 2013, the NCI spent $676.5 million for clinical trials. In addition, there is a long-standing debate in the biomedical research community about the publication of negative trial data, and the need for more widespread and effective use of all trial data – positive and negative.

At the same time, data sharing and data mining is a hot topic in cancer care and research and one aim of the Obama administration’s new “cancer moonshot” led by Vice President Joe Biden. Currently, there are “big data” projects underway at the NCI, the American Society of Clinical Oncology, the American Association of Cancer Research, and a project of NCI-designated cancer centers founded by Moffitt Cancer Center and the Ohio State Comprehensive Cancer Center. These projects aim to share genomic analyses, electronic medical records, cancer registry data, NCI trial data, and more.

“Negative trials matter because they tell us what doesn’t work – which can be as important as what does,” said Dr. Dawn Hershman, a study author from Columbia University Medical Center who also serves as vice chair for SWOG’s National Community Oncology Research Program, also funded by the NCI. “Negative trials are also critical for secondary research, which mines existing trial data to answer new questions in cancer care and prevention. Negative trials are used frequently in secondary research, and add great value to the scientific community.”

Along with Unger and Hershman, the study team included William Barlow, PhD, from the SWOG Statistical Center at Fred Hutch; Dr. Scott Ramsey of Fred Hutch; Michael LeBlanc, SWOG’s lead statistician at Fred Hutch; and Dr. Charles Blanke, the group chair of SWOG and a physician at the Knight Cancer Institute at Oregon Health & Science University.

Funding came from the Dr. Charles A. Coltman, Jr. Fellowship of The Hope Foundation and by the NCI’s Community Oncology Research Program (NCORP) Research Base grant.