When medical research reports negative findings: follow the money!

Big pharma has been taken to task on multiple occasions for selectively publishing studies showing benefits for their products and “burying” research with negative outcomes. So it’s surprising that there is now a trend to publish studies showing negative or null results. In fact, a journal specifically devoted to publishing studies of negative results closed up shop on 2017-9-1, explaining that “Since its inception…many other journals followed (its) lead in publishing articles reporting negative or null results” so there was no longer a need for a specific journal. Is this trend an initiative of the journals? Of researchers? Or of study sponsors? And why does it matter?

Let’s illustrate what I view as a problem with an example or two. The first has to do with the use of metformin to prevent or treat cancer. Metformin, the frontline drug for type 2 diabetes, is widely available for only pennies per day, so when it was observed in a multitude of epidemiologic studies that diabetic patients on metformin had lower risk for cancer and better outcomes, there was great interest in seeing whether metformin could benefit cancer patients without diabetes. As of July 2015, more than 100 clinical trials of this compound for various indications in oncology were said to be in progress by Kordes et al (2015)¹. Their article, the first report of a randomized, placebo-controlled trial of metformin with a survival endpoint for cancer treatment, was highly anticipated. Unfortunately, as clearly and unequivocally stated in their abstract, “Addition of a conventional anti-diabetic dose of metformin does not improve outcome in patients with advanced pancreatic cancer treated with gemcitabine and erlotinib”. Clearly a negative finding.

So I was more than surprised to discover, in the article itself, two very interesting graphs and the accompanying text which makes the meaning clear. In what the authors called “exploratory” analyses, for the metformin patients receiving an adequate dose, sufficient to lower insulin levels, survival was more than 3 times longer (median 18.6 months) compared to the metformin patients whose insulin levels did not go down (5.7 months), and more than double the survival of the placebo group (7.3 months).

One needs to ask, why was this result, so important from a patient’s point of view, minimized as “exploratory” in the article text, and left out of the abstract altogether? Keep in mind that most busy clinicians only read the title and abstract and use those to guide their clinical decisions. And the oncologists who might prescribe metformin for cancer patients will likely be influenced by online news stories addressed to them specifically, such as the ASCO Post (ASCO = American Society of Clinical Oncology), the Cancer Therapy Advisor, or XpertDox.

Here’s another example, this time a randomized trial in more than 15,000 patients with diabetes but without evidence of atherosclerotic cardiovascular disease (ASCEND Study Collaborative Group: Effects of n-3 Fatty Acid Supplements in Diabetes Mellitus)². One group received 1g capsules of omega-3 fatty acids, the other capsules of olive oil placebo. In the abstract of the article, the authors concluded “there was no significant difference in the risk of serious vascular events between those who were assigned to receive n−3 fatty acid supplementation and those who were assigned to receive placebo”. Again, a look at the body of the article proves instructive. Here it says, “In exploratory analyses, there were fewer vascular deaths (which represented 28% of all deaths) in the fatty acid group than in the placebo group”. In fact, the relative reduction in vascular deaths was 18.3%, from 6.0% to 4.9%, or an absolute reduction of 1.1 percentage points. This result is significant both statistically and probably clinically.

Of course, I’m not the only critic of how the results were presented. An expert on omega-3s, hired by the study authors to do blood testing, felt that baseline omega-3 levels were already quite high. Nevertheless, the study conclusion that “There is no justification for recommending fish oil supplements to protect against cardiovascular events” was touted by the University of Oxford, the American College of Cardiology, the European Society of Cardiology (ESC) at whose congress the study results were first presented, as well as online sources of news targeted to physicians (MedPage Today, MDedge, tctMD, CRTonline) or the public (EFACTS4YOU). Additionally, the ASCEND study conclusion appears to contradict other findings. For example, a recent meta-analysis³ based on 14 randomized controlled trials showed that omega-3 fatty acid “supplementation is associated with a modest reduction in cardiac death”.

In both examples, the authors used the word “exploratory” to describe results which are important but which did not make it into the abstracts. Is “exploratory” some sort of code word? And why did the editors of such esteemed journals (Lancet Oncology for the metformin article, New England Journal of Medicine for the omega-3 article) allow such misleading abstracts to be published?

My premise is that the publication bias towards positive results for drug-company-sponsored trials is easy to understand, as the companies have a vested interest in getting their drugs seen in a positive light so as to increase profits. On the other hand, when researchers and journals have a field day in publishing negative results, with accompanying big publicity, for medications or treatments which cannot earn big profits for anyone, one has to look for a different kind of bias. After all, if these treatments gave positive results, they would eat into the revenue streams of drug companies with patented medications. Are these drug companies influencing researchers and journals to publish the negative results, then? The mechanisms to do so certainly exist: for many medical researchers, an important source of income is provided by drug companies for conducting clinical trials on their products. This income stream also extends to the hospitals where the researchers work and where the clinical trials often take place.

As to medical journals, Richard Smith, an editor of the British Medical Journal for 25 years, wrote “The most conspicuous example of medical journals’ dependence on the pharmaceutical industry is the substantial income from advertising”⁴. Smith’s article gives other examples of how medical journals profit from drug companies, and concludes that medical journals are simply an extension of the companies’ marketing departments. Although Smith provides a fairly exhaustive list of methods for pharmaceutical companies to get the results they want from clinical trials, he does not include the possibility that they might encourage the publication of negative results about competing treatments.

But how? If one looks at the statements about competing interests made by authors, or about funding sources for these studies with negative results, everything looks clean. Authors are not required to disclose their income from big pharma for studies not paid for by those companies. And the funding sources look innocuous, often including not-for-profit organizations devoted to a specific organ or disease. But many of these groups purporting to represent patients take donations from big pharma, and one study found that a third of these organizations’ boards included pharmaceutical industry executives or former executives⁵.

So how much trust should we put in these negative results studies? I advise dosing with a grain of salt!

1. Kordes S, Pollak MN, Zwinderman AH et al. Metformin in patients with advanced pancreatic cancer: a double-blind, randomised, placebo-controlled phase 2 trial. Lancet Oncol. 2015;16:839-847. PMID 26067687 ↩
2. ASCEND Study Collaborative Group, Bowman L, Mafham M et al. Effects of n-3 Fatty Acid Supplements in Diabetes Mellitus. N Engl J Med. 2018;379:1540-1550. PMID 30146932 ↩
3. Maki KC, Palacios OM, Bell M, Toth PP. Use of supplemental long-chain omega-3 fatty acids and risk for cardiac death: An updated meta-analysis and review of research gaps. J Clin Lipidol. 2017PMID 28818347 ↩
4. Smith R. Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Med. 2005;2:e138. PMID 15916457 ↩
5. McCoy MS, Carniol M, Chockley K, Urwin JW, Emanuel EJ, Schmidt H. Conflicts of Interest for Patient-Advocacy Organizations. N Engl J Med. 2017;376:880-885. PMID 28249131 ↩

• ← Public Transportation: Is Montreal ready for the world stage?
• McGill Cutting Edge Lecture, 2018-10-11 →