The Lancet published a meta-analysis of 27 statin trials

Data, Drugs, and Deception: a True Story

 

Wednesday, May 23, 2012

      

Last week The Lancet published a meta-analysis of 27 statin trials, an attempt to determine whether patients with no history of heart problems benefit from the drugs—true story. The topic is controversial, and no less than six conflicting meta-analyses have been performed—also a true story. But last week’s study claims to show, once and for all, that for these very low risk patients, statins save lives—true story.

Actual true story: the conclusions of this study are neither novel nor valid.

The Lancet meta-analysis, authored by the Cholesterol Treatment Trialists group, examines individual patient data from 27 statin studies. Their findings disagree with an analysis published in 2010 in the Archives of Internal Medicine, and with analyses from two equally respected publications, the Therapeutics Letter and the Cochrane Collaboration.* Despite this history of dueling data the authors of last week’s meta-analysis, in a remarkable break from scientific decorum, conclude their report with a directive for the writers of statin guidelines: the drugs should be broadly recommended based on the new analysis.

As an editorialist points out, if implemented, the CTT group recommendations in the United States would lead to 64 million people, more than half of the population over the age of 35, being started on statin therapy—true story.

Where is the magic, you ask, in this latest effort? What is different? In some ways, nothing. Indeed just a year and a half earlier The Lancet published a meta-analysis of 26 of the same 27 studies, with the same results, by the same authors (true story, and an odd choice on the part of the journal). So the findings aren’t new. They are, however, at odds with other meta-analyses. Why? It is the way they calculated their numbers. This meta-analysis, like the earlier one from the same group, reports outcomes per-cholesterol-reduction. The unit they use is a “1 mmol/L reduction in low density lipoprotein (LDL)”, in common U.S. terms, a roughly 40-point drop in LDL.

That’s the magic: each of the benefits reported in the paper refers to patients with a 40-point cholesterol drop. Voilá. One can immediately see why these numbers would look different than numbers from reviews that asked a more basic question: did people who took statins die less often than people taking a placebo? (The only important question.) Instead, they shifted the data so that their numbers corresponded precisely to patients whose cholesterol responded perfectly.
Patients whose cholesterol drops 40 points are different than others, and not just because their body had an ideal response to the drug. They may also be taking the drug more regularly, and more motivated. Or they may be exercising more, or eating right, and more health conscious than other patients. So it should be no surprise that this analysis comes up with different numbers than a simple comparison of statins versus placebo pills. Ultimately, then, this new information tells us little or nothing about the benefits someone might expect if they take a statin. Instead it tells us the average benefits among those who had a 40-point drop in LDL.

But LDL drop cannot be predicted. Some won’t drop at all, some will drop just a bit, and some may drop more. Therefore the numbers here tell an interesting story about certain patients who took statins, but they have no relevance to patients and doctors considering statins. And yet, the latter group is the target of the study's concusions.

True story: in prior meta-analyses that found no mortality benefit the investigators simply looked at studies of patients without heart disease and compared mortality between the statin groups and the placebo groups. No machinations, no acrobatics, no per-unit-cholesterol. They took a Joe Friday approach (just the facts, ma’am), and found no mortality benefit.

Perhaps never has a statistical deception been so cleverly buried, in plain sight. The study answers this question: how much did the people who responded well to the drug benefit? This is, by definition, a circular and retrospective question: revisiting old data and re-tailoring the question to arrive at a conclusion. And to be fair they may have answered an interesting, and in some ways contributory, question. However the authors’ conclusions imply that they answered a different, much bigger question. And that is not a true story.

Guideline writers, doctors, patients, journalists, and policy makers will all have to pay close attention to avoid the trappings of deceptive data, dressed up as a true story.

*The Cochrane Collaboration analysis reports an overall mortality benefit with statins (RR=0.86), however their summary suggests that statins should be used for primary prevention “with caution.” In particular on p.12, after a discussion of the biases in many of the trials that led to their numerical finding, they clearly state that using statins for patients with anything less than a 2% per year risk of coronary events “is not supported by existing evidence.” This cutoff encompasses virtually all people that would be considered candidates for primary prevention.
=============================================================================================

Read the complete article here.