Extensive UCSF Study Finds Flaws in Previous Study’s Analytic Methods
A popular smoking cessation medication has been under a cloud of suspicion ever since the Canadian Medical Association Journal (CMAJ) published a study July 2011 reporting “risk of serious adverse cardiovascular events associated with varenicline.” Varenicline, also known as Chantix, blocks the pleasant effects of nicotine on the smoker’s brain and lessens nicotine withdrawal symptoms.
UCSF researchers, however, question the way the previous study was conducted, and their new analysis, scheduled to be published May 4 in BMJ, reaches a very different conclusion.
“We found no clinically or statistically significant increase in serious adverse cardiovascular events associated with using varenicline,” said lead author Judith J. Prochaska, PhD, MPH, an associate professor in UCSF’s Department of Psychiatry and researcher with the Center for Tobacco Control Research and Education. “The findings from 22 trials with more than 9,200 participants indicate a difference in risk of only 0.27 percent between those on varenicline versus placebo, or about a quarter of one percent.”
Prochaska identified the need for the re-analysis after reading the CMAJ publication and finding the conclusion inconsistent with the small differences within trials. Prochaska had received an “investigator-initiated research award” from Pfizer, the maker of Chantix, that spring for a separate study of varenicline in hospitalized smokers, many with heart disease. Prochaska says she was compelled to more closely examine the data to quantify the level of risk before using the medication on UCSF patients.
Differences in Meta-Analytic Methods
Prochaska and Joan F. Hilton, ScD, MPH, a professor in UCSF’s Department of Epidemiology & Biostatistics, conducted the new study, which, like the prior one, used meta-analytic statistical techniques to combine results from different trials of varenicline with tobacco users. The UCSF and prior analysis, however, differed in several fundamental ways.
The previous study, led by Sonal Singh, MD, MPH, assistant professor of medicine at Johns Hopkins University, looked at a sample size of 8,216 patients in 14 trials and reported a 72 percent (relative) increase in risk of heart attack or other serious heart problems.
The UCSF analysis included 22 double-blind, randomized controlled trials with 9,232 participants. More than half of the studies included participants with active or past history of cardiovascular disease. Eight trials had no events. The UCSF study found a 0.27 percent (absolute) risk difference, which it determined was neither clinically nor statistically significant.
“We identified eight separate trials with nearly 1,600 tobacco users randomized to varenicline or placebo that did not have a single serious cardiovascular event,” Prochaska said. “This information is meaningful and the Singh analysis excluded these trials.”
UCSF researchers say varenicline lasts in the body about seven days after a person stops using the medication. The new analysis examined events occurring during the drug treatment window or within 30 days after a patient stopped using the drug, rather than the entire trial period, which in many cases was a year in Singh’s study.
“The longer you follow heavy, long-term tobacco users – and in these studies, the average participant smoked a pack a day for 25 years – the more likely you will see serious cardiovascular events related to their compromised health,” Prochaska said. “In addition, 13 of the 14 studies in the Singh analysis experienced greater attrition in the placebo group than in the test group, which could inflate the treatment effect.”
Absolute Difference vs. Relative Difference
Singh’s study found that 1.06 percent of those who took varenicline had serious heart-related complications, compared to 0.82 percent of the placebo group. Although the simple absolute difference was only 0.24 percent – similar to the UCSF study finding based on meta-analytic methods – Singh’s paper reported the weighted, relative difference, which was 72 percent.
“What made the headlines is this claim of a huge increase, but it’s a relative measure; it’s a unit-less measure,” Prochaska said. “And so if you increase something small by a little bit, you can claim that it’s a 72 percent increase, but that obscures the fact that it’s still a very small risk.”
“For someone not accustomed to these measures, it might not be obvious that the 72 percent increase was relative to a very small base – interpretations the Singh study didn’t stress,” Hilton said.
The new meta-analysis provides a more comprehensive examination of varenicline by presenting and comparing the four most relevant summary measures for this type of analysis.
“Among these, we found that the methods used by Singh and colleagues produced the most extreme estimates of treatment effect,” said Hilton. “We identified the specific conditions under which this statistic is inflated. We recommended the summary based on the (absolute) risk difference because it is methodologically appropriate in our setting and gives a clear quantitative sense of the excess risk.”
Smoking and Cardiovascular Risk
Cardiovascular disease is the leading cause of death among smokers. Tobacco use also increases a person’s risk of stroke, cancer and lung disease.
“One of the most important things you can do for your heart is to quit smoking,” Prochaska said. “Clinical practice guidelines recommend the use of FDA medications to quit smoking. All medications carry some risk; however, we hope the FDA and other experts compare the Singh analysis to ours to determine what action, if any, should be taken with regard to varenicline and cardiovascular risk. Tobacco is a deadly addiction, and patients need effective treatments.”
Funding for this meta-analysis was provided by grants to Prochaska from the State of California Tobacco-Related Disease Research Program and the National Institute on Drug Abuse.
Hilton is funded by federal grants and has no current funding from any pharmaceutical company.
UCSF is a leading university dedicated to promoting health worldwide through advanced biomedical research, graduate-level education in the life sciences and health professions, and excellence in patient care.