The authors of one of the more notorious recent anti-vaping journal articles have issued a correction. Samir Soneji published a letter explaining that he and his colleagues used faulty inputs in their “meta-analysis” paper, which supposedly summarized the results of studies related to the mythical gateway claim. After fixing the error, their claimed final result dropped dramatically.
This error is ultimately unimportant compared to the other flaws in the paper. The methodology is inherently junk science, just as it was for a previous version of the same exercise. The interpretation of associations between vaping and smoking as causal (a gateway effect) ignores the huge obvious problem of confounding. In short, this exercise could never provide any useful information, and the paper is completely worthless. There is still no evidence that supports the gateway claim if it is interpreted honestly.
Nevertheless the implications of this correction are interesting.
First, it suggests that Soneji was trying to be honest in spite of recently becoming a clownish figure thanks to a recent hit-piece (worth a read as a master class in anti-vaping ignorance) in which he claimed he was booed when he presented at a vaping conference (he was not). Few tobacco controllers ever admit to errors in their papers, even when they are pointed out by others and indisputable. This error would never have been caught by readers. Of course it is possible that whoever actually caught the error communicated it to the entire large research team and some of them wanted to hide it, but others refused to do keep it a secret.
Second, this is a reminder that journal peer-review does not accomplish what many people believe it does. Among the many limitations of the peer-review process is that the reviewers do not get any more information about the study than any reader of the paper would. If there are underlying errors in the methodology (which is never reported in enough detail to identify most errors), they cannot find it, no matter how skilled and conscientious they are.
Third, this illustrates on how conclusions in papers relate to the study results. The correction dramatically changed the estimate of how many vaping teens and young adults initiate smoking, from about 30 percent to about 20 percent. Yet when correcting this number and other numbers it affected, the authors did not propose any changes in their discussions or conclusions about the results. This illustrates that the conclusions of tobacco control papers are not actually influenced by the results of the study.
Finally, the way the journal handled this is troubling. It simply changed the numbers in the online version of the paper without highlighting the edit. Only an eagle-eyed viewer of that page might notice the link near the top indicating that there was a correction, click on it, and learn that the text had been changed. This contrasts with the standard practice in online publishing (e.g., what this newspaper does), which is to add a prominent notice of changes as part of the article, making it difficult to overlook. Moreover, people still read the original print version of journal articles in libraries, and the archived online version – which will probably be read more times than the paywalled official version – still has the original numbers and is unlikely to update. While a prominent statement of correction, rather than a sneaky edit, would not reach all readers of those versions, there is some chance they would see it.
The authors deserve credit for the rare act of admitting to an error in their paper. However this laudable act did nothing to fix the fundamental flaws in their analysis. Its main contribution was to highlight deep flaws in public health publishing.
