I read the article in the NYT with interest and shared it with my sister (MD) and daughter (grad student in public health), and we were very puzzled. The conclusions and headlines didn't make sense or feel right. I shared your write-up, and it answered a lot of our questions.
I really appreciate the work you have done here and wish the NYT would have been more careful before giving the author such a public platform.
Bravo and thank you!
And this is one more example of the truly excellent work you do.
Thomas, thank you for this thorough analysis. The meta-analysis by Jefferson et al. was indeed bad science. They themselves stated that "The high risk of bias in the trials, variation in outcome measurement, and relatively low adherence with the interventions during the studies hampers drawing firm conclusions." If there's any lesson to be taken from this mess of a paper, it is that poorly enforced mandates with low compliance in which people use masks of variable quality, many worn improperly (below the nose!), and many used too long to remain effective, had a less beneficial effect in reducing transmission than hoped. It seems that Jefferson has an anti-masking agenda and used this poorly designed meta analysis to support his agenda. Letting one's personal agenda bias one's science is sure to result in bad science and weak conclusions. As you point out, the Bangladesh study and other, smaller well-designed studies show a clear benefit to mask use, both for reducing transmission and protecting the wearer. This is why surgical teams have worn masks in operating rooms for the past 100 years.
I'd like to comment on the extremely high quality of your work. Two of the giants of American medicine, Bob Wachter and Eric Topol, both referred to your article in their response to the mask debate. That is high praise indeed my friend.
A completely neutral approach to information interpretation does not exist. That's really the point of Kuhn's (and Kahneman's and Tversky's among others) work. Every human being has multiple filters, often ones of which they are not aware.
In that context, a decision to investigate the source of a set of information is not bias. It is research, and it is done for a very specific reason:
The issue is not whether crazy folks can have brilliant ideas. It's about whether there is a pre-existing agenda which is driving a particular author's information filter. That absolutely needs to be taken into account and is valid.
Finally, "ad hominem" implies an attack on an individual. Understanding the biases which an author brings to the table is not an attack.
Whenever I am sent a citation that seems to wildly contradict what I know, my first reaction (after calming down the demons of confirmation bias) is to ask - What's the source?
If I find out it's from Epoch Times, I don't even bother to read it.
The second question I ask is: Who are the authors? I.e., are they bringing their own biases to the table and if so, how well known are those biases?
I haven't heard anyone discuss the authors....
How well known are they in epidemiology/statistics?
What are their biases?
What is their agenda?
I suspect there's gold in them thar hills.........
Sorry, one more thing. Compliance and seal really is everything with masks. A surgeon is indeed usually protected. But look up what a surgeon does with his or her masking. And look up the research on what lowers the efficacy of a surgeon’s seal. Then compare that surgical procedure to what we did as a population. Most people masking weren’t actually masking during COVID. ZZ Top with a mask over his beard when he walked to his dinner reservation was signaling, not protecting. Sorry. And that had nothing to do with whether the pilgrims really cared about masking or not. We are lucky COVID wasn’t lethal for the vast majority. But if it were, I promise we’d have all been more fastidious and serious. Isaac Newton lived in the countryside during the Black Death, for example.
Woow! amazing work. You took a scientific paper and did a in-depth analysis to find strong biases on the paper.
Its exactly what a Cochrane review should avoid. From their website: "[Cochrane Database of Systematic Reviews] aimed at minimizing bias, to produce more reliable findings to inform decision-making"
The fact that is a Cochrane Review means they followed protocols, data has been scrutinized and several other scientist review it (PhDs with years of experience in the field). Nobody spotted these facts?
What does it say about the way we are doing science?
The work you did requires deep understanding and hell lot of time. I am used to judging every piece of information I read, but I always considered scientific papers the closest to "ground truth". Not any more. Which is a shame as I (and I guess most people) don't have the knowledge and time to do analyze scientific work the way you did.
Thank you for all the hard work of analyzing these various studies! Great job! As a Sociologist however, I take these kinds of statistical studies with a big grain of salt. This kind of statistical analysis of human "action"--and the act of putting on a mask is VERY contextual---ignore all sorts of "micro actions".....the "who, what, where, when, and why" of the putting on, keeping on, adjusting, etc. of mask wearing. As you allude to, these studies are not really about mask WEARING but about the validity of various statistical studies. I would offer that a statistic study, which assumes linear causality, ie. "wearing a mask will reduce the likely hood of a respiratory illness" is simply a poor hypothesis. Frankly, the best studies about the efficacy of masks----and not the "wearing of masks" are the studies that show how aerosols are prevented from moving by being blocked by something---and the "something" that blocks the transfer of "human juices" is a K-95 mask. I for one, wore a mask to simply add a layer between my respiratory system and the stuff that people exude when they talk, sneeze, cough, laugh, sing etc. What is really needed is a global, rigorous framework to study what goes into the "act" of wearing some kind of aerosol preventive mechanism.....
There is another way to approach the issue of whether masks (as opposed to mask mandates) work. In countries like Japan mask-wearing is culturally supported, having been employed against particulate pollution for decades. I believe the data on covid-related deaths post lockdowns in Japan supports the view that fewer deaths occur when there is general mask-wearing by the population. Perhaps you could look into this?
I seldom really learn from a blog or podcast, but in the case of Tomas Pueyo every encounter with his thinking has been memorable. Thanks Thomas.
I read the article in the NYT with interest and shared it with my sister (MD) and daughter (grad student in public health), and we were very puzzled. The conclusions and headlines didn't make sense or feel right. I shared your write-up, and it answered a lot of our questions.
I really appreciate the work you have done here and wish the NYT would have been more careful before giving the author such a public platform.
Bravo and thank you!
And this is one more example of the truly excellent work you do.
Thomas, thank you for this thorough analysis. The meta-analysis by Jefferson et al. was indeed bad science. They themselves stated that "The high risk of bias in the trials, variation in outcome measurement, and relatively low adherence with the interventions during the studies hampers drawing firm conclusions." If there's any lesson to be taken from this mess of a paper, it is that poorly enforced mandates with low compliance in which people use masks of variable quality, many worn improperly (below the nose!), and many used too long to remain effective, had a less beneficial effect in reducing transmission than hoped. It seems that Jefferson has an anti-masking agenda and used this poorly designed meta analysis to support his agenda. Letting one's personal agenda bias one's science is sure to result in bad science and weak conclusions. As you point out, the Bangladesh study and other, smaller well-designed studies show a clear benefit to mask use, both for reducing transmission and protecting the wearer. This is why surgical teams have worn masks in operating rooms for the past 100 years.
I'd like to comment on the extremely high quality of your work. Two of the giants of American medicine, Bob Wachter and Eric Topol, both referred to your article in their response to the mask debate. That is high praise indeed my friend.
A completely neutral approach to information interpretation does not exist. That's really the point of Kuhn's (and Kahneman's and Tversky's among others) work. Every human being has multiple filters, often ones of which they are not aware.
In that context, a decision to investigate the source of a set of information is not bias. It is research, and it is done for a very specific reason:
The issue is not whether crazy folks can have brilliant ideas. It's about whether there is a pre-existing agenda which is driving a particular author's information filter. That absolutely needs to be taken into account and is valid.
Finally, "ad hominem" implies an attack on an individual. Understanding the biases which an author brings to the table is not an attack.
Whenever I am sent a citation that seems to wildly contradict what I know, my first reaction (after calming down the demons of confirmation bias) is to ask - What's the source?
If I find out it's from Epoch Times, I don't even bother to read it.
The second question I ask is: Who are the authors? I.e., are they bringing their own biases to the table and if so, how well known are those biases?
I haven't heard anyone discuss the authors....
How well known are they in epidemiology/statistics?
What are their biases?
What is their agenda?
I suspect there's gold in them thar hills.........
Thanks, Tomas and team, for your investigation.
to your second point in opening remarks, 'what can we learn about knowledge from this debate?', this truly is a cautionary tale!
You performed great surgery to peel away the lies and deceit.
Great article. Thanks for digging into it and making it easier to understand.
Sorry, one more thing. Compliance and seal really is everything with masks. A surgeon is indeed usually protected. But look up what a surgeon does with his or her masking. And look up the research on what lowers the efficacy of a surgeon’s seal. Then compare that surgical procedure to what we did as a population. Most people masking weren’t actually masking during COVID. ZZ Top with a mask over his beard when he walked to his dinner reservation was signaling, not protecting. Sorry. And that had nothing to do with whether the pilgrims really cared about masking or not. We are lucky COVID wasn’t lethal for the vast majority. But if it were, I promise we’d have all been more fastidious and serious. Isaac Newton lived in the countryside during the Black Death, for example.
Woow! amazing work. You took a scientific paper and did a in-depth analysis to find strong biases on the paper.
Its exactly what a Cochrane review should avoid. From their website: "[Cochrane Database of Systematic Reviews] aimed at minimizing bias, to produce more reliable findings to inform decision-making"
The fact that is a Cochrane Review means they followed protocols, data has been scrutinized and several other scientist review it (PhDs with years of experience in the field). Nobody spotted these facts?
What does it say about the way we are doing science?
The work you did requires deep understanding and hell lot of time. I am used to judging every piece of information I read, but I always considered scientific papers the closest to "ground truth". Not any more. Which is a shame as I (and I guess most people) don't have the knowledge and time to do analyze scientific work the way you did.
Many thanks!
Thank you for all the hard work of analyzing these various studies! Great job! As a Sociologist however, I take these kinds of statistical studies with a big grain of salt. This kind of statistical analysis of human "action"--and the act of putting on a mask is VERY contextual---ignore all sorts of "micro actions".....the "who, what, where, when, and why" of the putting on, keeping on, adjusting, etc. of mask wearing. As you allude to, these studies are not really about mask WEARING but about the validity of various statistical studies. I would offer that a statistic study, which assumes linear causality, ie. "wearing a mask will reduce the likely hood of a respiratory illness" is simply a poor hypothesis. Frankly, the best studies about the efficacy of masks----and not the "wearing of masks" are the studies that show how aerosols are prevented from moving by being blocked by something---and the "something" that blocks the transfer of "human juices" is a K-95 mask. I for one, wore a mask to simply add a layer between my respiratory system and the stuff that people exude when they talk, sneeze, cough, laugh, sing etc. What is really needed is a global, rigorous framework to study what goes into the "act" of wearing some kind of aerosol preventive mechanism.....
Thank you, Tomas. It’s articles like this that keep me sane. Great work!
loved it!
There is another way to approach the issue of whether masks (as opposed to mask mandates) work. In countries like Japan mask-wearing is culturally supported, having been employed against particulate pollution for decades. I believe the data on covid-related deaths post lockdowns in Japan supports the view that fewer deaths occur when there is general mask-wearing by the population. Perhaps you could look into this?
Great analysis as always! This read is a thorough reminder to read past the headlines and check the sources.
I have two personal questions for you:
1. Until this paper came out, what was your personal mask wearing strategy?
2. After writing this post, has your mask wearing strategy changed and if so, how?