one assumption we're making is that the placebo group actually got saline. What if they didn't? What if they got lipid nanoparticles without the mRNA? What if they were injected with everything that's in the "vaccine" except the mRNA? At this point there is evidence that the lipid nanoparticles themselves are toxic even without the mRNA. If the placebo group did not get saline and instead got the vaccine concoction sans mRNA, then that would explain the serious reactions in the placebo group in the first couple of weeks.
Exactly! That's standard procedure in vaccine trials. They use an older vaccine in the worst case and just the adjuvants in the best cases (i cant find a citation right now, but they're out there). They almost never use a simple saline solution like they should. The old vaccines are more likely to give a lot of adverse events in the placebo, then once the trial drug is approved with this method, they can use the trial drug as the next placebo (in a future trial).
in thinking about it some more, it's still odd that the placebo group would have more adverse events those first couple of weeks even if they got the lipid nano's and other stuff that's in the injections other than the mRNA. Why would the lipid nano's by themselves cause a worse reaction than the lipids plus mRNA? There's so much we don't know about these experimental injections! They won't even tell us all the ingredients! And of course, there's the possibility of outright fraud.
If you combine the use of a non-placebo (lipids, adjuvants, etc minus mRNA) with the noted (above and whistleblower) over-reporting and result falsifying of SAEs for the placebo group; I believe together it would look something like these graphs.
They didn't. Look at those word clouds again. Clearly there's a lot more seemingly unrelated stuff included in the placebo group, whereas the known side effects stand out clearly in vaccine group.
First off, my guess is word cloud 1 for the vaccine group, given the size of "acute myocardial infarction" there and the size of "pneumonia" in the other cloud.
Second, thanks for this analysis. I've heard it tossed around that the original Pfizer trial had more SAEs of all types (including COVID) in the treatment group than the control group. I had also heard about the whistleblower from the Pfizer trial but hadn't see anyone dig in to the differences in the SAEs of the two groups and what they may reveal about the possibility of Dx tampering to "nudge" the trial in the treatment's favor.
Third, to steel man the regulators' position about the SAE rates: when someone says "not enough to draw conclusions from", there's a possible technical meaning to that phrase - "small enough to have occurred by chance" AKA "statistically insignificant". I'm not saying anyone actually did this calculation, and based on the prose in the paper it certainly seems plausible that doing so was purposely omitted. Essentially we're trying to estimate the parameter of a Bernoulli random variable. When I do that in a principled way with these numbers (Wilson interval for proportions), I get 95% confidence intervals that are significantly overlapping:
Vaccine: 0.00488, 0.00692
Placebo: 0.00425 - 0.00616
Those intervals are significantly overlapping, and so most statisticians would say therefore that the difference doesn't rise to the level of significance.
Now, to give the regulators a bit of a hard time, if we're going to use the same standard for severe COVID, there's even less certainty in the difference due to the smaller numbers (1 vs 4):
Vaccine: 0.00000815 - 0.000261
Placebo: 0.0000717 - 0.000474
Those intervals are also significantly overlapping.
But to steel-man them again, the difference here is that, unlike a drug trial where all participants have the disease in question, in this case the participants begin the trial without the disease, and the rate of the disease in the population is highly variable with time - and my prior would be that the kind of person who volunteers for an experimental vaccine trial is also pretty cautious and so their COVID infection rate is probably going to be lower than that of the general population. There is also the "healthy user bias" wherein this population tends also to be more educated than the general population, and so of higher socioeconomic status and thus healthier due to better access to healthy lifestyle choices and medical care.
So giving them the biggest benefit of the doubt, perhaps they just let their priors (that there would be increasing runaway COVID infection rates in the near future, that there is a healthy user bias in the trial participants, etc) overwhelm what is otherwise a pretty "meh" result - statistically insignificant for any reduction in severe outcomes from any cause.
All that being said, I tend _not_ to want to give anyone the benefit of the doubt here, owing to the terrible track record of Pfizer in these regards and the capture of the regulatory apparatus, at least in the US. I tend to put a lot of weight on the whistleblower's account because of my priors about Pfizer in this regard. Like Vinay Prasad said in his recent analysis of the "Thailand study" preprint on cardiac biomarkers in adolescents, I say that if these regulators had been doing their damn job, they would have done studies like that immediately following Pfizer's own trial.
And they would have found worse things than what this probably "nudged" trial data reveals.
I read the original Pf study protocol and related article, and I found only one point where the placebo was actually declared: one mention of the word "saline" in the article. Is that credible enough to accept that saline was actually used (e.g. as opposed to the AZ trials that used a meningitis vaccine to inflate control-group AEs)? I know - what else are we going to go on? But the control-group AEs seem remarkably high to me.
Thank you for your thorough work. If you want to talk Python data analysis, please feel free to get in touch.
In addition to pima’s comment, I would say the explanation for the distiribution of complications in the placebo group is as follows:
1. The placebo contained an active ingredient
2. The simple act of getting an injection (whatever the injectate) causes a reaction (maybe something akin to the surgical stress response) which results in complications
3. The complications were reported in a biased and/or fraudulent fashion
Maybe I’m not understanding something here but I would expect the SAE’s in the placebo group to be evenly distributed throughout the reporting period. In other words, if there were 111 SAE’s over a 10 week period then there should be roughly 11 per week.
By the way, a skewed distribution of SAE’s clustered around the first few weeks is actually ok if noted, discussed, explained and accounted for.
It certainly is bizarre but if an explanation is given the perhaps it is permissible.
I am presuming no such discussion is present in the paper
I may be wrong but I read somewhere that the placebo was thought to be an MMR vaccine because it is known to have some side effects and therefore assists in blinding the trial . Like others commenting here I very much doubt the placebo was saline.
Does anyone know why the adverse events reported in the MHRA report differs in many numbers from the adverse events reported by the article "Six Month Safety and Efficacy of the BNT162b2 mRNA COVID-19 Vaccine" (https://www.medrxiv.org/content/10.1101/2021.07.28.21261159v1.supplementary-material)? MHRA e.g.:reported 4 SAE and the 6 months study only 3 ?
IMO, unless there has been a previous study of saline injection vs a mixture of the adjuvants, lipid nano particles, PEG, etc., it does not make sense to use the adjuvants, lipids, etc as the PLACEBO. By definition, isn’t the placebo supposed to be inert, innocuous, “of no physiological effect”?? If we don’t know that PEG plus lipid nanoparticles plus adjuvants are harmless and of no physiological effect, it does not make sense to use that mixture as the placebo in a drug trial. Suppose that mixture alone harms people? Then how can you say anything about the safety of that plus the nRNA and further, approve the mRNA produce when you have no idea if the “carrier mixture” for the mRNA is safe or not? In addition, haven’t we already learned that the LNP (lipid nanoparticles) do collect excessively in certain body parts like the ovaries? IMO, only saline makes sense for a placebo and a separate study (previously done or concurrently run) would be needed to determine the safety of the carrier mixture if not already known.
Brilliant analysis! Everyone involved in this debacle deserves hard time.
one assumption we're making is that the placebo group actually got saline. What if they didn't? What if they got lipid nanoparticles without the mRNA? What if they were injected with everything that's in the "vaccine" except the mRNA? At this point there is evidence that the lipid nanoparticles themselves are toxic even without the mRNA. If the placebo group did not get saline and instead got the vaccine concoction sans mRNA, then that would explain the serious reactions in the placebo group in the first couple of weeks.
Exactly! That's standard procedure in vaccine trials. They use an older vaccine in the worst case and just the adjuvants in the best cases (i cant find a citation right now, but they're out there). They almost never use a simple saline solution like they should. The old vaccines are more likely to give a lot of adverse events in the placebo, then once the trial drug is approved with this method, they can use the trial drug as the next placebo (in a future trial).
Agree! Great point and thank you for raising it.
in thinking about it some more, it's still odd that the placebo group would have more adverse events those first couple of weeks even if they got the lipid nano's and other stuff that's in the injections other than the mRNA. Why would the lipid nano's by themselves cause a worse reaction than the lipids plus mRNA? There's so much we don't know about these experimental injections! They won't even tell us all the ingredients! And of course, there's the possibility of outright fraud.
If you combine the use of a non-placebo (lipids, adjuvants, etc minus mRNA) with the noted (above and whistleblower) over-reporting and result falsifying of SAEs for the placebo group; I believe together it would look something like these graphs.
They didn't. Look at those word clouds again. Clearly there's a lot more seemingly unrelated stuff included in the placebo group, whereas the known side effects stand out clearly in vaccine group.
First off, my guess is word cloud 1 for the vaccine group, given the size of "acute myocardial infarction" there and the size of "pneumonia" in the other cloud.
Second, thanks for this analysis. I've heard it tossed around that the original Pfizer trial had more SAEs of all types (including COVID) in the treatment group than the control group. I had also heard about the whistleblower from the Pfizer trial but hadn't see anyone dig in to the differences in the SAEs of the two groups and what they may reveal about the possibility of Dx tampering to "nudge" the trial in the treatment's favor.
Third, to steel man the regulators' position about the SAE rates: when someone says "not enough to draw conclusions from", there's a possible technical meaning to that phrase - "small enough to have occurred by chance" AKA "statistically insignificant". I'm not saying anyone actually did this calculation, and based on the prose in the paper it certainly seems plausible that doing so was purposely omitted. Essentially we're trying to estimate the parameter of a Bernoulli random variable. When I do that in a principled way with these numbers (Wilson interval for proportions), I get 95% confidence intervals that are significantly overlapping:
Vaccine: 0.00488, 0.00692
Placebo: 0.00425 - 0.00616
Those intervals are significantly overlapping, and so most statisticians would say therefore that the difference doesn't rise to the level of significance.
<continued>
Now, to give the regulators a bit of a hard time, if we're going to use the same standard for severe COVID, there's even less certainty in the difference due to the smaller numbers (1 vs 4):
Vaccine: 0.00000815 - 0.000261
Placebo: 0.0000717 - 0.000474
Those intervals are also significantly overlapping.
But to steel-man them again, the difference here is that, unlike a drug trial where all participants have the disease in question, in this case the participants begin the trial without the disease, and the rate of the disease in the population is highly variable with time - and my prior would be that the kind of person who volunteers for an experimental vaccine trial is also pretty cautious and so their COVID infection rate is probably going to be lower than that of the general population. There is also the "healthy user bias" wherein this population tends also to be more educated than the general population, and so of higher socioeconomic status and thus healthier due to better access to healthy lifestyle choices and medical care.
So giving them the biggest benefit of the doubt, perhaps they just let their priors (that there would be increasing runaway COVID infection rates in the near future, that there is a healthy user bias in the trial participants, etc) overwhelm what is otherwise a pretty "meh" result - statistically insignificant for any reduction in severe outcomes from any cause.
All that being said, I tend _not_ to want to give anyone the benefit of the doubt here, owing to the terrible track record of Pfizer in these regards and the capture of the regulatory apparatus, at least in the US. I tend to put a lot of weight on the whistleblower's account because of my priors about Pfizer in this regard. Like Vinay Prasad said in his recent analysis of the "Thailand study" preprint on cardiac biomarkers in adolescents, I say that if these regulators had been doing their damn job, they would have done studies like that immediately following Pfizer's own trial.
And they would have found worse things than what this probably "nudged" trial data reveals.
https://www.preprints.org/manuscript/202208.0151/v1
I read the original Pf study protocol and related article, and I found only one point where the placebo was actually declared: one mention of the word "saline" in the article. Is that credible enough to accept that saline was actually used (e.g. as opposed to the AZ trials that used a meningitis vaccine to inflate control-group AEs)? I know - what else are we going to go on? But the control-group AEs seem remarkably high to me.
Thank you for your thorough work. If you want to talk Python data analysis, please feel free to get in touch.
Thank you for doing this work and your analysis.
In addition to pima’s comment, I would say the explanation for the distiribution of complications in the placebo group is as follows:
1. The placebo contained an active ingredient
2. The simple act of getting an injection (whatever the injectate) causes a reaction (maybe something akin to the surgical stress response) which results in complications
3. The complications were reported in a biased and/or fraudulent fashion
Maybe I’m not understanding something here but I would expect the SAE’s in the placebo group to be evenly distributed throughout the reporting period. In other words, if there were 111 SAE’s over a 10 week period then there should be roughly 11 per week.
By the way, a skewed distribution of SAE’s clustered around the first few weeks is actually ok if noted, discussed, explained and accounted for.
It certainly is bizarre but if an explanation is given the perhaps it is permissible.
I am presuming no such discussion is present in the paper
Yes! I have a similar question below.
I may be wrong but I read somewhere that the placebo was thought to be an MMR vaccine because it is known to have some side effects and therefore assists in blinding the trial . Like others commenting here I very much doubt the placebo was saline.
Does anyone know why the adverse events reported in the MHRA report differs in many numbers from the adverse events reported by the article "Six Month Safety and Efficacy of the BNT162b2 mRNA COVID-19 Vaccine" (https://www.medrxiv.org/content/10.1101/2021.07.28.21261159v1.supplementary-material)? MHRA e.g.:reported 4 SAE and the 6 months study only 3 ?
IMO, unless there has been a previous study of saline injection vs a mixture of the adjuvants, lipid nano particles, PEG, etc., it does not make sense to use the adjuvants, lipids, etc as the PLACEBO. By definition, isn’t the placebo supposed to be inert, innocuous, “of no physiological effect”?? If we don’t know that PEG plus lipid nanoparticles plus adjuvants are harmless and of no physiological effect, it does not make sense to use that mixture as the placebo in a drug trial. Suppose that mixture alone harms people? Then how can you say anything about the safety of that plus the nRNA and further, approve the mRNA produce when you have no idea if the “carrier mixture” for the mRNA is safe or not? In addition, haven’t we already learned that the LNP (lipid nanoparticles) do collect excessively in certain body parts like the ovaries? IMO, only saline makes sense for a placebo and a separate study (previously done or concurrently run) would be needed to determine the safety of the carrier mixture if not already known.