Facts, not Fantasy

Tuesday, March 15, 2011

An interesting study of vaccinated vs unvaccinated children

A blog I follow called A Photon in the Darkness posted this.  I want to repost the entire thing for your benefit here, but please take some time to visit them and even follow them.  I will highlight something that struck me as particularly interesting/relevant.  I wonder if unvaccinated children also lead vaccinated kids in another important metric: death!?

An interesting study of vaccinated vs unvaccinated children

March 14th, 2011

Vaccinated children aren’t as sick as some people think.
The 18 February 2011 issue of Deutsches Aerzteblatt International had an interesting study [1] looking at the prevalence of allergic disease, asthma and non-specific infections (i.e. “colds”) in children ages 1 - 17. The interesting part was that they compared children who were completely unvaccinated to those who were vaccinated.

The study was carried out in Germany (some of you might have already guessed that) and looked at 13,359 children who had received at least one vaccination and 94 children who had received not a single vaccination. For those keeping score, the percentage of completely unvaccinated children was 0.7%. The subjects came from children covered in the Kinder- und Jugendgesundheitssurvey (Health Interview and Examination Survey for Children and Adolescents) carried out between 2003 and 2006.

What they found was that the unvaccinated kids led in one important metric: they had significantly more pertussis, measles and mumps than the vaccinated kids.

As for allergic disease (atopy), asthma and susceptibility to “colds”, there was no significant difference between the two groups.

So much for the “unvaccinated children have less allergic disease, less asthma and are sick less often than vaccinated kids” canard, eh?


No, I don’t actually think that the “true believers” in the virtues of communicable diseases will be convinced by this study. On the other hand, it is certainly “grist for the mill” for anyone who has an open mind.

So, what does this have to do with autism?
Now, I know that some of my faithful readers who have stuck with me through weeks and months of inactivity (’blog inactivity - I’ve been busy as heck doing other things) are wondering “What has this got to do with autism?” Well, the numbers are instructive for those who are proposing (or insisting on) a “vaccinated vs unvaccinated” study of autism. Even in Germany, which has a relatively homogeneous population, socialised medicine and centralised record-keeping, it was difficult to find very many completely unvaccinated children.
Still, the researchers managed to come up with statistically significant results, so what’s the problem? If they can do it with allergies, colds and asthma, it shouldn’t be that much of a stretch to do the same with autism, right?

Right?

Actually, no. It isn’t that easy.

Here’s the problem - problems, in fact.

First, the most obvious problem is that the small but hyper-vocal “horde” screaming for a “vaccinated vs unvaccinated” study in autism won’t be satisfied with any study that doesn’t find a correlation. We’ve seen this too many times before: a study shows no statistically significant correlation between autism and [fill in the blank] and the “horde” screams “Foul! Corruption! Conflict of interest!” etc.

To my mind, this is the single most significant reason to not do a “vaccinated vs unvaccinated” study: the only people who want one won’t be satisfied with the most probably outcome.

Secondly, there is the problem of statistical power. The statistical power (1 - β error) of a study is its probability of finding a real difference (i.e. rejecting the null hypothesis) when a real difference exists. The power of a study depends on a number of things:

[a] The α error probability threshold (often referred to as the “significance” level). This is typically set at 5% in medical and biological studies by consensus. The α error represents the probability of “finding” a difference when once doesn’t really exist. The more stringent the α error threshold, the lower the statistical power of the study will be (i.e.  the more likely it is to “find” no difference when one exists), given the same sample size.

[b] The size of the effect. When the effect is small or when there is a small difference in effect between the two groups, it is harder to detect. This should seem intuitive.

[c] The size of the sample. Because of random chance, it is always possible to select a sample from any larger group that is not representative of the group. Larger sample sizes give smaller β error, thus greater statistical power. This is because larger samples have a lower probability of being misrepresentative. For example:
If you have a bag containing 50% red marbles and 50% green marbles and only take two as your sample, there is a 50% chance that the two marbles you pick will both be the same color, giving you a false impression of the “population demographics” of the bag. If your sample is four marbles, your chance of getting all red or all green is less (12.5%).  
For the effect sizes and sample sizes used in the Schmitz et al study, the statistical power runs from 42% (allergic rhinoconjunctivitis 1 - 5 yrs) to 99% (pertussis). As expected, in the comparisons which failed to find a difference (i.e. all but the vaccine-preventable diseases), the statistical power was lower, primarily because the difference between the groups was small.

Let’s “run the numbers”!
To put this into the autism perspective, let’s “run the numbers” and see how the same samples would have fared in detecting any difference in autism prevalence between vaccinated and unvaccinated children.
If we assume that the prevalence of autism in the vaccinated population is 1% (given the small proportion of completely unvaccinated individuals in the general population - 0.7% in this study - that is a reasonable assumption), the sample in Schmitz et al is too small to detect a difference unless the unvaccinated group has a higher autism prevalence than the vaccinated group. This is because, with only 94 unvaccinated subjects and a general population prevalence of 1%, there are better than even odds that zero unvaccinated subjects would have autism, even if the prevalence was the same in both groups.

Why not just go out and beat the bushes looking for unvaccinated kids? The problem with that approach (similar to that tried in the past (and being tried today) by a number of anti-autism advocacy groups) is that the sample of unvaccinated children won’t be random.  So, what’s the problem with non-random selection? Let me explain.

In this case, soliciting for unvaccinated children, especially by or through groups researching autism or advocating for/against autism, could be expected to draw more attention from parents with autistic children. Moreover, those parents with unvaccinated non-autistic children would probably be more motivated to enter the study than those with autistic unvaccinated children. Thus, the sample would be skewed towards unvaccinated children without autism.

Possibly more concerning to those who desperately want to find a “vaccine-autism connection” is the possbility that, because they will be recruiting disproportionally among families with an autistic child, the prevalence of autism will be artificially high, even among the unvaccinated children.

The only ways to accurately determine the correlation of autism and vaccination are either to do a large random population sample (as in Schmitz et al) or to compare cases (autistic) to matched controls (non-autistic) for vaccination status.

Do want to win or do you want the correct answer?
Now, I’m sure that there are a lot of people in the “vaccines cause autism” camp that would welcome any result suggesting that vaccines cause autism, even if it was false. This is because they have confused science and sports. In sports, the most important thing (the only thing, if you listen to some fans) is winning. In science, however, the most important thing (again, some might say the only thing) is getting the correct answer.

To me, it is more important that we know whether or not vaccines cause autism than any personal preferences I have for any particular result. I may strongly suspect that vaccines don’t cause autism (based on data available to date), but I am perfectly willing to change my mind if the data go that way.

Bis später,

Prometheus

Citations:
[1] Schmitz R, et al. Vaccination Status and Health in Children and Adolescents. Dtsch Arztebl Int (2011); 108(7): 99–104

2 comments:

  1. Be careful what studies you believe. The study you are referencing was wrought with errors. I have indicated quotes directly from the study below.

    a) "...children who had by then received at least one vaccination according to their vaccination card were categorized as vaccinated." Children with only one vaccine were lumped in the same category as children with multiple vaccines, which leaves the possibility of improving the level of health in the vaccinated group.

    b) ‎"Study data regarding diseases may also depend on people’s ability to remember and on the assessment of those surveyed as well as on a medical diagnosis. We therefore cannot completely rule out that the true prevalence may be over- or underestimated." They did not use medical records to determine the level of health of the subjects. They used the recollection of the parents.

    c) "Subjects for whom at least one atopic disorder ever was reported were assigned the characteristic “atopic disorder.” There was no distinguishing sort of sick with really sick on the basis of atopic disorders.

    d) "Dr Schlaud was the lead investigator of an epidemiological study of deaths in children aged 2–24 months (TOKEN Study) in 2004–2009, which was jointly funded by the Federal Ministry of Health, the Paul Ehrlich-Institute, Sanofi Pasteur, and Glaxo Smith Kline.
    Dr Poethko-Müller was responsible for coordinating the study mentioned before." And as usual, the researchers spent 6 years working on another study funded by Sanofi and GSK - potential conflict of interest.

    ReplyDelete
  2. Since you did a copy paste of your questions "JJ", I shall copy/paste the answers from Prometheus:

    [a]“…children who had by then received at least one vaccination according to their vaccination card were categorized as vaccinated.”

    This is pretty much what the “We want a study!” crowd has been proposing: comparing completely unvaccinated children to those who have received vaccines. (LARIAN'S ADDITION: Again, never mind the huge ethical implications of such a study. Although, anti-vax pro-disease nutters don't seem averse to consigning people to death for their cause, so I can see how that would escape their ethics meter...)

    [b] While parental recall was used for atopic disease, asthma, “cold” and vaccine-preventable diseases, the vaccination cards were used for the vaccines.

    I think that we can be reasonably satisfied that parents would recall atopic disease, asthma and vaccine-preventable illnesses as these are pretty memorable. I’ll give you the point for “colds”, but I see that as comparing not actual incidence of “colds” but comparing the parental perception. Since we often hear that unvaccinated children “aren’t sick as often”, it was interesting to see that this perception doesn’t hold up to scrutiny (i.e. the parents of vaccinated kids felt, on average, that their children weren’t sick any more often than those of unvaccinated children).

    [c] “There was no distinguishing sort of sick with really sick on the basis of atopic disorders.”

    True enough, but atopic disease isn’t that easily quantifiable, especially as it waxes and wanes.

    [d] “And as usual, the researchers spent 6 years working on another study funded by Sanofi and GSK - potential conflict of interest.”

    Ah! The smoking “conflict of interest” gun!

    Sorry, but having other studies funded by pharmaceutical company doesn’t mean that this study was “corrupted”. Do you have some indication that the results are anomalous and may have been “tampered with”?

    In the research world - especially in Europe - no funding source is good enough to take the risk of fraudulent research. Look what happened to Andy Wakefield and Scott Reuben. If you have evidence of fraud or bias, please bring it forward.

    Bottom line - this study isn’t perfect, but it certainly argues (strenuously) against the oft-stated but never-supported claim that unvaccinated children are “healthier” than unvaccinated children. It also should suggest to the “We want a study!” folks what the likely outcome would be if they got their wish.

    ReplyDelete

Please keep posts here respectful. Those that cross boundaries will be deleted, and then placed in a special place for future ridicule.