March 14th, 2011Vaccinated children aren’t as sick as some people think.
The 18 February 2011 issue of Deutsches Aerzteblatt International had an interesting study  looking at the prevalence of allergic disease, asthma and non-specific infections (i.e. “colds”) in children ages 1 - 17. The interesting part was that they compared children who were completely unvaccinated to those who were vaccinated.
The study was carried out in Germany (some of you might have already guessed that) and looked at 13,359 children who had received at least one vaccination and 94 children who had received not a single vaccination. For those keeping score, the percentage of completely unvaccinated children was 0.7%. The subjects came from children covered in the Kinder- und Jugendgesundheitssurvey (Health Interview and Examination Survey for Children and Adolescents) carried out between 2003 and 2006.
As for allergic disease (atopy), asthma and susceptibility to “colds”, there was no significant difference between the two groups.
So much for the “unvaccinated children have less allergic disease, less asthma and are sick less often than vaccinated kids” canard, eh?
No, I don’t actually think that the “true believers” in the virtues of communicable diseases will be convinced by this study. On the other hand, it is certainly “grist for the mill” for anyone who has an open mind.
So, what does this have to do with autism?
Now, I know that some of my faithful readers who have stuck with me through weeks and months of inactivity (’blog inactivity - I’ve been busy as heck doing other things) are wondering “What has this got to do with autism?” Well, the numbers are instructive for those who are proposing (or insisting on) a “vaccinated vs unvaccinated” study of autism. Even in Germany, which has a relatively homogeneous population, socialised medicine and centralised record-keeping, it was difficult to find very many completely unvaccinated children.
Still, the researchers managed to come up with statistically significant results, so what’s the problem? If they can do it with allergies, colds and asthma, it shouldn’t be that much of a stretch to do the same with autism, right?
Actually, no. It isn’t that easy.
Here’s the problem - problems, in fact.
First, the most obvious problem is that the small but hyper-vocal “horde” screaming for a “vaccinated vs unvaccinated” study in autism won’t be satisfied with any study that doesn’t find a correlation. We’ve seen this too many times before: a study shows no statistically significant correlation between autism and [fill in the blank] and the “horde” screams “Foul! Corruption! Conflict of interest!” etc.
To my mind, this is the single most significant reason to not do a “vaccinated vs unvaccinated” study: the only people who want one won’t be satisfied with the most probably outcome.
Secondly, there is the problem of statistical power. The statistical power (1 - β error) of a study is its probability of finding a real difference (i.e. rejecting the null hypothesis) when a real difference exists. The power of a study depends on a number of things:
[a] The α error probability threshold (often referred to as the “significance” level). This is typically set at 5% in medical and biological studies by consensus. The α error represents the probability of “finding” a difference when once doesn’t really exist. The more stringent the α error threshold, the lower the statistical power of the study will be (i.e. the more likely it is to “find” no difference when one exists), given the same sample size.
[b] The size of the effect. When the effect is small or when there is a small difference in effect between the two groups, it is harder to detect. This should seem intuitive.
[c] The size of the sample. Because of random chance, it is always possible to select a sample from any larger group that is not representative of the group. Larger sample sizes give smaller β error, thus greater statistical power. This is because larger samples have a lower probability of being misrepresentative. For example:
If you have a bag containing 50% red marbles and 50% green marbles and only take two as your sample, there is a 50% chance that the two marbles you pick will both be the same color, giving you a false impression of the “population demographics” of the bag. If your sample is four marbles, your chance of getting all red or all green is less (12.5%).For the effect sizes and sample sizes used in the Schmitz et al study, the statistical power runs from 42% (allergic rhinoconjunctivitis 1 - 5 yrs) to 99% (pertussis). As expected, in the comparisons which failed to find a difference (i.e. all but the vaccine-preventable diseases), the statistical power was lower, primarily because the difference between the groups was small.
Let’s “run the numbers”!
To put this into the autism perspective, let’s “run the numbers” and see how the same samples would have fared in detecting any difference in autism prevalence between vaccinated and unvaccinated children.
If we assume that the prevalence of autism in the vaccinated population is 1% (given the small proportion of completely unvaccinated individuals in the general population - 0.7% in this study - that is a reasonable assumption), the sample in Schmitz et al is too small to detect a difference unless the unvaccinated group has a higher autism prevalence than the vaccinated group. This is because, with only 94 unvaccinated subjects and a general population prevalence of 1%, there are better than even odds that zero unvaccinated subjects would have autism, even if the prevalence was the same in both groups.
Why not just go out and beat the bushes looking for unvaccinated kids? The problem with that approach (similar to that tried in the past (and being tried today) by a number of anti-autism advocacy groups) is that the sample of unvaccinated children won’t be random. So, what’s the problem with non-random selection? Let me explain.
In this case, soliciting for unvaccinated children, especially by or through groups researching autism or advocating for/against autism, could be expected to draw more attention from parents with autistic children. Moreover, those parents with unvaccinated non-autistic children would probably be more motivated to enter the study than those with autistic unvaccinated children. Thus, the sample would be skewed towards unvaccinated children without autism.
Possibly more concerning to those who desperately want to find a “vaccine-autism connection” is the possbility that, because they will be recruiting disproportionally among families with an autistic child, the prevalence of autism will be artificially high, even among the unvaccinated children.
The only ways to accurately determine the correlation of autism and vaccination are either to do a large random population sample (as in Schmitz et al) or to compare cases (autistic) to matched controls (non-autistic) for vaccination status.
Do want to win or do you want the correct answer?
Now, I’m sure that there are a lot of people in the “vaccines cause autism” camp that would welcome any result suggesting that vaccines cause autism, even if it was false. This is because they have confused science and sports. In sports, the most important thing (the only thing, if you listen to some fans) is winning. In science, however, the most important thing (again, some might say the only thing) is getting the correct answer.
To me, it is more important that we know whether or not vaccines cause autism than any personal preferences I have for any particular result. I may strongly suspect that vaccines don’t cause autism (based on data available to date), but I am perfectly willing to change my mind if the data go that way.
 Schmitz R, et al. Vaccination Status and Health in Children and Adolescents. Dtsch Arztebl Int (2011); 108(7): 99–104