The International Academy of Classical Homeopathy (IACH) has chosen the study on the prevention of diarrhoea in piglets by Irene Camerlink and Liesbeth Ellinger as the best scientific article on homeopathy. This wonderful news was announced on the website of the Royal Dutch Patients Organisation for Homeopathy. The study has already got some attention on www.kloptdatwel.nl, a site of Dutch sceptics, but was not really discussed in detail. Now that this study has been awarded with the”IACH Research Award” by the institute of Vithoulkas himself, it’s time to take a closer look. Only for a moment, though, because it really isn’t worth that much attention.
George Vithoulkas is quite a hotshot in homeopathic circles. He received many distinctions amongst others in 1996 the Right Livelihood Award, “the alternative Nobel Prize“. Vithoulkas even dared to take up the ’1 million $ Challenge’ set by James Randi. This eventually never came to a real test and both gentleman blame each other of chickening out (JREF, Vithoulkas).
In 1995 Vithoulkas founded the International Academy of Classical Homeopathy on Alinissos, Greece. It’s mainly functioning as a summerschool for homeopaths, who come from all over the world for training. According to the website more than 9,000 homeopaths from 32 countries have visited the academy.
So it’s not that strange that being awarded the IACH Research Award is seen as a big applause by supporters of homeopathy.
The piglet study
Let’s take a look at the prize winning study now. Camerlink did the research as a student while doing a minor at the Organic Farming Systems Group of Wageningen University. The article was published in Homeopathy as “Homeopathy as replacement to antibiotics in the case of Escherichia coli diarrhoea in neonatal piglets” (here a freely available copy). Ellinger, a homeopathic vet, had been involved in other studies on homeopathic remedies by institutes and projects connected to the Wageningen University before. One of those earlier studies is on calves diarrhoea (2006).
The publication is about a trial in which 52 sows were divided into two groups. One group received a placebo. The other group received the homeopathic agent Coli 30K, a nosode prepared from various strains of E. coli bacteria. That’s one of those potions, which are diluted to such absurd levels that you wouldn’t be able to find a single molecule of the original stuff in the bottle you end up with. The main author, Camerlink, is well aware of this fact as she wrote to me in an e-mail. But she doesn’t see it as a big problem that there is no known scientific acceptabele explanation for the effects of homeopathy as claimed by homeopaths. Camerlink acknowledged to me that she expects a scientific breakthrough in the near future, which would give an explanation for the alleged workings of homeopathy.
Wrong blinding methode
Both placebo and verum were sprayed in the vulva of the sows twice a week during the last four weeks before giving birth. It’s more than likely that they used the same blinding technique as in the earlier mentioned study on calves diarrhoe: the “Wageningen method” as JW Nienhuys, secretary of Skepsis, calls it.
The researcher (or the farmer) gets two spray flacons labeled ‘A’ and ‘B’ and then sprays the sows according to random list with either ‘A’ or ‘B’. But this isn’t a good blinding method at all. In fact you reduce your test with 52 sows (N=52) to a test on two groups (N=2). The right way is preparing 52 different spray flacons, which contain either verum or placebo. It’s more work, but essential. You shouldn’t know which treatment a sow got, but you shouldn’t know which sows are in the same group as well.
Where’s the E.coli?
As outcome measure they looked at the number of the piglets, born from the 52 sows, who got diarrhoea caused by E.coli. At least, that’s what you would think considering the title of the article. In the article however, it is mentioned that they established E.coli infection by visual inspection of the faeces. Only one sample was sent to a laboratory to check properly for E.coli. And that sample came back negative… So we don’t even know if there was any E.coli caused diarroeha at all! Well, normally you could stop reading any further now to save you the trouble of searching for all kinds of methodological errors in the research, which are not very difficult to spot in this case, though.
Why many homeopaths think a RCT can help their cause and why they are wrong
The research as such didn’t interest me that much, because of these problems, but the attention given to the article by homeopaths, did. They mention the article as a true scientific proof for homeopathy. Because, they say, it’s a excellent performed Randomized Controlled Trial (RCT) which gave an undeniable positive significant result. And they proudly add that it was done at a university, with a very good reputation. For me this was the trigger to write an article to explain that such a claim (a single RCT with significant positive result is ‘proof’ ) is absurd in the case of homeopathy.
Even a perfectly designed and executed RCT can produce false positives. And with a verum which you can’t discern from placebo in any way (only by the label on the bottle), you’ll expect 5 percent of these RCT’s to be false positives. That’s still way too likely to happen and should prevent any serious scientist to conclude from a single significant positive outcome, that there really is a difference between verum and placebo and that we are just not scientifically advanced enough to measure it directly. The resulting article was “RCT, homeopathie en biggetjes: ‘The Good, the Bad & the Ugly’?“, which I didn’t translate yet. But you can find a similar argument in an article on the Science-Based Medicine blog: Plausibility bias? You say that as though that were a bad thing!
They needed some help on statistics
It was quite a surprise for me to read the news items on the websites of homeopathic organizations in the Netherlands concerning this award. A quote from the message on the site of the Royal Dutch Patients Organisation for Homeopathy translates as follows:
As result of the trial it was found that within the group treated with the homeopathic medicine only 10 out of 265 piglets got diarrhoea, in contrast to 63 out of the same number of piglets in the placebo group. This comes down to a significance p<0,0001 on piglet level and of p<0,05 on the level of sows.
The bold part of this quote is the correct statistical conclusion if you still want to analyze the data, although it does not represent what the researchers claim (because the lack of proof of E.coli and the faulty blinding). This outcome measure (diarrhoea on sow level) can’t be found in the article. The article only mentions the impressive p-value on piglet level. But that figure is a based on a serious misunderstanding of the statistics involved: it doesn’t take into account the fact that contamination between piglets of one sow is very likely. So it’s wrong to view the diarrhoea status of individual piglets as independent outcome values.
This was mentioned by myself in a comment on a blog on kloptdatwel.nl. Nienhuys, helpful as always, answered my remark with a calculation of the p-value on sow level, which if done correctly indeed shows p<0,05. But it’s a border case: if there had been only one more sow in the verum group with diarrhoea amongst its piglets, there would not have been a significant difference. Probably Camerlink picked up the better statistics from the blog after the discussion we had by e-mail.
Is this really the best thing homeopaths can show from recent research?
So, according to the International Academy of Classical Homeopathy this is the best study in recent time on homeopathy. At least it is the best of the articles send in for the IACH Research Award. How many did compete, I do not know (yet). Also I’m still unaware which famous and recognized homeopaths sat in the jury and whether there is a rapport of the jury explaining the choice for this terrible weak study. I’ve sent some e-mails to the institute in March and April. And every time they answered that they would put the information on the website soon. Until today I didn’t find any updates, however.
Original Dutch version Biggenstudie krijgt prijs: homeopaten blij met dode mus? published on April 4th 2012
header picture Flickr:woodleywonderworks (edited)
Published on May 20th 2012