Remedies that defy the facts

Robert Matthews

• Last Updated: November 28. 2009 7:42PM UAE / November 28. 2009 3:42PM GMT

Homeopathic medicines are made from extremely dilute solutions of the active ingredient but have never been tested in large-scale placebo controlled trials. AP / Merck / ho

Asked to name the biggest medical breakthroughs of the last century, many doctors would probably include EBM. It’s not a drug or some hi-tech brain scanner, it’s a concept: evidence-based medicine.

The very idea that a life-or-death practice such as medicine hasn’t always been evidence-based is alarming. More alarming still is the fact that while many therapies are now backed by hard scientific evidence, many others are not.

But as two recent controversies have shown, even when evidence does exist, it can fail to resolve often bitter arguments over efficacy.

Earlier this month, medical experts in the US came out against women under the age of 50 receiving routine X-ray screening for breast cancer. Put simply, they argued that the evidence that women in that age group benefit from such mammography is outweighed by the evidence of harm, in the form of false alarms, unnecessary surgery and exposure to radiation.

While the team stressed that mammography remained worthwhile for the over-50s, and could be justified on a case-by-case basis for younger women, its recommendations sparked a furore. Critics of President Obama’s US healthcare reforms insisted they were evidence of the government putting lives at risk by cutting back on medical services (despite the fact the expert panel is independent of both the government and health insurance companies).

More seriously, the panel’s evidence-based recommendations were immediately rejected by the American Cancer Society and the American College of Radiology, and, ironically, by the US health secretary Kathleen Sebelius, in a move condemned by some as a craven attempt to assuage critics of its reforms.

Meanwhile, medical experts pointed out that the evidence behind the panel’s recommendations was hardly new. It’s been known for years that just one life would be saved through the routine breast cancer screening of 2,000 women in their forties over the course of a decade, at a cost of more than 1,000 false alarms. Mammography lacks the reliability to overcome the relatively low risk of breast cancer among that age group. It’s a manifestation of the so-called “low base rate effect”, routinely taught in courses on medical statistics.

In short, the evidence may be simple and compelling, but it gets ignored because it doesn’t suit some people’s agendas. And that is one of the problems evidence-based medicine can’t solve: the reluctance of people to disregard evidence that doesn’t suit their agendas.

This is precisely the accusation that was levelled last week at practitioners of homeopathy, a form of complementary medicine currently being investigated by the UK parliament’s select committee on science and technology. The committee wants to establish whether Britain’s health service should be funding homeopathic treatment, which many scientists regard as useless.

Invented around 200 years ago, it gets its name from the idea of “like curing like”, with preparations of natural substances that cause, say, fever being used to treat a fever. What makes homeopathy so controversial is the fact that most of the preparations are made from incredibly dilute solutions of the original substance – so dilute, in fact, that not a single atom of it still resides in the phial of water or alcohol given to patients.

While various suggestions for how homeopathy could still work have been put forward, it’s fair to say that none of them is especially compelling. In any case, the real issue is not how it works, but whether it does – and many conventional medical experts reject this possibility out of hand.

Resolving the debate calls for a hefty dose of EBM, and over the years hundreds of studies have been carried out into the effectiveness of homeopathy. The most impressive of these are so-called randomised placebo-controlled trials, in which groups of patients are randomly allotted to receive either the homeopathic remedy or a placebo – a harmless solution containing no active ingredient.

More than 100 such trials have been performed, and according to medical experts attending last week’s select committee meeting, homeopathy repeatedly fails to show any benefit beyond that of the placebo. In other words, if the treatment does provide any benefit, it’s all in the mind.

Some might argue it hardly matters how patients get better, as long as they do. But leaving that ethical question aside, the evidence against homeopathy seems pretty compelling, and its implications can only be evaded by ignoring it.

There are, however, some perfectly good reasons why most of the supposed evidence on homeopathy can and should be treated with suspicion. Of the 100-plus trials, the majority involved fewer than 100 patients. Basic statistical theory shows that such small trials have little chance of detecting anything but the most amazing cure rates – far exceeding those achieved even by most conventional medicines. In essence, assessing any remedy via small trials is like trying to gauge the quality of a book by taking a random poll of a handful of people. You may well reach a result, but it is unlikely to be reliable.

Yet this hasn’t stopped sceptics of homeopathy stretching the powers of EBM beyond breaking point, by claiming that negative or inconclusive small trials is evidence that the therapy is useless. It may well be – but such trials cannot demonstrate it. To argue otherwise is to confuse absence of evidence with evidence of absence – a basic statistical fallacy.

One way around the inadequacy of small trials is to combine all their results to create the effect of a single, huge trial. This so-called meta-analysis has been used to great effect in conventional medicine; for example, it helped alert doctors to the benefits of “clot-busters” in protecting patients from strokes. But in the case of homeopathy, meta-analysis is fraught with difficulties. Many of the trials have quirks with the power to turn any combination of them into an exercise in GIGO: garbage in, garbage out.

The truth is that there’s no substitute for single, huge randomised placebo-controlled trials of homeopathic remedies, involving many hundreds or even thousands of patients. Such mega-trials are routine in conventional medicine, but there has yet to be one in homeopathy. That’s because such remedies aren’t produced by giant pharmaceutical companies with the money to fund such trials.

Maybe one day someone will come up with the hefty sums needed to carry out mega-trials of homeopathy, so the truth can begin to emerge.

In the meantime, doctors and patients alike must live with the fact that, while it’s better than anecdote-based medicine, EBM is still no panacea.

Robert Matthews is Visiting Reader in Science at Aston University, Birmingham, England