Why Therapies May Seem to Work (Even When They Don’t) – Part 1

Here’s a line of thinking that just makes me just nuts.

My horse was an absolute wreck until the (YOU SUPPLY THE TREATMENT HERE) fixed him.  Seeing is believing.  It works!”

Well, actually – no.  I mean, it could be yes, but it’s often a resounding, “No.”  And heaven forbid that you suggest that it’s, “No” to someone who is sincerely convinced that a product has an important and useful effect on some condition of the horse – at the very least, you’ll get a lot of comments.

One reason that many products and treatment methods remain on the market is because of this simple, and not entirely irrational, thought process:  “I tried it, my horse got better, so it works!”  Unfortunately, even when a horse’s problem improves following treatment, this, by itself, often cannot prove that the therapy was responsible for the improvement.  Here’s an example.  In other times, people did rain dances when the weather was dry and occasionally, it must have rained.  Thus, they kept on dancing.  That’s the same bit of logic I’m talking about as applied to determining whether treatments really work, or not.

There are some ethics involved here, as well.  I believe that people who provide treatments and therapies for animals have a moral and ethical obligation to prove, first, that they are safe and, second, that they are effective prior to selling them to horse owners (NOTE:  Not everyone agrees, as evidenced by the buckets of BS that are currently being peddled).  It’s usually not that hard to demonstrate that a therapy doesn’t do any harm to an animal – the treatment is given, the animal doesn’t die, and there you go.  Proving that it’s effective can be quite another matter, even if effectiveness is easy to claim.

It’s can hard to prove that something is really effective, for many reasons.  Complicating things is the fact that there are many subtle ways that honest and intelligent people can be led to think that a treatment has been effective when, in fact, it wasn’t effective.  This is a problem for both the giver and the receiver of the therapy. This fact applies across the board; false claims and jumping to conclusions about the effectiveness of treatments are a problem whether the treatment that is being assessed comes from scientific medicine, old folk remedies, or fringe treatments often described as “alternative” or “complementary.”

Here’s my philosophy.  Unless a treatment is tested for its true effectiveness, it is ethically questionable to offer that treatment to the public, especially if money is going to be exchanged.  As a controlled experiment, using an unproven therapy is another thing entirely, of course.  But unless something is tested first, you’re essentially paying for your horse to be in an uncontrolled experiment, and one in which no one is really keeping track of the results.  Of course, that fact seems to rarely dissuade people from trying the latest “new” therapy.  But keep in mind that the history of veterinary medicine is littered with all sorts of ineffective therapies that were sold on nothing but a promise.

SERIOUS ASIDE #1:  Unfortunately, when there’s money to be made, ethics often go out the window.  Economics trumps science just about every time.  It’s a problem.

First, we need to define two terms.  Something can occur in association with something else; a butterfly flaps its wings, and a hurricane appears in the world shortly afterwards.  That’s a correlation – you can’t say for sure that one caused the other (well, I suppose you could, but that’s a philosophical question).  A causation is simple; you hit your toe on the side of the counter in the kitchen, and your toe hurts.  One causes the other.

Lots of people confuse the two.  But mistaking a correlation with a causation is a basic fallacy of logic known as post hoc, ergo propter hoc (“It happened after, so it was caused by”). People have a tendency to assume that when things occur together, they must be connected. This is not necessarily true; if it were there wouldn’t be any point in writing this article.

For example, it’s been shown that people who drink diet soft drinks are overweight. Does this mean that the diet drinks cause obesity? No, it doesn’t.  They could, but the question is a lot more complicated than that.  Under any circumstances, when people count on their personal experience as the sole method of evaluating the success (or failure) of a therapy, it is virtually impossible to determine cause and effect.

In medicine, individual givers or recipients of a therapy can almost never know if things would have gone just as well if a particular therapy hadn’t been given (except in certain special circumstances).   That’s why scientists do studies – to see if therapies are really effective.  So, the obvious question is, “Why is it so hard to tell?”

There are several good reasons.

1.  The problem may have run its natural course. Ever hear the phrase, “Time heals all wounds?” One of the great things about medicine is that many problems take care of themselves (unless the problem is chronic, or fatal).  That’s the way the horse’s body works.  And thank goodness, too.  In fact, sometimes – say in a case of mild colic – the problem takes care of itself so quickly that it’s a challenge for me to get there before the horse recovers on its own (so I can get the credit).

Just because a horse seems to improve after a treatment, it doesn’t mean that the treatment caused the improvement.  In order to show effectiveness, you have to show, in a group of similar patients, that the patients that seemed to get better with treatment exceed the number of patients that got better without any treatment at all (or that they get better faster than if they were just left alone).  Unless records of successes and failures over a sufficiently large number of patients with the same complaint can be studies, you usually can’t know that a treatment was better than just leaving the condition alone.  It’s kind of a pain, but that’s what medical research is all about!

This is a problem for every therapy.  It’s as true for acupuncture and chiropractic treatments as it is for stem cells, platelet-rich plasma injections, or shockwave therapy.  You can’t know if a therapy works without testing it.

SERIOUS ASIDE #2 – You can sure sell a treatment without proving it works.  Every day.

2. Many diseases are cyclical. Conditions such as arthritis, allergies and gastrointestinal problems have ups and downs.  Your horse is better on some days than on others.  Most people look for treatment during the downturns, when things are the worst. Thus, treatments have an opportunity to coincide with upturns that might have happened anyway.  Heck you’ve probably seen this sort of thing when you take your car to the mechanic – the problem that was making you crazy for a week is suddenly gone when you take it to the shop (and then it comes back when you leave).

Bottom line?  It’s easy to misinterpret improvement due to a normal cyclical variation as a valid therapeutic effect.

3.  Placebo effects. You may have heard that the placebo effect does not exist in horses since they don’t know what treatment they are getting.  However, their owners sure know. In human medicine, research has shown that up to 70 per cent of medical/surgical patients reported good results from techniques that we know today are ineffective (at the time of the treatment, both the patient and the physician were convinced that the treatment was effective).  Furthermore, the pages of history are filled with examples of what, with the benefit of hindsight, seem like crackpot procedures that were once enthusiastically endorsed by physicians and patients alike.

Placebo effects are especially strong when it comes to treatment goals such as “better performance,” “improved disposition,” “increased healing,” or other crazy supplement claims.   To overcome placebo effects, it is essential that good scientific tests be conducted.

SERIOUS ASIDE #3 – Many products and services available for horse owners are nothing more than elaborate placebos.

4. Some problems are psychosomatic to begin with. I don’t know if this describes you, but I often run into horse owners who have horses that are in remarkably good health, but are still be concerned that something is wrong with his or her horse.  This is a huge concern among owners of performance horses, who immediately medicalize their concerns (e.g, “He’s not jumping right – he needs his hocks and stifles injected). Getting the hocks injected is apparently a lot easier than, say, learning to ride better, or giving your horse a break because he’s had a bad day.

I had a personal experience with this a number of years back.  A trainer asked me inject a horse’s hocks because it wasn’t jumping right, or was swapping leads, or something.  Let’s say I was supposed to see the horse on a Tuesday.  Tuesday came and went, and I didn’t get near the place because of emergencies.  So, I show up Thursday, and I’m greeted by the trainer, who says, “He’s jumping so much better!  Thank you!”

Since you’re wondering, yes, I did eventually tell her.

The point that I’m trying to make is that one of the difficulties in trying to measure whether a treatment works for vague complaints is that they can be alleviated by support and reassurance.   If the problem wasn’t there in the first place, supplying reassurance and support to the owner can help solve the “problem.”

Furthermore, some people seem to be just absolutely determined to find the cause of a problem, even if a problem can’t be shown to exist.  If conventional techniques fail to come up with an acceptable reason for the perceived problem, there’s always someone who can find some sort of “imbalance” to treat, a rib that is out of place, an acupuncture point to poke, a supplement to give, a joint to inject, or a vertebra to “adjust.”

That’s the downside to giving medical diagnoses and treatments for vague psychosomatic complaints.   It provides virtually unlimited opportunities for bogus products, devices, and treatments.  And, when someone’s concerns are alleviated, the product “works,” even if there was never a problem in the first place.  And so it sells, until the next newest and best treatment comes along.

There’s a lot more to this problem.  So much that there’s a whole ‘nother article with other reasons why therapies may seem to work, even when they don’t.  If you’re interested, CLICK HERE FOR PART TWO