Clinical Relevance

I thought it might be fun and, hopefully, thought provoking to bring up a term that’s thrown around in medicine, especially by statisticians, but rarely, so it seems, in day to day practice.  Statisticians, in case you don’t know one, happen to have one of the most impenetrable jobs around, but also one that’s very, very important, at least when it comes to analyzing the results of scientific tests.  And it’s a term that should be very important to you if you’re considering trying just about anything on your horse.

The term is, “Clinical Relevance,” sometimes also known as, “Clinical Importance.”  When people are doing investigations looking into whether this or that therapy is effective, one of the first questions that gets asked is, “Did the thing that you were studying make a difference?”  Researchers set up test conditions, test one thing or another, and then analyze the data to see if there is a difference in the outcomes.  All of that data analysis is done by statisticians, using one of any number of virtually incomprehensible mathematical methods.  At the end of the day, based on the results of the analysis, veterinarians like me, who have to put this stuff into practice – and who, by the way, usually aren’t very good at doing or understanding those very same incomprehensible math methods – usually get told that there was or wasn’t a “statistically significant” difference as a result of the study.  And that’s one way that some people make up their minds on whether or not to try a therapy.

Usually, when there is a statistical difference, this is cause for some sort of celebration back at the lab.  I mean, honestly, it’s not like veterinarians aren’t aware that in the battle for equine health, we are often poorly armed, so we’re always hoping that there’s going to be some new weapon come along to help beat arthritis, colic, or whatever.  When statistics indicate that there’s a statistical difference, we’re usually pretty happy to try to put that thing into practice.  (I can’t say what kind of celebrations most statisticians engage in – mostly, they just move on to the next study, I think.)  Of course, many therapies aren’t supported by good studies – or good statistics – at all, but that’s a whole ‘nother kettle of fish.

The problem, at least from a treatment standpoint, is that the wonderful new therapy often doesn’t make that big a difference.  It’s rare that there’s some wonderful new discovery that’s going to change equine medicine altogether.  There have been a couple.  Penicillin – the first antibiotic – was one of those things.  Anesthetic gases that horses could inhale revolutionized equine surgery.  But more often than not, what you get, when it comes to medicine, is something that might or might not make that much of a difference, even if there is some measurable difference in the results of a study.

This is where clinical relevance comes in.  When it comes to deciding whether or not to use a treatment – especially if it’s an expensive one – the question that you should be asking is how much of a difference a treatment will really make.  It’s not just important to know that there’s some difference – you want to know how big of a difference.  For example, you might find some treatment that has passed statistical testing, but every time you try it the horse tries to kick you.  Might be wonderful, just not practical.  Or a treatment or test may be too expensive.  Even if you use a particular treatment, the horse may not get that much benefit from it.  And that is a really important thing to understand.

It is always something of a leap of faith when it comes out to trying something on your horse.  For one, your horse wasn’t in the study.  But before you decide to go all in on some treatment, you want to make sure that the effects of the treatment are big enough to change your normal way of doing things.

Here are a couple of examples of what I’m talking about.  Let’s say that someone says that your horse should be treated with a laser (they seem to be everywhere, and I have to write an article about them before too long).  Without getting into all of the different kinds of lasers, and treatment regimens, and conditions that can be treated, when it comes to clinical relevance, the big questions are, “Am I going to end up anywhere different that where I would have ended up if I just let things take care of themselves,” and, “How much is this going to cost?”

I mean, let’s say that your horse has some sort of a cut.  In fact, let’s say your horse has a really BIG cut.  Let’s say he’s got this cut.  You’re going to get lots of opinions – you may have lots of opinions – on what to do.  Antibiotics.  Laser.  Hospitalize the horse.  Amnion.  Bandage.  Scarlet Oil.  Violet spray.  Silver spray.  Nitrofurazone spray (it’s yellow).  Some other color spray.

You get the point.

Here’s what was done.  The wound was washed out with a garden hose from time to time.  Healed up in about 10 weeks, as I recall.  Left a linear scar that didn’t bother the horse a bit.  He’s fine.  Kind of hard to imagine what all of that other stuff might have added, other than expense.

Once you start thinking in terms of clinical relevance, all sorts of lights go on.  Take the hottest trend in “sports medicine” right now, the so-called “regenerative” therapies.  This slyly named group of therapies – the name certainly implies that the therapies can return tissues back to some semblance of normal – holds all sorts of “promise” for the treatment of all sorts of injuries.  However, there’s currently a serious lack of evidence that they actually do anything important at all beyond the time that it takes for injuries to heal anyway.  So, if you’re considering using such therapies – and the appeal is undeniable, dating at least back to Juan Ponce de Leon, in the early 1500’s – it would probably be wise to ask if they are clinically relevant, that is, if they are going to make any difference on where your horse ends up in the long run (at this point, there’s little evidence that they do).

When it comes to your horse, there’s a lot of stuff that you can do.  What’s important, however, is doing the stuff that really makes a difference.  Just because you can do something doesn’t necessarily mean that you should.  Ask if the something that you’re considering is clinically relevant.  You might be surprised at the reaction.