The Bull Shit Detection Rules…

Every now and then I like to read some ‘other’ stuff to get away from the relentless physio research papers, and thanks to a recommendation by a friend I have been reading a book called “The Demon Haunted World”. Now this this isn’t some fantasy novel about dragons and warlocks, but is a best seller by Carl Sagan about the scientific method, and why we all need to develop critical thinking.

From the background reading I have done on Carl Sagan it appears he was an extremely interesting, intelligent and charismatic individual. Not only a brilliant scientist, but also a great philosopher, sociologist, psychologist, critical thinker, poet, sage, writer and orator with a fantastic way with words. His ‘pale blue dot’ being one of his most famous works, and is well worth a few minutes of your time to put you and your place in the world into perspective, listen to it here.

In ‘The Demon Haunted World’ Sagan describes some methods to help us distinguish between ideas that are considered valid science, and ideas that are pseudoscience of which there is a lot found in healthcare and pain treatment. In a chapter called ‘The fine art of baloney detection’ he discusses how we are all susceptable to many types of deception, from sales and marketing, to religious beliefs and faiths, and of course false scientific claims. Sagan also reminds us that falling for these things doesn’t make us stupid or bad people, but emphasises that we need to equip ourselves from being fooled time and again.

To do this Sagan suggests we follow nine key rules to detect baloney, what I rather less eloquently call bull shit, and after some recent discussions I have had on here on trigger points, dry needling, manual therapy, manipulations, neon coloured stretchy tape, corrective exercises, functional motor control etc etc, I thought I would share Sagan’s rules hopefully to help us all be a little better at spotting the bull shit often promoted as science in this profession of ours…

So here goes, the bull shit detection rules…

  1. Wherever possible there must be INDEPENDANT confirmation of the “facts.”
  2. Encourage substantive debate on the evidence by knowledgeable proponents of ALL points of view.
  3. Arguments from authority carry little weight — “authorities” have made mistakes in the past. They will do so again in the future. In science there are NO authorities; at most, there are experts.
  4. Spin more than one hypothesis. If there’s something to be explained, think of ALL the different ways in which it could be explained. Think of tests which you might systematically DISPROVE each of the alternatives. What survives in this Darwinian selection among the “multiple working hypotheses,” has a much better chance of being the right answer than if you simply run with the first idea that caught your fancy.
  5. Try not to get overly ATTACHED to a hypothesis just because it’s yours. Ask yourself why you like the idea. Compare it fairly with ALL the alternatives. See if you can find reasons for  REJECTING it. If you don’t, others will.
  6. Quantify. If whatever it is you’re explaining has some measure, some numerical quantity attached to it, you’ll be much better able to discriminate among competing hypotheses. What is vague and qualitative is open to MANY explanations. Of course there are truths to be sought in the many qualitative issues we are obliged to confront, but finding them is more challenging.
  7. If there is a chain of argument EVERY link in the chain must work (including the premise) — not just most of them.
  8. Occam’s Razor. This convenient rule-of-thumb urges us when faced with two hypotheses that explain the data equally well choose the SIMPLER.
  9. Always ask whether the hypothesis can be, at least in principle, FALSIFIED. Propositions that are untestable, unfalsifiable are not worth much. You must be able to check assertions out. Inveterate skeptics must be given the chance to follow your reasoning, to duplicate your experiments and see if they get the same result.

So there you go, some simple guidelines to help you work out the next time you are listening to someone explain how great this ‘thing’ is in helping people with pain or dysfunction, or are arguing the toss about how tape changes biomechanics, or fascia is released with a certain technique or tool, remember these rules.

Unfortunately for most claiming the effects or results they won’t hold up to scrutiny of these rules. In fact many are not able to go past rule No 1. And remember when you do question those claims and as often happens they or others then turn on you for being too critical, too skeptical, too closed minded or negative, first thank them for the compliment, and remember that there is NO such thing as being too critical or too skeptical, it is simply following the scientific method.

As always thanks for reading



4 thoughts on “The Bull Shit Detection Rules…

  1. Hi Adam,

    Great post as per usual. I love reading and/or listening to anything by Carl Sagan. In my opinion one of the great minds, and well ahead of his time. As many of the great minds are.

    I used a Carl Sagan quote in a recent “Professional Issue” paper that I wrote as part of some study I was doing:

    “When Kepler found his long-cherished belief did not agree with the most precise observation, he accepted the uncomfortable fact. He preferred the hard truth to his dearest illusions; that is the heart of science.”

    It would be nice if of all our profession (and science generally) would adopt this approach to practice and research. Pleasingly some already do, but many don’t……as we can regularly see in the comments section on this blog of yours! 🙂

    Thanks again.


    • Thanks for your comments Mark, and yes cognitive dissonance is an unpleasent feeling for many. I know I have encountered it many times and am sure I will encounter it again!

      As you highlight questioning is at the heart of both science and philosophy, which I find myself becoming more and more engrossed in, which I never thought possible, especially philosophy.

      Many wont and don’t like it and resistance will be strong, but as both a nerd and a geek, I will use a quote from Star Trek…

      Resistance is futile!!

  2. I would add an additional rule: Claims made by people who benefit directly in tangible ways from those claims being believed to be true should be subject to extra scrutiny. This is why we require statements of financial interest by authors in published scientific papers.

  3. Have just started following you. I recently read your dinosaur paper. After starting to feel a little disillusioned with my profession it’s nice to find out that there are plenty of other people asking the same questions and challenging the accepted.

Please leave your comments here...

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s