November 06 Research Article-Analysis-Comments

COMMENTS

The following comments have been made about this analysis from online yahoo chat groups;

USMedicalMassage Re: Reseach Fraud-Report or Ignor?

SEAN SAYS;

The questions I would like an answer to are;

1.) Is this research fraud?

After having to get through your rather verbose analysis, I still I'm unconvinced of "fraud" per se. It is apparent the study you cite is flawed, perhaps severely in some areas. Every study contains some flaws. It's always a judgement call to the reader as to whether these flaws will impact the logical conclusion that the reader might make to the study. Still, not fraud, just bad science, even if it is done repeatedly. Just like p values indicating outcomes of chance, perhaps there should be s values as to just how much researchers "suck" at what they do. Consistently "sucking" at what you do (research) is still a plausible reason as to why there are repeated flaws in a study, and why one could draw poor conclusions of the data. But still, "sucking" is not fraud. Same goes with the author defending her work. She just may suck at defending it.

While your diction throughout your analysis implies some bias in bashing the author, your paper does bring out classic problems in the studies of therapies in general. I disagree the blinding (esp. double) could have been attained even

if difficult. Unless you know something that I don't know, I were the therapist administering the massage treatment, I'd know I was administering massage treatment. Same goes to the subjects receiving the treatment. True, laser can be sham, however, touch is impossible.

Assessments (tests) that include a "self-reporting" section are always subject to criticism. They will never be 100 % valid, nor reliable, because of their subjectivity, however, when it comes to pain measurement, pain is now appearing to be more of a subjective experience rather than an objective physiological process occuring within the body. Ramachandran classified such experinces in the realm of "qualia", and even as a nuerologist, could never objectively quantify them. My point of all this is that sometimes when testing subjective experiences, all you have to work with is subjective assessments, esp when testing therapies. Is it good science? Not the best by anymeans, but to sit on our butts an not do any research until 100% valid and reliable assessments are invented stinks of it's own smell of unethical behavior do to inaction.

My opinion is you'll never produce a RCT with the Sackett level "I" you are searching for because of the nature of human interaction through therapies. It may be attainable in pharmacuetical trials, but not therapies. Being able to not attain this level of evidence, though, is not fraud.

Regards

Sean

Ted’s Reply;

Of course using the word fraud is a bit like holding up a red herring in that it may draw attention away from the more subtle yet equally important questions. For example, Is it important to establish a scientific basis for medical massage technique? What kind of scientific studies do we need to accomplish this task? Does the study in question do the job adequately? Is the goal of evidenced based practice desirable? Does this research facilitate evidenced based practice? Is it in the best interest of science, in general, and the medical massage therapist, in particular, to have ethical research? Is it important to produce research which physicians and the larger scientific community can trust?

This study was a first of a kind large scale historic study on the use of medical massage technique on chronic low back pain. According to its author in e-mails to me, there have been no follow-up studies of similar magnitude. Point is this is an important piece of research.

Sean and I probably agree that this particular study was on the suckie side as far as research design and methodology. Pedro rated this research as a 6 out of 10 possible. It got about a C +. Otherwise the current study does a fair job of establishing that medical massage techniques when combined with exercise are better than no treatment. Those of you who use exercise therapy with your medical massage technique already know this. What about medical massage alone was that as good as combining it with stretching/walking ect. This research did not establish this fact as successfully and after one month follow-up did not establish that at all. Clinical experience would probably lean most of us in the to believe that the combo is better.

Is this something you could tell your physicians colleagues/clients? Namely, that this research demonstrates medical massage technique with exercise is better than no treatment.

The researcher says no, you may not make that assertion unless you graduated from a Canadian massage school (or comparable) that teaches these techniques and are registered and experienced (10 years) by the Canadian College of Massage Therapists. Once those preconditions are established, the author further implies, again in the summary, that the comb was better even at follow-up.

Since most of you in this group are medical massage therapists from the USA you probably were not trained in Canadian massage schools not registered by the College of Massage therapists

I am asserting in my analysis that the research did not measure registration status, experience, and education. This means no conclusions can be made with regards to the influence of these variables on the application of medical massage technique to the subjects of this research project. Secondly, I am asserting that there is no proof that the combo was better at follow-up. The researcher put these false conclusions in the summary anyway. Because of this although those of you who are medical massage therapists may be practicing the very same techniques mentioned in the research if you didn’t get trained in Canada you can’t use this research to brag on yourself. Even those of you who received training directly from the masters. Studying Trigger Point Therapy, for example, directly with Janet Travell wouldn’t even qualify you to brag on yourself all because she wasn’t Canadian trained and neither were you.

If the hairs on the back of your head are perking up about now your ethical sensibilities may be nudging you.

If this kind of behavior, by researchers (knowingly putting false statements in research summaries) is tolerated and there are no disincentives it is likely to increase. This is especially true if you consider the unseen pressures on researchers to spike abstract summaries with spin/lies to gratify funding sources. In the case of this research the Canadian College of Massage therapists wants to recruit new students who can study in their schools and become registered medical massage therapists just like you, with one important difference. Since they were Canadian trained they are better and now they can prove it with research. Or more specifically they can lie and get away with it.

But as Sean says, so what, isn’t it the readers responsibility to discern the truth from lie. In the case of this research it is fairly clear to the trained eye that this research is bogus. It took me less than about 5 minutes to figure it out. Of course, I count my lucky stars that I have been fortunate enough to have several college level professional training courses in Research Design and Methodology and Statistics. This course work is not normally included in the vocational training programs of massage therapists and so most of you have not been trained. This would make it difficult to pick up the error especially without a full text article to review. Since most research of the online variety has only an abstract summary (no full text free articles available) you would just have to accept defeat at the hands of those darn Canadians.

If these practices flourish it would render the goals of establishing the scientific efficacy of medical massage technique, increasing the use of evidenced based practice, increasing the scientific communities trust of our research difficult/impossible to achieve.

Perhaps these goals are not that important after all. The cynic in all of us would agree.

Ted

P.S.

For those of you who want a more technical breakdown of Pedro’s rating system you can read my technical analysis linked below.

http://www.anatomyfacts.com/Research/Massage%20Journal%20Club/November06/novemberAnal06.htm

Ted Further replies;

Ethics aside, for now at least, sean makes an excellent point (“judgement call to the reader “) in ascribing responsibility on each an every one of us in quickly recognizing research flaws. If any of you have had a psychology/sociology course or two you might remember the concept of locus of control, changing ourselves probably works better than trying to change the world. The analysis of this study asserts the probability that several gate keeper organizations with some really smart people in them allowed this research to be published when they shouldn’t have. Disagreement on this point aside, if you accept this premise, it would be easier to change how we understand this research than change all of the organizations and all of the minds of the really smart people in these organizations. That is if we could become smarter consumers of research we would recognize the flaws and write letters to the editors of these publications which in itself is a disincentive to authors and publications.

Does it mean then that we all will have to take 2-4 college semesters of statistics/research design, learn complicated and mind bending/numbing statistical equations, conduct research, write thesis papers/dissertations? The purpose of the analysis of this research paper was less about bashing the author of the research and more about introducing in case study format some simple research concepts that will allow you quickly determine whether you can rely on research findings in extracting information for your practice. This allows us to achieve evidenced based practice without changing the world. This also allows us to achieve the goal of research literacy without lengthy study.

If research literacy is defined, perhaps over simplistically, as learning how to find, understand and critically analyze research the focus of my lengthy paper was on the understanding analysis part. What is the mean, standard deviation, P-values, confidence intervals, control groups, randomization, allocation concealment, subjective vs. objective measurement, spin tactics and so on. I have purposely avoided including equations because it is unnecessary in accomplishing the goals.

I realize this may not be the best forum (USMedicalMassageAssoc) for discussion of these topics and so I have established Journal Club online along with Adam Wexler who co-founded a physical Journal Club in the Seattle area about 1 ½ years ago. These folks are research scientists/massage therapists. I, as aforementioned, have some coursework in statistics ect. The goal of the free online group is simple; to promote research literacy by presentation and discussion of massage research. These efforts are in recognition of an increasing number of journal articles on massage therapy and other professions (medicine/physical therapy) awareness of the importance and value of evidenced based practice.

If you do a data search on pub med (online library of medicine), with a keyword of massage you will get around 8000 articles or so. As a rough index of how many massage articles out there it is still miniscule. Compare that number to study of the H-pylori bacterium (26,000) and you will realize how far we as a profession have to go in establishing the efficacy of our techniques scientifically. Make no mistake, in underestimating the importance of this on our collective financial bottom line. Professions that have established a scientific basis for there methods make more money than we do many times over. It is definitely a good business strategy to improve your research literacy because it encourages authors to produce good research and it increases our awareness and support of efforts to prove the effectiveness of what we do.

If our professions is just all about marketing exotic sounding techniques to capture thru novelty more customers there may be quick short term financial gain to the marketing innovators but long term respectably is lost to the professions that actually prove their professions methods. I would predict for example that research physicians were not impressed with the research article under review not only because of the poor research design but also because of a blatant use of science as a marketing tool. The positive findings of this research are then largely discounted by the scientific community. The research does serve business in dishing up a pabulum quotable research findings to increase consumption. This tactic though frustrates our long term goals of mainstream professional integration.

Hopefully we will increasingly value good science over marketing because long term financial gains from the former method are inevitable and certain marginalization from the later method are sadly predictable.

Thanks for your consideration, Ted