Rigor Mortis: How Sloppy Science Creates
Worthless Cures, Crushes Hope, and Wastes Billions

Reviewed by Dr. Panchajanya Paul, MD

Hardcover: 288 pages
Publisher: Basic Books; 1 edition (April 4, 2017)

We frequently come across new, exciting, and ground-breaking research findings in the media. But, most of the results later turn out to be false. Of the 7000 known diseases, only about 500 have treatments, many offering just marginal benefits. The cost of medical treatment is highest in the US. One of the reason cited by the drug companies are the increased cost of research needed to bring any new treatment out to the market. Medical research is a long and an arduous process. The first stage of the research is done on animals, and based on that data, clinical trials are designed for human beings. The quality of basic research determines whether the medical treatment based on them will be effective or not. We frequently read about innovative treatments about metastatic cancers, Alzheimer's, Amyotrophic lateral sclerosis, promise for a magic pill for obesity etc. in the media. But these drugs which show benefit to animals under laboratory conditions fail to show same success on human in real world. Not only that, some of the studies when repeated on the animals under same laboratory conditions fail to repeat the previous results. These are some serious concerns about the nature of biomedical research raised by Richard Harris in his book Rigor Mortis. Richard Harris is a famous science journalist, and has covered science, medicine, and the environment for the past twenty-nine years for NPR, and the three-time winner of the AAAS Science Journalism Award. The cover of his book says, "American taxpayers spend $30 billion annually funding biomedical research, but over half of these studies can't be replicated due to poor experimental design, improper methods, and sloppy statistics. Bad science doesn't just hold back medical progress, it can sign the equivalent of a death sentence for terminal patients."

Scientific truth rests on two fundamental assumptions- objectivity and reproducibility. The same results should be obtained by two different people following the same methods, and over time. Richard Harris takes us to the beginning of the scientific research, he writes, "careful science is new enterprise. Before the seventeenth century, natural philosophers, as scientists were then called, often relied on the word of authorities to sort out truth from fiction. For many hundred years, European intellectuals assumed that all knowledge already existed, and their job was to interpret the writings of Greek philosopher Aristotle. Around the time of Galileo, that edifice started to crack. Natural philosophers dared to conduct experiments in search for the truth...British philosopher Francis bacon formalized the scientific method: make a hypothesis, devise a test, gather data, analyze and rethink, and ultimately draw broader conclusions. This rubric worked reasonably well when scientists were exploring repeatable experiments in the realm of physics. But biology is a much tougher subject, since there are many variables and a great deal of natural variation. It is harder to see a phenomenon and harder to make sure that personal biases don't creep in".
Problems in study designs and analysis will produce false results. John Ioannidis first highlighted this problem in his widely cited paper called "Why most published research findings are false?". Scientists are smart people who are committed in their research, but they are also human, and prone to human follies. There are many causes for bad research. Richard discusses most them. Sometimes the study result is not relevant even if it is done accurately. "Improvements won't help in many instances when animals are poor stand ins for human disease. In the case of stroke...adolescent male animals often used in the studies may not be a good substitute for an elderly human having a stroke. Any drug doses for animals may differ dramatically than those for people. He cautioned that the extent of brain damage in an animal may not be a good surrogate for human disability or death." Animal studies can fail in human for several reasons. One example given in the book is about the wo sets of genetically identical mice, one group placed at a top cage near the light, and the other at the bottom cage away from light. Mouse prefer dark places, and thus they become more stressed and anxious, and more immune suppressed as their cage moves from the bottom of the rack to the top. Thus, even a small detail like the cage position of the animal can have a butterfly effect and lead the study astray. Sometimes researchers cut corners to finish the study early. As many researchers work on the same topic from across the world, there is an intense competition to be the first one to publish. The top-rated journals like Nature, Science, and Cell are more likely to publish groundbreaking and innovative studies. It thus pays more to be the first, than to be right. Also, publication in high impact journal is tied with tenure, professorship, salary, and academic appointments. This produces a lot of pressure and unhealthy competition among researchers fighting for the shrinking pie of research funding. Universities make it tougher by recruiting and preferring researchers who can bring grant money.

Finally, there is the issue of honesty, "... an element of nature that we develop as a child and never let go off. Our notion of right and fair doesn't form in a vacuum. People look around and see how other people are behaving as a cue to their own behavior. If you perceive you have a fair shot, you're less likely to bend the rules. But if you feel the principles of distributive justice have been violated, you'll say, Screw it. Everybody cheat; I am going to cheat too. If scientists believe that they are themselves being treated unfairly, they themselves are more likely to engage in less than ideal behavior", writes the author. Biology is complex full of grey areas. On top of it, the very nature of scientific process, means that there is no final truth. As new data comes and better theories are developed, the old ones are discarded. The author quotes McNutt "viewed through the lens of time, just everything that we write down we'll look back at and say - that isn't quite right."

In the end, Richard Harris gives numerous anecdotes, personal stories and interviews with top biomedical researchers. He lays out a plan about how the problems with biomedical research can be fixed. He says, "reproducibility of the studies will improve if scientists took simple technical steps, such as validating the cell lines, running proper controls with their antibody experiments, choosing adequate sample sizes for their mouse studies, deciding in advance what hypothesis they are testing and so on". The book also calls for is a more stringent p-values. It is accepted that if a study reaches a value of less than 0.05, the results are statistically significant. But these seems not to be stringent enough and many false results come out. Richard argues that those who use p-values should aim for an even stringent results of p value less than 0.005. These tougher standards will ensure that if the study is run again, the results will be 95 percent likely to be statistically significant. Richard also calls for the research process to become more transparent. This allows others with a more rigorous look to find out gaps that were initially missed. One idea is to reward the good guys and shame the rogues. Some journals have already begun to give badges to the scientists who are doing the right thing. Some journals are publishing the list of retracted articles along with the name of their authors involved in those flawed studies. Another step is to provide guidelines, templates and checklists to assist scientists working on animal models. This growing awareness among researchers and journals is a step in the right direction. In the end, biomedical research is a serious endeavor. It not only involves billions of dollars, but has the potential to save millions of lives done in the right way.