[Ed. Note: See Orac Lies Again. Part 2]
...the reasons we can conclude that the paradigm of animal modelling will never be a predictive modality, isolated cases of correlation notwithstanding.
Orac, AKA David Gorski, MD, PhD, has written a 2500+ word blog regarding
I would rebut his position on his own blog site but since I am not allowed to post there, and because my own response will probably be long, I will address his position here.
Dr Gorski starts his supposed scientific analysis of mouse avatars by committing the fallacy of poisoning the well. He relates that some animal rights activists have advocated for, and perhaps participated in, violent activities. While I too have denounced the use of violence in a civilized, democratic society I have not conflated the morally questionable activity of someone with his position on a scientific issue. If Hitler ever said 2+2=4 or “germs cause disease,” he was right, Auschwitz notwithstanding. Considering Dr Gorski’s intelligence, education, and hobby (refuting woo with science-based critical thinking) I can only conclude he used this fallacy on purpose.
Sadly, Dr Gorski continues his diatribe by using more fallacies. He groups anyone opposing animal-based research with anti-vaxers, deniers of anthropogenic global warming, and quacks. He accuses all those groups of cherry picking the data (and he is correct regarding anti-vaxers and global warming deniers and quacks). I will try to refrain from reproducing his exact words but the following is key, so here it is:
This generally involves cherry picking data and studies to paint as dim a view of animal research and its usefulness as possible, usually through a combination of exaggerating how bad the animals are treated and claiming that the research using animals doesn’t produce any useful results. One variant of this latter ploy is to claim that animal models aren’t “predictive” of human responses. Again, any animal models that are predictive are dismissed, and situations where animal models fall short are hyped to the max.
The “predictive” part refers to me, so allow me to once again point out the myriad lies in this representation. (Dr Gorski acknowledges having read my publications, so these are lies not errors of ignorance.) In our books and articles and even in this blog, I have explained many times that animal models can be successfully used in seven categories in science and research. In only two out of a total of nine categories are animal models not scientifically valid and both are when claims are explicitly or implicitly made that the model per se is predictive. Claiming scientifically validity in seven out of nine areas does sound like I am dismissing models that are not predictive. But I’ll let you decide.
Dr Gorski takes issue with my position regarding the two areas that claim to be predictive modalities. His position can be summarized as: “Ray Greek uses the word predict in a way that is inconsistent with science.” I will now address that lie for the umpteenth time.
The prediction argument is the one that scientists and nonscientists find easiest to understand both in terms of what it explicitly states and what it implies. Scientists and nonscientists have acknowledged that society supports using animals in research if and only if such use leads to safer drugs and better medical interventions. By that, they mean that if a drug is tested on a rat and kills the rat then the drug should not be given to humans because there is a high likelihood that the drug will also kill humans. This is so straightforward I can offer little to expand on the notion. Indeed, this is the basis for the requirements by the FDA and EPA that new drugs and other chemicals be tested on animals. The problem is that vivisection activists such as Dr Gorski commit the fallacy of equivocation and use the word predict in another context in an attempt to justify the fact that animal models clearly cannot be used as predictive modalities for drugs testing and disease response.
The word predict and its various forms (prediction, predictive etc) can be used in essentially two ways in science.
1. Scientists using the hypothetico-deductive method formulate a hypothesis in order to explain a phenomenon. This hypothesis suggests that various outcomes should be seen and other outcomes not seen if the hypothesis is true. In other words, the hypothesis predicts that certain things will happen if the scientists that thought of the hypothesis are correct. Thus, the hypothesis contains predictions about future events, or events that occurred in the past but can be discovered in the future, and these predictions can be tested and either confirmed or falsified. If confirmed, then the hypothesis is stronger, more likely to be true, but if the predictions are falsified then the entire hypothesis may be falsified as well. The statement: “Hypotheses generate predictions,” is a true statement in science.
2. The second use of the word predict is in the context of a test or practice as opposed to a hypothesis. When scientists, physicians, or nonscientists for that matter, use a test or practice in an attempt to discover reality, this modality can be tested in order to determine how well it actually works. For example, if you are involved in a car accident and are taken to the ER complaining of shortness of breath, the physician will order an x-ray of your chest to make sure you do not have a pneumothorax, a collapsed lung. If you do have a pneumo, the physician might need to place a tube into your chest, a procedure that is not without risk, so you want the chest x-ray to have a very high predictive value for diagnosing pneumothorax, lest you have a chest tube placed for what turns out to be no reason. You want the chest x-ray to be a predictive modality for diagnosing collapsed lungs. You are not interested in the ER physician generating a hypothesis, you want a definitive, or near definitive, way of determining what is wrong with you.
As I have written numerous times, the way one judges how well a modality, be it a practice or test, functions at predicting outcomes is by using the binomial classification table and formulas. This allows the practice or test to be evaluated for well it predicts a positive outcome when the outcome really is positive (the patient really has a collapsed lung) along with how well it predicts a negative outcome when the outcome really is negative (no collapsed lung). These values are called positive predictive value (PPV) and negative predictive value (NPV) and are taught to every medical student as well as students in college classes such as statistics. This is not only not controversial it is standard practice for determining the value of everything from a blood test for cancer to how well drug sniffing dogs are at finding drug smugglers. Anytime a modality is used to approximate reality, and the reality is eventually known, PPV and NPV can be used to evaluate how well the modality performs. Chest x-rays and CT scans of the chest perform very well for diagnosing a pneumothorax while Ouija boards and Palm Readers perform very poorly for telling you what the future holds in store for you. These things are objective and demonstrable.
So why would anyone conflate predictions that come from hypotheses with the predictive value (PPV and NPV) of a modality? As I said, society thinks there is an ethical cost to animal experimentation. (I happen to agree but this is not about my ethical position it is about the position of society in general.) So society has said animal experimentation should be allowed under certain conditions but not others. The following support this. An editorial in Nature 2009: “Animal-research policies need to be guided by a moral compass—a concensus of what people find acceptable and unacceptable.”  Giles, writing in Nature:
In the contentious world of animal research, one question surfaces time and again: how useful are animal experiments as a way to prepare for trials of medical treatments in humans? The issue is crucial, as public opinion is behind animal research only if it helps develop better drugs. Consequently, scientists defending animal experiments insist they are essential for safe clinical trials, whereas animal-rights activists vehemently maintain that they are useless.
In order to justify the use of animals in research in general, scientists routinely claim that the animal model, in whatever form/species, is a predictive modality for humans. Consider the following. Dr. Keith Cheng of Penn State's College of Medicine stated: “Animal tests are necessary for some research, such as testing drugs for toxicity. It would be, in my opinion, improper to release drugs for human use without animal testing.”  Rigmor Thorstensson, Head of the Department of Virology, Immunology and Vaccinology, at SMI in Sweden wrote an article titled: “Medical research on apes is no ethical problem for me.” 
The ethical reasons against animal testing must be weighed against the evidence that more and more people across the globe can have access to effective drugs and vaccines. If these were tested in clinical trials without first undergoing animal testing large numbers of people risking their lives in such studies and the development could also be delayed catastrophic. . . . For me it is no ethical problem of using monkeys in experiments, it is the only way to produce an effective vaccine against the major global infectious diseases, HIV, tuberculosis and malaria.
Heywood: “Animal studies fall into two main categories: predictive evaluations of new compounds and their incorporation into schemes designed to help lessen or clarify a recognised hazard.” Robert Vassar of Northwestern University states: “Chronic dosing in mice and monkeys is necessary to show the efficacy and safety of the antibody before it’s taken into humans.” The Council for International Organizations of Medical Sciences implies prediction when they state: “clinical testing must be preceded by adequate laboratory or animal experimentation to demonstrate a reasonable probability of success without undue risk.”  Ramesh Akkina, DVM, PhD, professor, Department of Microbiology, Immunology and Pathology at Colorado State University, Fort Collins, CO stated: “A major advantage with this in vivo system [genetically modified SCID mice] is that any data you get from SCID-hu mice is directly applicable to a human situation.” 
Zbinden states: “The three main purposes of experimental toxicology are (1) determination of the toxicological spectrum in selected laboratory animal species; (2) extrapolation to other species and prediction of adverse effects in man; and (3) determination of safe levels of exposure.” 
Biomedical sciences’ use of animals as models [is to] help understand and predict responses in humans, in toxicology and pharmacology . . . by and large animals have worked exceptionally well as predictive models for humans . . . Animals have been used as models for centuries to predict what chemicals and environmental factors would do to humans…. The use of animals as predictors of potential ill effects has grown since that time . . . If we correctly identify toxic agents (using animals and other predictive model systems) in advance of a product or agent being introduced into the marketplace or environment, generally it will not be introduced . . . The use of thalidomide, a sedative-hypnotic agent, led to some 10,000 deformed children being born in Europe. This in turn led directly to the 1962 revision of the Food, Drug and Cosmetic Act, requiring more stringent testing. Current testing procedures (or even those at the time in the United States, where the drug was never approved for human use) would have identified the hazard and prevented this tragedy.  (Emphasis added.)
Hau: “A third important group of animal models is employed as predictive models. These models are used with the aim of discovering and quantifying the impact of a treatment, whether this is to cure a disease or to assess toxicity of a chemical compound.”  Michael F. Jacobson, executive director of the Center for Science in the Public Interest noted in 2008: “We must test animals to determine whether a substance causes cancer.” Similarly, Huff et al. observe: “Chemical carcinogenesis bioassays in animals have long been recognized and accepted as valid predictors of potential cancer hazards to humans.” 
The above could be easily multiplied.
So, animal models are clearly and explicitly claimed to be predictive modalities by many in the animal model community. But unfortunately for this position, these claims have been falsified many times. Moreover, scientists themselves acknowledge that animal models cannot be used as predictive modalities. Many such statements have come from Pharma but some basic researchers have also acknowledged this. I will confine myself to two examples. Basic researcher James Hick of UC-Irvine:
This goes back to the '40s and '50s. Evolutionary biologists knew there was a problem. Later on physiologists also began to learn that there was a problem in the '70s and '80s. You couldn't make one-to-one relationships. It wasn't quite possible. There are species differences. . . . What motivates a lot of biomedical scientists is using animal models to generate novel and new ideas. However, what has happened -- and Dr. Greek is entirely correct -- is that to sell the idea, quote, unquote, sell it, I would -- I would agree that many biomedical scientists can be accused of over promising and under delivering. But the over promising is what's required or the -- or is what they have to define why are you doing this research and how's it going to directly benefit humans? . . . I do agree with Dr. Greek that -- that if someone directly thinks that there is a -- that they're using a CAM -- a real model, that they're studying a rat, that's really going to cure hypertension, I think that that's bad science.
Mullane and Williams state:
The difficulties in predicting drug efficacy from preclinical models have been of concern for more than two decades . . . Thus, novel findings apparently related to the systems and targets involved in disease causality; the delineation of the efficacy, selectivity and safety of NCEs; and the predictive relevance of biomarkers and animal model data to the human disease state, even when there is evidence for target engagement in humans, all frequently fail to enhance the success rate for new drug applications (NDAs).
But let's get back to the data that falsifies the claim that animal models are predictive modalities. The formulas for calculating PPV and NPV rely on a history of the modality being compared to reality. One simply cannot cherry-pick isolated cases of correlation between animals and humans and conclude that such cases mean that the animal model was a predictive modality this time. There is no such thing as a modality being predictive this time. If a nonscientist made such a statement, I would just pass it off as a statement based on ignorance. But when a MD, PhD states this there can only be one reason—the guy is lying! Dr Gorski knows that isolated instances are meaningless in terms of whether a modality is effective or predictive. One person might say he feels better after homeopathy. But test 100 in a double blind study and you will find that the effectiveness of homeopathy is no better than what would be expected from a placebo. THAT is science, not cherry-picking. Cherry-picking is looking back in time and finding an animal model that was eventually shown to react the same way to a drug or disease and ignoring all the times that same animal reacted differently and all the times another animal reacted differently to the same drug or disease. An example.
Offspring of the White New Zealand rabbit exhibited phocomelia in response to thalidomide administration to the mothers, just as human offspring did. But since the White New Zealand rabbit failed to correlate with humans on a variety of other teratogens and non-teratogens, it cannot be said to be a predictive modality for teratogenicity. The PPV and NPV of the White New Zealand rabbit was too low to be considered useful for predicting whether an unknown chemical would be teratogenic in humans. Ditto for non-human primates (NHPs). Most NHPs also exhibited phocomelia in response to thalidomide. But subsequent studies revealed that for all other chemicals NHPs were about as predictive as a coin toss. THAT is how one assesses a modality for its predictive value and Dr Gorski knows this because he uses this information every day. Rabbits were not predctive this time, for thalidomide, secondary to a failed history of correlating with human outcomes. To claim rabbits were predictive this time demonstrates a lack of understanding of how preditve modalities are assessed or just plain perfidy. I assure you Dr Gorski well understands how to judge the predictive ability for diagnostic and therapeutic modalities and other practices.
(For more on thalidomide see The History and Implications of Testing
Thalidomide on Animals:
For more on the concept of predictive models in medicine, see Are animal
models predictive for humans?)
I will address the remainder of Dr Gorski’s essay in Part II (coming soon), including the reasons we can conclude that the paradigm of animal modelling will never be a predictive modality, isolated cases of correlation notwithstanding.
1. Editorial: A slippery slope. Nature 2009, 462:699.
2. Giles J: Animal experiments under fire for poor design. Nature 2006, 444:981.
3. Q&A: Dr. Keith Cheng, researcher at Penn State's College of Medicine, shares views on using animals in scientific research [http://www.pennlive.com/midstate/index.ssf/2012/04/qa_dr_keith_cheng_researcher_a.html]
4. Thorstensson R: Medicinska försök på apor är inget etiskt problem för mig (Medical research on apes is no ethical problem for me). Infectious Diseases 2006, Kronikor.
5. Heywood R: Clinical Toxicity--Could it have been predicted? Post-marketing experience. In Animal Toxicity Studies: Their Relevance for Man. Edited by CE Lumley, Walker S. Lancaster: Quay; 1990: 57-67
6. Vassar R: Alzheimer's therapy: a BACE in the hand? Nat Med 2011, 17:932-933.
7. Council for International Organizations of Medical Sciences (CIMOS): International ethical guidelines for biomedical research involving human subjects. Bulletin of medical ethics 2002:17-23.
8. Anonymous: Of Mice...and Humans. Drug Discovery and Development 2008, 11:16-20.
9. Zbinden G: Predictive value of animal studies in toxicology. Regul Toxicol Pharmacol 1991, 14:167-177.
10. Gad S: Preface. In Animal Models in Toxicology. Edited by Gad S. Boca Rotan: CRC Press; 2007: 1-18
11. Hau J: Animal Models. In Handbook of Laboratory Animal Science Second Edition Animal Models. Volume II. 2nd edition. Edited by Hau J, van Hoosier Jr GK. Boca Rotan: CRC Press; 2003: 1-9
12. Longer Tests on Lab Animals Urged for Potential Carcinogens [http://www.cspinet.org/new/200811172.html]
13. Huff J, Jacobson MF, Davis DL: The limits of two-year bioassay exposure regimens for identifying chemical carcinogens. Environ Health Perspect 2008, 116:1439-1442.
14. Mullane K, Williams M: Translational semantics and infrastructure: another search for the emperor’s new clothes? Drug Discovery Today 2012, 17:459-468.