Epistemic Arrogance and the Randomised Controlled Trial
In the philosophy of science epistemic humility is the recognition that knowledge of the world requires interpretation by the observer, and therefore scientific claims must be based upon the idea that observation cannot lead to understanding of the world in-itself. Epistemic humility can be traced back to the work of Immanuel Kant who coined a similar argument in the Critique of Pure Reason (1781). But what is ‘epistemic arrogance’?
I would define epistemic arrogance as a lack of appreciation for the flaws of a method of observation, and a reluctance to qualify scientific claims to truth. I believe this ‘sin’ of epistemic arrogance is often committed in the context of randomised controlled trials (RCTs), not only in medicine but increasingly in the social sciences, psychology and economics. As I will go on to discuss the RCT is given a special status of “gold standard”. It is seen as the most effective way to assess treatment efficacy due to its ability to control confounding factors and sources of bias. This special status is unhelpful as I believe it has led to epistemic arrogance in the scientific community, and amongst the general public, whereby people ignorantly believe that the results of a RCT are guaranteed to be reliable or that RCTs are necessary for causal inference (i.e., determining if X causes Y).
I am not making a case against the use of RCTs, as I cannot deny the fact that they are inherently useful, I am critiquing the inflated authority it is given over other methods. Many scholars in philosophy, medicine, and sociology have argued that this inflated authority does not stand up to scrutiny. I want to shift from the arrogant trialist to the humble enquirer, to move from RCT predominance to a ‘cumulative understanding’ of science that acknowledges other sources of evidence (Nancy Cartwright, 2018).
“Colleagues can let us down, shared epistemic practices can be abused, and institutions can be corrupted. The virtue of epistemic humility therefore builds in, at the ground level, an acute sense of the fact that epistemic confidence is conditional, complex, contingent, and therefore fragile.”
— Philosopher of Science James Kidd (2017) discussing the ‘fragility’ of epistemic confidence
Background — A Brief History of Evidence Based Medicine and RCTs
The first medical RCT was published in 1948 by the Medical Research Council , it looked at the treatment of tuberculosis using streptomycin. One of the authors of this paper Austin Bradford Hill is credited with creating the RCT. RCTs gained traction in the 50s and 60s, especially after the fallout of the thalidomide scandal. Previously pharmaceutical companies were largely unregulated, RCTs provided a new effective method to test the claims of manufacturers when a new drug was being brought onto the market. Legal changes in America meant that the Food and Drug Administration (FDA) often required RCTs in order to approve a new drug. Over time testing efficacy and the randomised trial became analogous and in the 1980’s the phrase “gold standard” was attached to RCTs.
“any belief that the controlled trial is the only way would mean not that the pendulum had swung too far, but that it had come right off the hook”
— Austin Bradford Hill, the father of the RCT (1966)
Evidence-Based Medicine (EBM) was a movement that started in the 1990's to base treatment decisions for individual patients on the “best evidence” available rather than an over-reliance on the clinical judgement of doctors. From this movement, ‘hierarchies of evidence’ were created and a common theme among them was the supremacy given to RCTs (see below). Since the inception of the randomised trial as a method, the legal and philosophical framework of medicine has shifted so that RCTs are required to obtain “good evidence” for a given treatment, and the evidence derived from these trials is given authority over other methods.
What is a Randomised Controlled Trial Exactly?
In a RCT a people are recruited into the study, and the population is split into a treatment group (receives the new treatment) and a control group (receives alternative or placebo). Everyone involved including the doctors and the patients’ is unaware or ‘blinded’ to the treatment status of the patients until after the data reviewed. At the start of the study an outcome variable is pre-determined, for example, relief of symptoms (e.g. cough, diarrhoea). In the analysis the different variables are compared between the groups. If the difference in the outcome variable is statistically significant in the treatment group compared to the control, this treatment is considered to be effective.
This can be confusing. But you don’t need to know how to carry out a trial yourself! If you would like easy examples of real studies and their outcome variables click here.
Why is ‘Gold Standard’ Unhelpful?
Randomisation Does Not Control For All Confounders and Biases
The beauty of the RCT is its ability to control many confounding factors i.e. factors other than the target treatment that affect a patient's chance of recovering from a condition. For example, while recovery from condition X could truly be due to treatment with drug Y, other confounding factors could also be at play such as gender, age, other diseases, genetics, etc. This is why trials are randomised, to ensure that the difference in treatment effect that is observed is due to the treatment, and not due to other confounding factors. However, while the RCT can control for many of these confounding factors it cannot control for all of them.
The nature of many RCTs is that they recruit a small number of patients. The problem with a small population size is that it is possible for the treatment groups to be unbalanced. For example, it is possible that healthier or younger patients are randomised into the ‘treatment group’, and less healthy and older patients are placed in the placebo group. The trial could conclude that the treatment given was effective, but due to the small sample size and the imbalance between the groups hypothetically this treatment effect could have come from the fact that the treatment group was younger or healthier. In epidemiology, this is phenomenon is known as ‘random confounding’ (Cartwright, 2018).
Similarly, issues may arise even if population sizes are large. Concato and Horwitz (2018) use the example of a large RCT that investigated the use of tiotropium to treat lung disease (COPD). The study had 6000 participants, making it “the largest and longest randomised trial of tiotropium”. However, in this trial high risk patients were excluded; the drug was deemed effective despite not having a completely representative sample. As I will discuss next blind trust in any trial that is randomised, leads to problems of external validity. What do we do if a ‘high-risk’ patient needs treatment? Can we give them the drug on the basis of this trial?
The Problem of External Validity
Simply put external validity is the validity of applying the conclusions of a scientific study outside the context of that study, it is how much we can generalise findings to other situations and populations. So, in our tiotropium example above would we be able to give ‘high risk’ patients with COPD the treatment, even though they were not included in this reliable large RCT? It is important to note that even under perfect experimental conditions the randomised controlled trial can only show if a treatment is effective in the test population(s).
This highlights another external validity issue. Randomised control trials are conducted on a population level. When a doctor decides to use treatment X for condition Y he/she is making an inference from a population-based study as to what treatment to give an individual. There is no certainty that the individual receiving the treatment will benefit, RCTs show that on average the treatment confers a benefit, but it is also possible that the treatment is inert or detrimental to some patients that lie outside of this ‘average treatment effect’.
“The chicken noticed that the farmer came every day to feed it. It predicted that the farmer would continue to bring food every day. Inductivists think that the chicken had “extrapolated” its observations into a theory, and that each feeding time added justification to that theory. Then one day the farmer came and wrung the chicken’s neck. This inductively justifies the conclusion that induction cannot justify any conclusion.”
— Bertrand Russell’s Chicken Analogy, highlighting the problem of induction
The Results of a Randomised Controlled Trial Cannot Logically Be Proven To Be Correct
The first thing you get taught about science is that correlation does not equal causation. So why do RCTs get a pass? In a RCT if the treatment is correlated with the outcome variable, then the treatment must have caused the outcome. However, this chain of reasoning commits the false cause fallacy, it is a logical mistake as we have presumed causation. For treatment X to be a cause of outcome Y (under the probabilistic theory of causation) we must make several assumptions, for example that a) randomisation has been successful, b) the treatment groups are balanced, and c) confounding factors have been removed in this process. To call RCTs the “gold standard” evokes certainty that these trials are precise purely by design, and this has led to the idea that they are required for causal inference i.e. “good evidence” regarding the effectiveness of drugs. Humbly we must admit that a single RCT can only prove causality if all the underlying assumptions of the trial are met.
Conclusion
Earlier I said I wanted us to move from arrogant trialists to humble enquirers, we can achieve this by broadcasting what randomised trials can do and what they can’t do, and also understanding the contexts in which RCTs provide useful answers and the contexts in which they don’t. I’m unsure of whether we should dismantle evidence hierarchies, but I am convinced that the hegemony RCTs have is unjustified. The positioning of RCTs at the top renders other methods, and clinical judgements to be inferior or sub-par. As Nancy Cartwright (2018) says science has an obligation to build upon previous knowledge, and we should therefore consult other fields, other empirical methods, and other opinions to develop a cumulative understanding of science, not just a one trial fits all paradigm.
**DISCLAIMER: I do not claim to be a scholar on this topic, this is just a reflection of my reading thus far. I am open to debate or correction. Please message me. **
