The FDA approval process is in place to ensure that drugs are safe and effective before they are made available to the public. But do the clinical trials that lead to approval truly ascertain whether a drug is efficacious? Fully two-thirds of anticancer drug clinical trials leading to FDA approvals between 2014 and 2019 had at least one important limitation in trial design: lack of randomization, lack of significant overall survival advantage, inappropriate use of crossover, and/or use of a suboptimal control arm. In this interview with i3 Health, Talal Hilal, MD, first author of the study that recently reported these findings in JAMA Internal Medicine, shares his thoughts as to how and why the FDA's standards for approvals have been changing, how this change may impact patient outcomes, and how oncologists should account for the FDA's evolving standards when it comes to clinical decision making.
What prompted you to investigate the topic of limitations in clinical trial design and outcome?
Talal Hilal, MD: I think the main driver was just trying to quantify how often these things may be seen in modern trials, those done in the past five years. We all discuss some of these limitations on an individual basis, but I think there was a need for an analysis that tries to look at them all collectively; we were curious to see how often they may be seen, either individually or together.
Do you think that the proportion of FDA approvals based on trials with limitations has been increasing over the years?
Dr. Hilal: We didn't necessarily analyze them by year. Given that we used a five-year period, it was a little hard to see a significant change over just that time span; I think if we had a longer analysis of maybe 10 years, it would have been easier to see something like that. Ultimately, the main change during this five-year period has probably been that there are more single-arm trials leading to approvals, and a lot of those are accelerated approvals that are dependent on follow-up studies to confirm efficacy.
How might these limitations end up impacting patient outcomes?
Dr. Hilal: I think the main concern is that they increase uncertainty with a lot of the treatments we've been using, especially when drugs are being approved on the basis of single-arm studies. It may end up being that a lot of these agents are effective and their availability at earlier dates may be in the patient's best interest, but at the time of prescribing, without clear randomized data to confirm that efficacy, we're dealing with a lot of uncertainty. What that means is either that efficacy will be confirmed at a later date or that maybe some of these drugs are harmful and actually are not as effective as we thought. So we will end up having either treated patients with harmful agents or possibly having treated patients with effective agents before we really know for sure that they're effective: there are two sides to it, and I think it just introduces more uncertainty into the decision-making process.
Regarding the design limitations in randomized trials, their impact on patient outcomes or treatment may not be as readily measurable as what we might see with a single-arm study that ends up getting a follow-up randomized study. Here we have a question of suboptimal control: are we perhaps using more toxic agents, more expensive agents, that may not necessarily be better than what we have as a standard of care? If we're actually comparing these new agents to a control arm that we haven't been using in practice, how does that pan out?
And then the bigger question that I think is a point of debate is whether overall survival should always be the end point. I think no one would argue that when it's available, it definitely weighs more and has more impact, but can all cancers be measured that way? That's an important question. In our study, we tried to just see how many drugs in research and development are getting approved without that end point being met, even when a different surrogate end point is positive.
What do you think should be done to fix the issue of the limitations that we're seeing in clinical trials?
Dr. Hilal: I think the question of single-arm trials leading to accelerated approval is a modern sort of phenomenon; 10 years ago, that's not what the FDA used to do. We had a higher bar for approval, but that meant there were maybe some promising agents that clearly had single-agent activity that were not approved until we had the randomized trials, which meant a lot of patients may not have been offered these agents. Obviously, cancer is not going to wait, so some of these patients ended up dying without getting the benefit of the new drugs. So approving agents based on single-arm trials as an accelerated approval with the hope that a follow-up study is going to confirm efficacy is a tradeoff. I don't necessarily see a problem with that; I think the main issue is that the follow-up study needs to be well designed to actually confirm the efficacy. If it is a randomized trial, it needs to be in the right setting and the right indication, with the right control arm. If follow-up is done correctly, the accelerated approval being converted to a regular approval actually brings comfort to prescribers and to patients by confirming that the drug is clearly useful.
The other alternative is that if the follow-up study does not confirm efficacy, then that accelerated approval should probably not stay on the market. I think that's an issue: we're not necessarily seeing a lot of these agents' approvals being revoked.
Now, for the issue of design of randomized trials––control arms and crossover––that's a bigger question that has to do with the speed at which protocols are developed and trials are open for enrollment. Sometimes, protocols are written a number of years before the trial actually opens for enrollment. What was considered a standard control arm may not be standard two years down the road, so now you're randomizing patients to a control arm that's not adequate anymore. The best way to fix that may be limiting the bureaucracy that goes into opening these trials.
With the COVID pandemic, we have seen how when people really want to do that, it's very doable. COVID trials have opened very quickly without a lot of bureaucratic barriers. There's no reason we need to have those barriers in cancer trials; there's no reason for a cancer trial to be in development for a number of years and have the control arm be outdated by the time it's open.
In summary, we can help to solve this situation by speeding up the process, confirming efficacy, and being okay with revoking an approval after knowing that a drug has not been proven to be efficacious. I think that those would be the main points to work on.
Regarding the question of whether overall survival is the right end point, I think that needs to be carefully considered in the right cancer and the right setting. There are clearly cancers where the median survival is less than a year. It makes no sense that overall survival is not the primary end point in those cases: patients who have a survival of six to eight months want to know if the drug that they're getting is going to improve their survival a few more months versus just shrink the tumor longer. In contrast, for cancers that are clearly slow growing, where the survival may be five or 10 years, finding an overall survival benefit may not be as useful, and the end point may change there to quality of life. Even if the primary end point is progression-free survival, what is it coupled with? Is it less costly? Is it less burdensome on the patient? There are other metrics that can be measured there. So overall survival just needs to be carefully considered for every trial and every cancer.
There has been a dramatic increase in FDA approvals in oncology the last few months. Do you have any ideas as to what may be behind this change?
Dr. Hilal: It just seems to be a mindset of the FDA at this point. I believe that on a philosophical level, the FDA has basically allowed the burden of the decision of choosing the right therapy to shift more towards the prescriber. It used to be that the FDA would approve only agents that are clearly efficacious, meaning not just that they are safe, but also that they improve patient outcomes. I think the question of whether an agent truly improves patient outcomes is not always at the forefront, and it's not necessarily looked at as carefully, as long as the agent is "safe"––and that's a word that maybe we can define in various ways––but meeting a certain bar of safety seems to be enough now, which means that the burden is more on the actual providers. I don't know why, but I think it's a philosophical change. I'm sure that there are a lot of stakeholders that are pushing for it, and I think it's a little too early to say whether that's going to end up improving patient outcomes at the end of the day. I have my reservations about it.
What do you think should be the takeaway message for practicing oncologists, as far as how they should consider the data that's out there when it comes to treating their patients?
Dr. Hilal: My hope is to raise awareness of the fact that we're living in a time where the label of FDA approval does not necessarily mean that a new agent on the market is a great drug. That's not necessarily a bad thing, but it does mean that the burden is more on the prescriber and the patient––basically, the consumers of this research––to decide what drug to use for what patient at what time. It used to be that the availability of an agent and having that approval meant that efficacy was confirmed and oncologists were going to prescribe it. Now, just because something is approved does not mean it's going to be the right choice or that it needs to be prescribed.
These limitations highlight that we need to consider carefully the right patient and the right cancer for which to prescribe drugs; we should not just go through a list of agents because we can. I hope that this ends up leading to more people actually looking at the primary evidence of each of these agents that they're prescribing––actually looking at the trials and examining the data for themselves––and that it raises awareness that not everything we prescribe is perfect, and not everything that's approved is ultimately going to be beneficial for the patient.
At the end of the day, I think we all want what's best for the patients, and we all have different perspectives on what that means. I think being hopeful that a lot of these drugs are truly in the patient's best interest is what drives most of these approvals. The extent to which we're aware of the harms is what may end up leading oncologists to be hesitant about prescribing some of these agents. It's really that question: could we be making things worse? A lot of times, we don't know the answer to that.
About Dr. Hilal
Talal Hilal, MD, Assistant Professor of Medicine at the University of Mississippi Medical Center, is a hematologist/oncologist who specializes in the diagnosis and treatment of malignant hematological disorders, with a focus on lymphomas and chronic lymphocytic leukemia (CLL). He is a member of the American Society of Clinical Oncology (ASCO) and the American Society of Hematology (ASH).
For More Information
Hilal T, Gonzalez-Velez M & Prasad V (2020). Limitations in clinical trials leading to anticancer drug approvals by the US Food and Drug Administration. JAMA Intern Med. [Epub ahead of print] DOI:10.1001/jamainternmed.2020.2250
Transcript edited for clarity. Any views expressed above are the speaker's own and do not necessarily reflect those of i3 Health.