We peek behind the paper with the contributing study authors Jonathan Kish and Bruce Feinberg (both Cardinal Health, OH, USA): targeted agents or immuno-oncology therapies as first-line therapy for BRAF-mutated metastatic melanoma.
Please can you introduce yourself, your institution and your research focus?
The Real World Evidence (RWE) and Insights group at Cardinal Health Specialty Solutions (OH, USA) conducts observational research studies in the fields of oncology, rheumatology and other specialty therapeutic areas. We are a team of epidemiologists, health services researchers, clinicians and biostatisticians. We support biopharmaceutical companies in their pursuits to generate data and evidence that illustrate unmet needs of patients and the value (from both clinical and economical perspectives) of pharmaceuticals.
What prompted you to study the comparative effectiveness of the first-line targeted therapy and immuno-oncology agent for the treatment of BRAF-mutated metastatic melanoma?
For BRAF V600-mutated metastatic melanoma patients, there are two potential treatment options: targeted therapy (combinations of BRAF—MEK inhibitors), or immuno-oncology therapies, which can be prescribed regardless of BRAF mutation status. Guidelines from the National Comprehensive Cancer Network (PA, USA), and other societies, do not provide a blanket recommendation as to the setting in which one or the other of these two options should be used. Additionally, no head-to-head clinical trials have been conducted comparing these treatment approaches. Finally, effectiveness and efficacy, though related, are not the same: a therapy’s effectiveness is only demonstrable by real-world studies. Therefore, with patients having been treated with both agents as front-line therapy for several years, in the real-world, we believed the data may have been mature enough to begin to look at outcomes when BRAF—MEK inhibitors were used first, compared with when immuno-oncology therapies were used first.
“…effectiveness and efficacy, though related, are not the same: a therapy’s effectiveness is only demonstrable by real-world studies.”
What was the rationale behind the criteria you employed to define your study cohort and what limitations may these have imposed?
First, because we were interested in clinical outcomes including disease responses, progression and survival, we needed a data source from which we could directly access this information. As these data, to a large extent, are contained within the unstructured portions of a patient’s health record, we needed to employ a chart review methodology. To be as representative as possible and to locate enough patients to perform the analysis, it was imperative to cast a broad net and look at data from many different practices across the USA. However, in doing so, it was impossible to capture all of the potentially eligible patients treated at each practice. This can introduce patient selection bias; this is one limitation of the study.
Second, for real-world studies, the selection criteria imposed are often very relaxed compared with those for randomized controlled trials (RCTs). In this study, our only specification was that patients must have initiated first-line therapy with any FDA-approved, BRAF—MEK inhibitors or immuno-oncology therapies. Patients had to have started their therapy between 2014 and 2017. We specified these dates so that both treatment options were available at the time of the patients’ commencing one therapy or the other. Furthermore, specifying these dates ensured that there was sufficient follow-up time over which to measure the outcomes of interest.
Treatment choices may also be influenced by when patients initiated therapy and by patients’ clinical characteristics. In clinical trials, the stringent selection criteria ensure that at baseline the patients are as clinically similar as possible. We noted this in our findings; patients prescribed BRAF—MEK inhibitor therapy tended to have, what might be considered, more aggressive disease at baseline.
“Fundamentally, RWD/RWE are used to answer research questions regarding the comparative effectiveness of different treatment approaches, but not superiority or inferiority.”
How did you assess treatment ‘success’ in your study; was this in accordance with patient-stated preferences?
In our study, we relied on objective measures of clinical benefit, including disease response, time to treatment failure and patients’ overall survival. We also evaluated the duration of therapy and patients’ time off therapy. While not direct corollaries with patient-stated preferences, we believed that this data is also important to clinicians and patients. If the same survival benefit was observed for patients regardless of their duration of treatment, this would be an outcome of importance. From a patient’s perspective, the good news from our study was that many patients were still progression-free several years after therapy initiation. From the researcher’s perspective, this large number of patients without progression limited our ability to make a more definitive determination of which treatment was more clinically beneficial.
What was the importance of utilizing RWE in this study to answer your question; what could this provide that RCT data could not?
First, we acknowledge that RCTs remain, and rightly so, the gold standard for testing the safety and efficacy of pharmaceutical agents. However, research has shown that the effects observed in RCTs are not of the same magnitude and in some cases direction of those observed in the real-world. Real-world data (RWD) demonstrates an intervention’s effectiveness, not its efficacy. Fundamentally, RWD/RWE are used to answer research questions regarding the comparative effectiveness of different treatment approaches, but not superiority or inferiority.
What insights can RWE bring to the field of oncology?
In the USA, RWD/RWE have been utilized extensively in post-marketing safety surveillance as a result of the FDA’s sentinel initiative. More recently, the FDA has proposed a framework for using RWD/RWE in regulatory decision making for approvals of novel or expanded drug indications. Already, we have seen approvals for indications and expanded indications for the treatment of ultra-orphan cancers for which performing clinical trial are not feasible. Multiple, well-designed RWE studies are needed in the field of oncology to develop comparative evidence guidelines for understanding values (clinical, costs, patient preference) when there are multiple treatment options available for the same indication. Often, in these instances, RCTs — for addressing these issues — are unlikely to be performed within a reasonable time frame.
“Multiple, well-designed RWE studies are needed in the field of oncology to develop comparative evidence guidelines for understanding values (clinical, costs, patient preference) when there are multiple treatment options available for the same indication.”
What were some of the challenges you faced by employing a retrospective, chart review study design and how did you overcome them?
The biggest limitation to the approach employed concerned addressing potential biases. In a chart review study, treatment pathways may be dictated by clinical factors, payers, practice pathways and innumerable, unmeasurable, other confounders. Thus, the fundamental criticism of any research utilizing retrospective observational data is that any comparisons that are made are biased. This is because there is an imbalance of factors across cohorts. When examining our data, we realized that we did in fact see clinical differences between the patient groups receiving BRAF—MEK inhibitors compared with those receiving immuno-oncology therapies. We also knew from other market research that certain providers may have preferences for therapeutic sequencing of agents. We used a hierarchical statistical model, controlling for both patient- and provider-level factors, to address this confounding. Other frequently used approaches, such as propensity score matching and inverse probability of treatment weighting, were discussed; however, due to the relatively small sample size of our study, we were not able to employ these.
What may the implications of your research be on health policy, decision making and prescribing patterns?
From policy and payer decision-making perspectives, we hope that our data begin to form a foundation of evidence in this field. We believe that no single, retrospective, observational study should be viewed as powerful enough to answer the question of whether BRAF—MEK inhibition or immuno-oncology therapy should be preferentially employed as first-line treatment for the patient population we studied. Rather, we hope that this study becomes part of the research canon in this space, allowing decision makers, patients and providers to have other evidence streams for consideration. We have begun exploring this through surveys of oncologists in the USA; we have questioned their preferred treatment approach, showing them the data and asking if this impacts their treatment approach. We hope to publish the findings of this exercise soon.
“…we hope that this study becomes part of the research canon in this space, allowing decision-makers, patients and providers to have other evidence streams for consideration.”
Do you, or others, have further RWE studies planned in this filed, for example, for evaluating the comparative effectiveness of other oncology therapies?
Hundreds, if not thousands, of real-world research studies in oncology are ongoing at any time across the USA and the globe. Recently, we searched PubMed for “real-world oncology” and found nearly 400 manuscripts published in 2018; there has been an exponential increase in the number of real-world studies in the field over the last 15 years. As more and more therapies come to market, often with accelerated approval requiring post-marketing demonstrations of efficacy and effectiveness, this trend is likely to continue, with a particular focus on reducing the time from drug discovery to the bedside.
What unique challenges are there in using RWE in oncology?
With the increasing personalization of therapies, the sample sizes of cohorts continue to become smaller and smaller; this can limit the accuracy and precision of measuring outcomes using RWE research methodology. It is worth noting that such ultra-orphan disease populations also complicate RCT performance. This is because patient accrual can be long, slowing the time to approval of therapies. This has resulted in an explosion of RWE, control-arm trials. Additionally, the newer the biomarker or the mutation, the less likely the data points exist in structured datasets. This makes identifying the patient populations for study increasingly difficult. Coupled with the fact that analytical methods continue to evolve, and physician comfort with RWD remains low, there remains a large hill to climb with regard to bringing RWE to oncology. We believe that, fundamentally, it will be important to conduct multiple, well-controlled studies to enhance the position of RWD/RWE in oncology.
Bruce Feinberg and Jonathan Kish are employed by Cardinal Health. There are no other conflicts of interest.