1st GetReal Institute Annual Conference: Deep dive into Session 1 – a multi-stakeholder reflection of the current landscape, challenges and future of real-world evidence
On 16th March 2023, GetReal Institute held their inaugural conference in Utrecht, The Netherlands, focusing on ‘A European Outlook on Real World Evidence in a Global Context’. Here we take a ‘deep dive’ into the discussions that took place during the first session of the day focused on ‘Setting the Scene: RWE and the Global Research Landscape. What is the Future?’
The conference began with an introduction from Shahid Hanif, Managing Director of the GetReal Institute. Shahid welcomed the diverse and eclectic audience attending the conference that included people from various stakeholder groups including regulators, health technology assessment (HTA) bodies, academia, industry (both pharma and non-pharma), as well as patient representatives. After providing an update on GetReal Institute, Shahid introduced the first session of the day, moderated by Bart Barefoot (GlaxoSmithKline, UK) and featuring Andrew Roddam (Our Future Health, UK), Bettina Ryll (Melanoma Patient Network Europe, Sweden), Wija Oortwijn (Health Technology Assessment International, The Netherlands) and Peter Mol (CBG/Dutch Medicines Evaluation, The Netherlands).
Jump to:
Whistlestop reflection of RWE and the global research landscape?
Panellists’ perspectives
Q&A – How has the generation, acceptance and use of RWE evolved over the past 10 years, what progress has been made in putting RWE into practice and what are the remaining challenges?
Q&A – If you had to pick one thing to advance the field, what would this be and how can the GetReal Institute facilitate this?
Q&A – Do you see any roles for machine learning and artificial intelligence in any of your areas?
Whistlestop reflection of RWE and the global research landscape
Andrew Roddam kicked of the panel discussion by briefly looking back on the past 10–15 years, talking about why the field was established and how real-world evidence (RWE) has evolved.
“Often when we talk about real-world evidence, we focus very much on the end stage of the process where we think about developed medicines and evidence and value efficacy, safety, and how they they are used, but actually the opportunity and the pull through from the evidence in the discovery and the development of medicines and devices is really important, because it drives towards personalization of those medicines, and it drives towards appropriate use, and the better we get at that initial point will ultimately drive us towards the end point.”
Roddam explained that the availability of broader real-world data (RWD) sources, such as more routine monitoring of individuals through wearables and other technologies that track and monitor patients outside the healthcare system, has meant that RWE has grown significantly over the years. He emphasized the importance of always asking what question you are trying to answer through RWE and what is the best way to answer this. RWE should be used as part of a ‘package’ of information that is used together to address the thing that’s most important; that is, to develop new therapies, interventions or vaccines for patients to improve their outcomes.
Roddam went on the explain that there has been significant progress in the recent years, with numerous guidelines or white papers outlining the practicalities and best practices for RWE, as well as several large-scale demonstration projects, including one study in the UK involving 140,000 people looking at a new diagnostic test for early detection of cancer in the real world. Yet, despite these advances, implementation of RWE in regulatory decision making is more of a challenge. Natural heterogeneity of people and variation across geographies means that generalizability of data across the world is a real challenge. Whilst data collection is fragmented, initiatives like GetReal are helping to improve standards and ensure RWE is perceived as the same quality of evidence as gathered through randomized controlled trials (RCTs).
The session closed by looking at priorities for the future. The heart of this needs to be driven by the purpose for which you are using the evidence and what needs to be done to achieve this, whether it’s RCTs or RWD studies. The four main foundational elements to be considered are:
- Data availability and scaled quality – Is the data good enough?
- Global variability – How can data from different countries be used?
- Methodology – How do you use the data in the appropriate way?
- Reproducibility – Can you replicate the findings that you found (which is also key to determining the methodology)?
Panellists’ perspectives
Moving on to the second part of the session, the remaining panellists provided their own perspectives on RWE.
Wija Oortwijn began by stating that Health Technology Assessment International has been discussing RWE since 2016. From the HTA perspective, they experience many of the challenges highlighted by Andrew Roddam, including issues with quality and transferability of data. Oortwijn echoed the need to focus on what RWE is used for, an element they are discussing with the multi stakeholder groups within the global HTA community. Existing policy structures and information governance are not always enabling the use of RWD/RWE. Furthermore, coordination and consensus building is needed within Europe, especially with the new HTA regulation, to ensure there is adequate, consistent and reliable use of RWD/RWE in HTA practice. Many countries are establishing their own RWE frameworks; for example, NICE, IQWIG and Haute Autorité de Santé (French National Authority for Health), but these are not aligned and this is compounding the challenges.
Bettina Ryll continued with the patient perspective, providing her own experience in melanoma and the need to look at alternatives to RCTs to improve the outcomes for patients. RCTs are restrictive in terms of inclusionary criteria and do not often reflect the real-world experience of patients.
“Melanoma is so deadly because it likes to metastasize to the brain, and that’s the driver of mortality. But these patients were excluded from the trials…… if we start introducing new therapies and let them what I call ‘out in the wild’, we’re not learning from that experience where we are suddenly using new therapies in a very different population.”
Peter Mol followed with the regulatory viewpoint, explaining that patient stories are at the heart of what drives all decision making across all healthcare sectors. He welcomed the use of the term RWE, instead of observational and epidemiological data, to reflect the broader use of different types of data in decision making at various stages throughout the lifecycle of a drug. Mol then echoed Andrew Roddam’s point that it shouldn’t be a case of ‘either/or’ – both RCTs and RWE are needed and it is a case of determining when to use RWE and what data is needed. He explained that there is so much data out there, with the tools to analyze this data improving, and with projects like DARWIN EU and other HORIZON 2020 projects within Europe leading the development of the field.
You can also read more about DARWIN EU here:
- RWE in Europe is ‘evolving’ with the launch of new DARWIN EU® institution
- EMA announces first DARWIN EU® Phase 1 data partners and opening studies
Q&A – How has the generation, acceptance and use of RWE evolved over the past 10 years, what progress has been made in putting RWE into practice and what are the remaining challenges?
Several key developments were raised by the panel when discussing this question:
Terminology – Peter Mol repeated his previous remark about the adoption of terminologies. A highlight from the past 10 years in the EMA has been an initiative involving registries, talking to various stakeholders about how data from registry-based studies can be included appropriately in HTA decisions. Instead of developing a guideline on registries, they purposefully developed a guideline on registry-based studies, which provides advice on the design, data elements, quality and governance of a registry. In the future, a big step for the EMA now is what DARWIN EU will deliver to the European system.
Practical applications – from a HTA perspective, Wija Oortwijn explained that RWE has been used in practice by HTA bodies, but only post-launch for the evaluation of adverse effects or for cost estimation, and not for informing reimbursement decisions, for example. Post-launch evidence generation is fragmented in Europe. In recent years, the use of value frameworks to use RWE in combination with data from clinicals trials and other sources has evolved, a trend that Oortwijn hopes will continue in the future. One of the challenges is the generalizability of data and what kind of data or evidence the authorities need to make an informed decision, especially with regard to the price of a reimbursement which differs across Europe. Aligning frameworks, collaboration, governance structures, policy structures, ethical standards all remain a challenge from a HTA perspective.
Collaboration – Bettina Ryll noted that in the past decade there has been a shift in culture and integration of different fields, for instance, pharmacovigilance and epidemiology, as well as digital advances that have challenged the system and thinking. There is now a willingness between fields to think about how data can be used for good, but also what’s the risk associated with it. The Duchenne patient community and their patient registry was given as an example model to emulate in terms of data collection and handling. Looking at what others have done in terms of conceptual thinking challenges your own lens so there is more motivation to look at different datasets for evidence.
Evidence gaps – Andrew Roddam remarked that understanding and better clarity about who’s responsible for generating the evidence at different stages is key to the future. Evidence packages are generated for regulatory and HTA decisions but who becomes responsible for generating the evidence in practice?
Ethical frameworks – Wija Oortwijn pointed out that one of the challenges is how do we define the standards for data quality and who’s in charge of finding the standards for quality?
Data governance – Bettina Ryll also stated that from her perspective, another key challenge is that once the data is out there, what happens if something goes wrong? As a next step, legal frameworks need to be developed that also ensure trust.
Q&A – If you had to pick one thing to advance the field, what would this be and how can the GetReal Institute facilitate this?
Wija Oortwijn kicked off by staying that from a HTA community perspective, capacity is key – how to use RWE that is fit-for-purpose? Education and training are key to achieving this. It was suggested that the GetReal Institute could help to accomplish this and act as a broker to bring the different stakeholders together in a neutral space and to set the agenda for RWE.
Peter Mol explained that is his view use cases – where RWE should be used – as well as DARWIN, replication studies and harmonization within Europe, are all elements the GetReal Institute should focus on.
From the patient perspective, Bettina Ryll raised a discussion about the value proposition for patients in using their data for RWE. How data is used and the legal implications, for instance, in using genomic data. Ethical frameworks are vital.
Andrew Roddam added to the discussion by explaining that we need to move forward from policy-led discussions to real-world examples that demonstrate use from the beginning to the end, not just for regulatory or payers for example, but the whole pathway to improve patient outcomes.
Q&A – Do you see any roles for machine learning (ML) and artificial intelligence (AI) in any of your areas?
Peter Mol addressed this final audience question, explaining that by nature regulators are quite conversative. He stressed that any use of AI needs to be validated. One of the first applications of AI will be in the understanding of pathology slides but AI-driven analyses of data to support efficacy still has a long way to go. Oortwijn then asked the question to Wim Goettsch (University of Utrecht, The Netherlands) in the audience to offer his experience from H2020 HTx who have a project on AI and ML. Wim Goettsch explained that a conservative approach is being taken to learn to what extent AI can be used in HTA. Whilst there’s optimism, there’s still a long way to go. There’s a lot of variety in the algorithms used and data sources, and we need to know for sure what is going to happen to the data. He agreed with Mol that whilst in the very early stages for HTA, AI is still a long way off to even being considered for regulatory purposes.
Want regular updates direct to your inbox? Become a member on The Evidence Base, for FREE, now >>>