Ask the Experts: Pall Jonsson on the importance of transparency in real-world evidence
In this feature, Pall Jonsson, from the National Institute for Health and Care Excellence (NICE; Manchester, UK), discusses his presentation from ISPOR Europe 2019 (2–6 November; Copenhagen, Denmark) on the importance of open transparency in real-world evidence (RWE) generation and use.
As part of our ‘In focus’ on transparency in real-world evidence (RWE), we have posed some essential questions to industry experts in order to gain a better understanding of the current state of the field, and how we may work towards improved transparency.
In this instalment, Pall Jonsson, from the National Institute for Health and Care Excellence (NICE; Manchester, UK), shares his expert insights and discusses his presentation from ISPOR Europe 2019 (2—6 November; Copenhagen, Denmark) on the importance of open transparency in RWE generation and use.
Enjoy this post? Want regular updates straight to your inbox? Become a member on The Evidence Base®, for free, now>>
Please could you introduce yourself and your institution?
I head up the Science Policy and Research program at NICE. We have been active in the space of RWE and big data — working out how an organization like mine can make the most of the abundance of data that is generated outside of the clinical trial setting.
What is the importance of transparency with respect to RWE?
We are increasingly understanding that randomized trials don’t answer all the questions that, especially payers and health technology assessment bodies, need to answer. RWE has a role here, however there is a range of issues associated with RWE, including challenges associated with integrity, quality, risk of bias and confounding. Healthcare decision makers see lack of transparency regarding how evidence is generated outside of organized clinical trials as a major barrier to using RWE for high-stakes decisions.
“Healthcare decision makers see lack of transparency regarding how evidence is generated outside of organized clinical trials as a major barrier to using RWE for high-stakes decisions.”
One way to improve trust and confidence in these data is to be as open as possible about how the data were generated and how subsequent analyses were performed. These disclosures should happen before analyses even take place so that we can minimize concerns about selective inclusion of data, data dredging and ‘pre-looks’ — when hypotheses are developed after having looked at the data.
If we provide insights into the totality of the evidence upfront, stakeholders can assess both its reproducibility and replicability — both of which are important for establishing the credibility of RWE. This then ideally becomes a virtuous circle: over time, we should see decision makers establishing greater trust in RWE studies, elevating the credibility of well conducted RWE studies.
Saying that, it’s important to highlight that transparency in the generation and analysis of RWE does not automatically mean that the evidence is high quality. Transparency is one step towards understanding data quality, but we need to look more broadly at the evidence to understand the risks of confounding and biases, for instance.
What are some of the challenges associated with achieving transparency with RWE?
Conductors of clinical trials, and those working with clinical trials, are well aware of reporting requirements and the transparency that is required for conducting trials. On the other hand, RWE is, by its very nature, generated outside of a clinical trial setting; individuals collecting data or conducting analyses may not be as aware of what constitutes best practice and how best to achieve genuine and meaningful transparency.
“Currently, we have no universally agreed standards on reporting of RWE. We need to agree on minimum standards that can apply across jurisdictions — and ensure that key stakeholders are with us on this journey to help drive implementation.”
There are also additional challenges associated with secondary use of data because, in these situations, data are being used for a purpose that is different to that for which they were originally generated. Extending transparency into the whole evidence generation pathway becomes quite difficult in these cases.
We also need to think about commercial interests, academic interests and the protection of intellectual property. While transparency is a great principle, we may need to balance the requirements to ensure we are not stifling innovation by imposing requirements that could deter organizations or professionals from making the most of the value that lies in relatively underused data generated in healthcare systems across the world.
How may we work towards improved transparency in RWE research?
Currently, we have no universally agreed standards on reporting of RWE. We need to agree on minimum standards that can apply across jurisdictions — and ensure that key stakeholders are with us on this journey to help drive implementation.
The newly published recommendations developed by the International Society of Pharmacoeconomics and Outcomes Research’s (ISPOR; NJ, USA) RWE Transparency Initiative are a great first step. Rather than covering all RWE, the recommendations apply to a subset of studies — the so called ‘hypothesis evaluating treatment effect’ (HETE) studies. The Initiative recommends the registration of HETE studies based on secondary data use, such as of insurance claims and electronic health records. Why did we focus on this subset? HETE studies conducted on secondary data to establish comparative effectiveness or safety of interventions are most at risk for data dredging or cherry picking of results and perceived lack of credibility.
There is not a single path towards common practice, rather a combination of approaches that will need to be taken.”
We also need to make sure we identify a good home for a centralized registry — or alternatively a small number of interconnected registries that work according to mutually agreed data requirements and a common data model. It should be easy to both submit and search for information RWE studies.
How may we work towards ensuring the registration of real-world data study protocols becomes common practice?
There is probably a combination of ‘carrot and stick’ that will work to drive transparency into common practice; there could be positive incentives built into data use agreements to encourage registration of RWE studies. Regulators, health technology assessment bodies and payers could signal their preference for data that have been generated in a transparent way. Alternatively, journal editors could require registration as a pre-requisite for publication.
There is not a single path towards common practice, rather a combination of approaches that will need to be taken. To help to drive implementation, we should also initially consider what are the minimal reporting standards, rather than trying to capture every single aspect of all studies. Ideally, registration should not cause additional delays in the conduct and reporting of new knowledge.
How do you see transparency in RWE, and use of RWE, evolving in the future?
I am hoping we can move towards universally recognized standards for reporting, which will move us out of a ‘wild West’ situation to one that is more conducive to creating trust in the evidence that is generated in routine practice. After all, many data are generated in routine healthcare practice; it would be a lost opportunity not to use these data to generate learnings that can directly improve clinical practice.
Enjoyed this post? Become a member on The Evidence Base®, for free, now to receive regular updates straight to your inbox>>
Disclosures:
The opinions expressed in this feature are those of the interviewee/author and do not necessarily reflect the views of The Evidence Base® or Future Science Group.