How artificial intelligence explains real-world evidence: an interview with ChatGPT
The new AI-assisted chatbot, ChatGPT, is becoming one of the most talked about technologies in 2023 so far. As a scientific content provider, The Evidence Base was keen to understand more about this technology’s ability to answer questions and provide information on real-world evidence (RWE).
In this ‘interview’ we asked ChatGPT a series of general questions about RWE and share the answers to these questions, which we hope you will find of interest.
Please could you tell us about yourself and your role?
I am Assistant, a large language model trained by OpenAI. I do not have a personal identity or role in the way that humans do. My primary function is to assist users in generating human-like text based on the prompts that I receive. I do not have personal experiences or a personal history, but I have been designed to have a broad understanding of a wide range of topics and can provide information and answer questions on a variety of subjects.
What is real-world evidence?
Real-world evidence (RWE) refers to data that is collected from a variety of sources outside of traditional clinical trials. This can include observational studies, registry data, electronic health record data, claims data, and patient-reported outcomes. RWE is increasingly being used to evaluate the effectiveness and safety of medical treatments, procedures, and products in real-world clinical practice, and to inform decision-making about their use. The use of RWE is growing in importance as it can provide valuable information about how treatments and products perform in routine clinical practice, rather than just in the controlled setting of a clinical trial. This can be particularly useful for assessing the long-term safety and effectiveness of treatments and for evaluating treatments in patient populations that may not have been well represented in clinical trials.
How does real-world evidence differ from randomized clinical trial data?
RWE refers to data that is collected outside of the traditional clinical trial setting, in the “real world” as patients go about their daily lives. This can include data from electronic health records, claims data, patient-generated data, and other sources.
Randomized clinical trial (RCT) data, on the other hand, is collected in a controlled setting as part of a formal clinical trial, in which patients are randomly assigned to receive either a treatment or a control.
There are several key differences between RWE and RCT data:
- Population: RWE studies generally include a broader and more diverse population than RCTs, which are often restricted to specific age groups, health status, and other characteristics.
- Setting: RWE studies take place in the “real world” outside of a clinical trial setting, while RCTs take place in a controlled setting.
- Data sources: RWE studies may use a variety of data sources, such as electronic health records, claims data, patient-generated data, and others. RCTs, on the other hand, typically collect data directly from patients through structured assessments and measurements.
- Study design: RWE studies are often observational, meaning that they do not involve randomization or manipulation of the treatment being studied. RCTs, on the other hand, are experimental, and involve the random allocation of treatments to study participants.
- Bias: RWE studies may be subject to more bias than RCTs due to the lack of control over patient characteristics and treatment exposures. RCTs, on the other hand, are designed to minimize bias by randomly assigning treatments and using other techniques such as blinding.
How can real-world evidence guide drug development?
RWE can be used to inform drug development in a number of ways. For example, RWE can be used to:
- Identify potential areas for new drug development: RWE can be used to identify unmet medical needs or areas where current treatment options are insufficient. This can help to inform the development of new drugs that address these needs.
- Support regulatory decision-making: RWE can be used to support regulatory submissions for new drugs, including to demonstrate the effectiveness and safety of a new drug in a real-world setting.
- Inform clinical trial design: RWE can be used to inform the design of clinical trials by providing insight into patient characteristics, treatment patterns, and outcomes in real-world clinical practice.
- Enhance drug labeling: RWE can be used to update or expand the information included in a drug’s labeling, such as by adding information about the use of the drug in certain patient populations or in combination with other treatments.
- Monitor the safety and effectiveness of drugs in real-world use: RWE can be used to monitor the safety and effectiveness of drugs after they have been approved and are being used in clinical practice. This can help to identify any unexpected side effects or issues with the drug’s use in a real-world setting.
Do regulatory agencies use real-world evidence in their decision making?
Yes, regulatory agencies often use RWE in their decision-making processes. RWE refers to data that is collected outside of the traditional clinical trial setting, in the “real world” as patients go about their daily lives. This can include data from electronic health records, claims data, patient-generated data, and other sources.
Regulatory agencies may use RWE to supplement the evidence from traditional clinical trials and help inform a variety of decisions, including:
- Approval of new drugs and medical devices: RWE may be used to support the approval of new drugs and medical devices by providing additional information on the effectiveness and safety of these products in real-world settings.
- Labeling and labeling changes: RWE may be used to support the inclusion of new indications or expanded use for approved products, as well as changes to product labeling.
- Post-marketing surveillance: RWE may be used to monitor the performance of approved products in the real world and identify any unexpected adverse events or other safety concerns.
- Overall, the use of RWE in regulatory decision-making is increasing, and regulatory agencies around the world are actively exploring ways to incorporate RWE into their processes.
Is real-world evidence less reliable than evidence from randomized clinical trials?
RWE and evidence from RCTs are both important sources of information for evaluating the effectiveness and safety of medical interventions. However, each has its own strengths and limitations, and it is important to consider the context in which the evidence is being used.
RCTs are generally considered the gold standard for evaluating the effectiveness of medical interventions because they are designed to minimize bias and maximize the reliability of the results. However, RCTs are often conducted in highly controlled settings with carefully selected study populations, which may not fully reflect the “real world” in which interventions are used.
On the other hand, RWE is collected in the “real world” outside of a clinical trial setting, and may include data from a broader and more diverse population. This can provide a more realistic and comprehensive understanding of how an intervention performs in everyday clinical practice. However, RWE studies may be subject to more bias due to the lack of control over patient characteristics and treatment exposures, and may not always be as reliable as RCTs.
Overall, both RWE and RCT evidence are important for informing decision-making, and it is often best to consider multiple sources of evidence when evaluating the effectiveness and safety of an intervention.
How can we build trust in real-world evidence?
There are several steps that can be taken to build trust in RWE:
- Use high-quality data sources: The data used to generate real world evidence should come from sources that are reliable and accurately reflect the population being studied.
- Use rigorous study designs: Real world evidence studies should be designed in a way that minimizes bias and maximizes the reliability of the results. This can include using techniques like randomization, blinding, and control groups.
- Ensure transparency: It is important to be open and transparent about the methods and data used in a real world evidence study. This can help build trust in the validity of the results.
- Engage stakeholders: Involving stakeholders such as patients, clinicians, and policymakers in the design and interpretation of RWE studies can help build trust in the results.
- Use multiple sources of evidence: Combining real world evidence with other types of evidence, such as randomized controlled trials, can help increase the reliability and trustworthiness of the results.
- Use appropriate statistical techniques: Using appropriate statistical techniques, such as sensitivity analyses, can help ensure the reliability and validity of the results.
Why might patients be interested real-world evidence?
There are several reasons why patients might be interested in RWE:
- RWE can provide valuable information about the effectiveness and safety of medical treatments in real-world clinical practice, which can help patients make more informed decisions about their healthcare.
- RWE can help to fill in gaps in our understanding of how treatments work in a real-world setting, particularly for treatments that have not been well studied in clinical trials.
- RWE can provide information about the benefits and risks of treatments for specific patient populations or in certain clinical contexts, which can be particularly useful for patients who may not have been well represented in clinical trials.
- RWE can help to inform the development of new treatments and therapies, which can lead to improved healthcare options for patients in the future.
- RWE can help to inform clinical practice guidelines and decision-making by healthcare providers, which can improve the quality of care that patients receive.
What new technologies are being used to generate real-world evidence ?
There are a number of new technologies that are being used to generate RWE, including:
- Electronic health records (EHRs): EHRs are digital records of patient health information that are generated and stored by healthcare providers. EHRs can be a rich source of RWE, as they capture a wide range of data on patient diagnoses, treatments, and outcomes.
- Mobile health apps: Mobile health apps, which are used on smartphones and other devices, can collect patient-generated data on a wide range of health-related behaviors and outcomes. This data can be used to generate RWE on the effectiveness and safety of interventions.
- Wearable devices: Wearable devices, such as fitness trackers and smartwatches, can collect a wide range of data on patient activity levels, sleep patterns, and other health-related behaviors. This data can be used to generate RWE on the effectiveness of interventions for conditions such as obesity and sleep disorders.
- Electronic patient-reported outcomes (ePROs): ePROs are digital tools that allow patients to self-report on their health status and outcomes. ePROs can be a valuable source of RWE, as they can capture patient-reported outcomes that may not be captured through other data sources.
Overall, these new technologies are providing valuable insights into the effectiveness and safety of interventions in real-world settings, and are helping to generate more comprehensive and reliable real world evidence.
What is the future of real-world evidence?
It is difficult to predict the exact future of RWE, but it is likely to continue to be an important source of information for healthcare decision-making. As technology and data collection methods advance, it is possible that the amount and types of RWD available will increase, allowing for more comprehensive and detailed analyses of the effectiveness and safety of medical treatments and products in real-world clinical practice. Additionally, the use of RWE may become more standardized and the methods used to collect and analyze this data may become more standardized as well.
So what does The Evidence Base think?
ChatGPT was able to swiftly provide comprehensive and detailed answers to our questions in a matter of seconds. We were quite impressed with the structure/format of how these responses were generated as well. Questions do remain however with regards to the sources of the information and their reliability and how the information should/could be used. There’s also the concern if there are any copyright issues in terms of using the content generated by ChatGPT as well as how this published content can be used in the future.
How will ChatGPT be used in the real world? It’s definitely a case of watch this space…..
