Look behind the lecture: designing meaningful models

Written by The Evidence Base

In this interview, Daniel Malone (University of Arizona, AZ, USA) discusses his presentation from ISPOR (18–22 May 2019, New Orleans, LA, USA) on what it takes to build a meaningful model for a rare disease.

Please could you introduce yourself and your institution.

I am a Professor at the University of Arizona, where I have been for 20 years. I first trained as a pharmacist and then went on to graduate school at the University of Texas (TX, USA), followed by a post-doc at the University of Washington (WA, USA). My primary areas of research are health economics and outcomes research. I am also involved in cost effectiveness modeling, evidence synthesis and health technology assessment.

What is it that interests you about real-world evidence and outcomes research?

We have lots of different sources for data, for example, from clinical trials. Clinical trial data can be of great use for answering certain types of questions. The challenge is that it leaves us a little short on how that data or information technology is actually implemented in real practice. We have recognized that not every technology is used optimally; many technologies are even used in ways that were never studied or expected in the first place. Real-world evidence helps us fill in those gaps in our knowledge and expectations.

“…many technologies are even used in ways that were never studied or expected in the first place. Real-world evidence helps us fill in those gaps in our knowledge and expectations.”

Your talk concerns developing meaningful models for rare diseases. What do you feel makes a meaningful model?

A model is meaningful when we can use the information a model produces and predict results that inform our decision-making process. Models are not perfect, we know that, but the important questions are: is the model a close enough fit? Does the model get us to a better position to make a decision than we would be in without it? Are there other pieces of information that we should look for besides that which is in the model already?

I am a big advocate of employing models; sometimes it is better to have some information than no information. The challenge is that some people think the information a model produces is the only definitive information. All models have uncertainty associated with them and we need to represent that uncertainty in the results the model produces; sometimes we do a good job of representing that information and uncertainty and other times we don’t.

“…we are building Frankenstein’s monster; we are taking parts from different models, putting them all together and hoping we get useful information at the end of the day.”

Why is it difficult to model rare diseases?

The biggest challenges with rare diseases — and these are not unique to the economic model, but rather, to the conditions — is the fact that you have very few observations which you can use in your clinical trials, and very little information about their natural history. This is because, for some of these conditions, very limited numbers of patients develop them and the lifespan is relatively short; this can result in a lot of uncertainty about what happens when you introduce a new technology into the treatment of these conditions and what impact it has on the trajectory of those people’s lives. We are making assumptions about the benefits that these technologies have, when currently, we do not know if these will hold true or not.

Further, because we have studied these patients in lots of different ways, or studied these conditions lots of ways, sometimes we take surrogate measures from other conditions and try to apply them to these rarer conditions. Depending upon how well those surrogate conditions represent the specific, rare condition, this may, or may not, be a valid approach.

When you are putting together your models, do you use data from publicly available databases?

Most cost effectiveness models are constructed from a variety of data sources. First, we start with epidemiological and epiphysiological evidence. We also use clinical trial data, to the extent that they are available. We look for patient preference studies that may have been done at a population, or disease-specific, level, as part of the clinical trial development to support a model as well.

“…instead of having therapies that simply manage chronic conditions…we are likely to see the emergence of therapies that work to fundamentally prevent these conditions arising.”

A useful analogy may be that we are building Frankenstein’s monster; we are taking parts from different models, putting them all together and hoping we get useful information at the end of the day. Where this process breaks down is when those pieces do not fit together so nicely; perhaps, evidence that we have taken from one area does not match up well with another key piece of information we have tried to incorporate into the model.

The other challenge that we face with rare disease modeling is that most cost effectiveness models have expressed their benefits in terms of quality-adjusted life years (QALYs), which have been relatively modest. A change of two to five QALYs may once have been considered large. Now, however, we are seeing differences of ten, 20, 40 QALYs, depending on the degree of discounting we apply. 

These are large improvements and, yet, because these products are of high upfront cost, they are being deemed not cost effective despite their significant quality gains. That is a whole different ballgame to what we used to deal with. Some people suggest it is just a function of changing prices. Others argue it is a function of how we do our models.

What has been the biggest change that you have seen in this field since you started, over 20 years ago?

What I think has changed significantly has been the level of interest in the field. When I first started attending these annual meetings, there were perhaps 300–400 attendants. At the ISPOR conference this year, there are over 3500 attendants. Further, our level of impact has changed; we are having a significant impact both in terms of health, and healthcare decision making. We have invested a lot in our methods, which are getting more precise; we still have long way to go, but we are getting better.

“Competition will drive prices, yet, without the right product in the first place, competition will never arrive…”

What do you think, going forward, will happen in this field in the next 5 years?

We are starting to see a revolution in science in terms of our knowledge of how biology works at the very basis of our being — our DNA and RNA — and how we may be able to manipulate this. As a result, instead of having therapies that simply manage chronic conditions and mitigate, to some extent, their associated morbidity, we are likely to see the emergence of therapies that work to fundamentally prevent these conditions arising. That can be scary as there is always the potential of unintended consequences arising from such therapies; once they are introduced into the gene pool, we may see the emergence of long-term implications for humans that we cannot, as yet, anticipate. On the other hand, we may be able to reduce a lot of the suffering, whilst increasing the productivity and humanity, of many individuals with chronic, rare conditions.

Our ability as an entity to evaluate new technologies, fairly and accurately, can have a significant impact on how far we are willing to push those exciting, innovative technologies. I do worry that if we squash too many of these technologies early on, because they are deemed to be too expensive, for example, then there will not be any investment or incentives to develop these technologies in the long-term. Further, in the pharmaceutical realm, competition is key. With every breakthrough product marketed, what is next coming down the pipeline is not that far behind. Competition will drive prices, yet, without the right product in the first place, competition will never arrive, as nobody will be willing to invest energy into developing the compound themselves.