Peek behind the paper: how could we get more out of the EHR?
In this feature, we peek behind the paper with contributing study author Taylor Davis (Klas Research, UT, USA): could poor user training underlie the failure to derive benefit from the electronic health record (EHR)?
In 2019, we reported on some important research investigating how user training could impact clinicians' abilities to derive benefit from electronic health records (EHRs).
Now, take the opportunity to evaluate how far the field has come in this interview with contributing study author Taylor Davis (Klas Research, UT, USA).
Please can you introduce yourself?
I am the Vice President of innovation at Klas Research (UT, USA). Our organization is well-known for conducting executive interviews that drive a ‘consumer report’-type of reporting to the market.
What prompted this research to be conducted?
We had noticed that there was significant clinician dissatisfaction with the EHR. However, the degree and range of this had not been measured across organizations. Therefore, over 2 years ago, we initiated a pilot study; we worked with colleagues at five healthcare provider organizations to build an end-user experience survey that was subsequently sent to all physicians at those five organizations. When we got those initial survey results back, we noted substantial variation in users’ self-reported EHR use experience across organizations employing the same EHR software.
…the only way that the EHR can make a real difference in health care delivery, is if health care providers use it effectively.”
Since then, we have had 196 participant organizations send our end-user experience survey to their physicians, nurses and other clinical staff. We have continued to observe high response variation across different organizations utilizing the same EHR. Thus, today, there is an ongoing collaborative of provider organizations working together to learn the very best practices around the EHR so that we can identify what factors and actions drive its success and, ultimately, implement those to improve health care delivery.
What benefits may electronic data sources offer for improving clinical practice?
For many years, the EHR has had the potential to transform the practice of medicine: to reduce costs, improve health care quality and provide novel insights to clinicians. However, this promise has been only partly fulfilled, as demonstrated by the feedback that we have received from individual clinicians who often feel that the EHR is getting in their way and frustrating, rather than aiding, them.
What are some of the challenges associated with employing EHRs as data sources?
Traditionally, EHR vendors and provider organizations have been very focused on EHR system optimization – building an EHR that works better. Indeed, there is a lot of room for improvement with system optimization; we can make EHRs more reliable, more intuitive and we can create far better interoperability. At the same time, however, the only way that the EHR can make a real difference in health care delivery is if health care providers use it effectively. This is what we have found at the Arch Collaborative; whilst we have collected a lot of research concerning how the EHR system can be better optimized to meet the needs of end users, in that, we observe great variation in user experience reports from different organizations, at which the exact same EHR software is utilized.
This implies that user optimization is significant amongst the challenges that we observe. We have not worked enough, well enough and in the right ways to ensure that clinicians are fully proficient with the software they are using and that their software and workflows are tightly aligned. Organizations must be well-built teams utilizing their software optimally, together. Often, the governance of the EHR, and communication around changes with the EHR, becomes adversarial, bloated or democratized in such a way that slows down progress. The human component of the EHR is critical to our mastering of the EHR and our ensuring that the EHR achieves its potential to revolutionize the way that we deliver care.
What may some of the practical implications of your research be on the benefits able to be derived from EHRs?
Even organizations that report great success with the EHR have room to improve with their EHR use. However, these organizations often function as better teams, work more collaboratively to focus on improving care pathways and have clinicians reporting better EHR proficiency such that they feel more enabled to deliver better care.
The human component of the EHR is critical to our mastering of the EHR and our ensuring that the EHR achieves its potential to revolutionize the way that we deliver care.”
Measuring the quality of care delivery is very complicated in health care. However, we have observed correlations between organizations with better self-reported, end-user EHR experiences and health care delivery outcomes. For example, we have observed lower infection rates – reported by the Centers for Medicare and Medicaid Services (CMS, MD, USA) – at organizations with better end-user EHR experience ratings. There is also a statistically significant correlation between end-user experience scores and organizations’ leapfrog computerized physician order entry (CPOE) safety scores – that measure how safe the EHR is. This suggests that when clinicians have better experiences with the EHR, patients are more likely to have safer experiences and receive, overall, a higher quality of healthcare.
Currently, we would strongly encourage organizations to look at a variety of methods available for measuring value and assessing how the EHR is being utilized and adopted at their institutions. Organizations can employ tools – such as Cerner Advance – that assess users’ proficiency. Indeed, we see a positive correlation between organizations who use such proficiency testing tools and user-reported training rating experiences at those organizations.
Another thing that has been made very clear from our research, so far, is that we require more clinical content to be smartly built into the EHR. We need to get to a place where clinicians look at the EHR as a colleague; a colleague is smart about their insights. There is great potential to make the EHR much smarter in terms of content that is built into it. This is not something that is trivial. Recently, I met with one of the key EHR software producers and vendors and discussed some of the challenges that they were facing with regards to building greater clinical content, that currently exists in the literature, into the EHR.
…we are starting to see evidence that the learnings from this Arch Collaborative are leading organizations to improve their users’ experience with the EHR.”
Often, they explained, they face opposition to this as different societies, associations and researchers seek loyalty fees to build this knowledge into the EHR. There are situations where such charges are appropriate, as these fee funds the continuation of further research. However, there is some level on which we have almost staked proprietary over our medical knowledge and this is preventing the field from progressing forward. All of medicine needs to work together to build greater intelligence into the EHR so that it can be employed to deliver better care.
Encouragingly, we have re-measured our study parameters at 16 participant organizations. At 14 of those, we have seen their end-user satisfaction scores increase; at six of those, this satisfaction improved significantly. This demonstrates that we are starting to see evidence that the learnings from this Arch Collaborative are leading organizations to improve their users’ experience with the EHR.
How did you determine eligible participant organizations and how did you assess the quality of training that users at these received? What may the limitations of this method be?
Organizations that participated in the study self-selected to do so; we must keep this in mind and be cautious when analyzing some of the study results. Nevertheless, a wide range of organizations, which employ different EHRs, participated in the study.
To assess the quality of training end users received, we asked clinicians several experiential questions about their initial training, their ongoing training and how they felt about the ways their training was/is delivered to them. We asked a broad range of open-ended questions to ensure that we were observing correlation; the nature of these questions also ensured that we captured, fairly accurately, clinicians’ experiences. What we may not have captured well, however, is the true proficiency of clinicians in practice. Just because clinicians feel more educated, that may not mean that they are more educated practically.
How did you determine that variation in user experience was not due to differences in regulatory burden or programing design?
In order to assess this, we performed an analysis of variance. At the 196 organizations that participated in our study, the very same EHR software was employed at several of these. Therefore, the observed variation in end users’ EHR experience scores could not solely have been due to differences in software design. Indeed, 73% of the time, we observed at least two physicians, of the very same specialty, at the very same organization, with the exact same software at their disposal, report diametrically opposed opinions regarding whether the EHR enabled their delivery of high-quality care. At the Arch Collaborative, we believe that the main driver of this observed variation concerns the high variation in the ways that we have prepared physicians, and in the ways that they have prepared themselves, to utilize EHRs.
The EHR is one of physicians’ most important medical tools…”
For example, a very large health system in the mid-west told us that they require no training for new physicians who are to be utilizing the EHR. Instead, they encourage the physicians to shadow someone in their same specialty to see how to use the EHR. Understandably, in an organization like that, we measured especially high variation in user competency with the EHR between clinicians, as the level of user training at that organization was very poor. The EHR is one of physicians’ most important medical tools; we may permit a far increased number of medical mistakes to be made if we do not do a good job of educating and preparing users to be able to practice medicine with tools such as the EHR.
How do you think use of EHRs as data sources will evolve in the future?
I think we will recognize the potential that the EHR has to offer; we are already making positive progress toward this, but it is going to take us a while. As an illustration: when the automobile was invented, it could only go as fast over dirt roads as a carriage could. It took a long time to realize that a key limiting factor to the automobile’s speed, was the nature of the road itself. Similarly, we are still learning what the limiting factors to the EHR's success are. One, not the only one, but one factor is the human. As we look forward, we see a day where there is incredible intelligence built into the EHR. The humans that are utilizing the EHR must be able to know how to benefit from its intelligence and what the limitations to its use are.
Today, largely, when it comes to clinical decision making and alerting, humans do not have the support of their software. In large part, that is because we have not built and optimized the software well. However, it is also because we have not worked well enough with users. There is much room for improvement, but I do think we will get to a stage where the average physician or nurse feels very naked without the assistance of computerized technology as they are delivering care. Whether this is because they are worried they are going to miss insights from the patient, or that they really need something to help in their decision making and diagnoses, or that they need that watchful care to ensure that they don’t make a mistake as they are delivering care. Parts of that equation are a reality today, but there is still great potential to progress this much further
Read the full research article here:
Longhurst CA, Davis T, Maneker A et al, on behalf of the Arch Collaborative. Local investment in training drives electronic health record user satisfaction. Appl Clin Inform. 10(02): 331-335 (2019)