Streamlining Clinical Decision Support

Clinical decision support (CDS) helps referring providers order the most appropriate imaging exam for each patient’s clinical condition, reducing unnecessary and potentially harmful imaging.¹ The Protecting Access to Medicare Act (PAMA) mandates that all referring providers consult CDS when ordering outpatient advanced imaging studies, including CT, MRI, nuclear medicine, and PET, with financial penalties for noncompliance slated to begin in 2023.

But many referring providers, like Katherine W. Kent, MD, of University of Virginia (UVA) Health, have reported that CDS can be cumbersome to use.² In particular, they say that searching for structured indications in the dropdown menus that most CDS systems include is time consuming and limiting. They would prefer to enter free-text indications.

“Sometimes the indications and the clinical questions that I have are unique and not included in the dropdown menu,” explains Kent, associate professor of obstetrics and gynecology. “I don’t want to search through a list of structured indications, hoping for a match so that I can complete my order.”

As vice-chair of clinical operations and informatics for the department of radiology and medical imaging at UVA Health, Cree M. Gaskin, MD, supports technology investment and implementation, including improved ordering methods at UVA.

To make CDS more user friendly and increase its adoption among referring providers, radiologists at UVA have integrated an artificial intelligence (AI) tool into their CDS system that helps translate free-text order entries into structured indications. CDS systems require structured indications to score the appropriateness of exams based on each patient’s clinical condition and qualified appropriate use criteria.

Since implementing the AI tool in 2020, radiologists at UVA have found that referring providers used it 59% of the time when ordering imaging with CDS.³ They also found that it has encouraged ordering providers to use CDS for image ordering — positioning UVA for increased PAMA compliance.

“What we’ve seen is that this new AI-based approach is the preferred mechanism for choosing structured indications,” says Cree M. Gaskin, MD, professor of radiology and orthopedic surgery and associate chief medical information officer at UVA Health, who led the implementation and evaluation of the AI-based tool. “Referring physicians appreciate the opportunity to communicate with free text in their orders, and we continue to see successful results with our evolving clinical decision support system.”

Identifying a Challenge

UVA Health integrated CDS into their electronic health record in 2014. Ordering physicians anecdotally expressed discontent with the process of selecting structured indications from incomplete or overly lengthy lists, Gaskin recalls.

“Providers find picking from a list of structured indications frustrating,” says Gaskin, who is also vice-chair of clinical operations and informatics for the department of radiology and medical imaging. “They prefer to enter free text as they have for decades. Free text is more organic in terms of the way they think versus having to pick from a list. They want that freedom.”

This sentiment is not unique to UVA referring providers. A study published in the Journal of the American College of Radiology (JACR) reveals that 72% of unscored CDS imaging orders and 42% of total orders included free text.⁴ Gaskin says that the study results underscore providers’ preferences or ingrained ordering habits, as well as their frustration with being forced to categorize the unique patient narrative into a pre-formed structured indication written by someone else.

The inability to use free text can discourage referring providers from consulting CDS for image ordering, lead them to consistently select “other” from the menu of structured indications, or exit the workflow to bypass the system, says David S. Gish, MD, who completed a radiology residency and musculoskeletal imaging/informatics fellowship at UVA and helped evaluate the impact of the AI tool. These workarounds could result in inappropriate care and potentially cost radiologists and imaging facilities, who under PAMA will incur financial penalties when referring providers order advanced imaging for Medicare patients without consulting CDS.

It’s not just referring providers that prefer free-text indications. Radiologists also favor the narrative format because the information helps them better serve referring providers and patients, says Gish, now an attending radiologist at Sentara Rockingham Memorial Hospital in Harrisonburg, Virginia.

“Reading the patient’s story and understanding what the provider is looking for can influence our interpretation as we make a diagnosis,” he explains. “A succinct, sterilized structured indication doesn’t capture all the details you’d like to have. You lose that useful clinical history.”

Finding a Solution

Recognizing that the value of CDS is diminished when referring providers struggle to identify the appropriate indications from the structured list, UVA radiologists were hopeful when their CDS vendor approached them about integrating a new AI-based indications tool integrated into the CDS system.

“We were one of our vendor’s first customers and have remained highly engaged with evolving CDS technology. We evaluated the new AI-based solution and promptly decided to implement it, knowing that our ordering providers would prefer inclusion of free-text narratives about their patients’ conditions,” Gaskin says.

David S. Gish, MD, was lead author of the JACR article “Retrospective Evaluation of Artificial Intelligence Leveraging Free-Text Imaging Order Entry to Facilitate Federally Required Clinical Decision Support.”

The AI tool uses an algorithm to identify structured indications based on a provider’s free-text reason for ordering an imaging exam and information from the EHR, including the patient’s age, gender, and EHR problem list data. Once a provider enters a free-text indication, the tool immediately provides a list of predicted structured indications, those most likely a match, and additional possible, but less likely, indications.

Providers then select their choice from the predicted list. If the provider doesn’t agree with any of the predicted options, they can choose to search the full menu of structured indications, report that no matching indication was available, or exit the CDS workflow altogether.

“Once a provider enters the free text and selects a structured indication, the CDS system provides an appropriateness score for the exam,” Gish says. “Ultimately, the ordering provider can still order the exam that they feel is best to answer their clinical question. The tool just helps guide them.”

Implementing the AI Tool

While the radiologists were eager to deploy the integrated AI tool to improve the CDS workflow, they needed to test it before offering it to referring providers. “I knew the idea of applying AI to free-text was excellent in theory, since it would streamline the selection of structured indications, but we really needed to find out if the first-generation concept was robust enough to implement in clinical practice,” Gaskin explains.

To test the tool’s general performance, Gaskin and his team ran free-text indications from previous imaging orders through the system and determined that the algorithm offered reasonable predicted indications about 80% of the time, which the team considered acceptable for implementation. “The system did a good job, definitely solid enough to try it with ordering physicians,” he says. “It wasn’t perfect, but if it could guide them in choosing the accurate structured indications most of the time, we were willing to try it.”

Gaskin presented the AI tool’s capabilities at several high-level clinical meetings to ordering providers and administrators. They were accepting and sometimes enthusiastic about the idea of supporting free-text order entry and AI-assisted structured indication selection.

Following getting buy-in from the group, Gaskin worked with UVA Health IT’s training team to email ordering providers training materials, including tip sheets and slide decks, for using the AI tool within CDS. They also notified the UVA Help Desk about changes to the workflow, ensuring that their staff members were informed if providers reached out with questions.

After a brief pilot with a small number of ordering providers, UVA Health took the AI tool live throughout the system in February of 2020, allowing all ordering providers to begin using free-text when placing advanced imaging orders with CDS. Alternatively, they could continue using the existing CDS process of picking structured indications directly from a menu.

Increasing CDS Adoption

Three months after implementing the CDS AI tool, Gaskin and Gish performed a retrospective analysis of indication selection methods to assess user behaviors and preferences. The results, which were published in the JACR, showed that providers chose the new AI approach for 23,580 orders or 58.9% of the time. The AI tool yielded predicted structured indications in 91.7% (21,631) of the orders that providers submitted with free text. Providers chose AI-predicted structured indications in 57.7% of cases in which the tool offered them.³

“The advantage of the AI approach is that I can convey what I need to in free text and still satisfy the decision support requirements so that I can complete my order,” Kent says. “It keeps my workflow moving so that I can stay focused on patient care.”

While not part of the above published study, the radiologists anecdotally found that the AI tool also encouraged referring providers to use CDS more often. Before the radiologists implemented the tool, referring providers electively consulted CDS for approximately 50% of outpatient advanced imaging orders. After implementation of the tool, referring providers consulted CDS for roughly 70% of such orders, representing a significant increase, though a portion of this improvement could also be attributable to other changes made simultaneously to the ordering mechanism.

“Before this tool, we had a significant percentage of orders that did not get decision support,” Gaskin says. “Our PAMA compliance has improved significantly, which has been positive from an institutional standpoint. The more orders we have using CDS, the better.”

UVA’s referring providers are using CDS more despite a loophole that allows them to exit the workflow at the free-text order portion of the process, before decision support is applied. Gaskin and his team found that 21.3% of referring providers exited the workflow before CDS could score the exam. They are now working with the vendor to close the loophole, which Gaskin predicts will further increase PAMA compliance.

Leading Appropriate Care

As health systems nationwide implement CDS in line with PAMA, many are looking for ways to get ordering providers to use the technology for all outpatient advanced imaging orders. UVA’s experience shows that tools like the AI-based structured indications technology could help increase adoption of CDS and, in turn, improve care for patients.

“This initiative allowed us, as radiologists, to step into a facilitator role to improve upon the CDS process,” Gaskin says. “By leveraging AI to support free-text order entry to facilitate CDS, UVA saw benefits including improved communication to radiologists, reduced provider frustration with CDS, and increased institutional compliance with looming PAMA mandates. This model could definitely be applied at other institutions ready to upgrade their own existing CDS mechanism.”

This initiative also showcases the power of AI technology to improve patient care. “By enabling more referring clinicians to use decision support tools in a way that doesn’t disrupt their workflow, radiologists are doing their part to ensure more providers order the right exam for every patient,” says Gaskin. “Radiologists have long been the stewards of appropriate imaging, and these tools are helping us extend and enhance that role.”