“Healthcare needs AI,” says Nina E. Kottler, MD, MS, vice president of clinical operations and associate chief medical officer of clinical AI at Radiology Partners. “I am a huge fan of being an early adopter of AI. Most importantly, early adopters can help drive the direction of this disruptive technology; and this technology can transform the practice of medicine. In addition, early experience provides expertise allowing early adopters to avoid mistakes that others will make later — at a time when there will be far less latitude to make those mistakes.” As Curt Langlotz famously said, radiologists are not going to be replaced by AI, but radiologists who use AI and — Kottler adds — understand AI, will replace those who don’t.

Kottler’s group deployed AI algorithms several years ago and has seen positive results. “We have gained experience in identifying use cases that provide value for our patients and practice, choosing AI products and vendors, piloting vendors’ algorithms, and creating our own,” Kottler says. “Our radiologists have adapted to using the AI tools and have integrated them into their clinical workflow.”

GETTING STARTED

To ensure scalable, useful AI implementation, multiple steps are required — including preparing your data, your systems and your radiologists, Kottler says.¹ Radiology AI is still immature, with limited adopters and few use cases in practice. With this in mind, Kottler’s group has successfully managed challenges and experienced unexpected benefits from using AI.

“Deploying AI wasn’t a huge issue for us,” Kottler says. “We had a use case, similar to the ones the ACR Data Science Institute® (DSI) now collects and maintains.” Kottler’s group has gained experience in both identifying use cases that provide value for patients and in creating their own AI algorithms. Beyond image evaluation and detection, Kottler says, AI is helping manage radiology workflows, collect patient information, assist in scheduling and follow-up, create portions of the radiology report, and more.

We are already starting to see unexpected benefits of AI, Kottler says. One of the more common AI use cases is computer-aided triage. Many of the current FDA-cleared products are triage algorithms — software that decides which patient studies should be moved up the reading worklist to expedite evaluation. “We tested two such algorithms and found surprising benefits beyond the expected expedited time to review exams with critical findings,” Kottler says.

When asked about these additional sources of value, “Number one, the software improved patient care and outcomes,” Kottler says. “We identified 2.4% more intracranial hemorrhages and 4.4% more pulmonary embolisms that we were missing without the AI software,” she says. “Even though most of the missed findings were subtle, several of those cases progressed to significant findings, including a patient with a barely visible intracranial hemorrhage that returned with massive enlargement of the hemorrhage requiring surgical decompression.”

Another positive, she notes, is a decrease in the cost of care to the health system. The use of these triage algorithms decreased length of stays and improved ED throughput. In addition, “We saw radiologists’ efficiency shoot up by 11 or 12%. The reason: the algorithms are highly specific,” Kottler says.

According to Kottler, efficiency is also a source of value for radiology practices who are often struggling to keep up with escalating volumes. “The reason for improved radiologist efficiency doesn’t come from the positive cases — those are few and far between,” she says. “Increased efficiency actually stems from the negative cases. Why? If a highly specific AI algorithm provides a negative result the radiologist doesn’t perseverate on the study looking to see if there is something subtle they are missing.”

Kottler emphasized that when selecting any AI technology — if you are not building your own algorithms — you should focus on identifying vendors who are willing and available to work with your organization. Your first priority should not be to find the most accurate algorithm, but rather the best algorithm vendor. Accuracy measurements are based on a set patient and exam population that may not reflect your data. Moreover, the use of AI in healthcare is still new. With that, both the technical and clinical deployment of algorithms is not always seamless. It is helpful to have a partner who can help manage the challenges. “We don’t partner an algorithm, we partner with people who can help us effectively manage that algorithm,” Kottler says.

CHOOSING VENDORS

“It is becoming increasingly feasible for regular radiology practices to begin to incorporate AI into their workflows,” says Christoph Wald, MD, MBA, PhD, FACR, chair of the department of radiology at Lahey Hospital and Medical Center and chair of the ACR Commission on Informatics. “Some vendors offer portfolios of algorithms,” he says. Other AI marketplace options, he adds, aggregate AI from multiple vendors.

You need a good working relationship with your AI vendor, Wald says, and possibly a third-party workflow orchestrator to integrate AI algorithms into your clinical workflow. One really important issue for practices is monitoring real-time performance once the AI is deployed, Wald says. It is especially important for practices with no onsite data scientist. Practices need to remember that they will have to initially assess whether the AI is working as marketed. Each practice almost certainly has different scanners and protocols than those used to train the AI. Furthermore, diversity of patients in a given practice may differ from what the AI was trained on, creating bias, which can theoretically result in unintended health inequity consequences. Practices will need to monitor AI algorithm performance as their machines and protocols change over time. The ACR Commission on Quality and Safety and the Commission on Informatics will collaborate with domain experts over time on ways to foster the safety, effectiveness, reliability, and transparency of AI in radiology.

There will continue to be significant vendor consolidation in the industry, Kottler notes, which will make it difficult for practices to identify the vendors that will be viable in the future. The ACR AI-LAB™ can be a huge help for practices looking to move forward with AI, Kottler says. The ACR AI-LAB is a data science toolkit designed to democratize AI — enabling radiologists to develop algorithms at their own institutions, using their own patient data, to meet their own clinical needs. Some services through the ACR AI-LAB portal include exploring existing use cases for AI in medical imaging, proposing your own idea for a use case, learning how AI applies to imaging through a series of videos, and creating structured data sets around specific AI use cases.

“With AI algorithms, we are going to be gleaning more data from images than the visual information,” Kottler says. “We need to understand how to process that information and add context to it based on the patient.” In the not-too-distant future, she says, radiologists are going to use AI to predict patient outcomes — with precision guidelines for preventive health. “That capability would be huge, and of great value to patients,” Kottler says. “We will be busier, but we’ll be practicing at the top of our licenses.”

It is becoming increasingly feasible for regular radiology practices to begin to incorporate AI into their workflows.

UNDERSTANDING THE CHALLENGES

“I think the radiology community has moved beyond a fear of being replaced to a culture of accepting that AI will assist us,” says Scott J. Adams, MD, resident physician in the department of medical imaging at the University of Saskatchewan’s College of Medicine and an E. Stephen Amis, Jr., MD, Quality and Safety Fellow. Adams echoes Kottler in thinking that more radiologists are starting to understand the advantages of using AI.

“The hope is that radiology groups of all sizes will be able to use AI easily and cost-effectively,” Adams says. “I’m hoping we are going to see AI marketplaces become simple, so that as many algorithms as possible can be integrated as add-ons into existing systems — and that the integration aspect is seamless.”

“I think in the future, a tailored regulatory framework for AI will provide increased flexibility for AI developers and vendors,” Adams says. That would open up the market to more practices, of any size and scope, with far more choices. “For now, with all the hype around AI, there are overinflated expectations to an extent,” Adams says. While the goal is that off the shelf AI algorithms can be seamlessly integrated into a workflow, a misperception is that all AI algorithms will achieve desired performance across all practices, he says.

“We have to be careful about the way AI algorithms are trained — to consider their accuracy for certain populations,” Adams says. There can be bias in the data, he says, around geographical population, sex, race, and gender. Accuracy is also tied to the quality of a study, the image acquisition protocol, a practice’s equipment, and so on. “We have a tendency to think that once an algorithm is developed, it is going to be accurate across all populations and institutions,” he says.

“Monitoring accuracy and performance in real time is going to be easier for some practices than others,” Adams says. “If you have the bandwidth at your institution, it is possible to do that via real-time performance evaluation. But if you are a smaller practice, without a lot of IT support, there are going to be some real challenges.”

“The ACR DSI is really working hard to develop systems to monitor performance in real time,” Adams notes. “That is going to be huge for many practices to benefit from the work of a professional society. You may see vendors take on some of that responsibility as well, but you should be prepared with your own monitoring strategy.”

According to Adams, radiologists must continue to be involved — even step up their game — in terms of the safety of AI algorithms and their patient-centeredness. “AI is never ‘in place of ’ or ‘an alternative to’ radiologists — they will have a key role in the supervision of AI to ensure patient safety,” he says.

With so many AI use cases to choose from, radiologists are well-positioned to maximize AI’s strengths, Adams says. AI applications can support workflow efficiency and exam scheduling, image acquisition and interpretation, and report writing and communications for patient follow-up. “In each of these areas, there is tremendous potential for AI in the specialty — and many opportunities for this and the next generation,” he says.

SEEING POTENTIAL

“Medical students hear about AI doing some tasks extremely well and assume it is just a matter of time until it can do all things well — especially in radiology,” says Bibb Allen Jr., MD, FACR, the ACR DSI’s chief medical officer and a diagnostic radiologist at Grandview Medical Center in Birmingham, Ala. “What they may not realize is that as AI becomes proficient at thousands of narrow tasks it will become an invaluable assistant to radiologists and allow them to bring greater value to patients and  the healthcare system as a whole.”

The ACR DSI recently conducted its first annual survey of ACR members to find out how radiologists are using AI in clinical practice. The results indicated modest use of AI, but most survey respondents were satisfied with their experience and believed AI provides value to their practices and patients. The survey found the most popular AI algorithms are tied to screening mammography, PE, MR brain analytics, and brain hemorrhage. About 20% of practices not currently using AI plan to in the near future (learn more in the infographic on page 14).

“At the ACR DSI, we are pragmatic,” Allen says. The ACR believes AI and machine learning hold great potential benefits for patients and radiology practices — but only if those tools are brought into routine practice in the correct manner, he says. “We need to assume a leadership role in developing and implementing AI to do that,” Allen says.

“At the same time, we must ensure a trained workforce — capable of interpreting imaging examinations and guiding research and innovation into new imaging techniques,” Allen says. There is no better time to choose radiology as a specialty, he says, but AI’s complementary potential is a tough sell to many medical students.

REACHING OUT TO STUDENTS

“The first time I was introduced to AI in radiology, it was not depicted in the most positive light at all,” says Neil Jain, DO, integrated IR resident at MedStar Georgetown University Hospital and chair of the ACR Medical Student Subcommittee. “Many nonradiologists voiced that radiology would be entirely taken over by AI, leaving them jobless.”

“Of course, that isn’t true, but that’s the problem — we have outsiders looking in and interpreting our field,” Jain says. “This is partially our fault though. We are not pushing back hard enough and voicing all of the incredible aspects about our specialty and debunking this dark reading room myth.”

“Especially in IR, I see AI as a monumental advantage in visualizing visceral structures with higher accuracy and completing procedures with greater confidence,” Jain says. “AI has the potential to complement your daily tasks, making the time in the reading room or IR suite more productive and economical — ultimately making the clinical experience safer for the patient.”

Jain is committed to getting medical students excited about radiology. “Earlier this year, we hosted our first-ever virtual ACR Medical Students Symposium — with more than 300 students in attendance — where radiologists from across various subspecialties discussed their niche and showcased how diagnostic imaging goes beyond a routine chest X-ray,” Jain says. AI was a part of the conversation — with an introduction to what AI is and how it will be used in radiology. “To truly appreciate the future of radiology, we must first understand the challenges that we currently face. This will provide us with the necessary framework of where we are headed,” he says.

“The ACR Medical Student Subcommittee is also in the process of designing a mini-curriculum for AI in radiology,” Jain says. “As part of the medical students’ section on the ACR website, we’re developing an eight-module recorded curriculum for individual students with speakers talking specifically about radiology AI.”

“This material is geared toward medical students with the goal of setting the record straight,” Jain says. “Some of the topics we hope to touch upon include the economics of AI, how AI can be applied in a global health setting, and how AI plays into radiology careers.” On July 15, there will also be a webinar hosted for medical students entitled “Beyond the Hype: How AI Will Really Impact Careers in Radiology.”

“When you first begin learning about AI, it can feel strange and scary,” Kottler says. “As a new tool AI is unknown and unfamiliar, but we must be the authors of change.” Radiologists must become the experts understanding where AI works and where it does not — and share that expertise with one another, she says. “This collaboration  will boost our value and expand our role as consultants who referring clinicians rely upon to combine our expertise with the data AI provides.”

“Ultimately what you are looking for in an AI solution is how it can add value to the system,” Kottler says. “The best thing the ACR can do is to continue to ensure that the things we learn from and about AI in practice are shared. AI tools will fundamentally change how we do our jobs and adopting AI should not be a series of individual efforts.”