Radiology AI research and development of commercial products has undergone unprecedented growth, although implementation of AI tools in clinical practice remains limited. Radiologists face numerous challenges in evaluating AI systems and deciding which might be worthwhile in their practices. This includes which problems to prioritize and whether to build systems internally or to purchase applications from an external vendor. In this month’s column, we’ll discuss the key considerations or radiologists evaluating AI implementation in their practices.
Prioritizing Problems
The fundamental challenge a radiologist faces is prioritizing which problems to address with AI. Tools exist to triage cases by identifying critical findings, increase efficiency by performing time-consuming tasks or assisting with report creation, and improve detection by identifying potentially overlooked findings. Understanding clinical context and patient populations is essential. While an outpatient-only practice might benefit most from an AI solution to identify pulmonary nodules on screening CT chest studies, a children’s hospital could decide to prioritize an AI solution for bone age radiographs, and a comprehensive stroke center might prioritize an AI solution for detecting large vessel occlusion on CT angiography head examinations.
Influencing the Decision
Having established what is to be prioritized, the next question is whether to build a tool internally or purchase an existing solution. This consideration is particularly applicable to large radiology practices and academic medical centers, as smaller practices likely lack sufficient scale to justify developing and implementing in-house AI solutions. Key considerations include the availability of data and experts to develop AI solutions, the return on investment (ROI), and long-term goals.
Development of an AI tool requires sufficient labeled imaging data for algorithms to train, as well as machine learning experts to develop and evaluate performance. Building an AI tool can be done with internal teams of experts or via partnerships between radiology practices and outside vendors. Practices might find it faster and cheaper to deploy existing pre-approved or FDA-cleared tools rather than embarking on development of an in-house solution. In areas where tools do not yet exist, however, development and FDA approval of novel AI solutions could provide opportunities for new revenue streams.
Measuring the ROI
To date, it has been difficult to estimate the ROI from deploying AI solutions. Conventional performance metrics have largely focused on diagnostic performance, including specificity and sensitivity. For an organization evaluating whether to build or buy an AI tool, key questions include how it might translate to better patient care or improve efficiency and profit. Certainly, the New Technology Add-On Payment, which CMS established for large-vessel-occlusion AI tools, provides a valuable incentive for hospitals to adopt this technology; it remains to be seen whether other AI tools will be similarly reimbursable.
Making the Choice
The decision to build or buy AI hinges on expected value, risk tolerance, and organizational mission. A private practice looking to maximize short-term revenue and minimize investment risk might decide to selectively purchase tools that increase efficiency or partner with an AI startup company. Academic medical centers might decide that the development of internal AI systems is an important aspect of training and research ecosystems, offering opportunities for grant-funded research to explore applications of AI — which might not immediately be reimbursable or profitable, but which could eventually improve patient care. Ultimately, all radiologists are best served by staying informed of developments in AI and leveraging local knowledge and expertise to determine what solutions will work best for our patients.
Hundreds of algorithms are now FDA-cleared for radiology and that number will only grow. Whatever your role, you’ll want to understand what’s involved behind the scenes. You may be surprised at what you learn along the way.