Key Dates

Related Announcements

None

Issued by

Office of Strategic Coordination (Common Fund)

Purpose

Clinical imaging can play critical roles in prediction, diagnosis, treatment planning, and response evaluation across a wide range of health conditions. Artificial Intelligence (AI) approaches have catalyzed new opportunities in precision, or personalized, medicine, to integrate clinical imaging data with a variety of other health data, including data from electronic health records (EHRs), genomics, laboratory diagnostics, and other experimental imaging systems. The purpose of this RFI is to solicit public comments on the current challenges and opportunities in developing trustworthy, cost-effective, accessible, ethical, and sustainable precision medicine AI algorithms that integrate medical images with other patient-related data to support clinical decision-making.

Background

The NIH Common Fund (https://commonfund.nih.gov/) supports high-impact, cross-disciplinary programs designed to address challenging problems and seize new biomedical opportunities.  The NIH seeks to gather input on the current state and potential of precision medicine algorithms that integrate multimodal, medical, image-centric data.

Recent advances in the computational field of image analysis, including machine learning (ML)- and deep learning (DL)-centered methods, have emerged as critical components of biomedical data analysis, leveraging the abundance and complexity of medical data. ML refers to computer algorithms (a set of rules and procedures) developed to analyze and make predictions from data that are fed into the system. DL refers to a type of ML that can glean hidden intricacies in data by utilizing more (and more sophisticated) processing layers.  As an example, DL-enabled computer vision has been shown to capture information typically requiring a higher-fidelity modality for human interpretation. This is a promising source of information for precision medicine initiatives, as medical images contain dense, objective data that could be useful for phenotyping and personalized medicine.

In general, applications of AI in medicine have addressed narrowly defined tasks using a single data modality, such as a computed tomography (CT) scan or retinal imaging. However, health data is inherently multimodal, and a patient’s health status encompasses many domains (e.g., clinical, social, biological/genetic, environmental) that influence well-being in complex ways. Additionally, each of these domains is hierarchically organized, with data being abstracted from the macro level (for example, disease presence or absence) to the micro level (for example, biomarkers, proteomics, genomics). Furthermore, current healthcare systems add to this multimodal approach by generating data in multiple ways: radiology and pathology images are, for example, paired with natural language data from their respective reports, while disease states are also documented in natural language and tabular data in the electronic health records (EHRs). For the management of any disease, clinicians process data from multiple sources and modalities when diagnosing, making prognostic evaluations, and deciding on treatment plans. Thus, an avenue to transform disease prevention, detection, diagnosis, and treatment is the integration of rich multimodal data sets across the human phenome through the development of advanced AI/ML models.

It is anticipated that advanced AI/ML models will be able to:

• Integrate medical imaging (e.g., neurologic, cardiac, ophthalmic, orthopedic, gastrointestinal, urological) with various other clinical data types (e.g., text, wearable and other sensors, genomic, behavioral, EHRs, routine clinical and physiological measures, social and environmental determinants of health) to create coherent representations of patient phenotypes.
• Capture complex relationships between different data types.
• Scale to handle large, heterogeneous datasets and adapt to new data types to stay current with medical and technological advancements.
• Substantially improve information content and deliver interpretable results compared to conventional approaches by using techniques like attention mechanisms and feature importance analysis to reveal traditionally unreported image features (e.g., structural, functional, metabolic, etc.) and/or spatial/temporal relationships. These insights can inform more effective clinical interventions and patient outcomes, thus fostering clinician trust and adoption.
• Address data integration challenges, such as aligning diverse data of varying quality, ensuring comprehensive data utilization, and support real-time applications for dynamic patient monitoring and treatment adjustments.

By harnessing these advanced tools for clinical applications and disease prevention, the potential of multimodal data can be fully realized, leading to advancements in medical science and ultimately improving patient outcomes.

This RFI seeks comments from the community on the current challenges and opportunities in developing trustworthy, cost-effective, accessible, and sustainable algorithms integrating medical images with other patient-related data for precision medicine that support clinical decision-making. The NIH invites AI/ML and imaging researchers, healthcare providers, patients, industry representatives, patient advocates, funders, regulators, and representatives of populations that experience health disparities (as defined in section 464z-3(d)(1) of the Public Health Service Act, 42 U.S.C. 285t(d)(1) as “health disparity populations”) to comment on individualized, image-centric patient care through advanced ML techniques and data integration.

Information Requested

The NIH is seeking input on relevant aspects of multimodal AI systems, AI datasets, and AI algorithms that utilize diverse data types, and implementation techniques for these types of systems. There is no word limit for responses, and responses may include but are not limited to the following categories:

Imaging and related multimodal data integration approaches:

• Current challenges in ensuring quality and integrity of multimodal datasets.
• Methods for standardizing data normalization protocols across different medical imaging modalities that could improve ML outcomes.
• Advanced ML techniques that would be most effective for integrating multimodal data into a coherent model for clinical application.
• Best strategies for curation and integration of imaging data of the same modality, but that originates from different vendors/sources.
• Curation and integration methods for datasets to best prepare them for use with multimodal AI, with specific emphasis on techniques for labeling images with ground truth.
• Risks and benefits to utilization of synthetic data in multimodal algorithms for precision medicine.
• Other areas that could benefit from standardized methods and metrics that would aid in data integration.

Developing AI based clinical decision support tools that leverage imaging and multimodal data integration:

• Methods for ensuring that AI/ML models are interpretable, explainable, and transparent to clinical and/or patient users.
• Potential regulatory barriers to deploying multimodal algorithms for precision medicine in clinical settings, and strategies for addressing them.
• Ethical considerations, such as patient privacy and data security, that need to be considered during development of multimodal algorithms for precision medicine, and methods for ensuring patient privacy and data security.
• Bias identification and mitigation techniques that will ensure AI-driven healthcare tools adhere to ethical standards while also providing equitable care across diverse patient populations.
• Strategies for supporting prioritization of image-centric research in multimodal data integration and precision medicine at institutes, within industry, among clinicians, etc.
• Strategies for incorporating user input during iterative development of multimodal algorithms for precision medicine.

Demonstrating clinical utility of multimodal algorithms for precision medicine:

• Key factors for successful real-world implementation of multimodal algorithms for precision medicine, and strategies to support these factors.
• Methods for ensuring generalizability of multimodal algorithms for precision medicine across various clinical environments and/or patient populations.
• Methods for establishing effective collaborations between academic institutions, industry, and healthcare providers to accelerate development and clinical uptake of multimodal algorithms for precision medicine.
• Methods for measuring and demonstrating the impact of multimodal algorithms for precision medicine on patient outcomes.
• Regulatory frameworks to ensure the safe and ethical deployment of multimodal algorithms for precision medicine.

Other comments, suggestions, or considerations relevant to this RFI.

Submitting a Response

All responses must be submitted electronically on the submission website: https://commonfund.nih.gov/ai-rfi. Responses must be received by 5:00 pm (ET) on September 23, 2024.

Responses to this RFI are voluntary. Responders are free to address any or all the categories listed above. Please do not include any information you do not wish to be made public. Proprietary, classified, confidential, or sensitive information must not be included in your response.

NIH staff will carefully review all responses and will not provide comments to any individual respondent’s submission. Any identifiers (e.g., names, institutions, e-mail addresses, etc.) will be removed when responses are compiled. The NIH will use the information submitted in response to this RFI anonymously at its discretion (e.g., on public websites, in reports, in summaries of the state of the science, in possible resultant solicitation(s), or in the development of future notices of funding opportunity).

This RFI is for information and planning purposes only and is not a solicitation for applications or an obligation on the part of NIH to provide support for any ideas identified in response to it. Please note that NIH will not pay for the preparation of any information submitted or for use of that information.

We look forward to receiving your response. Please share this request broadly with your colleagues and community. 

Inquiries

Please direct all inquiries to:

Michele Ferrante, PhD
National Institute of Mental Health (NIMH)
Telephone: 301-793-2634
Email: [email protected]

Karlie Sharma, PhD
National Center for Advancing Translational Sciences (NCATS)
Telephone: 301-451-4965
Email: [email protected]