Key Dates

Related Announcements

Companion Request for Information: Critical resource gaps and opportunities to support Next Generation Sequencing (NGS) test development and validation, including the use of technologies such as artificial intelligence (AI)/machine learning (ML) to support NGS tool development and data interpretation (NOT-OD-21-162)

Issued by

Office of The Director, National Institutes of Health (OD)

U.S. Food and Drug Administration (FDA)

Purpose

The National Institutes of Health (NIH) and the Food and Drug Administration (FDA) are requesting information on what critical resource gaps exist for validation and use of AI/ML to support radiological tool development and clinical data interpretation. This RFI is being released in parallel with a companion RFI (NOT-OD-21-162) focused on resource gaps for Next Generation Sequencing (NGS). If desired, respondents may provide comments that encompass both foci where the fields converge (e.g., linking tumor features with sequencing data, merged datasets). The comment period on this Notice is 90 days. Response to this Notice is voluntary.

Background

Reference materials are needed to facilitate the development, rigorous performance assessment, and validation of AI/ML models (e.g., deep learning) across a full range of clinical radiological applications. A current challenge is the lack of large, ethnically diverse, clinically annotated radiology datasets of sufficient quality with associated metadata that are Findable, Accessible, Interoperable, and Reusable (FAIR) with the appropriate policies and controls in place to ensure responsible data sharing and data use (e.g., privacy protections, consent requirements, compliance with applicable laws and regulations). Data storage and analysis infrastructure and tools for clinical and translational research using radiology data are also needed. Such resource gaps, in general, are frequently identified as limiting factors that impede high-quality research, development, validation, and regulatory science. Addressing these gaps could foster the development and validation of the next generation of AI/ML algorithms (e.g., deep learning and continual learning models) capable of analyzing data from multiple clinical domains (e.g., radiological and NGS) to provide researchers, physicians, and patients with new big data insights on the detection, characterization, treatment, and drug resistance of cancers and other diseases.

Information Requested

The NIH and FDA are interested in receiving input on the greatest needs and opportunities for the development of high-quality radiological datasets and tools that can be used to support AI/ML development, particularly in relation to the three topic areas noted below. Since the algorithmic needs for the development and validation of AI/ML algorithms may go beyond the training aspect of AI/ML, NIH and FDA would be interested in information related to both training and real-world use of unlocked AI/ML algorithms. NIH and FDA welcome input from research investigators, study participants, professional organizations, and other interested members of the public. Respondents are free to address any or all of the information listed below or any relevant topic for NIH and FDA to consider. Respondents should not feel compelled to address all items.

Topic 1: Development of reference datasets, tools, and infrastructure to support radiological imaging analysis and interpretation using AI/ML

• Large, highly characterized imaging datasets linking radiological findings, clinical data, and genetic data
• Digital reference images and physics-based simulated images that include artifacts (e.g., phantom images) that may help to develop, standardize, or understand the limits of AI/ML algorithms
• Generation/acquisition of patient outcome data; extraction of information from unstructured electronic health record (EHR) data
• Tools for the generation of combined radiological, laboratory, and multi-omics datasets (e.g., NGS, proteomics, EHRs) with a reference standard/truth, using an appropriate protocol, best practices, multiple sites, availability/annotation of lesions and/or abnormality location for imaging data
• Linking genetic/imaging datasets to a common dataset for algorithm development and testing (e.g., NCI Cancer Research Data Commons Aggregator )
• Methodology and platform needs to curate, annotate, and pre-process data for learning in both health and disease
• Modeling disease characteristics, prognosis, progression, and treatment; prediction of the type of abnormal finding or risk
• Identifying predictive biomarkers, extracting/analyzing clinically relevant imaging features
• Performing real-world monitoring of adaptive/un-locked AI/ML algorithms

Topic 2: Existing Resources that could be leveraged to fill resource gaps

NIH and FDA are also interested in receiving broad input about existing resources that could be leveraged to fill gaps identified by respondents. When identifying any relevant, existing resources, commenters may wish to include the following information in their responses where applicable:

• Resource name and link
• Institution or body that manages the resource
• Disease focus, if any
• Specimen types and/or imaging modalities available (currently), and number of participants
• Primary data types available (currently) that could be used as ground truth and number of participants
  • For imaging data include type and time points at which images were acquired
• Future releases and timeline if available
• Supported/linked data types
• Availability of clinical outcomes data
• Level of characterization
• Access model (e.g., unrestricted, controlled access)
• Whether images/data were collected under an informed consent that supports broad and responsible data sharing and that protects the privacy and wishes of patients and research participants

Topic 3: General Comments

NIH and FDA welcome general information on any other topics with regard to critical resource gaps and opportunities to support radiological tool development and clinical data interpretation using AI/ML.

Submitting a Response

Responses should be submitted electronically by November 1, 2021 using the form at https://osp.od.nih.gov/rfi-comment-resource-gaps-for-radiomics. You may provide responses to one or all of the topics in the comment boxes. Responses received will be posted at https://osp.od.nih.gov/nih-fda-rfi-ngs-radiomics without change after NIH and FDA have reviewed all of the responses received. Please do not post any proprietary, classified, confidential, or sensitive information in your response.

This Request for Information (RFI) is for planning purposes only and should not be construed as a policy, solicitation for applications, or as an obligation on the part of the Government to provide support for any ideas identified in response to it. NIH and FDA may use information gathered by this RFI to inform development or modification of websites, policies and practices, processes and procedures, and supporting documentation.

Inquiries

Please direct all inquiries to:

NIH Office of Science Policy
Division of Clinical and Healthcare Research Policy
301-496-9838
[email protected]