Notice Number: NOT-MH-17-036
Key Dates
Release Date: July 14, 2017
National Institute of Mental Health (NIMH)
None
Issued by
National Institute of Mental Health (NIMH)
Purpose
The National Institute of Mental Health (NIMH) seeks input on the issue of statistical power in studies using human subjects, with the goal of improving the rigor and reproducibility of mental health research.
Background
It is essential that research methods are rigorous in order to generate reproducible results. One key element of scientific rigor and robustness of results is statistical power. Specifically, underpowered studies can yield inflated effect sizes, reduced chances of detecting the hypothesized effect (i.e., false negative), and reduced chances that a statistically significant result is associated with a true effect (i.e., positive predictive value). Additional consequences of inadequately powered studies include the ‘file drawer effect’ and populating the literature with biased estimates; both of which have downstream effects on meta-analyses and the planning and powering of subsequent studies and replications. These power-related issues are directly relevant to NIMH as power has been shown to be inadequate in neurosciences, psychological research, and clinical trials.
At the same time, merely increasing sample size will not solve all problems related to low power. For example, there are constraints on the feasibility of recruiting large samples at single centers, and multi-center trials introduce other sources of variance. Additional complications include, but are not limited to, low priority for publishing adequately powered replications in high impact journals, differences between clinical and statistical significance, and effect size accuracy (i.e., small confidence intervals) can require larger samples than those needed to nominally detect an effect with 80% power.
Information Requested
While the NIH has provided guidance for improving both transparency and scientific rigor to foster reproducibility in biomedical research, there is no specific information relevant to statistical power. As such, the NIMH seeks input on the issue of statistical power in the context of human subjects research designed to elucidate mechanisms of disease or to develop therapies for mental illness, including but not limited to those studies that employ neuroimaging techniques. Given the high importance and complexity of the issue of statistical power in clinical trials, the NIMH seeks guidance on approaches to improve attention to statistical power in mental health research.
Areas of possible comment include but are not limited to:
- approaches to innovative study designs
- solutions to optimize sample size while maintaining feasibility
- perceived barriers to submitting adequately powered studies
- guidance to applicants to improve rigor in applications
- guidance to the peer review process to attend to matters of statistical rigor
- solutions to facilitate the generation of clinical results that can be reproducible and advance mental health
Responses
Responses to this RFI can be submitted via email to RFIStatisticalPower@mail.nih.gov using the subject "Rigor in Mental Health Research". Responses will be accepted until August 6, 2017.
Responses to this RFI are voluntary. This RFI is for planning purposes only and should not be construed as a solicitation or as an obligation on the part of the Federal Government, the National Institutes of Health or NIMH. The NIMH does not intend to make any type of award based on responses to the RFI or to pay for either the preparation of information submitted or the Government's use of such information.
The NIMH will use the information submitted in response to the RFI at its discretion. Submitted information will not be considered confidential. Respondents are advised that the Government is under no obligation to acknowledge receipt of information provided and will not provide feedback to respondents. The NIH and the Government reserve the right to use any non-proprietary technical information in any future solicitation(s).
Inquiries
Please direct all inquiries to:
Abera Wouhib
National Institute of Mental Health (NIMH)
Telephone: 301-594-9195
Email: abera.wouhib@nih.gov
and Human Services (HHS)
