NIMH Clinical Trials: Portfolio, Progress to Date, and the Road Forward

Thomas R. Insel, M.D., and Nitin Gogtay, M.D.

In 2010, the Institute of Medicine (IOM) published a report on the National Cancer Institute’s (NCI) approach to clinical trials, A National Cancer Clinical Trials System for the 21st Century: Reinvigorating the NCI Cooperative Group Program . This report noted the slow pace of completion of NCI clinical trials and the lack of scientific innovation in the context of direct, tangible benefits to patients. A subsequent report in the British Medical Journal showed that fewer than half of trials funded by NIH were published in peer-reviewed journals within 30 months of trial completion; a third remained unpublished after a median of 51 months.¹ The United States Congress, concerned about the need for new treatments, authorized the Cures Acceleration Network  within the Patient Protection and Affordable Care Act of 2010, specifically to find more efficient pathways to new treatments. Moreover, in 2015, many members of Congress have been pursuing fundamental changes that could speed more effective, state-of-the-art health interventions to patients. Two such efforts are the 21st Century Cures Initiative  in the House and the Innovation for Healthier Americans  in the Senate, both decrying the slow pace and high cost of treatment development.

What changes are called for based on these reports? First, improved efficiency, measured by time to launch, recruit, and complete a trial. Second, increased transparency, measured by registration of trials in a public database, comprehensive reporting of results, and ultimately access to raw data. Finally, increases in efficiency and transparency need to be matched by adequate monitoring of human-subjects protection and privacy issues. NIH has been pushing for more efficiency in its clinical trials: recently released draft policies propose a centralized Institutional Review Board (IRB) for multi-site trials . In addition, NIH has begun mandating the sharing of aggregate results on ClinicalTrials.gov for a broader range of clinical trials.² Other funding agencies are also taking a similar course. The European Medicines Agency implemented a new data sharing policy  at the beginning of this year, with a goal of sharing of individual-level data for all clinical trials in the EU. The U.S. Food and Drug Administration has announced plans to expand sharing of data  submitted in regulatory applications.

These critical issues – efficiency, transparency, and human-subjects protection – are all relevant to NIMH-supported clinical trials. Elsewhere we have described the need for next generation trials focused on target validation using an experimental medicine approach .³ Here, we take account of the current data from the NIMH clinical trial enterprise, and comment on the road forward. In this paper, we are using NIH’s recently revised definition of a clinical trial : A research study in which one or more human subjects are prospectively assigned to one or more interventions (which may include placebo or other control) to evaluate the effects of those interventions on health-related biomedical or behavioral outcomes.

The NIMH-funded Clinical Trials Portfolio

As seen in Figure 1 below, NIMH funding of clinical trials has been dropping over the past five years. Under the 2014 revised definition of clinical trials, the funding for clinical trials appears to increase (Figure 1: red square), as many interventional studies of biomarkers or other exploratory outcomes, which previously had not been counted as clinical trials, now fall in the clinical trials category. However, applying the older definition to 2014 studies, it is evident that the progressive decline in funding continued through 2014 (Figure 1: blue line with diamonds).

Figure 1: NIMH Clinical Trials Funding FY 2008 ‒ FY 2014

Taking a closer look at only the 2014 NIMH clinical trials portfolio, analyzed using the new clinical trial definition, NIMH supported 246 clinical trials (61 new and 185 continuing from a prior year award) at a total cost of roughly $145M, about 10% of the total NIMH budget. More than half (52%) of these trials were R01 grants; 30% were exploratory (R21 or R34) grants (Table 1). Of the 246 trials in 2014, 72 were multi-site trials.

Table 1: Description of NIMH clinical trials by grant/contract mechanism; new and continuing awards in FY 2014

	New Awards	Continuing Awards	Total Paid FY 14
R01	27	101	$78,073,913
Exploratory Research	28	45	$20,198,468
Intramural	0	12	$17,888,856
Research Centers	1	7	$9,361,522
Contract	0	7	$9,412,609
Cooperative Agreement	1	7	$5,841,931
SBIR/STTR	4	4	$3,544,583
Other Research	0	2	$466,799
Total	61	185	$144,788,681

Both intramural and extramural funds support clinical trials. Within extramural funding, 88% of the NIMH clinical trial budget derives from two Divisions: the Division of Services and Intervention Research (52%) and the newly formed Division of Translational Research (36%).

NIMH clinical trials cover a diverse range of therapeutics across several diseases and disease categories (Figure 2). Over half of the NIMH clinical trials portfolio, including both intramural and extramural trials, focuses on psychological or psychosocial treatments (54%), with less emphasis on medications (18%) and devices (15%).

Figure 2: NIMH Clinical Trial Funding by Disease and Disease Category FY 2008 – FY 2013*

*2014 trials were not included, as the numbers could not be accurately compared to prior years due to the change in clinical trial definition.
**Other category includes studies on borderline personality disorder, somatoform disorders, life stress events or situations, social problems or issues, general mental health conditions, multiple mental health conditions, basic processes research, and neurological, psychiatric, and physical disorders not listed above.

Performance of NIMH-funded Clinical Trials

NIMH recently initiated several steps to improve efficiency and transparency of its clinical trials. Here we assess how the NIMH portfolio is doing on each of these measures.

Efficiency

Efficiency measures for a clinical trial include length of time to enroll the first subject, recruitment and retention rates during the trial, and time to completion of the trial. All of these measures need to be balanced with the complexity of the intervention, the quality and integrity of the research design, and the protection of human subjects.

NIMH requires milestones for recruitment and tracks subject recruitment, typically every four months, for trials with 150 or more subjects and for all contract clinical trials. In 2012, an internal review found that only 64% of NIMH tracked projects met a threshold of being within 80% of their recruitment milestones. In 2014, of the 246 total clinical trials, 132 planned to recruit 150 or more patients, of which 61 such projects were actively recruiting. Of these 61 projects, 79% were at 80% recruitment, and 39% exceeded their target recruitment milestones.

Clinical trials often encounter delays in enrolling the first patient, which increase the risk for delayed trial completion. For NIMH trials, the median time to enroll the first subject is approximately 10 months after the Notice of Grant Award, when funding is released. For trials that began in 2013, about 70% reported starting enrollment as of their progress report at the end of the first year of funding. The remaining 30% of trials with a lack of enrollment in their first year of funding, a problem which is not unique to NIMH trials, was associated with delayed completion of many of these trials. Of the trials completed in 2014, 84/101 (83%) were completed while in a no cost extension (NCE), meaning that most trials were not completed in the original funding period. In 2014, there were 103 trials in an NCE, of which 90 (87%) were in the first year of an NCE, 11 (11%) were in the second year of an NCE, and 2 (2%) were in the third year of an NCE. Even though no additional public funds are awarded during the NCE, the delay in completing a clinical trial means a delay in providing results to the public. A closer look at the trials in their first year of an NCE showed that a majority (63%) had a delayed first enrollment due to a truncated first year of funding, often reflecting NIMH administrative needs (e.g., review of human subject protection plans, etc.), and in some cases, additional delays in IRB approvals. Adjusting the recruitment targets to align with funding start up times, helping investigators navigate the IRB and Data and Safety Monitoring Board (DSMB) processes, and ongoing monitoring of recruitment progress will improve time to trial completion and also avoid the need for an NCE.

Transparency – Registration and Reporting in ClinicalTrials.gov

There are two aspects of transparency that bookend most trials: registration at the beginning and reporting at the end. Like investigators from other Institutes and in industry, NIMH grantees are subject to two broad policies for registration. The first policy, originally part of the FDA Modernization Act of 1997, mandated the registration of Investigational New Drug application trials for serious and life-threatening diseases and conditions. A public registry, ClinicalTrials.gov , was created by NIH through the National Library of Medicine to house this important new database. Subsequently, the policy was significantly expanded in 2007, through the FDA Amendments Act (FDAAA), which required that all intervention trials (e.g., drugs, biologics, devices, etc.), including early phase and behavioral trials, be registered in ClinicalTrials.gov. Observational studies were exempted. According to FDAAA, trials that fail either registration or reporting requirements are subject to a fine of $10,000 per day.

FDAAA also required that summary results be reported in ClinicalTrials.gov within 12 months from primary completion of recruitment.⁴ The NIH recently proposed an expansion of this mandate  to register and report results from all clinical trials regardless of the trial phase; thus there will be no exemptions previously allowed in FDAAA for NIH-funded clinical trials.⁵ NIMH is putting mechanisms in place to assure 100% compliance of its clinical trials with FDAAA requirements, including both the monitoring of registration of the trial and the prompt reporting of results following the completion of the trial. Among NIMH-funded trials awarded in FY 2014, 95% have been registered as of this writing. Prior compliance with ClinicalTrials.gov registration and reporting requirements will be evaluated for individual grantees during the funding decision process for future awards.

Transparency – Reporting Results in Peer-reviewed Journals

In addition to registering trials and reporting results in ClinicalTrials.gov, it is important that the results from completed clinical trials, both positive as well as negative, be published in peer-reviewed journals in a timely manner. As noted earlier, a recent study by Ross et al found that only 46% of NIH-funded trials (completed by December 2008) had reported results within 30 months of completion. Among those trials with published results, the process was slow, with median time to publication at 23 months (range 14-36 months), and a third remained unpublished after 51 months.¹ Similar analyses of NIMH trials completed in 2010-2011 (n=115, allowing up to 48 months of post-completion analysis time) showed a much better picture. As of this writing, 87% of NIMH trials had been published, although this included results as well as methods-oriented publications. Mean time to publication was 6 months (range 0-3.5 years since completion), and mean publication rate was 11 publications per trial, with a median rate of 6 publications per trial. Furthermore, it appeared that the publications were useful to the wider research or clinical community, as evident from the average citation rate of 18 citations per publication.

Although publication bias often precludes trials with negative results from being published, strongly encouraging data sharing for all clinical trials will ensure accessibility of data from negative trials for future use by researchers and clinicians. NIH requires a data-sharing plan for all projects over $500K/year in direct costs. Recently, NIMH expanded this policy to all trials, with investigators expected to share subject-level data during the trial. The process for standardizing, integrating, and sharing clinical trials data has already begun via a centralized data repository for clinical trials, called the National Database for Clinical Trials Related to Mental Illness (NDCT), which has recently incorporated data from approximately 13,000 patients from legacy clinical trials. NIMH plans to release data one year following the completion of a trial, defined as the time when the last subject has completed the protocol.

Human Subject Safety and Data Integrity

Each clinical trial must have a well-designed human subject protection and Data and Safety-Monitoring Plan (DSMP), with many high-risk trials requiring an independent DSMB. NIMH is taking several steps to facilitate data and safety monitoring under the newly instituted Office of Clinical Research (OCR). OCR has now developed risk-based monitoring guidelines to develop appropriate DSMPs for each trial. Almost 70% of trials funded in 2014 had proposed monitoring by either an external or NIMH-constituted DSMB. For trials overseen by NIMH DSMBs, OCR has added new DSMBs, thus minimizing the turnaround time for DSMB review and approval.

Multi-site trials are often delayed by IRB regulations. NIMH now requires multi-site trials to have a single IRB of record, which will significantly improve the time for IRB approvals and reviews. Finally, NIMH plans to enhance its site monitoring activities on select trials, based on risk-level assessment and trial design, throughout the duration of the trial including trial initiation, progress, and close out visits. These monitoring activities, along with more direct assistance by NIMH to investigators, are important steps for overcoming barriers to trial efficiency.

Summary: Increasing Impact

Based on the data presented here and prior analyses of trial efficiency and transparency, NIMH has implemented a number of new requirements for clinical trials. NIMH program officers will help investigators to define trial milestones, including a milestone to begin enrollment, and will carefully monitor the initiation, recruitment, and completion of each trial. Continued funding will be contingent upon meeting these milestones. NIMH will also consider a history of efficient performance in making funding decisions for new trials.

Recognizing the need to shift the balance from traditional efficacy trials to focus more on target validation and experimental therapeutics, NIMH clinical trials now require testing for target engagement as a potential mechanism of disease. These studies should prove informative even when the intervention does not prove to be efficacious; the goal of NIMH-funded clinical trials is now also to learn about the disorder and to disseminate the results and data – a negative finding is just as informative as a positive finding. The Requests for Applications (RFAs) with these guidelines were announced in 2014 and were reissued in March 2015.

Conclusion
Last year, recognizing the dire situation in treatment development for mental disorders, we set out on a path to fundamentally change how NIMH funds and manages its clinical trials portfolio. One part of this effort was to change what we fund by shifting our clinical trials portfolio to an experimental medicine paradigm. Another part was to shift how we fund by requiring all proposals to apply via an RFA. A third part, described above, was an increased focus on efficiency, transparency, and patient protection. As part of our stewardship of the public’s investment in clinical trials, we have taken several steps including better monitoring of recruitment milestones, careful risk assessment for each trial, and enhanced expectations for registration, reporting, and data sharing. Improvements in recruitment and reporting are already evident. NIMH and NIMH-funded investigators can make further improvements to optimize the impact of NIMH-funded clinical trials.

References

¹ Ross, J.S., et al., Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ, 2012. 344: p. d7292.

² Hudson, K.L. and F.S. Collins, Sharing and Reporting the Results of Clinical Trials. JAMA, 2015. 313(4): p. 355-356.

³ Insel, T.R. and N. Gogtay, National Institute of Mental Health clinical trials: new opportunities, new expectations. JAMA Psychiatry, 2014. 71(7): p. 745-6.

⁴ Anderson, M.L., et al., Compliance with results reporting at ClinicalTrials.gov. N Engl J Med, 2015. 372(11): p. 1031-9.

⁵ Zarin, D.A., T. Tse, and J. Sheehan, The proposed rule for U.S. clinical trial registration and results submission. N Engl J Med, 2015. 372(2): p. 174-80.