Workshop: Neurofeedback Intervention Development: Opportunities and Challenges

Transcript

Session 3

CHRIS SARAMPOTE: We are returning now, and I hope you had a good break. Before we turn things over for session three ‑‑ actually, as we turn things over to session three, I want to introduce Dr. Holly Lisanby, who you've already met, but she is the director of the Division of Translational Research at NIMH, and I'm going to let her lead off session three. So, Holly, take it away.

HOLLY LISANBY: Thank you so much. I'm going to start with a brief recap to set the stage for session three. And to me, the tag line of the day so far came from Masaya Misaki, who said, "Neurofeedback can translate neuroscience knowledge into brain intervention. Neurofeedback can evolve as neuroscience evolves." In session one we learned from Stephen LaConte that the heart of fMRI neurofeedback is the MRI, both its strengths and its limitations, and we learned that fMRI neurofeedback has a lot of knobs. From Masaya Misaki we learned that because there are a lot of knobs, a better understanding of the mechanisms by which this form of feedback works could inform how we set those knobs. We learned that the mechanisms involve learning, which engages distributive networks in the brain beyond the region of interest that might be targeted, and that with real‑time fMRI neurofeedback, we get two for one. We get both the neurofeedback signal, but we also get the whole brain activations during the training, which may be useful in understanding differences in clinical outcomes.

We learned from Michelle Hampson that real‑time fMRI neurofeedback has the potential to shift to long‐term trajectories. This may be especially important for youth, because it can represent a safe medication‑free way of leveraging the plasticity in young brains that could lead to a lifetime of benefit. But we also heard from her important challenges, like the need to support early‑stage studies to identify the best interventions and she argued for a reduced emphasis on statistical power.

In session two we saw some examples from Susan Whitfield‑Gabrieli about the promise that real‑time fMRI feedback could bring for neuro‑prevention, neuro‑prediction, neuro‑modulation, neuro‑triggering, and network therapeutics. We learned from her that it takes a village. We saw the products of multidisciplinary teams and decade’s worth of work, and we understood the potential of combination interventions that combine real‑time fMRI neurofeedback with other forms of intervention, like mindfulness meditation.

From Karina Quevedo we learned about how to hit a moving target and we saw really exciting data that real‑time neurofeedback intervention could decrease suicidal ideation in youth at risk for suicide.

And then from Kym Young we learned about challenges in this field having to do with dose determination and clinical implementation. We also heard a discussion about where real‑time neurofeedback belongs between the psychosocial intervention space and the device space. We heard about the pros and cons of each, and we heard about the importance of ensuring that neurofeedback work is optimally reviewed by those with the proper expertise and optimally supported on its pathway to clinical implementation, especially given its exciting promise.

Now, in session three we're about to hear from experts at the FDA from the Center for Devices and Radiological Health to address our questions about what role FDA plays in getting neurofeedback technologies from the stage of research into the clinic. And we're also going to hear from a team that has successfully navigated that pathway to bring neurofeedback technology to the market. Bottom line up‑front, the FDA provides a pathway to allow devices like neurofeedback to get to the clinic by being shown safe and effective for an indication for use in the treatment of a mental illness. The FDA does not control reimbursement, but it is a step in the process of getting treatments into the clinic. If you want to market neurofeedback technology for the treatment of an illness, you do need FDA approval. If you want to use it off‑label, FDA does not regulate the practice of medicine, but you will not be able to market or advertise that you are offering that technology. Insurers might decide not to cover the intervention in the absence of FDA clearance, but even with FDA clearance, they still might make their own determination to cover it or not to cover it.

Now I'd like to hand the session back over to Chris Sarampote, who will introduce our speakers and I'll be back to moderate our discussion.

CHRIS SARAMPOTE: Thank you, Holly. I appreciate your comments and the recap, and also thank you for joining us today and supporting this meeting today. We have three presenters and a moderator and a discussant today for session three. Starting with Dr. Talma Hendler, who's from Tel Aviv University. Dr. Hendler's also associated with Gray Matters Health, which represents a company in which she leveraged her research to develop an intervention that could be brought to market and impact mental health outcomes, and she'll be also talking about that work. Also, Dr. Doe Kumsa and Dr. Anita Bajaj from the Food and Drug Administration will talk about the regulatory pathway that exists for interventions like real‑time fMRI neurofeedback. And then finally, Dr. Luke Stoeckel from the National Institute on Aging will be joining the discussion that will be led by Dr. Lisanby.

So, with that, Doe, are you up next, do I have you leading us off?

DOE KUMSA: In the schedule I'm going second, but I could go if that's preferred.

TALMA HENDLER: I think I'm the first one going.

CHRIS SARAMPOTE: Very good. Talma, why don't we have you share your slides, and you can lead us off into the afternoon.

TALMA HENDLER: Thank you. Can you hear me?

CHRIS SARAMPOTE: Yes, yes.

TALMA HENDLER: Okay. So, my title is Crossing the Bridge from Idea to Biomarker, but I could also say that it's crossing the bridge from science to market, and this was thanks to a string of company that established for my work, and I will talk about the path that I took to get to this point.

So, this has been already mentioned. We are facing a great need, unmet need in mental health, and we have heard the numbers, really incredible numbers of patients that are looking for answers and their hearing options. Under these gray bubbles, actually we have quite a lot of science that has been accumulated in the last two decades in animals and humans, of course, and with respect to the [indiscernible] we could color these unmet needs with some notions, some scientific notions that we have about what is the underlying mechanisms of these different disorders. And just I mentioned here three, but we could think of others ‑‑ motivation behavior, which is from the positive domain, positive balance domain, the cognitive control system, which is the cognitive domain, and the emotional regulation system, which is the negative domain. And I was focusing myself on the blue one, the negative band domain, which is related to anxiety, PTSD, and stress resilience.

So, in my talk I will take you through the road of ‑‑ I need to move things here. I will take you through the path I took from clinical to lab work and then from lab work to clinical application. So, starting from scientific idea to biomarker. The beginning of this is in a prospective study I did on the first two scholars who were expected to undergo traumatic experiences, and we found that amygdala hyperactivity was predictive of how many symptoms and disturbances they developed after the exposure to a traumatic experience. And this was following us replicated. So, this is, by now, an established idea that the amygdala is important for emotional regulation, and it's probably related to the fact that it's highly connected subcortically and upstream through crucial cortices. And in addition to that, what we know is that from accumulated studies, the amygdala appears again and again in very different disorders and gradually it has been acknowledged that it could be regarded as a transdiagnostic neuromarker in psychiatry. And that was a good starting point in my mind to put forth the assumption that if indeed disturbances in the amygdala are related to psychopathology, we should try to modulate its activity. And since I saw the hyperactivity as a marker for trauma, the first direction I took was to try to down‑regulate the amygdala in hoping to see changes related possibly to emotional regulation.

So clearly the best way to measure amygdala, as we all know, is functional MRI and we heard a lot about it, but if we want to move to a scalable intervention ‑‑ and particularly, I was asked by the funding agent that helped me do this initial work to actually demonstrate that I can do the work in the field with first responders, I was thinking maybe I should move to EEG. But EEG, as far as I knew at the time, has poor precision and is not able to actually probe the amygdala, and that led to the idea that we should combine fMRI and EEG and try with machine learning to predict the fMRI from the EEG. And we did that more than a decade ago, demonstrating that we can actually obtain electrical fingerprint from the fMRI that can be used as a probe in neurofeedback. And you can see here the very first result we got, test versus sham, that people were actually able to reach this fingerprint mentally and down‑regulate it.

Then, of course, we had to validate and prove it and we invested a lot of work, almost two decades, with many papers that you can see some examples here, showing that if we bring new group to the lab and ask people to down‑regulate the fingerprint in the fMRI, we actually get back indication, regulation of their probe in the fMRI, and we could also demonstrate that it's probing a relevant process like emotional regulation using [indiscernible]. So of course I don't have time to go into the details, but importantly there is a lot of science behind that.

Another direction that we took in the development in the lab was related to developing interfaces for the feedback that could be good activators of the amygdala. And it was actually mentioned by Kym, and I was excited to hear that, that activation of the target is actually very important. And we were able to ‑‑ so we developed an interface, which is noise interface. Maybe you cannot hear [recording played] ‑‑ I'm not sure if you heard the sound. Did you hear it? Could you hear the sound?

>> Yes, we heard it.

>> Okay. So, this interface continued to change [recording played] ‑‑ I will stop it for the sake of time. And the more people down‑regulate the amygdala, the quieter the room becomes, and the other person sits down. So, all together, this brought us to the point that we can ‑‑ and of course, we demonstrated that people could use this to train their brain to down‑regulate the amygdala. So, I just briefly demonstrate to you how we went from a scientific idea, all the way to biomarker and on the way also improve the technology of the biomarker, but also with therapy.

And now I will talk a little bit on the second point of going from the lab work to the clinical application and actually demonstrating actually it has an efficacy in the clinical world. So, the first indication that we looked at was trying to see whether we can enhance resilience among first responders, which were our first target in developing the technology. The idea was, as already I said, that if hyperactive amygdala resulted in more post‑trauma symptoms, we should try to down‑regulate it. And we undertake a very large study in combat soldiers and we're able to demonstrate that, indeed, the soldiers in six sessions learned to down‑regulate the electrical fingerprint better than if we used another control condition, and most importantly, this was associated with improved alexithymia. Alexithymia is a marker of emotional regulation or people knowing what they feel and how they express their feeling or understand others' feelings, and it has been associated with stress and trauma. So down‑regulation of the amygdala biomarker resulted in both learning and a change in process. So I could say that we target a mental process that was relevant to stress, but we also were able to show ‑‑ bringing the individuals to the lab and ask them to now do real‑time fMRI on their amygdala, we were able to show that they are actually better in down‑regulating the amygdala, but when they do it, they're also getting better in connecting to the ventromedial prefrontal cortex. And this has been shown by others that actually down‑regulating the amygdala is related also to improving connectivity, which we can see is a very important node of emotional regulation.

So, we talked about, what is the outcome in the target? So here are two examples. And the next thing we did is to try to prove that it's actually valuable for patients, and we performed two randomized control trials on PTSD, on chronic PTSD: altogether, 93 patients. And we were able to show, looking at the CAPS, which is a common scale in PTSD for severity of symptoms, we were able to show, as you can see in the darker bars on the left and the red bars on the right, that people actually getting better and improving in their symptoms in a significant way, compared to no treatment group control. And when looking at PCL, which is a self‑administered scale, we were also able to show what has been already mentioned here very importantly that after six months, people even get better. So, if immediately we had 36% improvement in PCL in the test group, after six months we're at 65% of people getting actually a more than 10‑point change, which is significant clinically.

So, this was all the science we did before the company was established. So again, we did a lot of lab work and a lot of publications and then we established the company. And the company, the first thing they did is try to replicate this clinical study, which has just been published also, and you can see here the result that they received from the CAPS. So, 67% of the patients overall showed significant improvement, and in sample population, 80% of people were under other treatments like SSRI and 32% were actually in a remission rate with more than 60% in sample population. And very importantly, this also has been mentioned before, more than 90% patient compliance and only mild transient side effects. So, this was the study that we actually approached FDA for regulation approval.

So, the point I want to make is that we need the FDA, and as has been talked to you before, this is a device approval based on a 510(k), a predicate device. So, it's in the limitation of this predicate device and maybe this will be discussed further with the FDA people. But now we have a device prism for PTSD that is available commercially in the USA and also in Israel. So, to sum this part, from a clinical need to a lab work and from a lab work to a therapy and FDA regulation, it's a long road, a lot of hurdles on the way, but it's possible. This is what I want to say, but at the same time, I would like to ‑‑ and this was possible because of Gray Matter's help, the company. I would like to also say that it's not the end of what we have to do. There's a lot of improvement opportunities here, and I just mentioned it here briefly.

First, specifically we need to enhance target precision and disorder‑tailored interfaces, like I already mentioned a little bit ‑‑ for instance, trauma content or obsession cues. We need to improve suitability, which is identify good or poor modulators, optimize learning, and combined methods, like I already mention. We need to improve scalability. Right now, the treatment is being given in the clinic. Maybe we can go home, carefully and systematically study how to do it, the best way, and still be effective, and then we can improve certain settings ‑‑ preparation, engagement, and integration.

I will just mention briefly these two points about the future, how we can go farther beyond where we are now. And this can be done ‑‑ should be done in the lab, but the company can also help fasting some processes of this development. So, more targets and claims, and like I said, we started with PTSD, but we are thinking of other claims, of course, and these bubbles need to be met completely. We need to have actually a library of biomarkers that can be used to probe different disorders and the combinations of them, of course, as well. We need to improve precision of biomarkers, and this has been already mentioned as well; try to develop biomarkers that are more and more process‑specific and use networks, not only regions. And this was mentioned very nicely in the first and the second session. So, we should definitely go in this direction and definitely try to also challenge EEG. And the other thing is enhanced process probing by the interface. This is more about suitability, becoming more and more personalized about what exactly we activate when we are doing the neurofeedback. We already heard that this is important for the outcome. So again, we can use different interfaces that can probe different processes, like a phobic condition or social anxiety or addiction.

And lastly, we should increase personalization by accessibility, but also by understanding who the good modulator is, and this is just very briefly showing you here a study. And also, I wanted to finish and say that although we are now having a biomarker, an EEG‑based biomarker, we still think fMRI studies are incredibly important and we are trying to combine the two roads in order to understand the mechanism with the aim of improving and optimizing the learning. So, if you are doing many sessions now with an electrical fingerprint, with the biomarker, you can perform fMRI before and after and you learn about the mechanism that is related to the change in the EFP. And you can see here that people are actually with time ‑‑ this is EFP learning ‑‑ with time, becoming better. So, this is the capacity of neuromodulation, and combining a lot of data that we had in the lab with people undergoing between six to twelve sessions of EFP, down‑regulating the amygdala and also having fMRI neurofeedback, we pasteurize a network. This was also mentioned by Masaya. So, we pasteurized a network that is important for the modulation, and we demonstrated that this network is better at predicting than just the amygdala alone, the outcome of the EFP. So, there is the relationship between these two worlds that we can actually learn from one world to another and improve the EFP with the fMRI. And lastly, not least importantly, this also predicted, this network, the amount of change in alexithymia among these patients. So, it's also probing into the process, not only into the modulation. And this is capacity, how much you can modulate, but we can also think about ‑‑ we know that neurofeedback is a learning mechanism. There is a lot to study there and using models in order to improve it.

Thank you. I would like to thank a lot my group, people from my group and collaborators and funders. Thank you.

CHRIS SARAMPOTE: Thank you, Dr. Hendler. And let me now introduce Dr. Doe Kumsa and Dr. Anita Bajaj from the Food and Drug Administration. Doe, please feel free to upload your slides and take us into the next session.

DOE KUMSA: Okay, sounds good. Am I sharing the right screen?

ANITA BAJAJ: Yes.

DOE KUMSA: Okay. Thank you for having us. I'm Doe Kumsa, I'm a lead over here at the FDA. My colleague Dr. Anita Bajaj and I will be presenting regulatory perspectives, and we have no financial disclosures to report. We are located in CDRH, the Center for Devices and Radiological Health. It is one of the seven centers at the FDA and within CDRH, which has super offices depicted here.  We are in the Office of Product Evaluation and Quality, which itself is then ‑‑ the super office is then divided into several offices shown here and we are in OHC5, Office of Neurological and Physical Medicine Devices. So, this slide is just to give you an overview of the structure of our particular office, the Office of Neurological and Physical Medicine Devices. It depicts the two divisions within it and the multiple teams under the two divisions. 

There's a lot of information, but we wanted to highlight that depending on your device and indication for use, you will likely submit to one or more of the teams here and the assistant records for each of the teams are shown. We belong in the red highlighted one, the neuromodulation psychiatric team. We look at when they have psychiatric indications, deep brain stimulators, regional nerve stimulators, computerized behavioral therapy devices whenever they have a psychiatric indication.

So, moving on to the regulatory landscape. So here we are looking at different classifications. We have Class 1 to Class 3. The class a device is in determines the extent of regulatory control. There are three categories, and they were established through the FDA, the Food, Drug and Cosmetic Act. So, examples of devices in each category would be ‑‑ for Class 1, they could be bandages, medical gloves, surgical masks. For Class 2, there could be transcranial magnetic stimulation devices, transcutaneous electric nerve stimulators, biofeedback devices, such as neurofeedback devices. Class 3, they are high‑risk devices or devices where we could not come up with what we call special controls. So, when I say high‑risk devices, they are implantable, such as deep brain stimulators, implanted [indiscernible] and the like. There are regulatory controls in place for all devices and they are needed to provide reasonable assurance of their safety and effectiveness. General controls apply for all devices, and they're needed to provide reasonable assurance of safety and effectiveness.

For Class 2 devices, the general controls by themselves are not enough. So, in addition to the general controls, they are also subject to what we call special controls. This is because they just satisfy electronic establishment, registration, device listing. That's not enough to provide reasonable assurance of safety and effectiveness of the device out in the market. And special controls are usually device‑specific and indication‑specific, and they could include performance standards, post‑market surveillance, patient registries, special labeling requirements; they would be listed out in the performance standard. There could be pre‑market data requirements and there could be special guidances with respect to the type of device.

For Class 3, which are the highest risk, for those both general controls and performance standards by themselves are not enough. So, in addition to those, they have to have significant clinical performance data. Some devices, while their risk level might not be high, if we are not able to come up with special controls, they could be in Class 3. So that is kind of an interesting situation.

We recommend that you start interacting with us early on prior to starting your clinical study, so that we can give you feedback on your clinical study, desired approach. You're able to do this through Q subs. There are different types of Q subs. A pre‑submission, for instance, is one where you'd submit your questions to us, and you could get written feedback from us. It does provide an opportunity for you to interact with us early on and discuss with us your intended pre‑market submission. So, we don't do a pre‑review of data in a Q submission, but, rather, it'll be feedback on your approach. So, you could be asking us about a clinical study that you're designing, or it can just be an informational meeting where you share information with the FDA without expectation of feedback. Those are called informational Q subs. And after that, typically ‑‑ it's ideal that you do this before you submit an IDE or a marketing submission so that we can provide you feedback, so that you can come up with a well‑prepared submission, but sometimes it doesn't.

So, after this, you either submit an investigational device exemption or a marketing submission. So, an investigational device exemption ‑‑ IDE for short ‑‑ it's different from a marketing submission. It pertains to devices that have not been approved ‑‑ an IDE pertains to devices that have not been approved or cleared for marketing or devices that are being tested for a new indication. So, they're cleared and marketed for something else; they're investigating them for another indication for use. If the device or study is considered to be significant risk, then an IDE is needed.

So, when a study considers significant risk, at the onset the sponsor/investigator and the IRB are responsible to make a risk determination and they are supposed to look at based on criteria that are specified in a regulation. So basically, the criteria are ‑‑ one of the criteria, is it intended as an implant and represents a potential for risk to health or safety or welfare of the subject. If that's a yes, then it's a significant risk. If it's sustaining human life, then it's a significant risk device. If it's for use of substantial importance in diagnosing, curing, mitigating, or treating disease, then it's a significant risk device. And then there's a catch‑all caveat which says otherwise presents a potential for serious risk to health, safety or welfare of the subject, then it becomes a significant risk.

So, in fulfilling the requirements set by the regulation, we do take into consideration not only the device but also the target population the device is intended to be used in and the overall study design. So, if the target population includes patients who are suicidal, even with risk mitigations put in place in the study design, it could be a significant risk. So, it depends.

There are different types of IDEs ‑‑ early feasibility studies which are on the early side, the devices may not be final, they couldn't inform device design, it could be in a small number of patients. Then we have traditional feasibility studies where the device is further along than the earlier feasibility and its near final, but the focus is still on the feasibility and safety of the device. The pivotal IDE studies are where we are looking for safety information and effectiveness information. And then the other pathway is a marketing submission. So, you could come in with a 510(k), a de novo, a PMA or HDE. I will go into each of these as to what they require. So, a 510(k) is substantial equivalence; a de novo is when there's no precedent; and then PMA and HDE is for Class 3 devices, but I will go each one of these later.

So, we are showing you here a regulation that might be directly applicable to neurofeedback devices. So, this regulation has been around for a while. There are devices that have been cleared in the '90s and '80s even. So, note that you can have a device within this regulation that you're not specifically stimulating with electricity or magnetic field. So, the identification of the device, I think it states that biofeedback device is an instrument that provides a visual or auditory signal corresponding to the status of one or more of a patient's physiological parameters, so that the patient can control voluntarily these types of physiological parameters, and it does give you an example of what those are.

So, we thought it was important to put in this takeaway, that most mental health applications will likely not be exempt from a 510(k) and they will likely fall in this regulation as, one, there are precedents and, two, it does exist ‑‑ neuro‑regulation does exist; they are devices according to our device regulation definition. So, we wanted to kind of highlight that there isn't a confusion as to whether they're a device or not, and then there's also not a confusion about whether there are precedents that have been cleared under this regulation on the market.

So, I will hand off the presentation to my colleague, Dr. Anita Bajaj, who will present clinical study design considerations.

>> Thank you, Doe. I wanted to start by going over some review points to consider in clinical trial design, and just two caveats.  They're not intended to be guidance and we would really appreciate it if you reached out to us via the pre‑submission process that Doe touched on for specific questions about your regulatory approach.

Next slide. So, some clinical review points to consider; first of all, the need for a well‑controlled study. It's important to think about what is the appropriate type of control that you want to use in your study to minimize bias, which could be associated with a placebo effect. Ideally, you want to consider a sham controlled study. I'll talk more about that on the next slide. Another aspect that's important to consider is randomization, as well as blinding. Because of the high placebo effect that I mentioned before in many psychiatric disorders, it's important to consider blinding of not just patients but also investigators and study staff associated with the clinical trial. Having an SAP, or statistical analysis plan, is also important. It should include a formal statistical hypothesis and pre‑specified success criteria. Those are key elements to help support study implementation and success. Finally, determination of outcome measures is very important. We like centers to consider validated disease‑specific measures. It's important to pre‑specify the amount of change between arms needed to indicate success. We value objectivity of outcomes and functionality. For this reason, we prefer clinician‑ reported outcomes or scales based on objective measures. It's important to also define the time frame for evaluating the primary end point, and we, again, strongly encourage Q submissions to review the clinical protocols that are intended for marketing submissions before initiating the study.

Next slide, please. These are some of the factors that the FDA considers when evaluating sham devices. We like to see a sham which maintains blinding such that the patients in both conditions are unable to tell if they're in the active or the control condition. Also, promotion of subject retention. This is really important because participants in the assigned control condition often drop out of the study if they know that they are in the control condition. And lastly, matching time on task with the treatment application, as well as level of engagement.

Next slide, please. A few more points: In defining the target population, we want to consider the type of psychiatric therapy delivered. For instance, what diagnoses are used as targets? So, the type of psychiatric therapy delivered by neurofeedback might be, for example, major depressive disorder, obsessive‑compulsive disorder, substance use disorder, post‑traumatic stress disorder. We've already heard these discussed today in previous presentations. Even among these disorders there are also different levels of severity, anywhere from mild, moderate to severe; also, treatment‑resistant. It's important to be as specific as possible. Also important to note if the device is to be used adjunctively or with treatment‑as‑usual options or if it'll be used as a standalone treatment, not for use with concurrent medications or therapies. Some of these considerations will go into how the study is designed to support the indication for use and the target patient population. If we look at the issue of adjunctive versus standalone use, when a product is meant to be adjunctive or an aid to usual care, a patient should be on treatment‑as‑usual options; the protocol should state what the usual care constitutes; and if patients are allowed to be on medications, the protocol should include a plan for how medications will be managed and monitored to lessen the impact on ‑‑ to take into account the impact on the patient's condition when it doesn't confound the results. For a standalone device, the protocol should have a specific plan for patient safety if the patients are not going to be on medication or other usual care options. They should also have a plan for how patients will be washed out of the medications or usual care that they are receiving, and it's important to incorporate into the study design ways to show that the device is effective compared to usual care.

Next slide, please. For review of marketing submissions, we use a risk/benefit analysis where we consider other available treatment options for the target population, the potential of risk due to concomitant use of other medications or the interruption of medication usage, a risk analysis, a benefit assessment of the proposed intervention, and an assessment of uncertainty also needs to be factored in. The target population is also important to indicate. For instance, in a pediatric population it's important to look at the definitions that CDRH uses; for neonates, from birth through the first 28 days of life; infants, 29 days to two years; children are two years to less than 12; and adolescents are age 12 all the way through the 22nd birthday. That's a little different from some others. So, a study that uses persons from ages 18 to before the 22nd birthday is actually a pediatric indication as well. A treatment‑resistant population is also important to consider and it's useful to be specific about which other therapies have failed, and it's important to specify the DSM‑5‑TR diagnosis when applicable. It's also important to consider how the data obtained in the study will be leveraged ‑‑ this is for a pivotal trial, a marketing submission, for example ‑‑ and whether a new brain target is being studied, and if so, we need data on this novel brain location.

Next slide. When looking at how to evaluate safety and effectiveness in the target population, we consider the screening criteria and this means the inclusion and exclusion criteria, and they should reflect the target population mentioned in the IFU. If you are targeting patients who are already on some type of treatment, are you targeting a treatment‑resistant population, are you specifying the DSM‑5 diagnosis and how it was diagnosed, when applicable? It's essential to consider how to design your study to support the indication for use in the target patient population.

Next slide. Study outcome measures from a safety perspective should take into consideration specification of safety parameters, classification of adverse events, how adverse events are adjudicated or quantified, guidelines for reporting of adverse events, and long‑term plans for study participants. Study outcome measures from an effectiveness perspective need to take into account predetermined effectiveness measures with the rationale; alignment with the indications for use, again very important; time frames for evaluating primary endpoints; targeting acute versus chronic conditions; explicit hypotheses in studies with a statistical analysis plan, and the use of validated generally‑accepted measures. A robust protocol to monitor the study is also important ‑‑ usage of assessment schedules, clear informed consent forms with study time frames built in, case report forms with adequate detail, and the use of DSMBs or SMCs, which are data and safety monitoring boards and study monitoring committees. Safety monitoring boundaries, also known as stopping rules, are also important, and sample size is important, whether we're looking at safety outcomes or effectiveness outcomes.

Next slide, please. Again, outcome measures are critical. We must pre‑specify the study outcomes, define the time frames for evaluating the primary outcome measure, and we recommend the use of validated and objective outcome measures. Establishing clinically significant study outcomes is an important consideration to support study success. Ways to look at differences between groups and outcome measures could include a comparison of means or a comparison of response rates in which you need to define the responder and the remitter. Always keep the placebo effect in mind, as this can lead to bias in results. Keep also in mind that scales used in clinical practice might not be ideal for data generation in a regulatory setting. For example, in major depressive disorder we might like to see the use of a clinician‑reported outcome scale, such as the HANV or the MADRIS.

Next slide. I think I'm back to you, Doe.

DOE KUMSA: Yes, okay. Thank you, Anita. So, hand in hand with the clinical study design considerations that Anita had just mentioned, we also look at non‑clinical testing considerations and it's important to consider that they will be device‑specific. So, what is listed here might not apply to ‑‑ if it's a digital device, there isn't going to be an electrode or stability, but in general, these are the various elements that we take into consideration when applicable. If, for instance, a device is something that's already on the market, but it is being tested for a new indication and you're coming in for an IDE or something, we request that you include a letter of reference from the manufacturer for the device you're using so that we can leverage that information.

So, moving on to different types of marketing submissions. I had touched on them briefly earlier. So, we have what is known as, for instance, a De Novo petition, which they end up usually being Class 2 for a new device type with low to moderate risk. And when there isn't anything like it on the market that is Class 2 under 510(k), so it's a new intended use or it's a different technology that raises different questions of safety and effectiveness and the risk score is also different, you end up needing to submit a De Novo. Then we have our 510(k) pathway at pre‑market notification. So, it's typically required for most Class 2 devices. What they need to do is show that they are substantially equivalent to legally marketed devices. So, this is different from a standalone evidentiary standard. So, you are showing that you are just as safe and just as effective as the predicate device. If you end up extending outside of that intended use, then you're either a De Novo, depending on the risk level, or you might move into a Class 3 device. And then continuing with pre‑market submissions, the other ones are pre‑market approval, which are for Class 3 devices. It is based on determination of standalone device safety and effectiveness. So here, you're not showing you're as safe and as effective as something else. You are providing a standalone safety and effectiveness device, and it's a Class 3 device. And then we have the humanitarian device exemption. It is limited to less than 8,000 individuals. There are approval standards, safety information showing no unreasonable or significant risks, and then the effectiveness is probable benefit. So, it's just slightly lower than the PMA.

And then these are for any pre‑market submissions that you are going to be submitting. These are the different elements that it would need to include, as applicable. So, you need your indication for use statement stating what the device is going to do, what it's intending to treat. You have to describe the technology. You have to provide the device description, the different components, accessories. If you're adding something else to it as a component, that goes into the description of the technology. The labeling, of course, based on the outcome and the clinical study design that you have done, or any performance testing that you have done, it has to be reflected in the labeling as applicable, if it's sterilized or by compatibility. So, if it's a software device or a device that utilizes software or any kind of connection, we have regulations where software and cybersecurity documentation is needed. And cybersecurity documentation is becoming increasingly important, depending on ‑‑ making sure that cyber devices that are able to connect are still safe. Then also you will include your performance testing, whether that be bench, animal or clinical testing.

This is just basically a summary of the different types of pre‑market submission. So, if it's research and development ongoing, as we mentioned, we recommend you come in for a Q sub and ask about the pathways you're trying to pursue. You can submit a risk determination request as well. Your IRB might ask you to submit a risk determination request. You can ask for a classification. So, if you know which pathway you belong in, you can submit a 513(g) to get that. Investigational device exemption also belongs in this phase, it's in the research and development for ongoing. And then we have the cases where research and development are complete, so now you are entering the marketing arena. So, depending on what the device is and its indication for use, it could be a pre‑market notification, it could be a de novo PMA or humanitarian device exemption.

And this slide shows that the review process is a collaborative effort. So, it consists of interdisciplinary teams. The teams have several subject matter experts, including clinicians like Anita, different types of engineers and different statisticians. We do have extensive interaction with sponsors to make sure the submission makes sense. You know, if there's anything that could be resolved interactively, we try to do that. We do interact with our policy staff to make sure the decisions we're making are aligned with existing regulations and policies and guidances. And then from time to time we do have, like this meeting, external expertise, to elicit external expertise as well. We have patient listening sessions as well. And this is our contact information, and we want to thank you for giving us your time and we can take your questions.

HOLLY LISANBY: Thank you so much. So, I'd like to invite the rest of the panel members, Dr. Hendler and Dr. Luke Stoeckel from NIH, to join. I'm going to start with a few comments and then we're going to take a series of questions.

The NIMH mission is to transform the understanding and treatment of mental illness through basic and clinical research, paving the way for prevention, recovery and cure. Neurofeedback technologies, both EEG and real‑time fMRI neurofeedback, represent an exciting avenue to contribute to this mission, which is why we convened this workshop. Starting with our end goal in mind, that of prevention, recovery, and cure, we are interested in supporting research that will accelerate the translation of promising interventions into clinical implementation, like neurofeedback. That's why we want all of us to have clarity about what are the steps that it will take to get neurofeedback technologies into the clinic. Since in order to use neurofeedback for the treatment of a mental illness, whether it's a DSM‑defined diagnosis or a transdiagnostic symptom within a mental illness, FDA clearance is required to be able to market such a technology. So, we want to make sure that the research that NIMH supports will provide the types of evidence that the FDA will need to see to make a clearance determination. Waiting until after a confirmatory efficacy study is completed before consulting with FDA about what level of evidence will be needed to clear a device risk hitting a roadblock that might've been avoided if one had consulted FDA earlier, ideally before designing a confirmatory efficacy trial. This is not to say that you need FDA clearance or even an investigational device exemption before doing a non‑significant risk confirmatory efficacy trial, but rather consulting FDA when you're designing such a trial may accelerate ultimately obtaining the evidence that would be needed for a future 510(k) clearance.

Now, we've got a lot of questions from our panelists and also from our audience online and I'm going to start with a few, starting with a question to our FDA colleagues. "You showed us regulation concerning biofeedback devices. Do I understand correctly that the FDA considers neurofeedback technologies like EEG neurofeedback and real‑time fMRI feedback as a device?

DOE KUMSA: It depends on the indication for use. So, if they're trying to treat or diagnose a disease or psychiatric indication or any disease, honestly, yes, they would be a device.

HOLLY LISANBY: So, if you want to use neurofeedback for the indication of use for treating a symptom in a psychiatric population, that will require FDA approval. Am I understanding you correctly? I just want to be sure I'm ‑‑

DOE KUMSA: Yes, yes, because the question said is it a device. So, I was trying to answer both of them. It is a device, and then ‑‑

HOLLY LISANBY: Thank you. Next question: Am I correct that the types of EEG and fMRI neurofeedback studies that you saw presented in today's workshop likely would not require an investigational device exemption, assuming that the local institutional review boards had judged them to be non‑significant risk?

DOE KUMSA: So, it is likely. We do outsource that significant, non‑significant risk determination to the IRB. So, if the IRB determines that they are an SR, typically we don't come in and make it SR, but if there is a difference in judgment between the IRB and us, we are the final word in it, but typically it is outsourced to the IRBs. So, when IRB determines a SR, we don't come and say it's SR.

HOLLY LISANBY: Great. This next question is for Dr. Hendler as well as the FDA. "We saw an example of a neurofeedback technology that went all the way from the clinical need to the research, to clinical implementation and marketing approval using an FDA 510(k) substantial equivalency mechanism, and Dr. Hendler emphasized the important role of the industry partner, Gray Matters, in making those steps possible. The question is, what stakeholders should be engaged regarding the next steps in the translation of real‑time fMRI neurofeedback for therapeutic applications? For example, would MRI device manufacturers, should they be involved, would they be the ones to lead pursuit of a regulatory pathway?"

Anyone on the panel would like to answer that. Maybe Dr. Hendler?

TALMA HENDLER: [Indiscernible] the person to answer it, but I would say that having strong stakeholders in the game really helps to understand what is possible and what are the risks that need to be taken into consideration when you try to commercialize something. These are issues that, as scientists, we have no idea about. I've been learning a lot on the way about this. So, I think if someone is thinking about commercializing a procedure in the fMRI, it probably will be wise to advise with people who are dealing with these devices and have some experience and expertise and prior knowledge about what it takes to commercialize something like that, because it's about investment and risk and we are not experts in that. So, I would suggest being very specific about what you are ‑‑ actually the question is regarded to, what is your aim? If your aim involves fMRI specifically, you probably need to talk with the stakeholders that have any idea about it.

HOLLY LISANBY: So, Dr. Hendler's making a very good point. Marketing implies that you are a company that has commercialized a product that you want to sell, and you might be selling to health systems or providers who would then use that product in the treatment of whatever it has an indicated use for, according to what the FDA label says, and typically that's not the researchers unless they, like yourself, are engaged with developing industry partnerships, start‑ups that would then take the next step to commercialization. I see Dr. Stoeckel has a comment.

LUKE STOECKEL: Yeah, I mean it's not dissimilar. I also put this in the chat for the other panelists, but it relates to another related technology, which is prescription digital therapeutics, whether it's cognitive behavioral therapy or cognitive training elements in apps or in devices in‑hand, versus something like EEG that Dr. Hendler just presented. It's more a question about how the FDA thinks about approvals for those approaches differently. And the reason I bring that up is, is there something about the device itself, whether the EEG technology, the fMRI technology, that is important for investigators that see themselves down the commercialization path where they want to bring something to market that would help them think differently about how they're designing their studies and conducting their research to get to market. I use the example of prescription digital therapeutics because, to me, it's not dissimilar from the target mechanisms that we're interested in in fMRI‑based neurofeedback. So, I guess that's maybe a question for FDA more than Dr. Hendler.

DOE KUMSA: So, I'm not sure which industry is well‑posed to kind of address it. I agree that it's very similar to digital therapeutics like you're saying, but I'm not sure which industry is better posed to address it.

HOLLY LISANBY: Dr. Hendler.

TALMA HENDLER: Yeah, maybe just to add that it is also important to address not only the industries that are actually making the device and creating the product, but also with the target population or the insurance company. I think you mentioned that as well. It's really important because that also affects the product and the marketing possibilities and they influence the stakeholders in kind of mapping the risk, I think. So, this is in addition to when you are thinking about going ahead with commercialization.

HOLLY LISANBY: Great, thank you. I'm going to move on to another question here from Dr. Kym Young. And Kym, if you want to put on your camera, you're welcome to or I can just read this. "Why now?" Kym, would you like to put voice to your question?

KYMBERLY YOUNG: Sure, yeah. For the past 20 years we haven't been considered device‑based and we've been excluded from these definitions in literature of years and meta-analyses. And so, the question that really comes to mind is why now? Why is it now that we're being considered devices when we haven't been previously? At least in terms of ‑‑ I understand the FDA defines us as devices, but NIMH has not included us as a device up until today.

HOLLY LISANBY: So, thank you for the question, and as you saw in the FDA presentation, there has not been a change in how the FDA views the technologies that deliver neurofeedback. Specifically, there's already regulation concerning biofeedback and now an FDA 510(k) clearance for an EEG neurofeedback device. There has recently been an increased collaboration, I would say, between our agencies that's increased mutual understanding of how FDA views neurofeedback technologies, and we are attempting to be ‑‑ we, NIMH is attempting to be proactive to use that increased understanding to accelerate the translation of the very promising neurofeedback research into something that would be viable in the clinic. And learning that this is a device‑based intervention, learning that FDA approval is required for neurofeedback technologies to be marketed has been a very important point in shaping how we approach the intervention development pipeline for technologies that are viewed by FDA as devices.

And Dr. Young, you had a second question, if you'd like to pose that one.

KYMBERLY YOUNG: Yeah, I think it was just general confusion as to whether or not ‑‑ for research studies, for like R61's and R33's, if we need the FDA approval or if this is really when we're ready to go to market, which I think it has been clarified by now.

HOLLY LISANBY: Yes. And just to re‑articulate that, if the intervention is judged non‑significant risk by the IRB, that would be not requiring an investigational device exemption. Dr. Kumsa did mention that if there is a situation where there's a disagreement between FDA and IRB about the determination, FDA is the final authority on that, but you did also hear her say of the studies that we saw presented today, it did not appear that those would require an investigational device exemption. And I'd like to also further underscore for all of the audience listening today ‑‑ Dr. Young, you made this point very well in your presentation ‑‑ the safety track record of real‑time fMRI neurofeedback has been excellent. So that's a very important point to underscore. The next question ‑‑

DOE KUMSA: I do have ‑‑ one second.

HOLLY LISANBY: Sorry. Go ahead, please.

DOE KUMSA: From the studies that were shown, it does appear they are non‑significant, but we don't know; we haven't seen them come in. So, it's not an FDA recommendation or anything. They do appear non‑significant risk and the IRB called them non‑significant risk likely, but we don't know.

HOLLY LISANBY: Right, thank you. And Dr. Young, you did cleverly cite some of my own literature reviews on the topics of neuromodulation. They were not completely all‑inclusive. So please don't interpret my failure to include neurofeedback as saying that it's not important or that it's not defined by the FDA as a device, because in fact it is.

Did Karina raise her hand? Please go ahead, and then Michelle Hampson.

KARINA QUEVEDO: I think ‑‑ first of all, I think that the road to FDA is unavoidable. I know that it's going to happen, particularly if we continue to demonstrate that this has clinically significant results, which is what we hope for. I do want to caution that some of the populations that we work with are going to be naturally viewed by IRBs as very high‑risk for the institution that is hosting that particular research. And I'm not talking about just me, but many other researchers that really have insight to significant and salient mental health problems in our country and in other countries, like increased suicide risk in kids and adults, which is very, very pronounced, but it also carries risks to the institution to run research in those populations, because we can't control entirely the outcome of one or two participants, even if we do our best for it. So very easily the IRB will say, you know what, you're going to be a high‑risk study or more than minimum risk study, you need to bring your little fighting gloves and talk to the FDA, we'll leave it to you. So, the scientists are going to fly away from ‑‑ you know, "I'm not gonna touch suicide research, I'm not gonna talk those things with anything, I'm not gonna write any grants about that, because obviously it's not going to allow us to develop that area." I'm just giving a very simple example. I hope it's helpful.

HOLLY LISANBY: Yes, let's use that example. And your work is very important when we think about the unmet public health need of preventing suicide, specifically in youth at risk. So, a non‑significant risk determination, I'd like to ask our FDA colleagues here to comment. My understanding ‑‑ and you can correct me if I'm wrong ‑‑ is that the significant risk determination refers to the device when used in the specific study where it's going to be applied. And there are specific vulnerable populations, youth being one of them, that might be at higher risk for the side effects of a device when it's used in them. My understanding of that is typically ‑‑ let's say in my field of brain stimulation, if I was gonna use a transcranial magnetic stimulation protocol that might put youth at higher risk of seizure because they're a protected population, that might require an IDE, but in this case for the neurofeedback, the population is not increasing the risk of a side effect from the neurofeedback. 
I don't think that would ‑‑

TALMA HENDLER: No, but it's the perception.

HOLLY LISANBY: Well, I'd like to ask our FDA colleagues, because they can tell you more than perception; they can respond about the regulatory ‑‑

KARINA QUEVEDO: Thank you for willing to engage in our discussion. This is productive.

DOE KUMSA: Anita, you were gonna talk about the risks. Right?

ANITA BAJAJ: Sure, yeah, I'll start. We do like to look at the patient population, and of course, in a vulnerable population like [indiscernible], we would be very cautious about the potential for that to be substantial risk. If the device itself does not hold the risk, but it's the patient population and their ability to be part of a study that could increase the risk, we do sometimes come to the determination of substantial risk, and it's just to add an extra layer of safety and protection for them so that the FDA will be involved in the IDE process. I do know that it's a balancing act. Whenever you want to encourage innovation for devices that are serving these target populations that are very vulnerable, whether it's suicidal persons or in another context, pregnant women, children, whoever, the need is high, but the need for protection is also high. So, the FDA will be very cautious about those populations when making the determination of substantial risk. Did you want to add something, Doe?

DOE KUMSA: No, I think you covered it. I will put it in the chart; the regulation is 21 CFR 812.3(m). It talks about when something is significant risk, and the last point kind of says "otherwise presents the potential for serious risk to health, safety or welfare for subject." That's kind of broad, so it's possible that we might consider the significance.

HOLLY LISANBY: Okay, thank you. And let me just say that these regulations are meant to protect our human subjects. So, let's not view them necessarily as barriers but, rather, pathways to put the proper controls in place to protect safety, which of course we are all committed to.

I want to go now to a question from the audience. This is again for our FDA colleagues, a question regarding 882.5050. "Dr. Kumsa mentioned that most mental health indications for use do not fall under this exemption. However, current neurofeedback practice which targets various mental health conditions, including depression and anxiety, is conducted under the prescription use part of 882.5050 exemption. Thus, neurofeedback devices are sold to clinics without a specific indication for use. However, the clinicians are able to use the device for various mental health conditions based on their expertise. So as long as the device does not make marketing claims that include specific conditions in its indication for use, then it can be sold under that exemption. Is that correct? In the case of Prism, they made a specific claim for PTSD in their indication for use, which therefore required clearance. Is that correct?"

DOE KUMSA: So just to make sure I understand, if they're not making marketing claims with the indication for use, the specific indication for use, it wouldn't ‑‑ technically, you don't have clearance for it. So, it is kind of ‑‑

HOLLY LISANBY: This was a very long question that I just read aloud to you. I just put it in the chat. So, I think part of this is describing what's currently going on in practice. So, you might buy a neurofeedback device ‑‑

ANITA BAJAJ: Using something off‑label, basically.

HOLLY LISANBY: And use it off‑label, right. The practice of medicine is not regulated by the FDA; so, you can certainly use it off‑label, but if you were to put it on your website and advertise come get your neurofeedback for the treatment of XYZ disorder, that would not be consistent with FDA regulations. Am I correct about that?

DOE KUMSA: Yes, you're absolutely correct.

HOLLY LISANBY: So, it's not to say you couldn't do it. We don't know if you'd get reimbursed for it. As a clinician, it's on your medical license how you choose to practice. Let me just speak about our mission at NIMH. We're interested in supporting the research that would lead to the development of treatments that are safe and effective and available clinically to everyone, widespread. So, we're interested in supporting work that would ultimately lead to products that could be marketed such that they would be available long‑term for patients and reimbursed by insurance. Dr. Young showed some health utilization numbers about how much MRI costs and full courses of CBT and so on. If a person had to pay for that out of pocket, then you'd be talking about a very small part of the population that could access that treatment. 

Dr. Hampson.

MICHELLE HAMPSON: I just wanted to come back to the practical side of this a little for people who are doing really early-stage research, where they're not even doing confirmatory efficacy trials but are just at an exploratory stage. It sounded like you were saying as long as it's minimal risk, you shouldn't really need to deal with FDA issues, but of course ‑‑

HOLLY LISANBY: Yes, that's right.

MICHELLE HAMPSON: Okay. But if you have vulnerable populations or if you just have an institution that doesn't want to take any responsibility and they're just gonna market it as be super conservative and say okay, call it moderate risk so that the FDA signs off before we sign off, what is the ‑‑ what is to be expected and what is the most efficient route to try to get approval for something which is essentially pretty safe and very early stage? Like, do we go to a Q sub? I was a little confused with all the different terminology, but do you go straight to applying for an IDE or is Q sub a way you can just ask, does this even need an IDE?

DOE KUMSA: So, the best way would be a Q sub, because we can help you put together a good submission. Yes, a Q sub would be the first step. If your IRB says you need to go to the FDA, then you need to submit an IDE, but prior to submitting the IDE, we recommend that you submit a Q sub.

HOLLY LISANBY: I'd also like to clarify, session one really focused a lot on how exciting these tools are as a basic science platform. Our session right here, session three, is about using these tools to develop treatments that would be implemented in the clinic. So, what we're discussing here right now does not impact the early stage or the basic science use of these tools. As long as you're using it in a context that would be non‑significant risk, then you would not need an investigational device exemption. If you're further down the translational pipeline, where you're at the stage of a confirmatory efficacy trial and you're saying we believe the study would lead to a new treatment for depression that we want to have available for patients, that's when we need to starting thinking, okay, well, how are we gonna get that treatment to a patient? Is it a device? Is it gonna need FDA 510(k) clearance? Will we ultimately need a company that would market this? So that's kind of the transition from a research use of the technology into the clinical application of the technology. All of its important and I know it's confusing because we've covered a lot of territory, from the basic all the way to the clinical implementation in today's discussions. Dr. Hampson.

MICHELLE HAMPSON: Sorry, I'm still a little bit confused, because what's not clear to me is, if you ever reach out to the FDA and say in a Q sub, does this even need to apply for an IDE, do they ever say no? Like no, you don't need to apply for an IDE.

HOLLY LISANBY: I can tell you I've done it myself, and yes.

DOE KUMSA: You can submit a risk determination. So, you can say, give me the risk determination for my study and device. 

MICHELLE HAMPSON: Wonderful. Thank you.

HOLLY LISANBY: Yes. I have another question here from the audience. "Can we still submit a grant proposal for fMRI neurofeedback under a psychobehavioral approach? At what point does the NIMH plan to characterize fMRI neurofeedback as a device‑based approach? Will it be based on the purpose of the study, specifically the stage of the study, such as early, middle or late stage? For the NIMH's mechanistic clinical trials, they do not allow clinical outcomes to be included. What direction does the NIMH expect us to take as an early-stage investigator? I'm a bit confused." Well, thank you for that question. You've got a lot there in that question. So going back to, I guess, Dr. Hampson's question. So, if you're using any technology to ask basic science types of questions, like what does the circuit do, can we train people to modulate this circuit, and it's kind of the more basic question type of realm, no, you don't need to be generating the types of evidence that would support a 510(k) approval for a product. If you are trying to develop a treatment for a diagnosis or a set of symptoms within diagnoses and you're trying to enter the clinical trial development pipeline, the intervention development pipeline that we have at NIMH that supports early stage intervention development ‑‑ so it's not early stage tool development, it's early stage treatment development, that's specifically the R61, R33 type of application, then, yes, we are aware that FDA considers using neurofeedback technologies for the intended use of treating someone, they do consider that as a device. And we are currently at NIMH accepting such applications under our device‑based funding opportunity, the R61, 33. It's true that we have historically accepted them under the psychosocial intervention funding opportunity. We are currently accepting them under the device‑based funding opportunity because of an increased awareness of how the FDA views such technologies, and because even at the early stage of developing these interventions, we want to be sure that we are funding the research that will generate the level of evidence that will be needed ultimately to get these technologies into the clinic, and we've learned today that FDA 510(k) approval is one of those steps.

Yes, Dr. Hendler.

TALMA HENDLER: I would like to make a point here about academic freedom and the exploration that has been discussed by Michelle so nicely, that a scientist ‑‑ although it sounds good to say, oh, if I talk to the industry people beforehand, I could've done this and that or maybe other things, but at the same time it would have maybe kind of constrained my decisions to do something that is only safe and actually already see the target. Some of the success comes from just trying out something even crazy. When I thought about combining the EEG and fMRI, everybody said no, it's really not possible to see amygdala with EEG, and if I went at this point to an industry person, they might say this doesn't sound reasonable, maybe start with the motor cortex. So, I give that as an example, because it's easier to coordinate it with the fMRI and EEG. So, I took this very big jump without talking to the industry people and that actually allowed me to kind of discover a new possibility. So, it's a very delicate line between not going all the way crazy and doing things that are unrelated to the endpoint of a commercial product, but also letting researchers try things that may not immediately look as they can be FDA‑approved, but if they prove to be viable, they will. So that's my point.

HOLLY LISANBY: Thank you.

KARINA QUEVEDO: Can I bring just one small point? And this is a point that is, as Talma Hendler notes, kind of critical. In that movement from science to implementation is the importance of replication. And I tried to not very subtly put it into my talk, because it is something that does plague our field just by the fact that we often reward positive P values less than 0.05, but negative or absent results are not published or not known. And I was just going to take this opportunity to suggest to NIMH that you find a mechanism or a way, even if it's just putting posters or submitting a lost dissertation of a student that spent their entire career trying to find something and they didn't, the data gets lost, and it would really be helpful to have a site or a place for negative results, because they're extremely important to guide what comes after. And I know that I'm preaching to the choir here, but I just want to make everybody notice how much our milieu prizes first‑time results, results that are significant and how much non‑significant findings are squelched basically.

HOLLY LISANBY: You're making a very good point, and I want to parse a little bit about negative results. There are negative results that are informative because you have disconfirmed a hypothesis, and then there are negative results that might be because of being underpowered. And, you know, those are two different types of negatives, but the general point is that there is a publication bias and addressing that would be important for all of our aspects of science.

I realize we're gonna have time for just a few more questions. So, I do want to get to a few questions from the audience before we hand it back over to Chris for the wrap‑up. This is a question for our FDA colleagues. "You've been using the term disease. Can an FDA clearance be given for symptoms? How does the research demand criteria terminology fit into this?"

DOE KUMSA: I think it would be a clinical kind of definition. So clinically it's determined to be ‑‑ we've cleared devices for treating symptoms of a disease, so it's possible.

HOLLY LISANBY: So, you can in effect ‑‑ my understanding of the Prism approval is that it is for relaxation and stress in people with PTSD, not for curing all of PTSD. So that's an example of treating symptoms within a condition.

I have another question here. "Since we use many free software programs like AFNE for fMRI neurofeedback, I'm uncertain about how we can commercialize this approach. AFNE is intended for research use" ‑‑ and that's a nice segue to talk about the software part, because FDA has a whole classification for software as a medical device as well ‑‑ "as we saw applications of machine learning and artificial intelligence approaches to be able to do the signal extraction." Dr. Hendler?

TALMA HENDLER: It's a good point, and I was actually surprised that everything I did had to be redone in order to be submitted for regulatory. So, everything that the company is now selling is not done in my lab. It was looking at what I did and doing it again and maybe proving it on the way. So that's, I think, the answer, is it needs to be redone in a documented manner so it can be submitted as expected to the regulatory agencies.

HOLLY LISANBY: That's the type of roadblock that we're hoping to avoid by knowing in advance what types of evidence would be needed or documentation for the studies that NIMH supports, so that the NIMH support can optimally facilitate and accelerate the ultimate marketing and commercialization. So, thank you for that.

I do want to ask one more question. "Can FDA please clarify further their position on the need for controls? The Prism for PTSD device was cleared using a single‑arm study, though they were pilot controlled studies. The bridge auricular stimulation device was cleared based on a single‑arm study for opiate withdrawal. The NeuroStar was recently cleared for adolescent depression based on real‑world data, despite a failed multi‑center trial. When are controlled studies needed and when are they not needed? This is very confusing." And one point I do want to make about the recent FDA clearance for the adjunctive treatment of depression in adolescents with TMS is that that was an extension of a label of an already cleared device. It was already cleared for adults. Label extensions are treated somewhat differently than an initial approval of a device, but would our FDA colleagues like to comment on this question?

DOE KUMSA: And Anita, chime in. So, without going into specifics of typical sponsors or companies, we always try to ‑‑ we want to look at performance data that shows us that we can attribute effects to the device, and the main way we can do that for most psychiatric devices is we need a control that would help us account for the placebo effect. And if we can attribute an effect to the device, that is kind of like what we base our determination on, that and the benefit/risk analysis. So, we are always recommending, as Anita had said, RCTs that are placebo‑controlled, but I'll let the clinician speak.

ANITA BAJAJ: Yeah, I was just gonna say the same thing, which is that the gold standard is the double‑blind, sham controlled trial, but each indication has different considerations and might have a different type of requirement based on the risk/benefit profile that we come up with for that device. So, it's hard to give a blanket statement.

HOLLY LISANBY: I'd like to thank our panelists from sessions one, two and three for fielding these questions. And I'd like to hand it back over to Chris Sarampote for our wrap‑up and concluding remarks.

CHRIS SARAMPOTE: Thank you, Holly. And I'd like to add my thanks to every single one of the presenters who joined us today and who shared their research, shared their perspective, shared their history and their work to impact the lives of folks affected by mental illness. One of the things that struck me as we talked today is the range of issues. Even if you take a narrow-focused area such as intervention development of neurofeedback or real‑time neurofeedback, you see that the types of science that's involved, everything from basic to engineering to clinical trials or to services in interventions and regulatory implementation, it takes a wide range of expertise to develop an intervention and to make an impact. But not only that, it takes quite a bit of time, and there are people who have devoted their entire careers, and we thank you for sharing that, or people who are just starting their careers and we had a number of questions in the chat from folks who are just starting out and wanting to know how to engage in this field. So, we know it takes a lot of time, not only a broad range but also a lot of time to do so. So, we thank you again for all your work and for sharing that.

A couple quick things I wanted to add to that. At NIMH we are very eager to hear from researchers who are proposing to conduct studies and I would encourage you to contact program officers early and often to talk with us about your ideas and your aims. We're happy to give feedback. We find sometimes that people don't realize, oh, I could've contacted my program officer. We'd love to talk to you. It's one of the best parts of this job, is talking to researchers like you. Second of all is also reach out to FDA. FDA is also there to help you with your products and help you with your interventions and help you think about how you might get approval and how this might ultimately be a benefit to folks. I loved the discussion that folks have had here, and I know this is a tight‑knit community and probably no need to do this, but I would encourage you to keep working together and collaborating, particularly on the issues that you discussed today and particularly as you think about this last question, about how do we take issues related to regulatory aspects and think about them in how we design our studies early on. I think it's really important and an opportunity for collaboration.

As some folks mentioned earlier, we will have a summary transcript and a recording posted in approximately two weeks on the NIMH website. And for that, I wanted to thank our contractors, Deborah and Susan, and also our transcript folks who have been working with us today and who have been working behind the scenes before this meeting today. Thank you all very much. And then lastly, I'd like to turn it over to Michelle. I know you wanted to share just a little more information, and just to thank you, Michelle, for co‑chairing this. You were tremendous in helping us to think about neurofeedback and real‑time fMRI neurofeedback and its role in the future of NIMH funding. So, thank you.

MICHELLE HAMPSON: Thank you, Chris, for all the awesome work you put into this. What a great organizational feat. And also, to everyone, it was really helpful to have people from the FDA actually talking to us, which was a unique opportunity. I just wanted to share my slide for the meeting again, but I seem to not have sharing. Great, I just received it. I just wanted to put up a little reminder to everybody or if they missed my talk, that the big meeting in our field is called rtFIN, real‑time functional imaging and neurofeedback, and the next one will be held in November in Heidelberg, and we'd love to see people there. Thank you.

HOLLY LISANBY: And I'd like to thank our co‑chairs, Dr. Hampson and Sarampote for doing a wonderful job, along with all of our speakers, the logistical support, and all of you who attended today. To quote again Dr. Misaki's comment, "Neurofeedback can translate neuroscience knowledge into brain intervention and neurofeedback can evolve as neuroscience evolves." And as you've seen here today, regulatory agencies and funding agencies can also evolve as the science evolves, because we want to support your best ideas and move them forward towards improving the lives of people living with mental illness, their families and communities. So, thank you all.

CHRIS SARAMPOTE: Thanks, everybody.