Advancing Training in Suicide Prevention Clinical Care - Day 1

Transcript

STEPHEN O'CONNOR:  Okay.  We'll go ahead and get started.

Hello.  I'm Dr. Stephen O'Connor, and I chief the Suicide Prevention Research Program in the Division of Services and Intervention Research at the National Institute of Mental Health.  Welcome to day one of the NIMH workshop on Advancing Training in Suicide Prevention Clinical Care.  We're delighted that you will join us for two days of exciting presentations and discussion to assess the state of the science related to suicide prevention training in clinical care with the ultimate goal of preventing suicidal behavior through improved training of providers.  This workshop is sponsored by NIMH's Division of Services and Intervention Research and the NIMH Suicide Research Team.

You will find the agenda and brief biographical statements for each of our presenters, discussants, and invited attendees on the Eventbrite registration website.  Today's event will go until 4:00 p.m. Eastern time.  As a reminder, day two of the workshop will occur on November 8th and, again, will take place from 11:00 to 4:00 p.m.

Here's how the workshop is organized.  Each day will consist of four separate sessions.  Presenters have 10 minutes to 15 minutes to speak on their topic, and when all presenters are done, we will have approximately 30 minutes of discussion facilitated by the session moderator.  All panelists are invited to participate in the discussion of each session.  These workshops are being recorded and will be made available in the future on the NIMH website.

In advance of our meeting, I would like to take the time to thank many who have helped bring this workshop to you.  This includes my colleagues Clare Stevens, Rebecca Kurikeshu, Jane Pearson, Adam Haim, Denise Pintello, Jennifer Humensky, Dawn Morales, Andrew Nawrot, and Penny Kisner.  I would also like to thank TaRaena Yates and her colleagues at The Bizzell Group for all their work coordinating this workshop.  Finally, I would like to thank all of those who have chosen to participate in the workshop.  As you can tell, we have assembled some of the most informed leaders in suicide prevention to discuss this very important topic.  Several of our presenters have developed and tested interventions and services approaches that reduce suicidal thoughts and behaviors.  Now they are at the forefront of reshaping healthcare systems to implement evidence-based clinical care more successfully.  Other speakers provide clinical services in real world clinics, develop tools and technologies to enhance implementation of evidence-based practices, apply implementation science to broader topics in mental health, and are leaders in utilizing lived experience to inform clinical training.

Before we begin, I want to acknowledge the passing of Dr. Aaron T. Beck whose work developing and testing cognitive behavioral therapy transformed the practice of psychotherapy, leading to positive impacts on countless lives throughout the world.  His legacy is reflected in many of the presentations included in our workshop.

We encourage the audience to submit questions through the Zoom Q&A Tool.  Please note that we will not be able to answer every question, but we will do our best to address major themes presented in questions posed.

We will now turn to NIMH Director, Dr. Joshua Gordon, who will provide a formal welcome for this workshop.

JOSHUA GORDON:  Welcome to this NIMH workshop on Advancing Training in Suicide Prevention Clinical Care.  My name is Joshua Gordon.  I'm the director of the National Institute of Mental Health, and it's my pleasure to kick off this meeting with a few words.

Why are we having this meeting is really simple.  Previous research has demonstrated that most individuals who die by suicide have had contact with the medical system in the recent past.  Accordingly, NIMH has invested in suicide prevention efforts that apply to healthcare settings.  This includes efforts to identify those at risk in healthcare settings as well as the development and implementation of interventions that will reduce that risk.

While we as a field need to continue to improve the precision of our risk detection strategies and the efficacy of our clinical interventions, we also recognize that the clinical workforce must be adequately prepared to meet the demand to help individuals at elevated risk for suicide.  Providers across all stages of their career will treat patients with lived experience of suicidality.  For example, one systematic review indicated that 46 percent of psychiatry trainees lost at least one patient to suicide during residency.  About a quarter of psychologists and social workers report losing a patient to suicide during their careers.  And nearly all social workers and psychology trainees report working with patients experiencing suicidal ideation or suicide-related behaviors, and yet most clinical training programs do not offer rigorous preparation in suicide prevention clinical care or best practices.  This creates a system of care where providers don't feel adequately prepared and may rely on ineffective strategies for assessing risk and providing preventive measures.

This dynamic impacts everyone involved, especially patients and clinicians, but organizations and clinics find themselves in a tough position, too.  They assume responsibility to ensure clinicians are competent in suicide prevention clinical care and best practices while dealing with staff turnover, suboptimal training budgets, and all kinds of other obstacles.  Accordingly, NIMH has supported and continues to support research to improve clinician training in suicide prevention clinical care.  Several investigators who have conducted this research are included in this workshop.

For example, later today you're going to hear from the investigative team conducting a hybrid effectiveness implementation study in mental health clinics in New York State where an estimated 2,800 clinicians treating 80,000 patients have received training in best clinical practices and assessment intervention and monitoring that was funded through our Zero Suicide RFA in 2016.  More recently, we have an active RFA to support research projects that evaluate the effectiveness of service-ready tools and technologies that can be used to advanced training to monitor quality and quality improvement efforts and ultimately to improve the availability of evidence-based suicide prevention services.  I'll just point out that the final receipt date for this funding opportunity is June 15th, 2022, so get on it, apply, and you can help us meet the need.

Once again, with those notes, I welcome you to this workshop, and I thank you in advance for your engagement in the workshop and for the fruitful discussions that will follow.

STEPHEN O'CONNOR:  Okay.  Thank you, Dr. Gordon.  We'll go ahead and get started.  I'd like to introduce the moderator for our first session, Dr. Lisa Brenner from the Rocky Mountain MIRECC for Veteran Suicide prevention.  Lisa?

LISA BRENNER:  Hi.  Thank you so much, Dr. O'Connor, and it's so great to see this coming to fruition today.

The first panel session will be on Universal Skills, Core Competencies, and Expertise and Optimizing Clinical Workforces.  And we have a great panel of speakers:  Dr. Mitch Prinstein from University of North Carolina at Chapel Hill, Dr. Anna Ratzliff from the University of Washington, Dr. Dana Alonzo from Fordham University, Dr. Kelly Koerner from JASPR Health, and Dr. Virna Little from Concert Health.  And I will turn it over to Mitch.

MITCH PRINSTEIN:  Hello.  Thank you so much.  I just want to quickly make sure that you can hear me, see me, and see my slides.

LISA BRENNER:  Yes, you look terrific.  I see your slides.

MITCH PRINSTEIN:  Thank you so much.  I really appreciate the opportunity to be a part of this panel.  Thank you for the invitation, and thank you so much to NIMH for hosting this really important two-day workshop.  This is incredibly important, and I'm glad that we are talking about this.

I've been asked to address what approaches we could use to help ensure that recent graduates have core competencies in suicide prevention, clinical best practices, and there's lots to say here, but I've decided to focus my comments on just a few kind of barriers and hurdles to hopefully set the stage for the excellent presentations that will follow.

Ideally, we would find that every healthcare provider, and perhaps especially every mental healthcare provider, would have a few basic competencies to make sure that we are able to address the need when it -- for suicide.  Those competencies might include a number of factors at a bare minimum.  I've listed here perhaps a knowledge of different types of self-injurious thoughts and behaviors from NSSI, to suicide ideation, to a variety of different types of attempts  -- interrupted, aborted -- or attempts requiring serious medical treatment.

All healthcare providers, but, again, especially mental healthcare providers, would have some comfort or experience discussing and understanding the distinctions between these different types of thoughts and behaviors with their patients, that all could conduct a reliable and valid assessment of these types of behaviors to be able to determine the immediate risk or imminent risk of their patients, and could construct a safety plan and/or an appropriate triage for what would ideally -- would be a very large workforce ready to receive and address any imminent needs for children, youth, adolescents, and adults who might be experiencing thoughts of ending their lives.

I can tell you that for the last 25 years I have been conducting basic research on suicide, trying to understand the risk factors -- biological/social -- that might be associated with adolescents' risk for suicide.  And I've been doing most of that work right here in Chapel Hill, North Carolina where there is a mental health treatment provider on practically every corner.  And in our basic research, when we identify folks who are at very high risk and we -- not high enough risk where we need to bring them to the ED, but high risk and we want to ensure that they're in treatment, we have a remarkably hard time finding outpatient providers who will see suicidal youth.

I was mentioning before how challenging it is to find people who will see suicidal youth mostly -- and for a variety of reasons.  And to just address that problem, I do think that because there's a small base rate of those who are experiencing imminent suicide risk at the moment of an outpatient session, there are just perhaps few opportunities for a number of individuals to get extensive training.  But I think, more importantly, I'd like to briefly address two other areas where I think that we have some concerns that we could address and create a workforce that's able to work in the area of suicide.  One of those has to do with the idea that there are relatively few experts in suicide work that could lead trainings, and the second is difficulties in training on suicide within the curricula of many mental health providers.

When talking about mental health providers, I do want to acknowledge that I have expertise particularly in thinking about the world of psychology and psychiatry, which does represent only about a quarter of the overall mental health workforce in the United States.  And these data are a little bit old, but I believe these proportions have held true for -- during this time.  Nevertheless, I do think that the problems that I'll be discussing are relevant to all mental healthcare providers.

There are very few people who have dedicated their careers to the study of suicide.  For instance, using a couple of different metrics to understand this, if we simply look at the number of publications over a 10-year period, we see that there are, in fact, much -- there's much less literature on suicide compared to other areas.  And with less literature and less people spending their graduate studies in this topic, we simply have fewer people who are able to deal with suicidal populations clinically as well.  Another metric of course is coming from the work at NIH funding.  And although there's been tremendous at work in funding more research on suicide -- thank you very much to NIMH in particular for doing so -- still there are very few scientists and, therefore, very few graduate training programs where there is a concerted energy and emphasis on training in suicide as compared to these other areas and these other healthcare concerns.

In our own view of -- scan of the top 25 areas -- top 25 programs -- excuse me -- in health service psychology and also in psychiatry, we found only approximately 10 individuals across all of those different universities and institutions who are dedicated specifically to the study and, therefore, also treatment of suicide within the traditional training staff.  Again, there are different ways to think about categorizing faculty, but there just aren't very many people who work in this area.  I will say that when looking at this, and particularly thinking about scholars or practitioners from communities of color, the numbers are even more bleak.  So this shortage in the number of practitioners who address suicide are, therefore, training the next generation in suicide is a very big issue.

A second topic that I'll briefly cover is the difficulties in increasing the amount of time for training in suicide within a traditional training program at mental health practitioner types of training course.  Mostly this has something to do with the accreditation procedures that are dictated in part by the Department of Education and also the Council of Higher Education Accreditation, which in turn serves as the accreditors of accreditation programs in psychology like the American Psychological Association and also PCCAS, the Psychological Clinical Science Accreditation System.  These accreditation systems are -- require a competency-driven model, and a remarkable number of competencies that accreditors need to demonstrate their competence in in order for folks to graduate with a degree in the area of health service psychology, as an example.

There simply is no room or no time to increase an added area of emphasis on a topic that's high priority within our country, for instance, on suicide training.  As a result, we do not see that suicide training is included sometimes at all among the large number of people who are graduating with doctoral degrees in health service psychology.  Some recent data that I believe was collected by Marsha Linehan's group demonstrated that there are many, many trainees who have had no exposure to suicide clients when they enter their pre-doctoral internship, and very few have had extensive training in suicide by the time they complete their pre-doctoral internship and in some states become license eligible.  I'll also mention that another opportunity for training, of course, would come in the form of continuing education.

Not all jurisdictions require continuing education, however, and those who do, in some jurisdictions, there is a requirement for training in specific areas, for instance, in ethics.  But I am not aware of any where training in suicidality is required as a condition of ongoing continuing education, but this of course creates a number of possible solutions.

One, thank you so much to NIH -- particularly NIMH for the funding and a variety of different training mechanisms like T32s or other mechanisms within the R portfolio that allow for concerted efforts in training not only scholars, but also scholars with practitioner experience and those coming from communities of color to increase the workforce in these areas, in psychology, psychiatry, and other mental health practitioner areas, to make sure we have a larger workforce of folks who are interested and capable in treating patients with suicidality.  I would suggest that it would be interesting to think about ways that there could be partnerships with pediatricians or general healthcare practitioners that are not primarily mental healthcare to think about ways to extend our reach.

The second, of course, is to think about ways that we can use existing platforms.  For instance, HRSA has a graduate psychology education program.  There's a focus particularly on rural health and also on opioid use as a focus in that program to increase funding to have more psychologists who are trained to -- for those areas so we have a greater work force to address those high priority needs.  This could easily be amended to include suicidality as well and increase the funding and the availability of practitioners in psychology to address suicide.

There's also work with ASPPB which is a group that oversees all of the licensing boards within the United States and Canada for psychology -- there are comparable groups for other disciplines as well -- to really increase a focus on suicide both in licensing exams and also in CE, continuing education, requirements.  There's also the opportunity to work with accreditors to emphasize suicide training as something important for all students in mental health practitioner programs to be getting training in, and also the opportunity to create and fund the production of standardized training modules that might even include mock patients for those who don't have the opportunity for in vivo experience with suicidal patients that could be used across many disciplines and many programs to help create standardized training for our field.

I hope that that broad overview of what might be possible to do to increase our ability to address suicide prevention among all practitioners is helpful.  Again, so sorry for the technical issues there, and hopefully going too quickly -- not going too quickly to make that comprehensible for the rest of that presentation.  Thank you again very much.  I'm very excited to turn it over to Dr. Anna Ratzliff for her presentation next.

ANNA RATZLIFF:  Thank you so much.  It's really exciting to be here to share some of my experiences in working with primary care providers and thinking about suicide prevention.  I work at the University of Washington, and my main area of research is actually looking at primary care psychiatry.  And that is actually why I'm talking to you today because I want to talk a little bit about my experience in working with primary care providers and really engaging them in the conversation around suicide prevention.

Okay.  I guess my slide's up.  All right.  So I will go on.

I want to talk about really from a practical standpoint some of the things that I found helpful as I've engaged in the conversations with primary care and really helping to connect around why suicide prevention is an important topic for training.  I really appreciate the previous speaker speaking to the challenges of trying to fit in a suicide prevention screening, and treatment, and assessment, and planning as part of training programs.  I also direct our residency program in psychiatry, and it's amazing how many -- there's just as more and more accreditation standards all the time, and actually trying to make this fit in can be really challenging.  And that's in a specialty that's dedicated to mental health.

So you can imagine for our primary care colleagues, family practice providers, pediatricians, and internal medicine folks that actually fitting this in is really challenging.  So I'll talk a little bit today about things that I've found to be helpful especially as you go out into the workplace and work with practices to think about how to improve the training and readiness really of the workforce in primary care.

So I think there's three main things that I would focus on.  First, attention.  So really making a convincing argument about why as a busy primary care provider or medical provider you need to pay attention to suicide prevention.  Two, I think it's really important to give people actionable strategies, tools to actually be in action when they recognize patients that are at risk for suicide.  And ideally, three is sort of aligned.  So this is really the concept of, like, when do you work this into a provider's busy life?  And often I think the best opportunities are when there are other systems changes happening in places where people practice, so I'll talk a little bit about that as well.

The first thing I always like to start with is just a reminder to myself, and maybe to those of you on this call, of what the world of primary care looks like.  And I think it's really helpful to have this frame as you then try to think about suicide prevention.  A busy primary care provider -- you know, my whole world is a psychiatrist is actually mental healthcare, right?  I think about it all the time.  It's what I was trained to do.  I spent four years thinking about it.  Well, a busy primary care provider, this is -- you know, they have a gazillion things coming at them all the time, and these are data that kind of just show the sort of top 12 things that might be happening.

A typical primary care provider might see 20, 25 patients a day with a range of presentations.  And so I think often, one of the most important things is to just think about, yes, depression and anxiety are on this list, but it's one of a large variety of presentations that a primary care provider has kind of flying at them all day.  And so getting their attention about why to pay attention to this is a really important part or strategy, I think, in engaging people in really doing the training to be prepared to address suicide in primary care settings.

So I'm going to show a few slides that actually come from a suicide prevention training course that we created in Washington State.  I have some colleagues and I created this out of the University of Washington.  This was in response to that we do actually have a requirement for training in suicide prevention in Washington State.  All providers, and, specifically, I'm going to talk about medical providers, are required to do a one-time training of -- with six hours of suicide prevention as part of the -- their licensure.

So these are a few slides that actually come from the training that we have developed.  And one of the things we really try to capture people's attention with early in that training is why they would want to pay attention to this as a primary care or a primary medical provider.  So really sharing with them some of the statistics that, you know, these patients may show up in their practice, and so being prepared to address their needs is a really important part of their work.

We also found it really effective to use patient story, real people talking about the impact of suicide and the opportunities that may have been missed in some cases to actually intervene as part of our training to really engage our audience before we actually gave them the information about what to do about that.  So I think one of the things I'll say is I think it's really important to spend the time really creating this.  Why?  For primary care providers.  And then I think many people are convinced by recommendations, and to me, you know, the -- you know, the U.S. PSTF recommendations for screening for depression, which many primary care providers really take very seriously.  Also, you know, expect that there would be adequate systems in place to ensure treatment and appropriate follow up.  And to me, if you're screening for depression, that includes being able to address suicidality, so really connecting again to the "why," other things that they may be working on in their practices really driving them towards this.

Lastly, you know, the -- in this sort of section on "why," the PHQ-9 is now used quite frequently in primary care settings, and this does include a question around suicide.  So Question 9 on the PHQ-9 is listed here.  And I think that it's really important for primary care providers, when they start using these kinds of screening tools, you know, depression is common so they should be using a screening tool for it.  But they also have to be prepared if a patient reports suicidality or anything other than a zero on this Question 9 to do some additional assessment.  And I think it's really important to recognize a lot of the patients in primary care will have passive suicidal ideation, and so we also don't want the primary care providers to feel like every patient that has anything other than a zero has to end up in an emergency room.  So really creating some capacity to do that assessment in primary care is critical.  There also is just growing, you know, evidence base for suicide prevention in medical settings.  I was working with some practices recently, and they really found this recent article that came out of the American Family Physician really helpful in sort of convincing them that this is really becoming a best practice.

So I will kind of go on to my second point which is actionable, and this is that I think it's really important if we're going to ask -- if we're going to engage in training of primary care and medical providers, that we really try to give them actionable steps to take when they recognize a patient at risk for suicide.  In this case, this is actually the registration page for our training that we developed in in the University of Washington, and we really had that focus as we created this training.  We wanted to really engage people about why this was important and then give them some really practical tools that were very accessible for them to be able to take off the shelf and start using right away.  So I'll show you a few examples of what we did.

We talked a bit about why their role is so important and that they have, you know, several ways in which they might be really a critical part of the workforce.  And then I think we really talked about what are some of the ways that they can be in action when they recognize patients.  So this might be around sort of broadly thinking about public health messages around how to make home safer, right?  If you own a firearm, how do you lock and limit access to it?  If you have prescription medications in your home, how do you keep them locked up from, you know, being potentially at risk.  Then identifying patients at risk, being able to engage in some basic assessment and safety planning, or being able to connect people to next steps and care.  And then, you know, this last one, I think, is really important, which is a lot of our primary care providers become sort of the physician champions or provider champions to advocate for the infrastructure in their systems to actually support being able to effectively assess and manage risk.

I also think it's important to introduce them to standard tools.  These are the kinds of tools that sometimes primary care providers have just never seen.  So we introduced the SAFE-T in our training as well as the Columbia Suicide Severity Rating Scale, the primary care version, really introduced this and talk through how you can use this as part of your practice.  And then we also talk a bit about, you know, first-level screening is often that PHQ-9, but if that's positive, you do need to do additional assessment and that you can do that in primary care setting.

And then we actually in our course have -- the whole second half is really going through with a modeled example of a primary care provider.  And we actually used a real-life primary care provider with an actor going through a safety planning intervention after some passive suicidal ideation comes up in the context of a medical visit.  And so this particular part of our training really focused on giving people kind of something that they could relate to and see that it was possible to do this as part of their very busy life.  So really making people feel like these kinds of activities were actionable.  We do a lot of interactivity in this.  It's an online training so that people can take it at any point in time, and that really allows people to at least have an opportunity to practice in a virtual setting some of these skills.

The last thing that I'll focus on is aligned.  My area of expertise is collaborative care, and I spend a lot of time helping practices implement collaborative care.  There are data from collaborative care that the -- that that can actually in and of itself reduce suicidal ideation.  So we really consider when we're implementing collaborative care that it's an opportunity to also talk about their systems of suicide prevention in a primary care setting, and really make sure they're building that capacity along with that implementation, or adding capacity to address mental health disorders.

I think it's also important to make sure that people are aware that there are great resources out there around established protocols that also can include financing strategies.  The cost of doing suicide assessment is often one of the big concerns of primary care, just the time it takes that they're not spending billing for other kinds of care.  And so I think really addressing upfront some of those concerns and making sure that you are introducing training aligned with these kinds of strategies.

So those are a few ideas that I wanted to really seed this conversation with around from my practical experience in working in primary care settings.  I think it's really important as we think about how are we going to engage primary care providers to really come from the frame of a primary care provider or a medical provider, how busy their world is and how this can fit in.  Make sure you're providing really practical skills for training for medical providers.  One of my favorite comments that we got back around our training, which has now actually been delivered to over -- 3,700 people have completed that training, so it's a large number of people have completed that.  People talked about actually printing out, you know, a safety plan sheet that they could actually have in their office for the next time they had a patient that needed that.  And that was probably, you know, one of my most happy moment as a -- as an educator because we really were hoping people would feel like some of the things they learned word that kind of actionable.

And then lastly, I think there's a lot of opportunity as there's often work more broadly around integrating mental health into medical care settings to really add suicide prevention as part of those workflow changes and other systems changes that practices will be involved in.

Okay.  So I will stop there, and I will pass the baton on over to Dr. Dana Alonzo from Fordham University.

DANA ALONZO:  Yes.  Thank you so much, Dr. Ratzliff.   Let me get this up.  Okay.

I'm so pleased to be here with all of you today, and thank you to the panelists who went before me for your interesting and important presentations.  I've been asked to talk today about suicide prevention in low-resource settings and priorities for provider training.  And I think this is of particular importance as research indicates that while someone dies by suicide every 40 seconds globally, 79 percent of these will occur in lower-, middle-income countries.

So I thought I would start today by talking about some of the contextual challenges that serve as barriers to translating conventional approaches to suicide prevention and provider training to these low-resource settings to better understand why they have not resulted in either decreased suicide rates or increased workforce of suicide assessment and treatment--competent providers.

One of the most important things to understand is the context of armed civil conflicts that characterizes most low-resource settings, decades of violence, corruption, human rights abuses have resulted in, and continue to contribute to, community and family fragmentation, discrimination, marginalization, and overall loss of cultural identity, and a general lack of trust in the authorities and foreign experts, which serves as a significant barrier to implementation of evidence-based training models in these communities.  In addition, there's pervasive stigma regarding mental illness in these settings.  It's been noted that the degree of structural stigma in some low-income countries is so high that individuals are victimized and abused simply for expressing a mental health need.  Many low-resource countries also allocate less than one percent of GDP to mental healthcare provision, and little to no human rights protections exist to provide oversight over the few national mental healthcare facilities that do exist.  And those who choose psychiatry as their specialization are often mocked and labeled as lazy or unintelligent for that specialty selection, and psychiatrists, in turn, report holding negative attitudes towards their patients with severe mental illnesses.

Compounding this structural stigma is the common belief in many low-resource settings that mental illness results from moral shortcomings or weakness rather than biological causes.  Individuals with mental illness then often internalize and accept these beliefs and expect to be treated in a discriminatory way and therefore work to hide their symptoms rather than to seek help for them.  Qualitative research of people affected by mental illness in low-resource settings indicates that the effects of this combination of structural and internalized stigma are often more burdensome and distressing than the mental health conditions themselves.

In addition to the stigma and lack of funds and protections is a general lack of providers and an absence of specialized training on suicide assessment and treatment curriculum in mental health training programs and medical schools across low-resource settings, such that even where -- when providers do exist, they're ill equipped to manage the needs of clients presenting with suicidal thoughts and behaviors.

So without this recognition of the contextual challenges that are prevalent across low-resource settings, the priorities of conventional approaches to suicide prevention and provider training often miss the mark when trying to be translated to these settings.  Largely, they're grounded in a Western perspective which then doesn't incorporate mechanisms for addressing those long-term systemic inequities and structural issues, like the lack of funding allocated for mental healthcare provision, which then ultimately undermines the long-term impact of any prevention model, or they might be focused on implementing outside models that have evidence for effectiveness in high-resource countries, but were not designed with the unique values, expectations, and beliefs, and experiences prevalent in low-resource settings and, therefore, make translatability limited.  They also often incorporate training or supervisory structures that then limit scalability and sustainability, largely because they rely on foreign specialist experts for ongoing supervision and case consultation.

So then what should the alternative priorities be?  Well, first, I think it's important to identify and harness de facto sources of support and care in the community, and this requires engaging local stakeholders to identify who in the community can take on this position.  In some of the work that I did in Guatemala and Peru, this involved engaging clergy, teachers, staff of school offices, bodega owners, right, whoever the local stakeholders and community leaders identify as meaningful individuals in that community.  Another priority should be to maximize specialty care, so this could look like a chain of care where lay providers are trained to manage low- and moderate-risk cases while reserving higher-risk cases for that small network of specialty mental healthcare providers that does exist.  And in this way, those contextual limitations related to lack of human capital and lack of training are then addressed.

Alternative priorities should also include mechanisms for sustainability and growth that don't foster dependence on foreign experts but rather empower communities.  So this requires building in incentives for that small network of specialty care providers to take on the role of supervisor and developing in-country networks for peer consultation, and decentralizing care, and expanding reach to those higher-risk marginalized indigenous communities.  And lastly, it requires establishing countrywide standards of care so that in any region, individuals are able to access providers who have a core set of basic skills for addressing suicidal thoughts and behaviors.

So what does an approach built with these alternate strategies look like?  So in my work, I've translated this as a strengths-based community capacity building approach that's grounded in a social determinants of health framework that I've used throughout countries in Latin America called the Engaged Community Action for Preventing Suicide, or ECAPS model.  ECAPS has as its starting point the recognition of local communities to identify the problems that are relevant for their community related to suicide risk and the resources available to address them, and to develop, implement, and maintain solutions to their own problems.

It's grounded in a community-based participatory approach that preferences the experiences, knowledge, and needs of the community.  It supports cross-cultural partnerships and sharing and exchange of knowledge and resources.  It builds on the unique community's needs and assets.  It positions community leaders to maintain, sustain, and grow the model through a training of trainers model.  And it views training through a process of mutual learning where training participants are considered agents of change rather than just recipients of outside foreign expertise and knowledge.

It acknowledges the multiple and cumulative forms of trauma that were mentioned earlier that lead to that distrust of authority and outsiders by positioning local leaders to guide the development and implementation process.  It incorporates mechanisms for peer supervision through the development of that chain of care and providing incentives to trained specialists for serving in a supervisory capacity, like an advanced certification or recognition by licensing and accrediting bodies within the country.  It leverages technology as a means to provide and support training supervision and case consultation, and incorporates a mechanism for addressing the social determinants that drive the structural inequalities and disparities that promote long-term -- or to promote long-term systemic change.

I'll go back.  ECAPS is a seven-step process, and I won't go through all seven steps, but what I'd like to highlight is step four which is developing a change strategy because that's where the training of providers comes in.  And so you'll notice that that's midway through the process, right?  It's not where we start off.  We're not coming in with an outside training model that we're going to bring to the community, but we're developing that training out of collaboration with the community who identifies what is relevant and what needs to be included in that training model.

So in my work in Latin America, we developed a training model called PEDIR, which in Spanish means to ask, which trains identified community stakeholders to ask about suicidal thoughts and feelings and then be prepared to answer the -- to address the answer they may receive.  And I'm happy to share what the actual content of PEDIR training is and what the modules look like in our discussion.  But I guess I'll wrap up my time by saying that what's unique about PEDIR training is that it's not a predetermined evidence-based model that's just adapted to the community by modifying language or including relevant cultural idioms of distress or explanatory models of mental illness.  Rather, the actual content of PEDIR training is it's self-driven by each community so that no two trainings will be identical as the relevant social determinants of health related to suicide risk present in each community will vary, as will the risk and protective factors for suicide and the resources for addressing suicide risk.  So in this sense, PEDIR training is not only individualized to each community, but designed around each community's unique needs and assets as guided or identified through that ECAPS process.

So I will stop here, and I will turn it over to Dr. Kelly Koerner.

KELLY KOERNER:  Thank you so much.  That is amazing.  I might even say that most of the settings in the U.S. -- the healthcare systems in the U.S. that we work with would qualify for low- and medium-resource settings and was very, very moved by your talk Dr. Alonzo.

So I'll just say that Stephen asked me to talk, I think, because of the different types of training experiences I've had and some of the conclusions that I and my colleagues here have been drawing.  Some of my career has been focused -- I'd say probably the majority of it -- on suicide-specific care and training for providers, really especially focused on caring for people who are at higher risk -- the very highest risk for suicide.  And then a whole other set of research has been focused more straight dissemination implementation science research, but all with the spirit of trying to understand and develop new models that really get at clinical expertise at levels of skill that can really move clinical outcomes and then also that can be sustained.

So just so you know where I'm coming from, I've trained research therapists in clinical.  I've had the opportunity to work with juvenile justice staff who have high school degrees.  I've led for a number of years a learning community that was international with people from different regions where there -- there's less access to evidence-based, mostly social work, and then also experimenting with a number of different, I would call them, kind of novel, tinkering, turn everything on its head type training models just to see what really can work.  So that's sort of where I'm coming from.

And I just say as a disclaimer at the beginning, I and Linda Dimeff, my partner in all things work-related, have moved almost solely to technology-first approaches to training, but also to treatment delivery, even in suicide care, and I'll tell you more why.  But I think -- I think that that is -- I'm super glad, Stephen, that we have a number of panelists who will be talking about technology-assisted care because I think -- I'll tell you why I reach this conclusion.  But I think that's the direction we all need to really focus.

But on the -- on the topic of training and sustaining clinical expertise, I just want to share a few things that I am learning in the hopes that some of these ideas will be useful to the group as we talk amongst ourselves in these next two days.  The first thing, whenever you're training for clinical expertise, you have to reckon with solving what's called the amateur plateau.  I think folks may be aware of this.  This is one of my more favorite representations of this concept.  But whenever you're developing a new skill or ability over time, you -- some feel frustrated and decide it's just too hard.  And I would say from my perspective, this is the majority of the folks we train in suicide care, that they decide that it's too hard for them and they opt out, and they opted out as soon as they can.  As soon as the requirements lift, they screen those folks out and they don't look after them anymore.

Then there's another group who kind of get up to pass the "suck threshold" is how I think about it, and then they plateau because our environments in which we work don't actually train to a level of expertise.  They train to being just good enough, and people continue to persist the way -- this way, and this is the group -- I tend to focus more on postgraduate training.  This is the group who I mostly am interacting with.  And then there are a few who get off their strandedness in the kind of amateur plateau past the "kicking ass threshold," and actually do keep pushing themselves and develop, but there are very few of us who have the conditions, back to the -- all of the points the previous speakers have made.  It's very seldom that you get people who have the conditions that will build expertise, and that's because experience alone doesn't develop expertise across all fields, be it surgery, chess, skiing, sports, behavioral health.  You see a pattern of improvement in skills for a few months to maybe a year, and then people plateau out because they get good enough to meet the demands in their setting, and it becomes automatic.  You just do what's easy, and there not really incentives to improve.

People who -- and just an analogy.  Anybody who's got sort of amateur level guitar, you know a couple songs -- "House of the Rising Sun" -- around the campfire and then you pass the guitar.  There's just not enough in it for you to raise, and I would say that's the case primarily with suicide care for most average providers.  They don't see enough people that it's worth raising their skill.  Experts, however, their pattern looks more like this, that they continue to grow and you can continue to see improvements for up to 10 years.  And the reason is because, like Steph Curry and others who are at the peak of their game, they have deliberate practice that slices up the competencies and they have informative feedback.  Our training models in suicide care should always bake in deliberate practice and informative feedback.  That should just be a starting place for development of expertise.

This may be easier for you to just quickly skim and read, just an example of what this might look like.  And we've used our group, different collaborators and I -- I'll just talk here just about some work with Asha Puspitasari and others.  One of the ways we've tried to work with this basic premise is have all didactics done on your own, use the end-time with the expensive trainer/supervisor for practice with feedback, and actually bake in research assessments as part of the training model.  This could be done in CECCME-type formats.  But you can look at the Puspitasari article in JCCP for the basic model.

But the thing that you have to do is have a session that has performance expectations, very friendly.  Like, if you were going to a great lab or a great gym, you learn the skill on your own with the demand to perform it in the next session, and you use adherence measures to rate each other's performance live.  There's bug in the eye feedback.  You know, there's lots of ways to play with this concept.  But the idea is train it on your own, have the demand of performing it, and then use feedback and have participants, learners give feedback as a way to rapidly up skill people.  So again, just a basic instruction and modeling all provided on your own time.  The expensive trainer reserved for that unique training, feedback, and practice and rehearsal, so one idea here.

The other thing I would say is what should be trained.  I want to just throw out this idea that has  -- I've really moved this way myself, which is all training should be of modular experimental therapeutic competencies, in my view.  That will lead to faster science.  Let me just unpack this idea a little bit, but I think we should abandon this idea that we're training a package and instead think that we're training modular competencies that are really tightly tied to experimental therapeutics models.  So super just concept here.  Take any kind of context that -- therapeutic moment that you're working in and that intervention, and then you have a bunch of therapists who self-identify in these different ways.  They say they're doing DBT or CAMS or whatever.

If you actually look at what those therapists do  -- fidelity measures, rating sessions, and so forth -- what you actually see is that the package labels mask what's actually effective.  Some therapists have outcomes standard deviations above others, and that I think is where the action is.  So if you look at a bunch of DBT therapists, you know, they really distribute themselves in a particular therapeutic moment.  Some may go way more cognitive, some others more skills training more toward exposure.  So and knowing not the label, but that more specific experimental therapeutic move, I think, is where the action is.

So when we think about suicide care, we really want to focus on not only acute risk, but also treating the long-term drivers, right?  And we have people who come in with really different histories of how long they've thought about it, how much they have proceeded to make their plan, the things that actually will transition them for ideation to action, all that.  So if we're thinking about how we're training folks here, I think it's -- it would be helpful to think about training to detect a client marker, say, urges just for example, the interventions you do, and you're looking for a little move in the change process, right?

So our training research could be more tied to this level of loop out of the package into the process so that you are identifying what actually moves the little O's is how I would think about them, these little changes, and you can build training models around this so that you're training to see the client marker, training to do the intervention, looking for the little O.  And then, of course, you know, for training research you have to build around is that behavior -- are you going for adherence, or competence, or my preference that it just moves the little O.  You're training until you see the therapist can move the little O.  And then, you know, there's other aspects of getting so that you don't have to do this with session ratings so that you can kind of work it back into some standardized role plays.  But nonetheless, I think this model could really advance things a lot faster.

So just to build it out, I'd refer you to Kristen Grabill, I think, has done some fantastic work on pediatric OCD with this experimental therapeutic model.  But you can do this for chain analysis of urges where you specify because lots of -- I'd say most of the different treatments out there have some way in which you're understanding what sets off urges, and then therapists are supposed to do something in response that.  But you could actually tie this really nicely to test different hypotheses about which little O's did you move.

The other thing I'll just throw out here in the last couple minutes is I'd love to see us move, too, away from training just in our current protocols and train instead towards algorithms.  The predictive algorithms that are out there -- computer adaptive testing methods of doing assessment -- are so much better than us humans at predicting transition from ideation action or who's going to be really at risk in the, you know, next six months, that sort of thing.  I'd love to see us actually train so that we could understand what helps people release autonomy and understand when the algorithm is actually better than we are and make this adjustment so that we move more quickly to adopt AI and other important advances.

Last point I would just want to share is -- has more to do with the environments in which we practice also do not shape the type of expertise that we would like to see.  So just the way that this typically talked about is in terms of kind versus wicked.  In kind environments, you can detect patterns and you get immediate feedback.  I think of this as like a NICU nurse with fluid management.  They immediately see the urine output.  You put fluid in, you see it out, you see the kid pink up, you know.  Like, they get immediate direct feedback whereas our emergency department social worker who does suicide crisis management gets no feedback.  She sees people who recidivate back to the hospital.  They have no idea if that safety plan really worked.  And the conclusions they draw from the folks they see back have not much to do with what should you do more of, what should you draw from your experience.  And this is another way, I think, that we need to improve things.

So I'll just point you to Hogarth, which is the best, I think, place to start if you're interested in that idea.  But training should always include measurement-based care.  So like in DBT, there's the skills diary card that helps you see the marker and see if it moves.  And CAMS, there are very specific ways you're measuring each session and you see if it moves.

So I'll just -- I just want to say the last thing here is I hope that we can move perhaps away from thinking about instructor-led training or training a whole workforce because it's expensive.  The complexity of problems that need to get addressed are hard, and our workforce is -- shortages are going to persist.  So I kind of feel like this little pug here.  I'm not sure who we're going to train given the shortages that we're seeing.  I don't -- I focus myself much more on feasibility.  So as we think about what we're funding, if training research, I hope it will be thinking of training as treatment development, as experimental therapeutics in action.

So I'll leave things there and pass it to Virna Little.

VIRNA LITTLE:  Thank you so much.  Very much appreciate it.  I'm just going to swap some screens here.

So thank you, everyone.  I think my presentation is going to look a little bit different and want to talk about some practical experience that we've learned on the ground through a suicide prevention and primary care training that we actually came up with that was a part of a project funded by Centene and in conjunction with the Association for Clinicians for the Underserved.  And we managed to train primary care providers around the country, and we started really working with primary care providers in the Zero Suicide Academies, which I've loved being a part of.  And having spent a lot of my career in primary care, realized that what we needed to do was try to develop some training that would really work for providers sort of on the ground, and came up with a suicide safer care training.

And we have had the ability to do that, really set some very basic groundwork for primary care providers, and essentially mirrors a primary care visit, and goes through the steps of what you might do for a patient at risk for suicide in conjunction with the similar workflows from primary care.  And so do a lot of things like level setting in the very beginning, talking about, of course, the prevalence and to mirror what Anna said earlier, like, why should primary care care about suicide safer care and caring for patients at risk for suicide.  Doing some work around language, certainly working with some language from Ursula's website, Now Matters Now, so that we can really set a little bit of a foundation.

This is the outline of what that training looks like.  The training generally takes about an hour, and mostly because we learned over time that that's the amount of time we had, whether it was rounds or part of a staff meeting, that to do anything longer was just not going to work.  And again really trying to cover these pieces that follow a primary care workflow, and reviewing with them, like, why we're focusing on health settings, what that really looks like, and why they should really pay attention to suicide safer care.  And so talking a little bit, and actually, I realize that these are not the slides that are going to be helpful for some reason.

And so one of the things we've done is we've been able to train 3,000 primary care providers in about 28 states.  So 3,000 primary care providers and their team members will really focus it a little bit sort of based on who attends the training.  Oftentimes we'll get a lot of primary care providers.  Sometimes we'll get a little more nurses or administers.  And essentially, what we try to do is talk very directly to some of the things that you often hear in primary care.

So there's a couple of things that we hear.  One is really talking very specifically about having the knowledge to intervene.  As we've talked to providers around the country, about 95 percent of them from all disciplines said they had thoughts about suicide safer care being a part of their role.  So in other words, really feeling like, yes, this is part of my role.  I really feel like I could be helpful in suicide safer care, that I'm ready and willing to sort of take some information and try to change my practice a little bit.  Most of them, more than half of them, said that they had not gotten any training either in their current positions or in their training programs.  The other piece that came up, interestingly, was that 44 percent of the behavioral health providers in these primary care practices said they didn't feel comfortable or confident to be able to intervene.

So some of the things that we will do is talk about what do we want to say when someone says that they're thinking they're at risk for suicide.  So I know some of the other speakers talked a little bit about how primary care is using the PHQ or the PHQA, and we will literally role model what it means to go through the PHQ and talk about -- talking someone through.  So I see that you're having difficulty sleeping, and that's a problem for you most days.  I also see that you have thoughts that you're thinking about hurting yourself or would be better off dead.  Let's start there and let me ask you some questions, and thank you for telling me that you're thinking about suicide.  So that what we're doing is giving providers some very specific language.  We call them storage statements to say two or three sentences when someone first tells them that they're at risk for suicide or for responding to Question 9 on the PHQ.  And what that's really done is given them some very concrete language to say.

We also talk a lot about addressing the amount that providers concerned that they won't have enough time.  And so one of the things that we do as much as possible is make very specific comparisons to other primary care workflows.  So as an example, if someone came in and they were struggling with asthma, we would give them a treatment.  Potentially we would talk to them about whether or not they've been to the ER.  They've used their rescue inhaler.  We would talk to them about environmental triggers.  We would talk to them about emotional triggers.  We might give them some education, teach them how to use the spacer.  We might even give them something that we have onsite that might be helpful for them, a peak flow or something.  We would do an asthma action plan.  We would figure out whether or not they needed specialty care, maybe to see a pulmonologist or to have someone go to the home, and then we would do an asthma action plan.

And so when we think about somebody at risk for suicide, we're really taking the providers through some of those very same steps, only applying it to suicide safer care.  And I think what that's meant is that many of the providers have said, oh, I already use this workflow.  I understand.  My staff does this all the time.  We respond to patients who come in who are struggling with asthma, as an example, and then giving them very concrete things to apply to the same workflow.  So giving them storage statements if someone says yes.  And then, as an example, taking them through the ASQ and the four questions or through the Columbia Scale, and then teaching them, say, okay, if someone scores positive on the Columbia Scale, here's what you might be able to do.  And giving them some very specific tactics, such as putting the lifeline in their phone so that they can share it with patients and talking to them about the Now Matters Now website so that they're able to do that and really think about what that looks like.

And then talk to them about appropriate levels of care, so not that you see, you know, that comes in that's having chest pains needs to go to the emergency room.  What are some of the things you do?  What are some of the outside factors that play a role?  And what we've found is that providers have said a couple of things.  They have said when it really gets put into what they already do in their workflow, it feels very possible.  The other thing that we've heard from providers is that if they feel like it is a suicide prevention workshop, some of which they might have had before, that they don't necessarily pay attention, that they feel it is too broad or not specific, and doesn't have things that they can come away with.

We also talk very specifically about what they can do to start to track their population of patients at risk for suicide, which include adding suicide risk to the problem list and making it very similar to what providers are doing around diabetes and having a diabetes registrate, and we encourage them then with language that they can use.  So I really care about you, and I know that we've had a chance to talk about your thoughts of suicide today.  I just want to let you know that I've noted our conversation in your chart because I want to make sure that every time I talk to you or someone from my team talks to you, we make sure you're safe.

And so giving providers very specific language that they can use when they add suicide risk to the problem list, and talking to them about paying attention to what we call two populations.  So the population that comes in, you identify at risk for suicide, what do you do.  What's your pathway?  What's your response?  But also what do you expect to happen when that person comes back for a primary care visit?  If I have diabetes and I come back to primary care, there's a workflow and a process that each time you asked me about my diabetes, you check my sugar.  There are certain things that happen.  So what are the expectations when somebody comes back for a primary care visit?  And so those are some of the things that we cover.

What we have found across the board in some follow-up with providers is that they have actually had some very specific things that they could take back to their practice.  So what we're hoping to do is to continue the project.  It has been funded through the balance of this year to really train healthcare providers.  It's an hour and it offers CME, and we're hoping to take this learning and continue to develop some trainings that will further our ability to train primary care providers on the ground with some tactics that they can take back and use immediately in their practice.

So I apologize for the slide mix-up today.  Apparently, it's going to be a little bit of a struggle today with some things.  But, again, wanted to share this information with you, wanted to share some of the learning, and look forward to answering any questions about this information or the project.  And I'm going to turn it over.

LISA BRENNER:  Hi there.  Great.  Thank you so much, everybody, for all the presentations, and I love how they all fit together.  And, Dr. O'Connor, great job in selecting the panel.  Wonder if people who are in the last session can at least turn their cameras on.  I think that's what we're supposed to be doing at this point.  So and, Dr. Little, since you are available, I'm going to start with a question for you, okay?  And also thanks to the -- to the presenters for their resiliency this morning.  I think, you know, it's all part of the deal.

So and this is a question for Anna and Virna.  "Curious about how to do this upstream and how much foundation do people need before we actually" -- and I guess anybody could really chime in.  "How much foundation do people really need before they can do this suicide-specific training," and how do we help people that, like, maybe that this feels really foreign to them?  Or, you know, I know sometimes a lot of times we find people don't think that this is in their lane or not in their competency lane.  So how do you -- I guess I'm expanding on the question now.  How do you help people understand that this is part of their job?  And two, what's the basic foundation that folks need to then have this suicide-specific training?

VIRNA LITTLE:  So I can just start off and talk a little bit about we're doing a project now in Montana where we're doing, like, a train the trainer from different healthcare organizations.  And we brought them together, talk to them about the training and gave them both tips on training and also tips on being able to be a train the trainer, and went through and did the rehearsal and talked to them.  And one of the things that I think is super important is that they developed their own, I'm going to call them stories, from their own healthcare settings where, you know, either the team has been impacted by suicide or stories that they can relate to in terms of workflow, and how the practices have certain workflows around certain conditions that can be related.

And what we found is that the day-long training and then some follow-up really gave people a foundation to be able to do this.  But we really had to work with them closely to personalize it and make it their own.

LISA BRENNER:  And it sounds like you want to chime in here, too.  That would be great.

ANNA RATZLIFF:  Yeah, I guess I would say I think, you know, in our state, because there's required training for your license.  That is highly motivating for people to show up.  But I think the real opportunity is then to do exactly what Verna talks about, which is really make sure that this -- that they actually kind of come in and own, that this is part of their scope of practice.  And I think because we're -- suicide is a relatively rare event in primary care, it's really important to actually really connect with them about what their experiences have been.

And I think in our survey, we do some surveys before and after our training, and, again, we've had over 3,000 people.  And most people have either had a patient die by suicide or know someone in their life personally that's lost someone to suicide.  I'd start it with that to remind them about like why and kind of prime them to be really paying attention is helpful.  And, again, people don't want to hear about problems that they don't have solutions for, especially in primary care.  So if you don't give them some really actionable things to leave that training with.  I mean, I love, Virna, the kinds of things that you're talking about.  Like, this is -- this is the analogous thing that you may be dealing with all the time, and so that's how you can kind of learn about this, are really important strategies if we're going to engage that, like, broader workforce, sort of the layer outside of behavioral health where this is a more common problem that you're addressing all the time.

So, you know, I think I really appreciated the question about, like, who is our workforce.  And I think we're all having to really think much more broadly about who is the behavioral health workforce.  And all of those people are going to at least need, I think, basic skills in recognizing someone at risk and what to do as a first next step when that happens.

LISA BRENNER:  Mm-hmm.

LISA BRENNER:  And, Kelly, I wonder just to extend on this a little bit, you know, I think that use of technology can be an extender, but it can also sometimes take people out of the process.  And so, you know, people kind of maybe, you know, don't have to ask the hard questions or ask them less frequently.  How do we help people get comfortable because I think there is a certain amount of exposure to doing this that is needed and use technology to extend?  And I wonder if that -- I hope that's a clear enough question.

KELLY KOERNER:  Totally is, yeah.  The way I tend to think about it is as technology assisted, so what are humans good at versus what is technology good at.  You know, humans are good at being kind when they are not pressured and they're not having to check off their EHR, C-SSRS form, you know?  So if I as a bot could actually gather that information for you and let the person privately share in a way they're more comfortable with and then serve that up to you provider, you skim it.  You hone in on where you need.  And then you can just be kind.  Like, I'm super sorry you're at this point.  Let's focus on how to get you stabilized and into treatment for the drivers.  So that's how I tend to think about it.  Does that make sense, Lisa?

LISA BRENNER:  Yeah, that makes great sense, and I like that idea and think about that a lot, too, like, what are computers good at, what are humans good at, and then how can we -- and I think maybe a little bit in healthcare and maybe in mental healthcare a little slower with that then than other folks.  I'm hoping we're gaining some traction.  Dr. Alonzo, how long is the training that you -- that you -- that you described, and can you also expand on how do you tailor it, and could it be incorporated in a research project?  So that's three parts.

DANA ALONZO:  Yeah.  So part one or one of the parts -- I don't know if I'll get them in order.  But it's tailored based on the information that is received from collaborating with the community, right, through the focus groups in step one of that ECAPS model where the community stakeholders, whether that's service providers, community residents with lived experience, family members, school personnel, whoever those community stakeholders are, present from their perspective what are the major challenges in their communities, right?  And then identify not just what the assets are that are in the community for managing those issues, but also what might not be so well developed that needs to be enhanced or modified or restored, right?

But it's community driven, right, which is why that -- developing that training doesn't come in until step 4 because you have that basic information from the community where you're -- where they're giving you their expertise about what challenges the community is facing.  You can't develop the training model, right, because we don't have the information on which to identify the relevant content.  And that's why the model varies from community to community, right, because it will specifically address those unique risk and protective factors, and the unique social determinants of health related to suicide for that specific community.

So is there research based on it?  Yes.  I have done some preliminary research on it, and I'm hoping to be supported by NIMH.

LISA BRENNER:  Okay.  Let's go on.  Let's fund this woman.

DANA ALONZO:  Yes, please.

(Laughter.)

DANA ALONZO:  Make that happen.

LISA BRENNER:  No control over it.  Just putting it in play.  All right.

DANA ALONZO:  And then part three I don't remember.

LISA BRENNER:  You answered them all.  That was great.

DANA ALONZO:  Okay.  Good.

LISA BRENNER:  Thank you.  All right.  And I think just, you know, back to -- and, Shelby, I see you.  I think you have you something want to jump in on.  Can I ask a follow-up question and we'll will jump to you?  Okay.  So part of, I think, the challenge is, we're still in early days trying to identify best ways to screen even with computer help even with technology.  And, you know, I think we're getting better and better at screening and evaluation and having that be evidence based, and certainly way better than we were, I don't know, 10 years ago, 15 years ago.  Yet, you know, most of the work that's still being done is being normed and evaluated on, you know, specific populations and may not apply to other folks.  So how should we think about race and ethnicity factors when we're thinking about taking screening tools that have some evidence and moving into other settings?  How do we get to enough evidence.  Yeah, how do we get to enough evidence, and what do we do until we get there?  That's a hard one.  Anybody feel free to chime in.

KELLY KOERNER:  I'd defer to Dana on this one.

DANA ALONZO:  I think it's challenging because of the way evidence is defined, right, or what is considered acceptable, or good evidence is defined, and RCTs and some of these communities just are not the approach that will be accepted or give -- have the greatest impact.  And so I think there needs to be a sort of shift in that definition as a part of the way to address that problem.  And then I think -- well, maybe I'll stop there, yeah.

LISA BRENNER:  Okay.  And I think -- you know, I think this is something we're all struggling with is trying to figure out how to get enough of the right kinds of evidence so that we feel like the practice is evidence based enough, whatever that exactly is, and then how do we keep expanding.  And so this is a challenge we can all -- Virna, do you have a comment on that?

VIRNA LITTLE:  Yeah, I would also say just practically speaking, you know, like, I tend to just be very practical and in the weeds.  The reality is oftentimes you have to flow with what's in the electronic health record because a lot of places don't have the resources or the ability to change what's built in.  Fortunately some of the EMR, you know, companies now are putting some standard, you know, tools in that are part of suicide safer care.  But oftentimes it's about figuring out, because if you're going to operationalize it, you have to build it in with what they're already doing in the EMR.  So you have to really think about what is doable for them, what is the best possible scenario that they can do to either get it built in or some sort of, you know, work around --

You know, I know one time I had an EMR and I was ordering the safety plans.  And, you know, safety plans were -- is a letter because I could modify them and I could attach them to a visit, right, even though there wasn't a template built in.  And so I think you also have to be really practical as well.

LISA BRENNER:  Yeah, practicality and creativity are definitely two keys with all of this.  Sorry.  Kelly?

KELLY KOERNER:  Yeah.  No, I was just going to come back to something that Mitch said as well, which is I think the lack of diversity in trainers and researchers, in the folks who bring along junior people, it's really moved us more toward people with a lived experience stance.  That's just another reason why we have moved that way to paraprofessionals because there are -- I think there's good evidence that that group as a workforce are really talented and effective.  So just a -- just another plug is if we expand who we think about who we're helping, there may be more a rapid path for something that's more on that side of the house for our workforce.

LISA BRENNER:  Yes.  No, I think that's a really important diversity issue that we sometimes -- we often don't talk about enough.  And I'm going to put a plug in also for folks with health conditions and disability and bringing them into the conversation, too, the challenges they may be having in their lives.  Reasons for living may be very, very different if you're living with a chronic health problem, and we just need to find better ways to talk to folks about hopeless days, not hopeless days.

I recently was talking to somebody who's living with a brain injury, and she's like I feel hopeless sometimes that I still have to live like this, and I'll have to live like this forever, but I still want to live.  And so how do we kind of really respond to that and meet her with kind -- the kindness that you talked about, Kelly.  Not overreact, not underreact, you know what I mean, and hold that and then help her work on that hopelessness.  Shelby, sorry to make you wait.

SHELBY ROWE:  That is okay.  Listening to all the speakers this morning, I think that, you know, we can have some of the best trainings, and I really liked what Dr. Koerner and Dr. Little, you know, were talking about.  We can have those bite-sized measurable trainings because to develop that expertise that is essential and needed, we have to have really clear-cut, tangible learning objectives and goals.  You know, when we're talking about physicians, their whole training, you know, it's a class of people with perfectionism as a trade.  And I think we have that elephant in the room is convincing our healthcare providers that suicide care is lifesaving healthcare.

You know, I'm listening to speakers going, I'm not sure how, you know, we convince our healthcare professionals in the country that they should care about their suicidal patients because once they realize this is a life-threatening health event that I can prevent, I think a lot of these other issues will fall to the wayside, you know.  Because when we think about treatment and diagnosis for cancer, you know, those tools are real expensive and inconvenient, and they take time and it's a challenge, but they do it because they know it's going to save the life of their patients.  It's going to promote the lives.  And when I see and hear, like, well, you know, the standards for training don't include suicide care because there's no time, you know, there's time for lifesaving care and the training when it's a priority.

And so I'm not sure on that attitude change of shifting, this is something that I can do to help save the lives of my patients.  And then those training objectives because we can have some of the best training there, the most evidence based, but if they're seeing this as an option and not as this is essential lifesaving care for my patients, you know, I'm not sure how we get that attitude change because I think the skill level will change that desire to have an expertise, you know, the not allowing people to stay at that amateur plateau.  You know, I wouldn't go to a heart surgeon that opted out of learning the best techniques.

LISA BRENNER:  Yes.  Yes.  I agree.

SHELBY ROWE:  So if we have that attitude change, we'll see that skill change in our field, and so maybe we're getting closer to that tipping point.  And I'll stop talking because I see lots of hands up.

LISA BRENNER:  Yeah, there's lots of heads up.  I mean, I think one thing we have tried, and I'm interested in the other folks on the call today, you know, people often talk to us about workflow and how this can disrupt workflow.  And, you know, we say, like, what if somebody comes in with cardiac symptoms and all clammy and, you know, crazy off-the-charts cardiac emergency that also disrupts workflow, they're like, oh yeah, you're right, but that's a disruption that they had anticipated being part of their practice.  So I think, you know, how do we -- how do we help folks realize that they do have these, you know, not frequent, but emergent events that happen already and then switch that skill set to suicide prevention?  Tony, chime on in.

ANTHONY PISANI:  Hi.  This is a really helpful discussion.  Appreciated all the presenters.  Just adding another element to both the kind of attitudinal changes that people were talking about as well as the sort of, like, how do you approach these skills, one thing that we know from a lot of dissemination and implementation research is that the people around you, the network of people that you interact with, know, like, and trust really influence a lot about what you do.  And so, you know, sometimes it's not just information that people need.  It's actually interaction that they need.  So I just wanted to maybe add that additional element in here because sometimes we -- when we think about training, we think about a whole bunch of individuals rather than about an interconnected network.

LISA BRENNER:  Yes.  DeQuincy, so nice to see you.  Please.  Love to hear from you.

DEQUINCY LEZINE:  Glad to see you, too.  I think one of the things that I would jump in and add is just a reference that folks might want to take with them.  Even in their national strategy and the appendix on groups with increased risk, it indicates that a number of medical conditions are associated with medical risk, associated with an increased risk for suicide as well.  In one of the books that's highlighted there, conditions with increased risk list, like, 25 different medical conditions that are associated with increased suicide risk.

I know when I got diagnosed with tinnitus, just kind of ringing in the ears, when you go to the association website, it lists the suicide prevention lifeline as a resource because a lot of people get so distressed from it, or insomnia, or other things.  So when I see the list of all the things that providers are working with, all the things that primary care providers are paying attention to, a lot of those things, once they reach a certain point of causing distress among their clients, could lead to increased suicide risk.  So it should be incorporated into that whole training set as something to be aware of that their clients might need this.

LISA BRENNER:  Yeah, DeQuincy, I think that's super important and I could not agree more.  Anna?  Anna, there's also a question for you specifically about alcohol use, so I'm going to pop that in here and let you talk about -- we're going to talk about people -- there's a specific question about how do you manage in primary care settings -- let's say somebody comes in.  They've got a substance abuse problem and suicide.  Both seem really imminent.  How do you -- how do you manage both?  So sorry to add to your --

ANNA RATZLIFF:  It's all good.  That's what primary care is actually like.  So I did put some resources, and there's actually some great sort of protocols and for primary care specifically around addressing substance use disorder and alcohol use disorder, especially from SAMHSA.  So I really encourage people to check those out.  There's some nice protocols that are available.  I mean, some of them are a little older, but most of the content is still good, so I think that's a great place to look for that.

I guess it brings up, though, what I actually wanted to say, which is, and I think we haven't talked about it explicitly.  But I think a big reason why people don't take this on as still stigma, and I think we actually have to take that one head on.  I'm doing a big clinical trial trying to address collaborative care for co-occurring opioid use disorder and common mental health disorders, and we were trying to introduce screening, and we've had huge challenges with primary care providers even being willing to take that on.  And it's complicated, stigma, right?

So for example, medical assistants are worried that if they screen for this and realize that somebody's prescribed opioids, is actually at risk, they won't -- they might lose their prescription and they care about that person and are worried about them, right?  So, I mean, it's not just as simple as, like, we don't want to take care of people who have complex problems.  But I think any of these trainings really has to take that on directly and start to talk about that so that we can actually do that because it isn't as simple as just telling people you should do this.

(Laughter.)

ANNA RATZLIFF:  I mean, I just --

(Cross talking.)

ANNA RATZLIFF:  Yeah, you know what I mean?  I just want to say, like, I think people often do know this.  It is this, like, either a lack of a skill of doing something or maybe there's some attitudinal barriers to actually engaging and making a change, so, because I think that's what -- I love Virna's example of, like, asthma as another example because when you do talk to primary care providers individually about that kind of thing, like, how -- you know, what are other kinds of things that come in and takes up time, and how do you work on those, and can you use that same approach to suicide?  They start opening their mind to it, but I think, you know, the first thing, it's like, that's not me.  I need one thing that's not me, you know?  So I think that that's also the challenge.

So alcohol specifically, I think it's important.  You do need to address it.  It's a really important risk factor.  It commonly is presenting to primary care.  I encourage people to look at some of those resources around that.  I think especially people who present with alcohol use disorder really do need to be screened for suicide also.  So, I mean, I think it is a really, really important thing to look at some of those co-occurring.

And I couldn't agree more about the medical conditions.  I think that's actually one of the big ways that we hooked people or tried to engage people when we had to do this.  I mean, I was put in charge of giving -- creating this for the University of Washington, and I'll just say, like, it's a little anxiety provoking to be the person -- the face of the six-hour training you didn't want to take that you now are required to do for University of Washington, so I'm just --

(Laughter.)

LISA BRENNER:  Yeah.

ANNA RATZLIFF:  But so we spent a lot of time thinking about, like, how do we make this something that people will actually get a little hooked into.  So really using those good educational adult learning theories to get people engaged in the content is important.

LISA BRENNER:  Awesome.  Hey, Jay.  You've been so patient.

JAY CARRUTHERS:  Hi, everyone.  Great discussion.  I guess I just wanted to come back to kind of the observation that I think everyone appreciates that, you know, suicides are low-frequency phenomena, base-rate phenomenon.  And often clinicians don't know when they're actually -- their therapeutic stance or what their intervention is actually moving the needle in the right way.  You know, if you're thinking about training, you can have fantastic trainers like Virna and Anna in primary care, but in primary care, in particular, where, you know, it's not their core competency and they're not doing the volume.  You know, like, surgeons it's all about the number of cases that they do.  They develop that competency, and they -- obviously they get feedback a lot faster in terms of whether the -- and the post-operative course, you know, it's always a struggle.

And we're spending a lot of time trying to support health systems and, you know, getting beyond -- this is getting into my talk a little bit, but, you know, check the box.  You know, was a safety plan done, yes/no, versus actually was one done that was, well, you know, high fidelity, high quality.  So love to hear the group's kind of thoughts about that.

LISA BRENNER:  I think there's a paper there called -- titled, "Getting Beyond Checking the Box."  Mitch, you've been so patient.  Please, and feel free to respond to Jay and/or your own thoughts.  I know you've been waiting a while.

MITCH PRINSTEIN:  Oh, no problem.  Thanks so much.  It's a great conversation.  I just wanted to make the pitch for really focusing on youth for a couple different reasons.  One is it's the best way to change an entire cohort in the way we're talking about this and really, you know, have a sustainable change.  The second is, you know, the data are suggesting that we're now seeing much younger youth engaging in suicide attempts, particularly within the black and African-American communities.  So I think for every educator that is listening, I think for every person -- for every parent that is listening, I think for every clinician regardless, of your field, I think that we're talking about one out of every five young people at least of high school age is seriously considering suicide.

If that was anything else -- sexually transmitted infections, anything else -- this would be universal prevention -- fluoride in the water -- every health class would be talking about this.  Every pediatrician would be having this conversation, and every parent would be receiving instruction on how to have the suicide talk with their kids the same ways we have the sex talk and the drug talk.  And I think if we create a culture where it is normative for everyone to be talking about this, it is a very, very common phenomenon to experience ideation.  I know that the completed suicide and that the attempts are lower obviously, but serious suicide ideation is an outcome in its own right that we could all be talking about.

I think that when kids come freely into their doctors' offices to talk about this, you know, and when this is part of just the standard annual checkup in a way, and we're thinking about this in every mental healthcare setting -- it should be asked to every patient -- that's when I think we start to really move the needle and create system-wide change.  And that means partnering with paraprofessionals, and with clergy, and with community leaders in ways to get this out as well to disparate and diverse groups.

LISA BRENNER:  Well, I think -- I could not agree with you more, and people in the chat are agreeing also.  And I think the -- another added bonus, Mitch, and I've seen this with my own kids, is they have less of a stigma issue.  There's more parity.  They grow up to be adults and long after Dave Jobe's and I retiring, you all are talking about, you know, things besides stigma.  That's a goal for the future.

And I think we are at the time.  Thank you all so much, and thanks for participating.  Just a good conversation. 

STEPHEN O'CONNOR:  We are off to such a great start and we're on time.  So I'd like to introduce the moderator for our next session, Dr. David Jobes from Catholic University of America.  Dave?

DAVID JOBES:  Thank you, Stephen, and thank you so much to the folks at NIMH for organizing this incredible panel, these presentations, and for everybody that's involved, and, of course, people who are in attendance.

We're going to be talking about how to create a system of care that reflects patient preferences and better support for clinicians.  That isn't a mutually-exclusive kind of thing, so we're going to try to grapple with that.  And so we have a stellar lineup here with Dr. Kate Comtois from the University of Washington leading off, Dr. Kathryn Turner -- who's affiliated with Metro North Mental Health and Zero Suicide, Dr. Ursula Whiteside from the University of Washington, and Dr. Julie Goldstein-Grumet from the EDC and Zero Suicide Camp.  So I'm going to just kick it over to Kate, and then each successive speaker will then introduce the following speaker.  So Dr. Comtois.

KATE COMTOIS:  So what I want to do is talk a little bit about the issues with training and how we think about it as a clinician in the system as well as a patient in this system, and really kind of talk a little bit about kind of what are some of the tensions that we face in suicide care, and how they kind of shape and maybe limit what we can do with training.  Some of this already came up in the previous presentation.  And then I'll talk a little bit about how I see the training challenge, and some things we've tried, and some research opportunities.

So, you know, I really want to kind of highlight, as a clinician who works day to day with suicidal patients and is sort of the local suicide expert for the team at my clinic, who gets lots of questions, I think there's a core tension of this experience of feeling out of control because clients can't predict whether their -- clinicians can't predict whether their patient will die by suicide.  They can't predictably prevent it.  It's not like an antibiotic.  They don't have that level of control over the problem in front of them.  Simultaneously, there's a desperate need for control.

I was just consulting with a psychiatrist on Monday who has been traumatized by a patient suicide, has a very suicidal patient she's working with who just hasn't been able to move the needle and the patient made a suicide attempt.  And then, of course, there's, you know, the fears of being held liable and it is your, you know, basic living and how you support your family.  So I think that tension is just a big part of what it's like to be a clinician in the system and I think needs to be acknowledged as we think about training.

The other thing to think about is sort of the experience of the patient and the clinician inside this kind of world of healthcare.  And as my colleague Jeff Sung has pointed out, when you think about how our systems are structured, you know, I've done, you know, most of my -- spent most of my career and sort of how is this suicidal patient responded to as they move through the health service system.  There's this very strong risk management focus that we see, and it comes from the suicide loss perspective, right?  You know, you're both -- you know, who would sue, but even also just like are you doing enough for my sister, are you doing enough for my husband, are you doing enough for my child to prevent them ending their life?  And so I think therapists become very focused on justification of their actions and control over the suicidal person, and there's kind of a lost opportunity cost there with therapeutic alliance.

The flip side of the coin, and I kind of came up through this as I came up through DBT.  And my training and learning about suicidal patients is this focus on the experience of the person who is questioning whether life is worth living and is questioning and asking the question is -- should I stay for this?  I think it really drives clinicians toward the therapeutic alliance, toward recovery.  We've got to get this person out of this.  We've got to get this person a life worth living, and it leads to kind of dilemmas in the health service system because often that's a higher tolerance of risk than the system or some clinicians can handle, and it has much more of a focus on patient autonomy and less on control which, again, can bang up against the healthcare system.

Of course training is necessary, right?  If you don't know something, you can't do it, and I think we can overly rely on training if they only knew, they can do it.  But as has been highlighted in the "Crossing the Quality Chasm" and elsewhere, it's probably not the clinicians.  Their lack of training is probably not what's driving care, and I almost want to ask the opposite question.  If it was easy enough -- suicide care is so challenging.  There's so much more work involved.  You want to make sure you're consulting with others, you're getting second opinions, your -- you want to talk with family.  There's a lot of other uncompensated care, a lot of things that take more time.  As Lisa mentioned, you got to remind folks that, you know, other things do take time.  It's not just suicide.  But at the same time, all of that is uncompensated.

So, you know, you almost kind of wonder if all of that really was compensated at an adequate level and there were sort of informatics to make this easier, would the training problem kind of solve itself?  How much are we asking training to do to kind of overcome the implementation issues?  And as I'm saying that, like, suicide care is not like cardiac care, right?  So we have a very fixed budget inside of mental health.  It can't just expand with the number of patients who need higher levels of care, so everything needs to be smaller.  We have to have enough time.  There has to be a way.  It has to be scalable.  Even at the NIMH level, there's a real focus on this kind of practicality, which is -- totally makes total sense in the sense of we've got to be able to scale.  And at the same time, much of the evidence in healthcare for suicide prevention is in psychotherapy, and they're not always easy or inexpensive to train, and in a less money and less time environment where folks aren't paid that much and suicide care is hard, you see a high turnover.  And then health systems have to train people again, and they sort of give up on training.

And kind of come to the question of, you know, why train on something that isn't going to be available anyway?  So, you know, these -- that we're really facing this right now with the pandemic and the mental health crisis.  Access to care requirements mean you've got to have that first session, but if you're giving more and more and more first sessions, there's less and less and less follow up, right?  So care is coming down to that one session, and so I think that's where a lot of what we've seen already, right, what we need to be able to do, teach people to assess and manage, but that leaves kind of treatment behind, right?  So we're not actually necessarily treating the problem to make the suicide risk go away, and if we don't do that, why would we expect the risk to go away?  And then it's a huge ask for training to think if training is going to improve suicide outcomes when all we're going to train on is something very, very scaled down.

So those thoughts just out there.  I'd be curious what other people think in response, but let me talk a little bit about what I think we can do and should be doing in our -- in our training research and our training implementation efforts.

So I think the big thing is reach.  So reach is the first in the implementation model, RE-AIM, and it refers to one of two things:  either the percent of clinicians trained who are actually using the innovation or the percent of patients who need the innovation is actually receiving it.  And to me, this is the point of -- this is the core point of training, is that we get to this level of reach.  And most of the time, that's a real limitation in training research is we're not seeing reach as an outcome.

So if we just think about that in terms of a couple of training models.  So DBT has a very effective traditional training model.  It has been wildly more successful than a lot of other trainings.  People often say to me why are anxiety trainings or behavioral activation trainings not as widely available as DBT.  I mean, I do think that's in part because DBT is an effective treatment.  I think that training models for DBT have tended to use DBT as part of the training and that's been pretty effective.  And there's cost savings for a very chronic population that creates kind of a pull to bring training in.  But limited -- despite all of this enormous training, there is really limited reach, right?  You can't find a DBT program that easily for a friend or family member if you need to, and many programs are just too small to be useful.  So you've trained a whole team and they're seeing, like, five patients.  They're seeing 10 patients, and there's, you know, 100 patients who need to be seen.

The other possibility is sort of online training.  I won't say a lot about that.  People have talked about it before.  If we think about it in terms of Caring Contacts, which is an intervention I've been closely involved with -- here are some, you know, just training examples -- again, the training is going to have limited reach if the intervention is logistically too complex.  So most places that are doing Caring Contacts are doing postal mail, and it's sent by administrative staff.  So that's often happening in settings where there some way to afford an administrative person who figures out which letter is going out to which person at what time, right?  And then I've been dealing with this in a more complicated way because we do Caring Contacts by text message, right?  So we're sending out a message and we're more likely to get a response, either a positive response, which is most common, or occasionally a more distressed response. And so the complexities go up when recipients are going to respond more.

And we don't see Caring Contacts by text done by anybody.  We've had lots of consultations, lots of people interested, overwhelmed by the logistics.  So our focus has been trying to figure out how to sort of build an app for that kind of approach.  Can we use technology, as Kelly was saying, to kind of integrate it with the treatment itself so that it is -- it's easy to send and respond to Caring Contacts as it is to receive them as a patient?  And I think once we've got that, let's train that, right, because that's got what we need for it to work.

A couple other quick ideas.  One is to train by experiencing the treatment rather than training about it.  So we did this with our preventing addiction-related suicide intervention that was developed by Rick Reese.  He trained it by simply giving the intervention to the chemical dependency counselors saying essentially the exact same thing that they need to do.  And with no other effort, we were successfully able to get them to administer the treatment.  We showed outcomes in our step wedge trial for both the counselors and the patients taught by counselors who were just trained once by experiencing the treatment, and it also led to our successful adaptation using this model.  So I think that's something we might consider.

Another alternative is how can we get this technology for training kind of embedded in the job site.  So Doyan Darnell, who has a K Award at our -- here at University of Washington, has been focusing on making some really nice training tools in terms of chatbots, standardized patients, adherence feedback, accessible for busy acute care nurses so that they can learn safety planning skills right on the job.

So my concluding thoughts, I think that we need to do a heck of a lot more training research.  It's really limited.  When you look at the literature for rigorous trials, there's just not enough.  We need to study Reach a lot more explicitly.  What percentage of trainees are using the innovation to a meaningful degree?  What percentage of patients who need whatever got trained are actually getting it?  And thinking of Reach as the outcome for cost-effectiveness of training models.  I think we should, as I mentioned earlier, consider incentive programs for Reach.  Could we get better training options because the incentives are there to actually do the treatment rather than sort of convincing people that they should be interested.

I think we could really consider can we either stabilize and assure implementation or can we vary the amount of infrastructure or funding to somehow control for implementation in training trials, because a lot of training trials are either very unrealistic because they're in -- they're not in the kind of environments where they need to be done or they don't have enough infrastructure to do it, and so you can't tease the implementation versus training outcomes apart.

And then I think we should further explore the possibility of giving some of our interventions to the clinicians as the way to teach them.  In addition to PARS, there's been some work of using acceptance and commitment therapy this way.  I've done some exploration of teaching DBT skills to clinicians.  And so I think that might -- you know, what do we know how to do as treatment developers?  We know how to change behavior.  Can we turn that effort toward changing clinician behavior rather than telling people about it?  And I'll stop there and give it to Dr. Turner.

KATHRYN TURNER:  Great.  Thanks very much, and thanks very much for the opportunity to present today.  It's been great to hear from all of the speakers.  So I'm going to be talking about implementing a restorative just culture within health service, but a particular focus today on the training aspects of that.  So I'll talk a little bit about the issues and the drivers for change, a little bit about restorative just culture, our incident response framework, the training that wrapped around all of that, and the evaluation of it.

So why did we go down this route?  So we had -- this was back a number of years ago, probably about four or so years ago -- four or five years ago.  There were concerns about the incident review process and about learning improvement, concerns about a culture of blame, about the level of engagement of staff in the learning process following an incident such as a suicide, quality of the reviews following incidents, and quality of recommendations, and also the consistency of support for staff following incidents.  And it was in the context of thinking about and then implementing the Zero Suicide Framework, and people were particularly concerned that -- of the risk of the language around Zero Suicide and whether there was a risk of worsening a blame culture.

We know that it -- that losing a patient to suicide can have significant impacts on clinicians and -- which can be for some severe and long lasting.  It can have physical and psychological symptoms in addition to impact in clinical confidence and even considering about choice of career.  And so clearly it can be a negative, although when well supported may actually contribute to positive growth.  So we engaged -- went through a significant engagement process with staff and feedback from staff and also reviewed the literature.

So the restorative just culture moves away, and this is just a very brief just got a description of it.  It moves away from traditional just culture, which is the frequently sort of used just culture which is more backward looking -- it's retributive, it's focusing on individual actions, violations, and consequences -- to a forward-looking accountability.  It accepts that failures are due to complex systems but balances that with individual -- with a forward-looking individual accountability that contributes to a healing, learning, and improvement process, and importantly, supports the empowerment of victims.

So the -- so restorative just culture, as I said, it's a forward-looking accountability.  It focuses on hurts, needs, and obligations of all who have been affected, and it asks the questions who has been hurt, what do they need, and whose responsibility is it to meet those needs.  So all parties come together to address those needs, and it promotes the healing of trust relationships and people, and it understands that forgiveness can take time and compassion, and involves openness, honesty, and reparation.  And really important, it's very action-oriented, so it assigns roles and responsibilities for all who have a stake in in the event.  And it's a forward-looking accountability that moves away from blame, and it's -- as I said, it's focused on healing, learning, and improvement, and the empowerment first, and second victims, and that it's really important, that concept of engagement of all stakeholders.

So the components of that, so as part of the healing process, certainly from a service perspective, it was really important to ensure that we are, you know, very quick to engage with consumers if they're still alive if it's not a suicide, and families and caregivers, and provide that clinician disclosure, but also linking them in with the supports and postvention support.  It's also about an open disclosure process following a review of an incident and also about how we respond with evidence-based interventions, such as staff/peer responses.  It's also about the learning process, so looking -- the really important -- the importance of bringing everyone together as part of that learning process, so having the team involved in the incident as part of that learning process, having the family as part of that learning process, and the leadership of the organization.

It's about avoiding proximal linear approaches, and it's -- the restorative just culture sort of has within it -- what works very well within that is embedding safety to -- or resilient healthcare principles rather than more traditional safety one principles.  And it's about how we can -- we've got accountability for that forward-looking improvement as well, so ensuring that we've got high-quality strength -- quality and strength of recommendations and that we're accountable for monitoring, and evaluating, and of sharing lessons.

So what are the training requirements?  Well, there's a -- there's a need for everyone to understand restorative just culture, and we've found that that can be sometimes very challenging for people.  Some people understand it very quickly.  Other people do grapple with the concept.  It's about ensuring that we've got training around clinician disclosure for all staff, that they are skilled at engaging with caregivers for the loss of a consumer or other incident, that they're aware of how to link families into postvention support.  It's about having some staff trained and skilled in open disclosure framework and developing training based on psychological first aid for the peer responses, so the responses to staff.  And also, you know, sort of increasing evidence that it's important to increase knowledge in a way, and that's processes following critical incidents.  There are possible reactions of staff following an incident and strategies that may assist, so getting that as part of the training requirements.

It's about also training in the incident review process in certain groups of facilitators, the training of the reviews, but also staff in general understanding those processes.  So understanding about human factors, about safety, too, and resilient healthcare, about concepts of work as imagined and work is done, and the efficiency thoroughness tradeoffs, for example.  And then it's about understanding how we can get high-quality and strength of recommendations and effective sharing of lessons and tracking with implementation and evaluation.

So how did we do the training?  So we did some presentations to the board and the executive and to the statewide leadership because getting people at higher levels in leadership positions to understand what restorative just culture is and what we're trying to achieve was very important.  So that was part of our sort of rollout of education.  But also to getting out to all staff and training them, you know, in whatever time we could get, you know, whether it's 30 minutes in a, you know, a staff meeting or up to 60 minutes where we try to convey an understanding of restorative just culture.  Particularly some training around clinician disclosure and a particular approach to that broadly about the incident review process and the staff support process.

There was a requirement to get some staff trained in formal open disclosure, and we did that through an external firm that actually came in to do that training.  And then training and incident reviews, so our facilitators and facilitators from across the service were trained -- had a day's training in incident reviews.  And then volunteers volunteered for the staff peer response program, and they were trained in a range of methods, including cycle of -- you know, the core of that is psychological first aid.  And this is in addition to the standard suicide prevention training that we provide, which is some -- a few hours online in a day, a face-to-face training for all staff.

So just briefly, evaluation of that.  So we -- we've been able to demonstrate that we've had an improvement in reports of just culture, second victim.  Fewer people fear disciplinary action and blame.  There's more trust involvement with -- our Peer Response Program is associated with increased perception of organizational support, and those that were involved in the incident review process actually had significantly improved measures of perception of just culture and support.  Less second victim distress, intention to leave, and reported absenteeism.

And we were also able to demonstrate that we are in line with safety, too, reviewing a larger range of incidents, not just the most severe incidents, a greater variety of review methodologies.  And we moved away from the use of root cause analysis which were -- we didn't feel were very satisfactory.  And, you know, there's, you know, there's growing evidence for, you know, better approaches than root cause analyses.

We did a -- an audit of recommendations as well and a review about process, and we were able to demonstrate that we were achieving what we were hoping to achieve in terms of better engagement of stakeholders, learning from what's going well, improve quality and strength of recommendations.  And, you know, feedback from staff was very positive.  Restorative just culture isn't an easy process.  It can be actually a tough process because it is about staff being engaged in that learning process and feeling safe to be able to talk about how things, you know, can be improved or how we may have been able to do things differently, and that -- you know, that's not an easy discussion to have.  But as you can see from the comments, you know, people have found it a positive process.

So embedding restorative just culture, and Safety II concepts, and improved approaches to support staff is associated with improvements in culture and second victim experiences, and as well as that improvement in our strength and quality of the recommendations that come out of our reviews.  We focus on a learn anything rather than just looking at what's just happened before the suicide because we think, you know, that very linear proximal focus on incident reviews can actually, you know, potentially cause more harm because it leads to more restrictive practices, and we retreat from recovery-based principles.  And we often learn a lot more important lessons when we look at the broader picture and learn anything.  And it also provides opportunities to proactively learn rather than just learn once harm has happened.

But there are many challenges with this.  The cultural change is tenuous and aspects can shift very quickly when there's changes in leadership at higher levels, particularly if there's not a good understanding of what we're trying to achieve and the benefits of it.  So, you know, constantly thinking of how can we communicate this better and get people to understand, and I think, you know, doing this evaluation.  And so I guess one part of that step to be able to demonstrate that it has got better outcomes.

Do we -- you know, one of the questions we have is do we need a corrective experience to shift our views?  People have often been quite traumatized by incident reviews in the past and their losses and incidents in the past, so culture will be slow to change.  And so then the question also is can we use perhaps simulation exercises to give this kind of a corrective experience.  And, you know, what would be the most important sort of experiences to simulate?

How can we measure restorative just culture changes?  There's some good tools for just culture but perhaps not so much for restorative just culture.  And we also need to look at how can we further embed resilient healthcare principles.  And one of the biggest challenges, of course, is how we do this -- all of this in existing resources because we're really, you know, funded for Safety I, which is a fairly narrow focus on safety, and if we you want to do a more proactive and broader approach to safety, it's very challenging within existing resources.

So thanks very much for the opportunity, and I'll now hand over to Ursula Whiteside.

URSULA WHITESIDE:  In the chat there, I just share with you a one-minute survey that will be using as part of the time that we're together, so when you get a chance open that up and we'll do that together.  This is Baby-Cat.  This is who brought this morning an offering, so I think that's sort of in support of what we're doing today.  Once in a while she brings in things from the yard, and today was one of those days.  She announces it with loud meowing so you kind of know what's happening.

I'll be talking about lived expertise as guidance for the work that we do in supporting patient preferences and clinicians as well.  The sources of information that I'm using today have to do with those who I consider advisors to the work that I do, lived experience advisors, patient-reported outcomes that were used to identify people that we would do qualitative interviews with.  And by "patient-reported outcomes," I mean, those questionnaires that you get in health settings, like primary care or behavioral health settings.  And then training surveys, like the survey that you're taking now.  The survey that you're taking now is one that I give at almost every training that I do, and I train a lot of primary care and mental health clinicians.

My mentor and the mentor of other people who are here today -- David Jobes and Kelly Koerner -- is a woman named Marsha Linehan who developed dialectical behavior therapy and was herself someone with lived experience and who came out in around 2011, which has helped transform the field of suicide prevention.

In around 2012, I created a team of people with lived experience to help create resources that would be freely accessible to people who are struggling with suicidal thoughts, so struggling with those day-to-day thoughts all the way up to struggling with attempts and repeated hospitalizations and such.  You can learn more about that at NowMattersNow.org.  And you can see here a couple of our youth advisors.  So we have a new group coming in who help guide our work, and they provide their experiences of how they got through hell and which coping skills from dialectical behavior therapy that they used, and suggestions and advice.

In the chat, I'm going to share with you a -- really overview of language recommendations from those with lived experience.  So this isn't just my team.  This is in general.  I just want you to have this resource, but essentially what we're doing is moving our language away from assuming we know what people's intentions are to describing what it is that's true.  What is the behavior that we see, and then trying to learn more.  With my team of people with lived experience from Now Matters Now and my fabulous friends in the suicide prevention community, and also my own lived experience, we've come to these conclusions that we try to make sure we include every time we have a training.  We ask what's your personal experience, and then we say at the end of the day, these are the things that people really say are most helpful.  The clinical tools are great. My advisers say, yes, go ahead, train all day, that's fantastic, but please make sure you include these three points that when I'm in a high-intensity, hot-hot moment of suicidal emergency, these are the three most important things that someone can do in that moment.  They cannot panic, they can be present, and they can offer hope, and that last one is one you want to be careful with because it's not just -- or I would say it's not "things will get better."  That's not the communication you have in that moment.  It's let me help you get through the next five minutes.  That's the hope.

So the interesting thing is that this is for the most hot emotional emergencies when people are on fire emotionally, but they also fit for the day-to-day suicidal thoughts, the less intense.  And this is especially true for that healthcare provider who uncovers somebody as having suicidal thoughts on their patient health questionnaire or their PHQ-9.  What do you do in that case?  These also apply in that case.  At the end of the day this is what's most important.

Coming back to that patient health questionnaire, the PHQ, lots of folks use this -- primary care, behavioral health -- and there's an item -- last question, thoughts that you'd be better off dead or of harming yourself in some way and that is asking over the last two weeks.  What we know is that this is actually not a terrible indicator of who's at risk for suicide.  It's as good as some other health indicating -- indication scores, like cholesterol is for cardiac event.  Not perfect.  But when we asked people who completed these questionnaires, in particular, people who had experienced suicidal thoughts, we got this qualitative feedback.  They said, yes, asking this standardized question is appropriate and valuable.

They also indicated that their major concern was a hope for help, but the balance with the fear of consequences, "consequences" meaning, like, I don't know what the provider's going to do with this information, and you just gave me a questionnaire and it didn't say what you were going to do with this information.  And then it does matter how the provider interacts.  Listening and caring make a huge difference.

So if you've completed that questionnaire that I provided for you, that actually incorporates some of this information that we gathered from patients, which is you see this orientation of saying that this is -- experience is not uncommon, and also what this data will be used for, and that's really important.  People are generally afraid that they would be involuntarily hospitalized and that does happen still.

Well, what about for people who say they're not having suicidal thoughts, but then go on to attempt suicide?  So funding from the American Foundation for Suicide Prevention allowed us to interview some people who had this experience.  It turned out they actually were not lying.  I think sometimes that is the first assumption, but not for everyone.  The first assumption when you see the situation where I say, no, I'm not having suicidal thoughts, and then in the weeks and months after I make an attempt, what is it that's going on?  Well, overall with the 26 people that we interviewed, people were generally having worsening symptoms of depression and life circumstances were getting more difficult, but they weren't necessarily having suicidal thoughts or they hadn't had them in the last two weeks.

So this was in the months and days leading up to their attempt, and we ended up calling these unplanned attempts because I think, you know, while you want to keep in mind this is qualitative data, all but one person had not had any plans to harm themselves until the day of their attempt.  So this is this interesting phenomena where many people do not have plans to attempt suicide or to die by suicide, and that this information really gives us plans for next steps.  How do we prepare people?  How do we prepare people in elementary school for what to do when they become on fire emotionally?  How do we prepare them to know what steps to take to bring that down and to know that they'll get through it?  And this has really helped inform Now Matters Now work to come up with a stop, drop, and roll for overwhelming crises.  You're welcome to check that out on our website.

Okay.  So you filled out that brief survey, and on that survey we asked about experiences with suicidal thoughts.  So this was how many of you in your lifetime -- and not you; I mean, of the 2,000 people I've asked in previous trainings -- have had suicidal thoughts.  And of those working in primary care, it's about 45 percent.  Of those working in specialty behavioral health, that's 55 percent.  I use this as a training tool because what does it say?  It says, yes, you are working with people who are suicidal and you are sitting next to people who are suicidal, so when people tell me, like, I don't really see those clients, I just want to like knock them on the front of their forehead a little bit.  What you might not be seeing is people who end up dying by suicide, and when we look only at that outcome, we really miss this whole other area.

Dr. Koerner was talking -- Kate Comtois.  Kate also worked obviously with Marsha Linehan.  Kate was talking about how, you know, your -- I lost my train of thought where I was talking about that.  But, you know, basically, like, with these experiences, they're not uncommon.  It happens a lot and that needs to change in the way that we think about this, that this is the spectrum and that building a life worth living -- that's what it was -- is an important focus and it's also tied to important healthcare outcomes like depths of despair and loneliness that we know influence medical outcomes and early and premature death.

So another resource that I want you to just make sure you have is the advice from people who have -- you know, who've been there.  This is my advisory team.  Essentially, this should be some of the guidance informing our next steps for treatment development is, like, what do people wish they would have known?  What do they think would be helpful?  And so here are some takeaways and ideas from that.

So in doing this work with people with lived experience, people want to know what you're going to do with their information.  They want to know that you can hang, meaning you're not going to freak out.  And also it's important that we understand that this is part of the human condition, and that knowing that can help reduce our own anxiety and make sure we include this as part of the treatment and work that we do on a day-to-day basis.

So with that, I will turn it over to my dear colleague, Dr. Julie Goldstein.

JULIE GOLDSTEIN:  So thank you so much for having me here.  It's really a pleasure to be here.  I'm Julie Goldstein-Grumet.  I'm the director of the Zero Suicide Institute at the Education Development Center, and I'm going to be talking about how prepared the workforce is to identify and care for people at risk for suicide, something pretty much all the presenters have already been speaking about.

We know that -- you know, what we've been hearing is how people lack training in the workforce, and so I'm going to talk a little bit about what -- how is it that people's own perceptions of their knowledge, of their comfort, or their skill in providing care, where that really stands, but with -- in the hopes that maybe it can give us an opportunity to think about how might that be impacting care and outcomes.  What can we learn from that?  Where might we be able to improve in target areas?

With Dr. Whiteside, she mentioned these three things, right, that people at risk I wonder if I shouldn't -- you know, I shouldn't be ashamed, or there's something fundamentally wrong with me, or maybe there's another path I can take.  And actually I was thinking I wonder if that applies to the workforce, too.  Are these questions that people who are delivering care ask themselves or experiences they might have, and how might that frame where we could do better.

So to set the stage, I'm going to briefly describe the Zero Suicide Framework since training is a really integral part of this model.  Zero Suicide -- several people have mentioned it -- it's an aspirational goal.  We want to reduce suicide for people who come to care, and it's a set of evidence-based practices.  The wheel represents those practices.  It's things you're familiar with:  screening, assessment, safety planning, lethal means counseling, treatment, caring contacts -- but it's more than just doing those practices.  It's also creating a culture that no longer finds suicide as acceptable.

And many of you say -- you know, may feel like, no, we never felt that way, but I think that's not true.  I think that for years healthcare systems often thought if we treat people who have mental health illness or, you know, other behavioral health disorders, chances are we will lose somebody to suicide.  It was kind of expected.  And I think this really has to stop because you can't try to do better if your expectation is that you may or may not really achieve better.  We have to strive for zero.  So you have to set aggressive but achievable goals that eliminate suicide attempts and deaths, and then modify how care is delivered in the system and the infrastructure, and build the supports accordingly, and that includes data collection and continuous quality improvement.  Are you doing what you say you're doing?  Is there fidelity in these practices?

So Zero Suicide is really a patient safety initiative.  It should look no different than trying to drive down wrong-site surgery, wrong medications delivered, patient falls, things like that.  Suicide care should be just a standardized and supported by a prepared workforce with clinical workflows that support the work they're doing just like we do for medical conditions, and Dr. Little already talked about that.  If you come in for asthma care, you should know next -- any provider in that system what happens next.  If you break your ankle, there's a routine set of care practices.  If you come in with heart disease, there are certain medications you're put on and certain tests that are run to track your health.  Just like when we go into our -- get our physical, we have our blood pressure taken, our height and our weight tracked.  Physicians and providers in those practices know what to do.  If our blood pressure is significantly high but I came in for headaches, they would manage my blood pressure.  If I came in with a rash they would say let's talk about your blood pressure first, and they would pivot and they would focus on the issue at hand.  So we have to kind of remove the burdens and the idea that suicide is an add-on, but rather it's just part of holistic care, and doing quality suicide care is necessary if we want to save lives.

So the element of TRAIN in the Zero Suicide Framework is really integral to every aspect of how care is delivered.  Yes, I've spoken about screening, and assessment, and safety planning, and lethal means, and all of those are really critical pieces of what I'm talking about when I say we have the Zero Suicide Framework, but it's more than just the what.  It's also the how and the why.  I think that's really the difference.  A lot of systems might be saying but we did this care -- you know, we did screening tools and safety plans three years ago.  How is a Zero Suicide Framework going to change things?  And it's about how these pieces are tied together in a bundle, that it's more than being able to say, oh, we completed a screen, so if you audit our chart you can see that we did a screen.

It's the understanding that doing those pieces of evidence-based care leads to better outcomes, and here's why and here's how.  And that's more than just doing the pieces of care that we're familiar with.  It's some of the things that you almost think are like the soft areas.  Like, how do you engage patients who don't want to come back to care or have real obstacles in their lives to routine care?  Telehealth really opened up a lot of opportunities, but part of it is really increasing transparency about what does that care entail.  Why should you come back?  Dr. Whiteside talked about hope.  I think that's a really critical piece of training.  Why should patients trust us to come back?  It's important that clinicians are trained both in the care practices we've been talking about, patient engagement strategies, but family involvement, and the role of lived experience and how to really weave that throughout the care that we provide.

So a lot of people have said, you know, people lack training, and here's the data.  They do lack training.  They're not getting it in grad school even if they're licensed, and this lack of training really influences care practices.  So in a recent survey of providers, 22 percent report they are likely or very likely to use no suicide contracts.  Hopefully, that's making you gasp because we know that no suicide contracts are not effective.  They can be very damaging, and there is absolutely no evidence that they were, so that's really alarming.  We also know that asking about lethal means was reported being used only by about a third of clinicians, and removing access to lethal means is one of the single best strategies to reduce suicide, but it's only being used a third of the time.  So there are gaps in training and we need to -- we need to focus on that.

We created -- this is free, publicly available on the Zero Suicide Toolkit, zerosuicide.com, a workforce survey where those who work in healthcare systems, and this includes every kind of healthcare system, not just behavioral health, their self-perception of how -- their knowledge, their skill, their comfort.  This is also perception, but it's an opportunity for the system to kind of get a pulse on how do people in my system feel with regard to their preparedness.  It's 65 items in this -- these six domains.  It takes less than 10 minutes to complete, and each organization gets its own unique link, and it's anonymous.  And then the organization gets outcomes across all of their providers what their results are.

So I'm going to talk just a little bit about what we found.  Sixty-eight thousand respondents.  So it's a pretty robust group of individuals who have taken this survey over the last several years.  Four hundred and seven health and behavioral health organizations.  About 60 percent were from outpatient care and the rest were a mix of crisis, inpatient, hospital systems, primary care, with the most common roles being nursing and behavioral health clinician.  Eighty-three percent directly interact with individuals at risk for suicide as part of their role.  So recognizing risk, really kind of asking, you know, how would you know if somebody is at risk for suicide, and this is a Likert Scale.  That's how respondents are responding, strongly disagree to strongly agree.  All self-report as I said.  Fewer than one-third of the workforce strongly agree that they're knowledgeable about warning signs, know their organizations' procedures for what to do when someone is at risk, or confident in their ability to respond.  And you can see that most respondents fall in the "agree" category.

Okay.  What about screening?  We've talked a lot about the importance of use -- of screening for suicide risk, and we're talking in this those who do the screening, so mostly clinicians.  Fewer than half strongly agree that they have the knowledge and the skills to screen people for risk, and you can see only 38 percent report using a standard tool instrument or rubric for screening.  And generally speaking, the research definitely says that standardized screening tools -- the ASQ, the PHQ, the Columbia -- are far more reliable at detecting suicide risk than is anything a clinician might develop.  So I think the fact that that many more individuals say I know how to do this and I'm confident and I'm comfortable, makes me wonder what tools they're using since they're not using standardized tools.

With regard to treatment, so here we're looking at those having had the training to deliver evidence-based treatment.  Less than 50 percent report they've had training and suicide-specific treatments, things like collaborative assessment and management of suicide, cognitive behavior therapy specifically for suicide prevention, dialectical behavior therapy.  Eighty-three percent report feeling comfortable and feel they have the ability to provide treatment.  But again, it really makes me wonder what are they providing treatment in if fewer than half have actually had suicide-specific treatment?

So I don't want to dismiss the incredible work that clinicians and providers are doing to take care of people at risk for suicide.  It is very demanding hard work and done by an incredibly dedicated workforce, but I do think this is a disconnect where people are not getting the training they need to provide the treatments that we know work.  So I think these results are a little bit of good news/bad news.  It's great that providers generally agree or strongly agree that they're comfortable and confident, but only fewer than half report receiving training.

You know, one of the things I also think about -- here's another point.  Only about a third strongly agree that they're confident or comfortable providing treatment to individuals with suicide risk.  And to me, this is one of the most alarming statistics because if somebody that we cared about had to have open heart surgery, or cancer treatment, or some invasive care for a really acute problem, and the physician said, you know, they were going to try their best but they didn't necessarily have the training involved, we wouldn't go to them.  And, in fact, for medical issues we often go for second and third opinions to make sure that we're getting the best care possible.  But it's hard for most people to know the questions to ask their providers to make sure that they're getting the best treatment possible.  And, you know, it makes me wonder why do we accept this in suicide care.  We definitely -- we wouldn't accept it.  If possible we wouldn't accept it for medical treatment, so why do we accept it for suicide care?

People had a chance to report on what they think their training area needs and gaps are, so these probably aren't surprising.  Most want suicide prevention and awareness, more on identifying warning sides, how to talk with patients about suicide, and also the 40 percent want to better understand their staff roles responsibilities.  And I think this really gets to the point that it isn't just about individual training, but it's also about what is the organizational infrastructure that supports the care practices that we want -- we want used.

A few other findings, some probably not surprising, but some a little bit more so -- so psychiatry at higher capacity.  In all those six domains, psychiatry at higher capacity.  Inpatient and crisis providers also had higher capacity compared to outpatient providers, probably what we'd expect.  People are getting more -- they're probably seeing more people at risk for suicide more often.  The staff who identified as physical health providers, so this would be primary care, GPs, other types of specialty medical providers as well as administrators or support staff had lower capacity in almost every measure by comparison to behavioral health.  And on its head, that makes sense except when you consider that 64 percent of people at risk for suicide are seen in primary care in the year before their death.  Other studies have said the general practitioners just have less comfort and skill in conducting comprehensive suicide risk assessments and, in fact, often don't use comprehensive suicide risk assessments even in patients who have screened positive for suicide.  So I think more training for physical health providers is really critical, and several of already talked about this in today's -- during today's event.

Some of the implications, though, for training and systems, there's a positive skew to all of the items that I've talked about and many that I didn't have a chance to share with you today.  Fewer than half report strongly agree, though, across skill, and comfort, and training, and knowledge.  And really that represents the difference between good care and great care, something that if it was ourselves or our loved one, we would definitely strive for.  We want great care, and I think it's incumbent on us to give families and people at risk for suicide the vocabulary to ask providers what kind of care they can provide, what they've been trained in, provide them with questions that they can ask because, again, most people don't know what to ask, so they don't know what kind of care they're going to get.  And then they don't -- and there are a lot of incredibly well-intentioned providers, but you should be able to provide training that -- or treatment that directly targets those thoughts of suicide, and we need families and their loved ones to have the right questions to ask.

We also know that the greater the fidelity to using comprehensive suicide care practices lead to fewer suicide behaviors among clients, so these care practices have to be used as a bundle.  So we have to link individual training with the organizational changes that others have already spoken about.  This includes measurement-based care.  I think if you're a clinician in a system, you want to know not just how well you're doing, but how will your system is doing.  Is it changing outcomes for those that you work with, and so the system needs to be transparent about outcomes.

And I think Dr. Turner was talking about that.  That's part of what a just culture is, and a just culture would include here, you know, what are the dynamics.  I think this is an area for continued research.  What are the dynamics, and the procedures, and the infrastructure that must be in place to really make the workforce cared for?  How do we improve clinician sense of comfort and confidence?  This is kind of a snapshot of how they feel, which is most fall in the "agree" category.  We need to figure out what we can do to bump people so that they feel that they can strongly agree.  What is it that's missing?  What is the training?  What is the support from the system that would help move them from "good" to "great."  And as we think about a just culture, it's not just requiring and offering training for the system, and it's more than just offering system-wide change, but in a year following COVID and burnout, we e also have to think about support for the team doing this hard work.  How are we supporting the healthcare workforce?  How are we making them feel that they can access care if they need it, and what can we do to promote their own sense of agency and coming forward if they need help?

This is the toolkit I've mentioned, zerosuicide.com.  It's free and publicly available, and most of the resources I've talked about are there.  So thank you, and I'll turn it over to Dr. Jobes.

DAVID JOBES:  Thank you, Julie.  I would invite the panelists to turn on their cameras for our Q&A.  And also other panelists of the presentation are welcome to turn on the cameras and ask questions along the way, and we've -- we certainly invite the audience to send along some questions in the chat and help our conversation get rolling.

I guess one of the things I wanted to start with just in the role of moderator is that, Kate, you talked about, you know, the control issues of I have no control, I need control, on the provider side.  And then we've also had this discussion about lived experience and how much people with lived experience of suicide have not liked being controlled or had that experience.  And so we have sort of this juxtaposition that we're, you know, really encountering in the field.

And the one thing I would sort of observe that hasn't come up in our discussions around control issues is the issue of liability.  How many clinicians and systems of care are just abject -- at least in United States, and Katherine, I remember we talked in Australia about, you know, this coming your way one day -- but how many professionals and systems of care are just terrified of liability for wrongful death tort litigation.  So does anyone want to sort of weigh in on that because these seem to be sort of moving forces that really then create this clash.

KATE COMTOIS:  I do think that we're getting a very -- I mean, not only the sort of perspective of those of us who kind of came up through, like, lived experience-oriented treatments like DBT, but also when we do surveys, and I've been doing some online research recently talking about what would -- what would be helpful if you were suicidal that might come by on your Google feed that might, you know -- and it's the lights and sirens that everybody doesn't want.  That they just don't -- they don't want that response, as Ursula was highlighting.

And at the same time, the clinicians can't -- like, when I was recruiting participants, I've had great difficulty over the years recruiting participants for trials from the emergency room, not because nobody's asking the participant, but because the clinicians have the inpatient to refer to.  And if they have an acutely suicidal person, they refer to the inpatient unit and that feels more secure than sending them to an outpatient, or caring contacts, or some other kind of outcome.  But once they're in the inpatient unit, I can get referrals really easily because they have nowhere to go, right?  They can't sort of hand that responsibility off to somebody else, and they're thrilled that were there and happy to grab the patient.

So this question of who's got clinical responsibility is really critical, and the reason that I put the two control issues of sort of being traumatized by a suicide and liability together is that I think almost all the same dynamics play out, right?  So I think that even if we solved the liability issues, people are still going to have the fear of losing someone to suicide and having that kind of tragedy.  And honestly, I mean, I just felt -- I didn't bring it up because I feel like it's outside of kind of perhaps the scope of what we can do within the research field.  But it's absolutely the case that we have set -- you know, things are set up in a way where defensive medicine is just rampant.

DAVID JOBES:  Yeah.  Ursula, do you want to weigh in?

URSULA WHITESIDE:  I was going to say, and others might want to speak on this, too, but, you know, your  -- with these new national recommendations for suicide care, the Joint Commission supports them.  You know, they're -- the Action Alliance, their Zero Suicide supported.  You know, we should be -- clinicians should be more worried about not asking than about asking and following these plans, and the -- and the plans do not indicate that people be hospitalized unless they're at a certain level.  And so if you're documenting and I'm following the standard of care, you're in a very good spot compared to not asking someone at all who dies by suicide, I would say.

DAVID JOBES:  Right.  One other question I had to direct to Julie and harkening back to what Mitch talked about in the first panel is the incredible confidence that clinicians have even if they don't know what they're doing is effective, and the idea of training younger clinicians who are less interested in being right and just don't want to get in trouble and, of course, want to do the right thing as well.  But I -- in my experience, it's been much easier to train younger clinicians than clinicians my age because the clinicians my age are very confident and very assured that what they do, it must be effective.  So can you just expand on that, those -- that confidence factor and what do we do about that?

JULIE GOLDSTEIN-GRUMET:  You know, I think it's a few things.  I mean, I think you've said what the -- what the challenge is, but, to me, a lot of that is about what is the system messaging about that.  So how is the system telling clinicians to identify barriers, come forward about things that they don't know?  Where's the transparency in that?  It's okay that this is hard work and that new research is evolving, and it may -- you may not know all of that research because you went to grad school 30 years ago.

So to me, it's really incumbent that the system messages on that repeatedly as their staff turnover and does that with data.  Here's where we've gone.  Here's where we've come.  Here's where we're going.  Here's where we've been.  This is what our data is telling us, our strengths and our weaknesses, and how do you all think that we can get better in that, right, not just, like, we're going to feed it to you.

And ideally, you know, I think you get -- that includes making the younger clinicians who might have more comfort to some extent taking on these kinds of evidence-based trainings, find their voice within the organization.  So it's not that they're disproving the more senior clinicians who have tons to offer, but how do you make that a little bit more of a level playing field so that both/and are providing a sense of contribution to the system.

DAVID JOBES:  A question I want to combine with a question of Dr. Turner from James Bywater:  "We are a Zero Suicide grant recipient, and by far our biggest hurdle has been getting organizational approval and buy in.  What can you recommend for this?"  And, Katherine, I'm just thinking about how you pretty much single-handedly, as I recall, got Gold Coast Zero Suicide up and running, and I'm sure you had a lot of selling to do to convince people about that.  You can give words of wisdom.

KATHRYN TURNER:  I think you can get lots and lots and lots of conversations, face-to-face conversations.  So yes, and I've recently moved from Gold Coast after being there for many years, and I'm in a large service in Brisbane now, so starting all of those conversations again.  So I think it's about engaging, you know, at the highest levels at the board, executive, engaging the leadership group, getting them all on one side and understanding the nuances of Zero Suicide and what we're trying to achieve.  And then it's about going and having conversations with clinicians and allowing them to express their concerns.  I think that's really important first, and then -- and then being able to address those concerns.

And I think now what we didn't have when we started at Gold Coast I think was a lot of -- we had a lot of hope for Zero Suicide, but not a lot of evidence.  But I think now we've got some compelling evidence, we've been able to, you know, show some significant, you know, improvements at Gold Coast that we've been able to publish, and then there's other growing evidence internationally, so I think that certainly does help.

But so yeah, I think, you know, lots and lots of conversations with many stakeholders to understand all of that, you know, and then all of the training that we need to do, et cetera, and the development.  I think engagement, so not taking something and then just saying, well, this is the model, but actually looking at the local context and getting clinicians engaged in tweaking that and working out what's -- so that they've got some ownership and feel that it is right for their context.  I think that's a really important part as well.

DAVID JOBES:  Yeah, a certain amount of persistence to be sure.  Silvana asks about -- I wonder -- so much of our focus today and in our health care systems is on acute risk.  And for a lot of people, if they've given them a Columbia and a safety plan they're good to go.  That in their mind is sort of like, you know, a job well done.  So what are your thoughts about more chronically suicidal -- people who are more chronically suicidal and not just being focused on acute risk only beyond safety planning?

URSULA WHITESIDE:  Yeah, Kate, I think you'd be good to talk about this, too.  We both use this term "building a life worth living," and part of the reason people remain suicidal over time is because their life hasn't changed and they remain in the circumstances.  And we -- you know, many people have more limitations than others around their circumstances, things that typical mental health and primary care don't come close to scratching, right?  Social economic stuff, you know, being -- having a trauma history, having -- you know -- yeah, being discriminated against.  So I think that the focus there on how do we -- do we help people build a life worth living in whatever environment they're in has probably got to be key.  Dr. Comtois, what would you say?

KATE COMTOIS:  Well, I would first highlight that it's not clear to me that in -- I think that there's -- there is something potentially curative in telling the story of your suicidality and doing the -- some sort of clinical narrative, some sort of story about how you came to be suicidal.  I don't think that shows up on the safety plan unless you, like, do the whole safety planning intervention or you do something from the Aeschi model, like a clinical narrative or like the SSF or a DBT.  We do a chain analysis.

There's equivalence of this everywhere, but I think the -- I think we often think we've treated somebody when we've done the risk assessment and the safety plan, and I think without the clinical narrative in it, which I think might actually -- we haven't done really good, precise research on this.  But I have a suspicion it has a curative effect in and of itself, and I've proposed some research on this.  I think it's -- I think that's an area that should be studied.  But I think -- other than that, it is not clear to me that doing the risk screening, the risk assessment, and the safety plan that we're actually treating the problem.

I think we are managing the problem.  We're assessing the problem and we're managing the problem, and I think that distinction -- management and treatment -- is critically important because it gets us to think we treated this.  We didn't treat this, right, in my opinion by doing this.  We need things that actually treat it to follow this initial management plan.  And that doesn't happen in the chronic case or even in the acute care case where we've done the safety plan.  It's like helping people get through rather than helping people get better.  Like, we would definitely not want to have an equivalent in diabetes care, right, where -- okay.  So we've assessed.  We've done the hemoglobin A1c.  We've figured it out.  We know you have diabetes.  We have given you some tools for crisis management of your diabetes.  So, you know, if your sugars go really high, you know, go get on the treadmill.  If your sugars get really low, drink a bunch of juice.  And, you know, good luck to you, right?  Like, that is just not what we do, right?  We actually get insulin on board.  We actually figure out how to treat the problem.

And that's something that's just been freaking me out about how care is happening for suicidal folks is that element, which I think is incredibly important just for all these systemic reasons.  We got to make it smaller, we've got to make it simpler, we have access to care, you only get one session, all those things that I've been highlighting.  I just think we ought to be frank that we're not actually doing it let alone training it.

DAVID JOBES:  Dr. Alonzo, and I totally agree with your points, Kate, you know, about the difference between management and treatment and what we think of as drivers.  From our friend Kelly Koerner, she's asking, "If you had a magic wand but couldn't change low-moderate system to high-resource systems, what training and research would you do?"

KATE COMTOIS:  I would definitely try giving the interventions to the people who are helping the suicidal individual, and I wouldn't limit it to clinicians.  I don't know that I think clinicians are necessarily all that brilliant at this.  If you're going to -- there's some treatments that I think take a certain sophistication.  And, you know, in DBT we often get handed folks who had no graduate clinical training, and they have a bachelors, and we successfully train them to do a lot of helpful things.  I think the same could be true for a lot of peers.  I know a lot of other areas of mental health resource low -- in low-resource countries we're very effectively using paraprofessionals for treatment of anxiety and depression.

So I think the key might be if we could teach the competence in dialectical behavior therapy skills in whatever intervention we're doing, if we could sort of provide that treatment, would it actually result in, okay, I now have this competence, so when a patient comes in front of me, I kind of just know what to say.

DAVID JOBES:  I'd note that Dr. Stanley chatted that "Both safety planning intervention and crisis response planning have suicidal narratives baked into the discussion of those interventions," so that narrative component certainly rings true to me.  Dr. Alonzo, you had a question.

DANA ALONZO:  Yeah.  Actually I was just going to comment on or build off of what Kate was saying.  I think that, you know, when we think about -- at least from through the lens of low-resource settings taking evidence-based practices, there are a couple of key challenges related to that.  A lot of them are, you know, when we think of, like, taking an evidence-based practice from a high-resource setting, they have a predetermined kind of training contact -- content, so that doesn't account for the cultural values and experiences of those in low-resource settings.

They often also have a sort of predetermined target of prevention, like means restriction.  Well, in Peru, for example, like, one of the most common methods of suicide is to ingest poisonous naturally-growing plants in the Amazon.  Means restriction just isn't viable in that kind of a setting.  Or they require that kind of strict adherence to the procedures for fidelity purposes, yet when thinking about how to implement them and resource in low-resource settings, adaptation is crucial, right, to account for all of those other contextual issues.  And so it creates this challenge where we can't just take the evidence-based model and train people to use it effectively because it just doesn't translate as intended, but needs to be delivered as intended to have its effectiveness, right?  So it's sort of like a Catch 22.

DAVID JOBES:  Kathryn, what are the differences that you see, you know, in Australia in the systems of care?  Are the commonalities stronger or are there unique differences that you see at the cultural level?

KATHRYN TURNER:  Well, I think -- I think there's lots of commonalities.  There's clearly, you know, significant differences in the systems that we have in terms of, you know, where we've got, you know, a complex system in terms of our -- what's funded from the federal government, what's funded from state government, what's in the private sector, et cetera, et cetera, that's quite different from the United States.  But I think all the -- lots of the cultural issues are the same in terms of the some of the cultural issues.  Yeah, so there's -- I mean, I think there's lots that I've learnt from, you know, all the great work happening in the United States that works very well here.

Some of the cultural differences, as you said, some of that liability concerns is different, but there's other concerns that's similar.  You know, people are more concerned.  They talk about, you know, concerns about the coroner and investigations from the coroner in a fearful way, in a similar way that people are concerned about litigation perhaps in the United States and, you know, the impact that that has on people being risk averse and retreating from, you know, recovery principles and et cetera.  So, you know, I think that there's lots of -- lots of alignment.

ANTHONY PISANI:  Just to add to -- just to add to that, I think a couple of people here mentioned the importance of lived experience.  I think, you know, in Australia, my experience there has been that the whole culture and movement is a few steps more developed around the peer workforce and the role of lived experience.  I think there's a lot to learn there as well.

DAVID JOBES:  Barbara, did you have a comment for us?

BARBARA STANLEY:  Yeah, I would just say that I think that it -- I think we can't think about things like crisis response planning and safety planning as management tools.  Anybody who has heard me speak knows that I always start out with -- in my talks about talking about two types of risk.  There's acute crisis risk and then there's the more kind of long-term underlying risk, and that we need to address both, and that crisis response planning and safety planning are good at addressing acute risk.  It's not management.  It is actual treatment, and but it shouldn't be treatment without addressing underlying risk.  And I think it's really important that we keep it in mind because I actually think that by saying it's like a management tool, it kind of diminishes it and the power of it.

But at the same time, no one, I think, who does safety planning in the -- in a trained way would think that that's the beginning, middle, and end.  Institutions have kind of set it out that way, like, that's the -- we do risk assessment, safety planning, boom, boom, send them on their way.  That's the beginning.

DAVID JOBES:  Thanks.  I'm mindful of time.  I guess one comment that I would make is that when you look at, for example, American Foundation for Suicide Prevention's 2025 plan to reduce the suicide rate by 20 percent by the year 2025, so much of the focus is on healthcare systems.  And that's the focus for this panel, whether it be jail systems, or EDs, or primary care.  And what -- even though a lot of people who are suicidal come in contact with primary care, in particular, and also mental health providers, the majority of people that will die today -- by the end of the day will not be actively engaged in mental healthcare.  So from my vantage point, this is where we need lived experience.  We need people like Ursula and DeQuincy, you know, creating websites because a lot of people who are struggling do not want to see mental healthcare providers.  So we can change our systems and we can change how we do providing.  We also can create these alternative models if we're going to try to reach more people.

So with that, Stephen, I'll turn it back over to you, I think, for our break.  Thanks, everybody, for your participation.

STEPHEN O'CONNOR:  I'd like to introduce Dr. Gregory Aarons from the University of California-San Diego.  Dr. Aarons, I'll turn it over to you.

GREGORY AARONS:  Thank you very much, Dr. O'Connor.  So happy to be here and moderating this panel, so let's get right into it.  And our presenters will be Dr. Jay Carruthers from the New York State Office of Mental Health, Dr. Barbara Stanley from -- excuse me -- Columbia University, and Dr. Christa Labouliere from Columbia University.  So with that, I will turn it over to Dr. Carruthers.

JAY CARRUTHERS:  So good afternoon, everyone.  Thanks for -- to NIMH for putting this on.  It's already been a really stimulating conversation.  So my name is Dr. Jay Carruthers and I'm the director of the Suicide Prevention Center at the New York State Office of Mental Health, so we're the lead mental health agency or authority for New York State.  And you can see my title sort of reflects my bias.  I mean, we're talking about training, but I really think in the sort of science-to-service gap, implementation is under resourced, and you can't really talk about training without implementation support.  That's sort of a preview of my talk -- my brief talk today.  I really think it's so important.

And so really quickly, I think high level, I was asked to talk about the existing infrastructure of our training -- you know, program and training infrastructure that was mapped onto by the research project.  That's the very large implementation of Zero Suicide that my colleagues Christa and Barbara are going to talk about.  So I'm going to provide a high-level overview of that existing training infrastructure in New York State and pick and choose some sort of lessons learned that were learned from that project, but also some other of our mostly grant-funded projects in this space.  Okay.

So the Suicide Prevention Center of New York is really a technical assistance center nested in OMH, and, you know, we have a broad mandate to really promote, coordinate, and -- strategically advance suicide prevention with the end goal of reducing deaths and attempts -- suicide deaths and attempts among New Yorkers.  But, you know, when -- there was a thorough review done of our whole portfolio, and one of the lessons that was sort of highlighted, or one of the gaps that was highlighted, was we could be doing more strategically to support training in suicide prevention.  So really I started this job with a kind of explicit charge to ramp up support for, you know, clinical training and suicide prevention.

So and really, the main tool by which we do that, the main infrastructure, is our colleagues and really at the Center for Practice Innovation.  So the Center for Practice Innovation, or CPI, really is set up as a technical assistance center that has been designed to support uptake of evidence-based practices across -- for the mental health workforce, so not just in suicide prevention, but in a number of different areas.  So ACT, I'm sure people are familiar with ACT, an evidence-based approach.  And focused-integrated treatment combines the integration of mental health and substance use disorder treatment.  IPS is supported employment, wellness self-management.  OnTrackNY is a first episode psychosis program.  I'm going to talk today or reference today really the work by Barbara's group, which is the Suicide Prevention, Training, Implementation, and Evaluation Group, SP-TIE for short.  But there are other areas that -- but in essence, this is really the main kind of tool by which we have pushed out training and really tried to increase uptake of evidence-based approaches, not just in suicide prevention, but in other areas for the mental health and the behavioral health workforce across New York State.  Okay.  Lisa Dixon is the founder of -- or the director of CPI, and prior to her was Susan Hassack, some of you may know.

Now, SP-TIE, Barbara's group, really has done just an absolutely fabulous job creating, you know, really a panoply of resources, mostly modules -- you know, interactive modules and webinars that do the heavy lifting for us.  And a lot of them are organized around the AIM model, kind of a helpful framework that we use to remind clinicians, you know, "A" for assess, "I" for intervene or intervention, and "M" for "monitoring between episodes of care."  So we talked about sort of AIM for Zero.  I talk about AIM for Zero, kind of combining some of the concepts that Julie Goldstein-Grumet talked about.  Really, AIM focuses in on the clinical components of the Zero Suicide Model.

So we're constantly having conversations about, you know, what are the needs, where are the gaps in training, and making revisions to the library of modules and webinars that we have housed at CPI.  And you can see we just recently revised one for the C-SSRS module.  That was earlier this year.  Currently in production there's something on assessing and managing suicide risk through telehealth.  Obviously that's been -- you know, providers of had to radically and, you know, really abruptly shift to that, and people – you know, speakers have already talked about the fear that most clinicians bring.  And then you throw managing suicide risk and assessing suicide risk from a telehealth perspective, and obviously that increases materially, so.  And then we are revamping a safety planning module.  You know, we’ve learned so much since the previous module was developed, so we’re looking forward to releasing that later in 2022.  But that just gives you a sense of we constantly have something in the pipeline that and evolving to meet the growing needs of the mental health workforce in New York State.  So incredibly productive.  You know, 56 trainings in the use of – those aren’t unique individuals per se, but really just a tremendous amount of training since coming online in earnest in 2015.

All right.  So the first lesson, I guess I was sort of new to the job and eager to – we had some money that was, you know, kind of a one-time bucket of money, and I was eager to do some work with evidence-based practices.  And so we partnered with Dave Jobes, and Greg Brown, and Kelly Green, and we wanted to offer CAMS and cognitive therapy for suicide prevention, you know, two evidence-based approaches.  And we offered tiered kind of resources to clinicians across – in the public mental health systems across New York State.  So it started, you know, kind of easy access – accessible online training or introductory webinars, then a smaller cohort went to live full-day trainings with role plays.  And then we offered a more intensive smaller group kind of supervision after going through the first two tiers.

So obviously from my title you can see where I’m going with this, “If you offer it, they may not come.”  And you can see sort of graphically that really you can focus on, you know, how many people actually completed each tier.  So completed foundational online turning – online trainings, we had about 400, and then you go down to the next kind of plateau for the black line and you can see completed in-person workshop, about 150.  And then to the third tier we really just struggled mightily getting anyone to participate in the, you know, applied supervision, and I think there were a variety of issues there, the time commitment.  I think some people perhaps were intimidated by having to record sessions or -- but we really just simply didn't get a good uptake in the most -- you know, the most intensive support for learning these evidence-based approaches.

So in addition to CPI and kind of pushing things out through that Technical Assistance Center, a lot of our training has also been driven by an implementation that's been driven by grants.  So we've been fortunate enough to try to push out, you know, and disseminate further, and increase competencies through really predominantly -- you're going to hear about the NIH project that -- later, but we had a few SAMHSA grants that we've had over the years that we really used to try to push out evidence-based practices and what we call sort of AIM for Zero.

So I would say that the dip -- you know, SAMHSA has shifted from three here.  You can see the National Strategy for Suicide Prevention grant was only -- was only three years.  They've since shifted to five-year grants, and that, to me, is a huge difference and makes it a little more -- I wouldn't say it's easy, but I think three years is simply too short to get sustainable uptake of evidence-based practices working with different grant sites across the -- across the state.  And of course, you know, the -- if you're working intensely with just a few providers, it's difficult to scale that.  We have, you know, limited bandwidth for technical assistance, so.

So lesson number two, sort of less is more.  And I think we have -- you know, in part maybe me because I learned lesson one that I think doing full-blown psychotherapy and scaling that up is extremely challenging.  So we've made a very deliberate decision to focus on what I think Stephen O'Connor rightly characterized as brief but mighty, kind of brief intervention, predominantly safety planning, and we're also piloting ASSIP in Central New York.  I think some of you may have heard of that, which is a -- really a three-session intervention, and we've worked with the developers in Switzerland to try to get that -- pilot that in Central New York.

Okay.  So I really think this side is probably the crux of it that -- lesson number three, that, you know, training in the absence of broader systemic support is really not going to go very far.  And you've heard kind of, I think, some of the other speakers have emphasized this that, you know, can you recognize quality both at the individual and the clinician level.  That's so critical, and we've been working really hard with predominantly our grant sites to see if we can -- we can support them in doing that.  And we talk a lot, I mean, about moving from check-the-box level of care.  You know, was the safety plan done, yes/no.  That's easy to verify, but it's a lot harder to make sure that it was done well, you know, high quality, high fidelity safety planning.  And so that's where a lot of our efforts have gone into over the last year.

I think, again, some of the other speakers prior to me of talked about, you know, it's so important to be able to provide feedback to the providers when they're doing good work and help them identify when the quality is not where it should be.  So audit and feedback -- that's kind of a term from the implementation science literature that you hear -- I think absolutely is critical to this work.  And we've really assigned implementation specialists to work -- from the grant team to work with each of our grant sites, so it's pretty intensive.  You know, just about every two weeks they're meeting with these grant sites across the state, so it does require a lot of resources and support to, you know, really to build process and structure around safety planning.  It's simply not enough to do the basic trainings and engage them in plans.  What are they doing to build process and structure to support, you know, things like high-quality, high-fidelity safety planning?

So just about out of time here, but I wanted to just kind of tee things up for Christa in that -- so back to the project that we're going to talk -- they're going to talk a little bit more about and some of the lessons learned from there.  It really -- the context is that, you know, historically we've had a bucket of money to support CQI projects, and that's been led by our colleague Molly Finnerty, who leads the PSYCHES team at the Office of the Mental Health.  And so basically, I was groveling and sort of advocating for suicide prevention to be a CQI project, and, you know, Molly to her credit is very data driven.  She had a webinar and holds kind of some of the clinicians across our outpatient mental health clinics, and the number one issue they wanted support on was suicide prevention.  Then she -- but she didn't -- she still didn't trust the data, so she did a broader survey across all Article 31s, and again, suicide prevention came back as the number one issue.  So I think it's sort of aligned with what you heard, again, from Julie Goldstein-Grumet, if you caught her session, that, you know, clinicians really want this support and are calling for it.

But real quickly, clinics that participate -- outpatient mental health clinics that participated got a Medicaid rate enhancement.  Basically, any claim that they dropped, they got a 3.84 percent bump while they're participating in this CQI project.  And it was 165 public mental health clinics to really do training and implementation of a high-risk pathway.  And, you know, we were providing technical assistance and implementation support, performance metrics for quality improvement along the way.  And this was going to happen regardless of whether or not we were fortunate enough to get NIMH funding to do the project and evaluation you'll hear more about.  But I just want to underscore that, you know, we were moving ahead with this because of the demand, and so this is sort of the backdrop to the NIMH project that you'll hear a little bit more about.

All right.  So I want to thank -- of course everyone knows suicide prevention is a team endeavor and just a list of our of many of our collaborators.  So thanks very much.  I'm going to stop sharing my slides and turn things over to Christa Labouliere. 

CHRISTA LABOULIERE:  All right.  Thank you so much for inviting me to be here today.  I feel like I've already learned so much, and hopefully we can share some of the things that we have learned here in New York State.  And so I'm going to be talking today a bit about the project -- the statewide project that Jay just referenced and talking about some of the barriers and facilitators that we found when trying to implement Zero Suicide here in our behavioral health outpatient clinics.

So as you heard earlier on, Zero Suicide has these seven essential elements.  Ultimately four of these elements focus on how clients should be treated while the remaining three are related to implementation.  And so we did our best to engage leadership, develop a training protocol for the workforce, and engage in data-driven quality improvement processes as part of the implementation focus.  But we also defined a clinical model that was going to be implemented here to target those other four essential elements.

Jay referenced this.  This is the AIM Model which stands for Assess, Intervene, and Monitor for suicide prevention and really focuses on broad screening and risk assessment for everybody who's coming into our outpatient clinics, then providing a higher level of intervention for folks that are identified as being at elevated risk, predominantly brief interventions like safety planning intervention, but also suicide-specific interventions, and then providing a higher degree of monitoring for folks who are at risk, so follow up, making sure that they're having clean transitions with warm handoffs.  And then also following up to make sure that screening and risk assessment are an ongoing process for everybody who is in our clinics.

So the project that we were doing, we implemented Zero Suicide in 165 freestanding outpatient behavioral health clinics and 280 affiliated satellites across New York State.  So something that was kind of unique about our project was that these folks did not share leadership.  They did not share an EHR.  They were all freestanding independent clinics of diverse size, diverse client demographics, and diverse geographic regions all across the state.  So these clinics represented over 100,000 clients served, and all clients over the age of 10 that were enrolled at these clinics were part of our project.

Prior to implementing this clinical model, we offered both training and ongoing implementation assistance throughout the project.  So the trainings were distance learning modules through the CPI Learning Management System.  As Jay already had mentioned, you know, we have this great system here where we are able to offer free clinical trainings to clinicians here in New York State.  And so we leveraged that system to provide suicide prevention trainings on risk assessment, safety planning, our clinical pathway, and also adaptations for clinicians who were working with youth.  We also offered ongoing monthly learning collaboratives to both leadership and site champions who are clinical supervisors to offer additional clinical and implementation assistance.

So our study had three phases.  We collected some data at baseline before we began the process of implementing these things.  Then we had about a year-long implementation phase where we were engaging actively in these learning collaboratives, and then finally a maintenance phase to see if we could sustain these gains once that heavy touch of implementation assistance was rescinded.  So we were very lucky that we were able to collect data at multiple levels.  So we had clinic monthly reporting on their implementation processes.  We were able to do an annual workforce survey of all providers in our clinics looking at both provider attitude, self-efficacy, and intentions as well as more organizational factors that would influence the Zero Suicide implementation.  And then all clients, we were able to obtain data from their -- from administrative databases that are maintained here in New York State with things like their C-SSRS scores or safety plans for high-risk clients, service utilization rates, and then also some outcome factors, such as suicide attempts, ED visits, or inpatient hospitalizations.

In relation to this presentation specifically, one of the nice things is that we were able to obtain both qualitative and quantitative assessment of barriers and facilitators from both leadership, site champions, and, you know, frontline clinicians.  So we were able to get really robust information on barriers and facilitators.  So probably, you know, the first and foremost barrier, you know, and you've heard a bit about this in a number of presentations today, is that there is a high need for training in this population, even with licensed mental health professionals.  At baseline, 51 percent of our providers reported insufficient training in suicide prevention.  Their suicide prevention knowledge was relatively low.  Their attitudes towards suicide prevention were not great, and unfortunately self-efficacy was only moderate amongst our clinicians.

When we asked folks specifically about what the barriers to implementing evidence-based practices for suicide prevention were, many of the barriers they reported were related to training, such as needing more training and experience with these protocols.  Many clinicians reported apprehension or discomfort or that they needed more supervision and support.  And of note, these training-related concerns were more commonly expressed than the things that you would think of as being big barriers, you know, things like time constraints or feeling overburdened.  So really the need to training rose to the top as something that needed to be addressed.

We also did qualitative coding of the responses that folks gave during our learning collaboratives.  And the most common barriers expressed by leadership and by site champions were also training issues, so things like clinician discomfort with suicide, a reluctance to use new tools or protocols, and also a perception that a protocol can sometimes hinder therapeutic alliance, and that if these new protocols are not integrated well, that this can actually be damaging to client rapport, and so that there needs to be greater training so that clinicians can implement these procedures with sensitivity in a way that's really client centered.

Now, the good news is that there is a solution to this.  Training initiatives are effective.  The top facilitators that we saw reported by clinicians were that, you know, additional training and supervision really helped thing.  You know, in our subsequent years, significantly more clinicians endorsed sufficient training to assist their clients.  Their knowledge increased notably as a result of these trading condition -- the training initiatives that we had, and attitude self-efficacy and their use of evidence-based practices all increased significantly from baseline to post-implementation and maintenance.

Another barrier that was related to training was that we had a very high level of turnover in the public mental health system here.  Forty percent of our clinicians in year two were new clinicians that were not present the year before, and we only had about 20 percent of clinicians that were present across all three years of the initiatives, so we had very high rates of turnover.  And, of course, this poses difficulty for training, but also poses difficulty for continuity of care.  You know, understaffing can result in lengthy wait lists.  Client rapport may be disrupted by frequent changes of clinicians due to turnover.  And it also means that you can't just have training be a one-and-done initiative where, you know, like, you do a big trading initiative at the beginning of your project and then you're done.  Instead, you really need to have a focus on ongoing training, and this can be particularly challenging in a fee-for-service environment where a lot of our clinicians in New York State are really only paid for the time that they spend face-to-face with clients, and that sometimes means that, you know, we have to get creative in our approach to training.

So ultimately, clinics need established procedures for ongoing training, particularly in regards to suicide prevention protocols.  There is a need for clinics to continually provide training in the basics while also balancing more senior clinicians' needs for advanced, more nuanced training.  And having high-risk meetings or more informal peer consultation can provide helpful support, especially for novice clinicians, and really can help keep suicide prevention in the forefront of clinicians' minds.

So it was really nice to see how some sites got really creative in how they approach this, doing safety plan reviews for quality improvement, you know, using supervision for the opportunity to role play new skills or challenging situations, and really providing opportunities for blame free consultation as opposed to a perception before that, you know, like, sometimes bringing this stuff up in supervision could be fraught.  Lastly, leadership really needs to be creative about finding ways to support training.  And one of the things we found was that leveraging existing time that's set aside for supervision or staff meetings to facilitate training for fee-for-service staff can be one approach that can be useful for doing this.

Of course there are other barriers.  You know, time constraints and demand for resources are a huge barrier that you also have heard earlier today.  Things like time constraints or feeling overburdened by paperwork, lack of resources were all commonly reported barriers by our clinicians.  Now, the nice thing is with our implementation assistance, perceptions of positive organizational climate, leadership support, and supervision all increased significantly over the time of this project.  However, we were asking them to do a lot and perception of these barriers also significantly increased.  And that means that we really need leadership to take the extra time and support the extra time it's going to take to provide high-quality suicide safer care.  Leadership really needs to support clinician self-care and wellness because this can be a challenging population to work with, and there needs to be a focus about it's a lot easier to onboard procedures at intake than it is to kind of maintain ongoing procedures that clinicians may need to remember.  Things like quarterly screening were much harder to uptake than, you know, the sort of things like screening at baseline.  That was easy to remember.

One thing that did help was that EHR integration and technology can be a huge help and -- you know, a good reminder to clinicians, but this is only true if it's done well.  Clunky EHR and integration sometimes can be more of a curse than a blessing, so this needs to be done well.  And leadership needs to be careful that procedures cannot always simply take and add more approach.  Clinicians really need flexibility and good guidance on how to complete the paperwork and procedures across a few sessions, how to prioritize things that need to get done, and how they can obtain, you know, flexibility in regards to providing sessions of longer duration if suicide prevention procedures need to be done in the first few sessions.  Sometimes things like task-shifting involving support staff or peer specialist interns, their assistance can be invaluable in this regard.  And also we had some success with contracting with outside organizations to sometimes support things like follow up and monitoring between levels of care that may not have always been feasible for primary clinicians to do.

So the good news regarding one final challenge is that we were able to implement Zero Suicide pretty effectively.  The percent of clients screened that were identified with a positive screen was eight percent.  It was pretty consistent across the course of the of the project.  And, you know, while that is a big number, that is not overwhelming.  It is not going to freeze our mental healthcare systems.  And once these folks were identified, the re-attempt rate was very low for folks who were receiving our Suicide-Safer Care Pathway interventions.  So once clients are identified, the majority were, in fact, getting the enhanced pathway supports they were supposed to be receiving if they remained in treatment.

Now the not so good news is that client-related barriers continue to be reported by clinicians, and, in some cases, actually increased in salience.  So getting lack of client buy in for treatment, losing contact with clients continue to be an issue across the course of the project.  And really client engagement and retention might actually be some of the most challenging issues that we need to address going forward with our work in outpatient clinics.

So I'm going to hand the floor over to Dr. Barbara Stanley who was the PI of this grant.  She's also a professor of medical psychology at Columbia University, and special shout out and thanks to our collaborators on this project as well.

BARBARA STANLEY:  Okay.  Thank you, Christa.  So I was asked to talk about how do the results from this grant inform future research and statewide training efforts.  Before I get started, one thing I'd like to say is that we were the very fortunate recipients of the Zero Suicide -- Zero Suicide grant from an RFA that Dr. Gordon mentioned earlier today, and this allowed us to work within a system that was actually very rich in infrastructure to be able to do this project.  And I think without both of these things, the rich infrastructure that New York State had and this grant, we wouldn't have been able to really mount this effort.

And by "infrastructure," I mean, we have resources within the state where we have a really good group that does analysis of Medicaid data, and so that group was existing that we just had to then tap into.  And then also that New York State has a system, as Jay mentioned of doing quality improvement projects that enlists programs and clinics throughout the state.  So I didn't have to go out and find 165 freestanding programs on my own.  They were there to be invited.  Okay.

So I'm going to start with the end and then I'll go through some of the data that we got from our project that kind of informs why I say these are some of the next steps and research needs.  And before I say this, one of the things I will -- I want to underscore is I think that there is a great need for more research on what is the -- what kind of training is the most effective training that we should be doing in suicide prevention because in preparation for this, I started  -- I did bit of a literature review and there really isn't a lot out there on, like, this training works better than this training for clinicians in this particular circumstance.  So that's kind of like the prelude here.

So here are the -- kind of like the five takeaways I think, that we need to develop and implement scalable, on-the-job suicide prevention trainings to compensate for the lack of suicide prevention trainings during graduate education.  And this underscores what Mitch Prinstein said earlier, and I'll show you some data on that.  We need to develop a system of incentives and accountabilities to increase training uptake.  We need to develop models to site champions being alternative models -- I'm sorry -- to site champions being responsible for transmitting information to clinicians which was the model that we used.  We need to develop strategies to train staff on client engagement and retention, which Christa already referenced, and we need to develop more scalable suicide prevention training approaches that achieve quality.  And by "scalable" here, I mean not just that are clinician accessible, but that are “expert trainer accessible” that don't call on expert trainers to be running all over the country or Zooming all day long doing their -- doing trainings.  We need to figure out something better than that.  Okay.

So the clinician training needs, prior training.  So this is what they came into our study looking like.  So we see here that 20 percent of them have never -- had never received training in risk assessment.  Twenty-eight percent had never received training in how to discuss access to lethal means.  A quarter of them never had any training on how to develop a treatment plan, and 43 percent had no evidence-based suicide-specific treatments.  So they came in deficient, so we could go back and say, okay, we need to do better in our graduate training which we certainly do, but these are people who are out there now.  We need to figure out what to do.

Okay.  Suicide prevention knowledge over time.  The good news here is that knowledge increased from baseline to implementation and maintenance so that we can at least get our knowledge up for the different aspects of suicide prevention intervention that we think are important, and so that's really -- that's kind of a good news thing, so it is possible.  But clinicians -- the percentage of clinicians endorsing receiving training increased significantly over time, and more clinicians endorse sufficient training to assist suicidal clients post-implementation.  So they felt like they got more training that led them to be able to deal more effectively with clients in this project, which is very good.

But this is what the sad thing is.  So clinics enrolled in this project, as Jay mentioned, they got an incentive to participate in a quality improvement project.  We were one of three that year that they could choose from.  So we had -- we had web-based trainings.  They were asked to participate in two web-based trainings and a third if they treated children and adolescents.  And so one was just kind of like a foundations of suicide care management.  This was our protocol.  And then we did -- we gave them our pathway, and then we did an adaptation of the AIM model.  And you can see here that only a little over half of the clinicians at baseline completed the trainings. So we did this implementation going in and looking at outcomes knowing that only half the staff were -- received training -- the basic training that we asked them to take, and this was with multiple reminders and, you know, whatever kinds of ways that we could get them.

And so this comes back to the need for incentives and accountability, so somehow we have to figure out how can we make this more incentivized?  And I think, you know, states like California and Washington have come up with mandating at least some sort of suicide prevention training for clinicians.  And unfortunately, I think that that probably is the way to go, to make some sort of mandate.

Okay.  So we need to figure out some way -- we realized in our project that staff turnover was a big issue.  I won't go into that because Christa mentioned that, and so we -- training has to account for staff turnover, so training has to be an ongoing, as you come into a system you need trainings in X, Y, and Z.  The problem is, you know, when we start a new job, we all have a zillion trainings that we have to take and they become kind of meaningless, so we have to be creative.  And I think this is one place where we can use our creative forces to figure out how can we incentivize this and deal with high staff turnover.  And then we have to figure out some alternative models to using psych champions as being responsible for transmitting information to clinicians.

So what we had was, we had a learning collaborative where a dedicated -- a designated person or two from each site came onto these learning collaboratives every month.  We did a didactic.  The didactics were typically led by Greg Brown and Kelly Green from the University of Pennsylvania.  We happened to spend a lot of time, because we were requested to do this, on safety planning, but so we -- so the site champions then were charged with going back and sharing the information to their staff, and then this is what the site champion said.  Only 20 percent of respondents stated that they always share information from monthly calls directly with their clinicians, and 44 percent of them said they often shared it.  So even though this was their so-called job, some of them -- it didn't really get through.  And so we've been toying with how can we do more direct clinician training that maybe we'll have better uptake which is a very, very big task.

Okay.  So as Christa mentioned, one of the things that we actually, I think, in our project we didn't pay enough attention to, was not really part of the AIM model.  We didn't really pay attention to making sure they were engaged in outpatient care.  And so this, to me, was a really, really sad figure, that 40 percent nearly of people, once they were identified -- they came into the clinic, they had a suicide risk assessment, and -- they were identified as being at risk, as high risk.  They were put on the suicide care pathway, they got a safety plan, they were to be monitored more closely, they were to have at least weekly appointments, and 40 percent of them never came back for a second appointment.  That is very scary to me.

And then one of the things that our pathway recommended was, and this seemed very, very reasonable, that somebody who had made a recent suicide attempt or had been to the ED for a recent suicide crisis and they entered outpatient care, we should at least think about seeing them weekly for the next several weeks.  So we chose 12 weeks which is the period of high risk  -- a period of high risk.  And we found out that you can look at the graph here and it's very sad.  Only two percent of our patients attended 12 weekly sessions as recommended, and fewer than 30 percent received six or more sessions during the 12-week period.  And so one of the things that we are thinking about is, okay, we need to do better training on this, but we also need to think more creatively about, well, was our pathway the right thing?  Were we asking too much of both the client and the clinician to be seen weekly.  Okay.

And then I'm just going to end here by talking a little bit about quality.  And so we spent a lot of time within our learning collaboratives on having -- helping people learn how to do high-quality safety plans.  And despite having spent a lot of time, and I won't go into all of the details here, but we didn't get great quality.  And so the first -- this piece of data shows our -- how complete safety plans were.  So you can see that for Step 6, which is access to lethal means, they got a score of zero to 2.  They got a score -- average score of 1, so about 53 percent rating there.  They did better on things like identifying warning signs and distracting activities, and then in terms of overall quality, same thing here.  They didn't do that well in terms of overall quality.  And so we have to figure out some way of doing -- of increasing quality and fidelity more than what we are doing.

So the last thing I want to say is we need to think about scalable trainings for evidence-based intervention, and I'll use the safety plan as an example here.  We spend a lot of time in online training and learning collaboratives with improving quality of safety planning, but the quality still needed improvement.  So we didn't do this kind of intensive training in our project because it just wasn't feasible.  But when we do trainings, if somebody wants to become an expert or be able to say that they are competent in doing safety planning, this is what it entails: a didactic online module, a demonstration role play that the trainers do, a practice role play, at least one that the clinician does, expert review and rating of recorded or live safety plans, expert feedback, and additional review and rating if needed.  This requires -- and I'm not even going to talk about what it requires of the trainee, but it requires three to four hours of expert trainer time, and it is actually not really feasible on a large scale.

And so I'm going to come down in the same way as some of our earlier speakers, which is we need to think about doing things in a more automated way.  And so we've been toying with this and playing with this idea of an alternative model of training safety planning, most of which is automated.  So the didactics can be an online module.  This is what we've been -- this is the model that we have worked with.  So first, when they do a didactic, they learn what safety planning is.  Then the next thing that we do is we teach them how to do adherence ratings, and this also can be automated.  We haven't automated it yet.  And then based on that, trainees view and rate fidelity of pre-recorded interventions, so in this case, safety planning.  So it's a good and a bad example of safety planning.  They particularly love reviewing bad examples after they've learned how to do it adherence ratings.  This can also be automated.  And then we share with them expert fidelity ratings.  This can be automated.  Then the expert can come in, and this is not automated, and lead a discussion of the fidelity ratings and have a Q&A about it.

And so you can see that what an expert needs to do is very much pared down, expert trainer.  And so the advantages of this is it cuts down on expert trainer time dramatically, and it eliminates role plays that many clinicians actually, I'd say dislike here, but many clinicians actually hate doing them.  And so I thought, okay, this is probably -- this is not novel.  Probably other people have done this.  I went to the literature I cannot find any other place in the literature where they are doing this.  Maybe it's done, but I couldn't find it.  But this needs to be -- if we're going to do this, we need to see, okay, how -- we need to evaluate it.  How well does it do by the gold standard kind of training model.

And so I'm just going to come back to these are the five things that I think that we need to do in terms of next steps.  And again, I'm very, very grateful to NIMH for funding us to do this and for being fortunate enough to be working within the New York State Office of Mental Health.  Thank you.

GREGORY AARONS:  Thank you very much.

BARBARA STANLEY:  And now I'm turning it over -- I'm sorry -- to Greg Aarons.  Sorry.

GREGORY AARONS:  Thank you, Dr. Stanley, Dr. Labouliere, and Dr. Carruthers.  Yeah, what an amazing project, and you've proven again that, you know, there are great ideas often, hard to deal and manage with the complexity of real-world services.  And I'm thinking how, you know, listening to the presentations, thinking about how this fits into some of our implementation frameworks.  So, for example, in our Exploration, Preparation, Implementation, and Sustainment Framework, we think a lot about that outer context, the system context.  And I know, Dr. Carruthers, you alluded to that.  And, you know, having a Medicaid rate enhancement, for example, in the outer context and how can that facilitate and support an initiative like this in the inner context.  And we also heard about, you know, different components of the implementation, you know, the Learning Collaborative, the site champions.  And one of the things we've been thinking a lot about in EPIS Framework is this idea of bridging factors.  How do we really communicate the -- and align our activities that communicate the importance of initiatives like this while also working through the workflow issues, the turnover issues.

One of my NIMH projects recently, we were working to address a similar issue in terms of both the training and workforce, and we had this idea of and actually implemented this interagency collaboration where clinical teams across agencies work together.  But at the core of it, there was a seed team that had the expertise in that catchment area that was developed and supported by the system for ongoing training, fidelity assessment, and coaching.  And as there were turnover in the system, that team could also do that that training.  And it has to be somewhat localized, I think, so that those resources are available.

We keep coming back to this idea, and Dr. Koerner talked about this, you know, use of AI, use of more automated methods, and there are some really good advances.  So, for example, thinking in the area of substance abuse, there's an app-based contingency management program called Dynamic Care that's used for engaging providers and clients and helping managing the process.  Things like that could be available.  And then, you know, on the cognitive behavioral treatment and motivational interviewing side, we see developments by Dave Atkins and colleagues at the University of Washington with a platform that's called LYSSN -- L-Y-S-S-N -- that provides near real-time feedback about quality of -- CBT quality of motivational interviewing and also uses AI and computational linguistics to identify issues around suicidality that are talked about in session, where problems occur at work, at home, with family, with friends, and other things that emerge out of conversations.  And these platforms, I think, really hold great promise and go along, Dr. Stanley, with what you were really proposing which is how do we automate this?

But in thinking about that, I think we can automate things, but we have to do it in a way that's engaging.  So when we really look at the drop off of clinicians and counselors, that slide that you showed, Dr. Carruthers, of, you know, engaging and then through the various steps of sort of the training cascade that drop off, how do we do things in an engaging way that that could keep folks involved, keep them really invested in the training.

And it was great, Dr. Labouliere, when you were talking about, you know, assessing, you know, attitude, self-efficacy, and intentions.  So how do we keep those positive and high through that training process?  And, you know, I think my challenge is in thinking about how we align those activities engaging policy to support this and aligning with what organizations are doing and how they're incentivized.

So I'm -- I just want to throw that out to the groups, and maybe I'll direct this first question to you, Dr. Carruthers.  You talked about aligning initiatives, and what are -- what would your recommendations be for thinking about, since you're at the State office and also, you know, working with these organizations, thinking about aligning initiatives across, you know, service providers, organizations that provide direct care and the service systems?

JAY CARRUTHERS:  Well, one of the complaints we get is that -- you know, we have a sister agency that's a substance use, and, you know, a lot of the provider systems say that they're being bombarded with directives from our sister agency.  So I think we could do a lot better in that regard of actually aligning, and there's a real need for that.  But I -- going back to sort of the point Barbara made about automation, I do think it's an empirical question, right, in terms of what is the right ratio of automation-to-human kind of engagement to optimally manage suicidal individuals.  I think that's really the question that we do have to figure out.  And depending on who you talk to, I think they'll give you a different answer, but I think it would be beneficial to see more research in that area to figure out how we can optimize that combination.

GREGORY AARONS:  Great.  Thanks.  Dr. Labouliere, I was thinking as you were talking and describing the learning collaborative and the site champions and leadership, I just wanted to know more about that, you know.  Who were -- who were the site champions?  How did the learning collaborative consider levels within organizations because often a site champion may be identified, but there may be levels that can be engaged.  So what are your thoughts about how that went and how you might maximize that effectiveness?

CHRISTA LABOULIERE:  Yeah.  So when we started our learning collaboratives, we initially met with leadership, so we were looking -- we were meeting with folks that were at the top to kind of engage them, get their policies and procedures on board, you know, talk to them about their workflows.  And then as we started to move towards training and finalizing their new protocols, we had leadership identify site champions, and all that we asked was that these folks had some sort of clinical supervisory position so they would have the -- you know, a little bit of authority to do training and supervision, you know.

But, you know, then we wanted folks that wanted to be cheerleaders.  We wanted folks that wanted to do this work.  And so leadership assisted in that process of identifying these individuals, but we got a really diverse group of individuals, you know.  Some folks were very seasoned clinicians.  Some were new to these roles.  And really what we wanted was to have the opportunity for there to be, like, boots on the ground that got additional training, that had some expertise, that if a clinician had questions there was somebody there that they could turn to.  And so we requested that there be two site champions to handle the kind of ongoing turnover issue we had even amongst our leadership.  And yeah, it was a very rewarding experience to work with this group over time.

I do think one of the drawbacks of this approach, and Barbara touched on this, though, was that we were really kind of envisioning this to be more of a train the trainer model where, you know, we would impart additional clinical training, and then they in turn would turn to their frontline clinicians and, you know, bestow that knowledge upon them.  And it really did not work out that way unfortunately.  It was very hard within some of our systems for that knowledge to disseminate from our site champions to our frontline clinicians.  So I think across the course of the project, we had to get more creative.  And, you know, maybe in the future, more direct-to-clinician work, you know, in addition to those original trainings, you know, I think that probably -- you know, that kind of a hybrid approach probably would've been more effective.

BARBARA STANLEY:  One thing I'll just add to that, you know, in fairness to these site champions, is they were clinicians.  They had full clinical loads.  They had a job to do, and this was their volunteer job, and so I would say that they probably did, for the most part, the best that they could.  And, you know, when you're talking about like a, you know, a truly diverse set of clinics that we were working in, you know, it was -- some were more able to give, like, so-called release time to the people who were doing this.  Others were completely unable to do that.  So that was -- that's one of the constraints in working in the real world.

CHRISTA LABOULIERE:  Yeah.  Having some sort of formal approach, you know, to -- you know, for information dissemination or to be able to have some protected time was an enormous facilitator.  And, you know, some places could do that, and, unfortunately, other places could not.

JAY CARRUTHERS:  And I think that's where I sort  -- if they have, you know, a robust quality improve -- you know, do they have staff whose job is quality improvement and to provide that feedback and audit, you know, some of the kind of things that are key in implementation.  But, you know, as some of the other presenters have highlighted, you know, the range from high-resource to low-resource L systems, you know, it's highly variable how much -- you know, whether systems have those physicians on staff that it really can assist at the system's level supporting all these quality improvement efforts.

GREGORY AARONS:  Yeah.

BARBARA STANLEY:  I would say also this harkens back to what Mitch said first thing this morning, which was they are -- we are not giving out this kind of training when we should be giving it out.  When you're a student, you're there to learn.  When you're a clinician, you're there to treat, not so much to learn.  Like, you're not being paid even though we can think about it that way.  We're not learners then.  We're supposedly experts.  And so we need to think about how do we -- how do we push more in terms of having our clinicians come in with more of a basic knowledge.  And so, I mean, this is all -- this whole workshop is about suicide prevention training and ways it could go, and I think that should be one of the take-home points.  We need to have a more shovel-ready workforce as they enter the workforce.

JAY CARRUTHERS:  Yeah, that requires a lot of political will.  I mean, I think Mitch did highlight the barriers in terms of, you know, no curriculum time.  But, you know, I guess going back to your original question, Greg, like, how do you align some of this, I mean, you know, accreditation -- the accreditation standards of the Joint Commission taking this on has really helped.  We use that when we're engaging health systems.  That softens the ground.  I think we should be doing the same thing, you know.  And when we approach trying -- you know, Zero Suicide, we sort of try to be sensitive to Zero Suicide fatigue when we're working with grant sites because I do think it's real, you know.  They do have full-time jobs, and we keep coming at them asking them to do more.  But the way I approach it is that it's really fundamental just to providing good clinical care, so it's not just suicide care.  I think it's much more -- it's much broader than that.  And it really should be part of the core curriculum milestones for all the mental health graduate school programs, but that does require, you know, concerted kind of political effort to really go after that.

One of my other hats is running our Grand Rounds Program and just trying to get continuing education credits.  The standards are different for social workers versus psychologists versus physicians.  So, you know, bringing a standardized approach to all that, those are, you know, can be real-life barriers, too, if there was more alignment there.  But I think definitely we should make a concerted effort to make these, you know, core standards of graduate school.  It's really got to be.  It's got to happen.

GREGORY AARONS:  Yeah, there's so many things to respond there.  I'm loving this discussion.  And I think, you know, Barbara, you're absolutely right.  The core curriculum, we need to train our upcoming workforce, but I also think that we can consider, you know, and there's a lot of work in business and management around the learning organization, to have that culture of learning within the organization so it's not totally alien.  And it's a big challenge to embed that.

The other thing with your study, you know, you've been alluding to some of the high-resource organizations and lower-resource.  There's a wonderful opportunity, it sounds like, to really understand, you know.  As I'm thinking about this, it's almost like we're talking about low-income and high-income countries, low-income and high-income organizations, and how could training be tailored for these different organizations.  And I think there may be ways to go about that.  I wonder what your thoughts are just about the variability that you were seeing.

JAY CARRUTHERS:  And I do think that we -- like, conversations with colleagues at CPI, it often is, you know, we're really bending over backwards for what does often boil down to their organizations that are -- that just don't have the implementation support kind of infrastructure.  You know, weak versus strong organizations, and how much of our approach is going to be dictated by that.

GREGORY AARONS:  Yeah.  One of -- one of the things that we really focus on in the work that I do is thinking about the organization as a whole.  So it's great to have champions, and often our executive directors may delegate or designate or ask for volunteers.  But unless we can align that leadership support so executives are communicating the importance of this -- they talked about in their newsletters, they're hiring people with expertise.  Really making it part of the culture, I think that can help.

BARBARA STANLEY:  Well, I have a couple thoughts about that.  One is, I would have to say I can't think of a clinic that we work with that was high resourced really.  They were -- they might be medium resourced and low resourced, but mostly they were really stretched, and they were kind of like running on empty or an eighth of a tank of gas.  And so the other thing I have been thinking about is how can we expand our workforce in a way that is -- that is kind of economically sound.  And so one of the things that we've been thinking about is the role of peers and people with lived experience and how can they come in and help the effort here.

JAY CARRUTHERS:  You know, Christa highlighted the baseline, you know, 40 percent turnover.  That was pre-pandemic, so our sites are saying they've had greater than 50 percent turnover of their staffs over the last year.  So workforce issues are ubiquitous and really, really challenging, I think, right?  All the training designs has to assume extremely high turnover, and I have to agree with Barbara.  We're talking about the public mental health system.  There really is no high-resourced.  It's really low and moderate, so, but the workforce challenges are just -- they are daunting.

CHRISTA LABOULIERE:  Well, and it's one thing for leadership to talk the talk, and it's another for them to walk the walk, you know.  I think it's -- yes, it is important for them to be emphasizing this as part of their culture and in their newsletters, but there also needs to be a system that accommodates the extra effort that it takes to work with suicidal patients well.  And many of our systems are not built to do that effectively, and it's not necessarily just leadership being inflexible.  We need to be more creative in our approach towards how do we task shift some of these things?  How do we bring in different economic revenue to give our clinics more of a safety net so that they can give their clinicians more of a safety net so that that can trickle down to better care for our clients.

It's tough.  It forces you to think about things that, you know, are not necessarily targets of treatment, you know.  It's bringing in broader economic forces and, you know, looking for better equity and care.

JAY CARRUTHERS:  Yeah.  I mean, I think we've really struggled, particularly with transition care where the regulatory -- you know, it's kind of the doughnut hole, financing and regulatory.  You know, just to do follow-up phone calls post-discharge has really been challenging.  You know, we're seeing like 10 percent -- maybe 10 percent they're able to actually connect and answer the phone, and they're not getting paid for any of that.  They're making twenty calls and they'll connect to one, so that's been really challenging.

GREGORY AARONS:  Thank you.  Dr. O'Connor and then Dr. Jobes?

STEPHEN O'CONNOR:  Okay.  Thank you.  It's been a great presentation.  I'm struck by a unique component of your model which was technical assistance, and I was wondering if you could say a little bit more about that.  I see Dr. Brenner is still part of the workshop.  I'm not sure if she's available.  But, you know, I think about what they provide at the Rocky Mountain Myrick in terms of almost on-demand consultation.  If you were working with a suicidal veteran, you can call and you can talk to an expert, and they can help you conceptualize what's happening for that person and how to help them.  And I was just wondering if you could explain a little bit more about how this technical assistance worked in your program at the state level.  Is that more for the organizations or is that for the clinicians?  And I would also welcome, you know, Lisa, if you wanted to touch upon this as well.

BARBARA STANLEY:  Well, so I can say a little bit about this, and maybe, Christa, you might want to add more.  But the technical assistance was amazing actually, and this was part of what was built from the quality improvement group within the state.  And technical assistance came from two implementation people who were available all the time to sites, and it was mostly to site leadership, not to -- not direct to clinician kind of thing.

We have thought about gosh, you know, could we -- could we provide like a console -- like an on-call consultation service for clinicians in New York State who have questions about suicide prevention for their clients or thorny issues.  That wasn't part of this, but it was -- it had a lot to do with just how to get this program up and running if there were issues with what does it mean to do this kind of assessment.  That was the kind of technical assistance we provided to sites.

CHRISTA LABOULIERE:  I think we were also very responsive.  You know, like, we wanted to make sure that the data we were collecting from providers and from site champions subsequently informed our learning collaboratives.  So we did not have, you know, the sort of thing where a clinician could call us up on-call if they were working with a suicidal client because we had, you know, over 5,000 clinicians in this project.  But we did have a site champion at each site that supposedly should have been able to be in this role as part of this supervision, and if they had questions, then they could in turn come to us.

And so a lot of times, subsequent sessions of learning collaboratives were covering particularly challenging cases, or this was an issue that comes up in our clinic a lot, how do other places handle this.  And so I think, like, that was a really nice synergy that developed in our learning collaboratives was folks not only turning to experts in the field for information, but also being able to help each other and share, you know, their successes and their challenges.  And I think that that was a really nice, unique experience for many of them.

GREGORY AARONS:  Great.  Thank you. Dr. Jobes, your question?

DAVID JOBES:  So I really appreciate hearing about this project because I've known about it for a while.  Jay, it was really cool to hear the follow up.  The thing that I'm reflecting on sort of at the end of the day is I'm pretty discouraged, frankly, about systems of care, about clinicians' willingness to change and learn and grow.  And the thing I'm preoccupied a lot in the research that we do is thoughts of suicide, and that's 12 million American adults and another three million teenagers.  So that's 15 million people with serious thoughts of suicide in the 30 days prior to the SAMHSA and CDC surveys, and we don't have enough clinicians.

And that's where these peer models or, like, Alternatives to Suicide, which is a peer-based group support, or lived experience are so compelling to me.  And 988 is coming like a freight train, you know, this three-digit lifeline number, and we already cannot handle the capacity of the -- of the calls to the lifeline.  Now, we make it simple by having it 988 and we're just going to be absolutely overrun.  The good news is lots of awareness, lots of excitement, lots of energy in that space, but no research about effective crisis response is on -- is on the drawing board.  And I just feel like we need a massive paraprofessional workforce, engage people with lived experienced and consumers as being part of the solution.  Thanks.

BARBARA STANLEY:  Dave, that's kind of what I was trying to say by using more -- engaging peers -- peer specialists more, but I think it can expand the paraprofessionals.  But the thing that we have to be careful about is that our professionals don't do that great a job working with suicidal people.  And the other thing I would say about this project is I don't feel the least bit discouraged about this.

One of the things that we did, we focused here on barriers, and where we can improve, and where we can do new training and all of that kind of stuff here.  But if I showed you some of the results, we went from a very low screening rate to almost 90 -- you know, almost full screening for suicide when people come into clinics.  We went from very few safety plans being done on admission for high-risk people to nearly 80 percent of people getting safety plans.  So we can implement Zero Suicide in this kind of, you know, herding cats type of environment.  So I feel very optimistic about being able to do this and being able to do better in outpatient care.

CHRISTA LABOULIERE:  And I have to just herald the clinicians that we had in our -- in our study, too, you know, like, working in very low-resourced settings, just doing amazing work, you know.  And yes, like, folks started out with high training need and, you know, not so great attitudes and knowledge, and then they blossomed.  They did a really good job.  And it was just kind of amazing to see this improve both quantitatively, but also qualitatively.  Like, folks who went out of their way to institute peer consultation programs or to join our State Suicide Prevention Conference to share their work, you know, like, just folks doing really, really valiant work, you know.

So, like, yes, are we understaffed and under resourced?  Yes.  But you also have to just, you know, send kudos to those folks that are still doing amazing work anyway.

JAY CARRUTHERS:  We really emphasize also to some of the provider systems, and I think Craig Bryan, you mentioned this in the workshop that you had with Barbara talking about safety plans and crisis response plans that -- you know, the follow-up rate is so low for -- and, Christa -- or, Barbara, you highlighted the engagement rate is low.  That's obviously not just for suicidal individuals, but for all mental health.  But we emphasized this may be -- this safety plan may be the only evidence-based intervention that these patients receive, so, you know, to really get them to focus and pay attention and really apply themselves in providing it.  So and I agree with you, Christa, that a lot of them really answer to that and, you know, met the challenge.

GREGORY AARONS:  We have time for just one more.  Dr. Bryan?

CRAIG BRYAN:  I guess I was -- had some thoughts, in many ways, I think, kind of echoing some of what Dave was getting at, which, you know, as I hear the discussion and thinking about sort of all these systemic barriers.  And I have often reflected, especially doing legal work on malpractice claims, consulting with healthcare agencies, I'm often struck how the answer that most systems institute often involve administrative tasks, more documentation more, screening.  So actually, Barbara, when you said, well, the increased screening may increase safety planning, I'm like, oh, that's easy stuff, right?  Of all the things we could be doing, you know, giving a -- giving a screener to patients who come into the ED is something that's, relatively speaking, low cost and easier to do.

But I wonder then sometimes are we almost like chasing our tails because, you know, clinicians don't have time to do high-quality care because they're often asked to do a lot more administrative tasks as a way to document that the care that we're providing, you know, it really is good enough.  And so there's more paperwork, more forms, which then interferes with the quality of care, and then things don't get any better, so then we impose more administrative requirements.  And I've noticed that it just seems like systems never remove strategies.  We don't, like, get rid of things like screening strategies or techniques or interventions.  We just add more on top which is -- I do think it crunches the time for the clinicians, again, and I just -- I don't know.

I don't know if it's a -- it's not a question.  I mean, I guess it's more just food for thought as we, you know, get close to get the end of the day and we have another day coming up, that maybe we're sort of doing this to ourselves in some respect because we just keep adding more and more things that clinicians are expected to do, but we're not creating the space for them to actually engage with patients and do high-quality interventions with good protocol fidelity.  And then we come back and we have workshops on why can't people do high-quality interventions, so.

GREGORY AARONS:  And I think that's the case for engaging, you know, policy and funding, and thinking through that entire chain.  So thank you, everyone.  We'll close the session with that.  Great session.  Thank you, all the presenters, and thank you for your questions.

STEPHEN O'CONNOR:  Yes, thank you very much.  Okay.  So we are in our fourth and final session for day one of our workshop, and we're going to be joined by DeQuincy Lezine and Shelby Rowe, who will be providing some helpful reflections on content from today.

DEQUINCY LEZINE:  Thank you, Stephen.  I'm going to go ahead and get started with this by talking about language because I think that language really matters, especially when we're doing clinical care.  I think the language we use with each other as professionals, it kind of reflects the way that we have conceptualized people, and the concepts in our heads can have implicit bias.  It can have stereotypes.  And then the language we use with our clients, it unintentionally impacts how they see themselves.  So we keep in mind that someone who is seeking care, if they likely already have a negative bias coming in, and if they're -- especially if they're at the point where they're considering suicide.  So they're already primed for interpreting things in ways that kind of line up with possibly a negative self-view or with self-judgment, and we're trying to counter that.  We often don't really have a lot of time to accomplish that, so all the words that we use really matter.

And so just kind of using some reality to clarify what I mean by this about how we phrase things. You know, it's one thing when you have problematic language that's filled with negative stereotypes and pejoratives that might get overheard by patients who are in the waiting room or patients who are in emergency department rooms or out in the hallways even, but there are also less obvious ways that language kind of impacts the efforts that we have in trying to provide help for people.

Some of the language problems, they come from the unintended connotations that happen because of the way that the terms are used outside of the suicide context.  So I'm going to reference some.  That Now Matters Now slide on language, we don't need to show it, but it's just shown up a couple of times during the day.  One thing that was on there was having died of suicide or died by suicide instead of saying "committed suicide."  And some folks have asked kind of why that is for committed.  Most of the time the clarification that's given is that "committed" seems like committing murder, and so it involves this kind of legal judgment aspect to it.  But I'm going to add that we also have this positive connotation to “commitment”, to being committed to something as being something that's socially desirable, something that's positive.  And we don't really want to have suicide within either that legalistic context of the same as being committing murder or that positive context of being something that we want people to commit to.

I'm going to add to that, though, “completed suicide” because living should not be the incomplete.  Living should make me feel more complete, not less, and if "completed" means death, we should just say "death."  The same with suicide versus successful attempt.  We like being complete.  I like being complete.  So do we really want death to be what we convey to people as completement, as being commitment, the completion, the success.  Meeting the standard for a negative outcome in the research context where death is the outcome that folks are looking for does not need to translate into the positive outcome that clinical care wants to achieve.  Similar is saying something is manipulative or a gesture.  We're passing judgment.  We're minimizing behavior.  We're making some kind of negative comparison between the person who is seen as serious, meaning they have more pure suicidal intent compared to somebody who might have more ambivalence.

And what it sets up is this objective view of medical severity being the same as somebody's subjective intent.  And it really undermines the reality that somebody could have significant intent to die, and they can change to a more lethal method which, for some people, the type of minimizing that happens when you say it's manipulative or a gesture pushes them more towards that lethal action because you're saying that they are not yet serious.  So focusing then on that actual behavior and exploring the inner experience makes a lot more sense.

Another thing that's on that slide is about saying that somebody is something, so they are borderline.  They are schizophrenic or they are bipolar.  And I'm going to add to that saying that somebody is a suicidal patient.  It makes it into something that's a fairly stable characteristic.  It dehumanizes the person.  It puts the person into a category of being the "other."  And I can understand that for a lot of clinicians, maybe they want that type of psychological distance to have somebody be considered "the other" and less of the same type of person that they are because it might make them feel more comfortable, make them feel more calm.  But the interactions that that clinician has is going to reflect that.  We have tons of research in psychology about in groups versus out groups and how that has a negative impact on the entire interaction for both people.  So we need to re-humanize the people who we're trying to help.

When we're working with somebody, somebody who currently has suicide risk, not that they are inherently at risk, but they currently have suicide risk, as somebody stated earlier, the idea of navigating the suicide risk instead of managing or controlling or monitoring it.  We're trying to help the person get through this suicidal period and also acknowledge that feeling suicidal is not something that needs to be completely done away with.  It is something that needs to be navigated and understood within the context of that person's life.  I'd also suggest we heard earlier "suicidal thoughts and feelings," and another way to say this would be experiences and suicidal intensity being something that's more inclusive, and, thus, it's more accurate instead of just saying suicide ideation.  It's not always ideation.  It's not always a cognitive thought process.

There are definitely also high-intensity feelings, and those can override somebody's anti-suicidal thoughts.  So there could be times where I'm thinking don't, don't, don't do it, don't die, but the feelings might kind of overwhelm that.  There are also some experiences that won't really qualify as thoughts or feelings, and I wanted to make that tangible for folks to try to understand.  There have been times when it was images or sent memories of where I had planned to die.  There are times where it has been the taste memory of putting something in my mouth that was metallic, or it has been the sensation memory of swallowing large quantities of something in a short amount of time.  So if we as experts can shift our language and different ways about suicidal experiences, then it serves as a reminder for us and for others about it being more than thought and about the people we're working with being humans, not being something that we have to push aside into some other category.

The language that we use is going to impact whether or not somebody is going to listen to the rest of what we have to say.  Listening to the guidance, or support, or therapeutic counseling that we're going to give can be turned off because of the language that we're using.  If somebody fails, like, you're not really understanding them or if they feel like you're not really taking them seriously as a human providing care for another person, it's going to impact whether or not they decide to go to that follow-up appointment, if they decide that they want to go to see you again or see any therapist again, or if they want to call the Lifeline again, or if they wanted to call your crisis center again.  That initial experience, particularly when somebody is feeling particularly hurt and particularly negative about the world, can have a powerful impact on the follow-up care that they might seek or choose not to seek, and it reflects the underlying attitudes that people have about others with lived experience and the attitudes that are in our health, and mental health and behavioral health systems, and Shelby is going to provide some further comments on this topic.  Thanks.

SHELBY ROWE:  Thanks, Quix.  You know, I feel that Dr. Lezine has spent time talking about language, and I want to, you know, close up the day talking a little bit about attitude because our attitude tends to steer our language, and then vice versa, the language we use reflects our attitude and oftentimes those unconscious biases that we may have.

I was really struck by one of the slides in Dr. Whiteside's presentation where, you know, nearly half of the folks that she surveyed that provide care say that they personally had thoughts of suicide, that they have that lived experience.  So we talk about that's so important, and so my curiosity falls in why do we make it unsafe for them to bring that to the forefront?  And I think that a big role in that is that stigma, the discrimination against individuals with mental health struggles, and specifically those who have experienced suicide intensity.

I think that we still struggle as a field in suicide prevention and in mental health of really having that patient-centered focus with our care.  I think, you know, there's a lot of encouraging growth.  I am a huge fan of the research and work of so many of our speakers today.  I could, you know, fan girl a lot of their names.  But, you know, we have so much opportunity for growth, which is a much nicer way of saying there's a lot of shortcomings, a lot of failings.  And shifting to patient-centered versus provider-centered, yes.  Getting new training is always going to be cumbersome, but if it's training that can help save the life of our patients, you know, what is in the best interest of our patients that we are serving, then it is worth it, you know, taking that extra time.

Our healthcare field and the last two years made a complete pivot in how they manage contagious virus, and lethality shifted how they staffed and operated healthcare facilities and staffing.  So it is possible for systems to make dramatic pivots when they know that it is lifesaving care.  And I think, you know, maybe bringing in more people with lived experience to help bring the emphasis of, you know, the good news is most of us are surviving, and I don't know.  I'm open to conversations with anyone.  How do we bring more awareness to that heroic commitment to recovery in the face of no hope when life circumstances are bad, when access to healthcare is minimal, when in the last year, individuals may have had three different therapists, and who may act like, you know, I would love to do this screening, but it would be really inconvenient, and if I do this other test, it's more billable hours, and shifting away from, you know, what's really going to help that provider clinic.

And I know that everyone needs to be paid, and that's a whole other conversation of why isn't it equal, on par with doing a screening for cancer, doing a screening for blood pressure and heart disease?  Our brains are just as healthy.  We need them to function really well.  Why aren't we making and demanding that insurance billable rights for doing screenings for suicidality are not getting compensated because they should.  But it's also, you know, lifesaving.

So I think sort of shifting our attitudes, and yet I can be on that soapbox all day, and I think Quix and I may have more to say.  But do we -- do we want to take care of people at risk for suicide?  I say that knowing that there's a resounding "yes" from our speaker panelists, and we have a lot of great champions in our field that of course we want to.  But how do we change, you know, to really practice, to do it like it we mean it, to train new clinicians like we want them to be effective at saving lives?  How do we equip our healthcare facilities, our mental healthcare facilities so that they're in the best place to save lives, because individuals at risk for suicide, their lives are worth saving, and we know that most of them are surviving.  And by improving the quality of care through things like the safety -- effective safety planning, doing good screening, collaborative assessment and management of suicidality, dialectical behavior therapy, we know there are things that work.  Let's demand excellence in our field so that more lives could be saved.  Quix, other thoughts?

DEQUINCY LEZINE:  No, you hit it well.  Thanks.

SHELBY ROWE:  Any thoughts or comments from any of our other speakers from today or Dr. O'Connor?  Does anyone have questions for us?

STEPHEN O'CONNOR:  I don't have -- I don't have a question in mind, in particular, but I did want to just sort of reflect how much your inclusion in this conversation means to overall benefit of the workshop, and it really reflects the values of NIMH, in particular, the Division of Services and Intervention Research where we, you know, really place a high value on stakeholder engagement.  And we're really looking for sustainable, acceptable approaches to addressing mental health needs.

And I think that, you know, just having you part of the conversation throughout the day and, you know, I guess demonstrating equality and kind of an egalitarian approach here, in some ways, I think, it's kind of modeling, you know, what it is that we're hoping to do as well, you know, as clinicians when we're working with people who have suicide intensity, is kind of saying, you know, I'm here.  I want to help.  I'm willing to hear the feedback.  I'm open to doing whatever we need to do together, you know, to achieve this shared goal.  So I so appreciate the comments that you all have provided.

You know, one thing that kind of struck me over the course of the day is that how connected each of the different sessions have been.  There is synergy here.  We absolutely need to understand what the core competencies are, where those can be facilitated, thinking about organizational factors that -- across the spectrum of suicide prevention from upstream to postvention that impact the clinical care workforce.

And then we've had this massive implementation study that's happened throughout New York State because we have to go ahead and we have to solve these issues now.  Even though we might not have all of the best approaches, you know, we need to go ahead and study things so that we can generate more data, and then we can continue to iterate and improve upon this in the future.

So I was just sort of reflecting on that, and I'm hoping that the future work that happens in this space will really utilize the expertise of lived experience, consultant investigators to really be part of the research design and help figure out acceptable and feasible solutions.

SHELBY ROWE:  I appreciate that, Dr. O'Connor.  And, you know, one thing is just really recognizing individuals with lived experience and individuals who are receiving care as being those active partners in recovery and treatment, that they are not -- that they are carrying the lion's share of the burden of that recovery, and that they are committed to recovery, and that as their healthcare providers, you know, yes, it's a big ask, and there's a lot that can be done.  But being supportive of their recovery and knowing that they are walking that path of recovery, and as those treatment providers, we're helping and assisting and providing guidance on their recovery journey and helping to support that, and that they are an active partner.

I think sometimes because of the acuity when we often see people in our emergency departments or walking into an inpatient unit, of forgetting their ability to recover and thrive, and reminding ourselves that they can recover and that most will, and that we  -- you know, and being there, yes, it is intense, and, you know, that vicarious trauma.  But there's also that vicarious hope that you can gain from your suicidal clients.

DAVID JOBES:  Well, it feels, for me, a sort of synergistic moment.  There was a group that Julie Cyril and Desiree Stage and Justolman Raney got together with a PCORI grant that brought researchers and people with lived experienced together.  And to me it was transformative, not because it necessarily produced the perfect recommendations for research, but it just took me out of my seat of presuming to know for other people what they would, of course, want in terms of a treatment, and also even questioning the relationship to life and death.  And it sounds kind of corny to say that, but not presuming that for every single person, getting rid of suicide is necessarily their number one agenda.

And what I mean by that is, it is -- it is a relationship of sorts, and for some people, getting rid of suicide is a really daunting, scary thing, and for others it's something that they're desperate to find.  And so I feel like as a researcher, my heart has been open with humility as much as I can bear it, you know, bring it to really learn from -- directly from people with lived experience about those perspectives.  And so I really applauded that effort because the goal was to create research guidelines that are informed by this perspective.  And I think you've seen with DeQuincy and Shelby, I mean, this is just such a valuable perspective that we for many years didn't even think to include.  And so I'm just grateful for that.

STEPHEN O'CONNOR:  Okay.  Well, thank you for sharing that, Dave, and thank you, Quix and Shelby, for excellent reflections and feedback on our first day of the workshop.  So we have five minutes left.  What I'm going to do is just wrap up here, and we will end on time at 4:00 P.M.

So again, I want to thank all of the presenters and moderators for excellent presentations, really thought stimulating discussions.  Thank you to the audience for your participation today throughout.  We could feel the energy and excitement and appreciate the questions that you posed as well.  So this is a two-day workshop.  The second day occurs on Monday, November 8, and, again, it will take place between 11:00 A.M. and 4:00 P.M. Eastern Time.  It will be new content.  We have whole new set of sessions, some of which touch upon issues that were addressed today, including kind of the pros and cons of utilizing service-ready tools for suicide prevention.

We're going to hear from people such as Dave Atkins who Greg Aarons mentioned earlier, Craig Bryan, a number of other people who attended today that will talk about the topic, who are going to speak about barriers and facilitators around training clinicians in suicide prevention best practices, including considerations when working with specific groups of individuals.  And then we're going to hear from several thought leaders in this field about potentially paradigm-shifting opportunities related to advancing clinical care in suicide prevention.

So we look forward to seeing you all again next week on Monday, and I wish you all the best.