School-Based Suicide Prevention: Promising Approaches and Opportunities for Research

Transcript

BELINDA SIMS: Good afternoon. Welcome to the webinar on school-based suicide prevention, promising approaches, and opportunities for research. I am Belinda Sims, a program official at the National Institute of Mental Health and moderator of today’s webinar.

Before we begin, I will review the housekeeping notes for this webinar. Participants have entered the meeting muted, in listen-only mode, and your cameras are disabled. For more information on today's speakers, we invite you to read their biographies on the event registration website. Please submit your questions via the Q&A box at any time during the presentation. Be sure to address your question to the speaker you'd like to respond. If you have technical difficulties hearing or viewing the webinar, please note these in the Q&A box and our technicians will work to fix the problem. You can also send an email to events@1sourceevents.com. This webinar will be recorded and posted to the NIMH website for later viewing.

The goal for today's webinar is to provide information on innovative practices in school-based suicide prevention. To start off the webinar, we will hear from Lynsay Ayer, a suicide prevention consultant with NIMH who will provide a brief overview to set the stage. Following the overview, we will hear about ongoing school-based suicide prevention activities in four school districts. Our first group of speakers from Michigan will present on activities in Detroit City Schools and Washtenaw County Public Schools. The speakers are Natalie Rodriguez-Quintana, Jill Paladino, and Robin Jacob. Next, we'll hear about activities in the Boston Public School System in Massachusetts. Our speakers are Shella Dennery and Andria Amador. The last presentation focuses on work going on in Baltimore City Public Schools in Maryland, and our speakers are Stacey Davis, Rebecca Lee, Akil Hamm, and Patricia Roberts-Rose.

Please remember to submit your questions via the Q&A box at any time during the presentations and be sure to address your question to the speaker you'd like to respond. We will now hear from Dr. Lynsay Ayer.

LYNSAY AYER: Thank you, Belinda. Before we get started, I just want to remind everyone that if you or someone you know is having thoughts about suicide, help is available. Here are the numbers for the National Suicide Prevention Lifeline [800-273-8255] and the crisis text line [Text HOME to 741741].

Youth suicide has been an increasing concern in recent years, and studies suggest that the pandemic has probably exacerbated risk for suicide among certain groups of youth. As just one example, here are some recent data from a study of emergency department visits for suicide attempts among youth. The top graph displays trends for girls and the bottom graph is for boys.

I'd just like to call your attention to the solid line, which represents emergency department visits for suicide attempts in 2021 through May. You can see in the top graph that that solid line is significantly elevated compared to prior years' trend for girls, which we see is substantially elevated, whereas in the graph for boys, we don't see a substantial difference so far in E.D. visits for suicide attempts.

That's just one study among several that show that this is an increasing concern. The government, the media, and the health care sectors have all recognized this and called for more attention to youth mental health and suicide risk in particular. And within each of these calls - just some examples here on the slide - there is a real emphasis on the importance of schools as a setting for delivering prevention and intervention programs.

Simultaneously, we've seen a lot of increased spending from the federal, state, and local governments on school-based mental health and suicide prevention efforts. So, as a result, potentially schools have a unique opportunity to implement new and innovative suicide prevention programs with more funding than they would normally have. However, this is also a time when schools are facing truly unprecedented challenges.

I just want to take a moment to underscore some of the reasons why schools are seen as key to youth suicide prevention. First, school is a very common place for youth to seek mental health support. Recent data show that about the same proportion of kids get their mental health treatment in schools as from specialty mental health settings.

Furthermore, we know from studies that leaving mental health and suicide prevention in the domain of parents and caregivers really risks missing potentially the majority of kids at risk. For example, one recent study showed that of nine- and 10-year-olds who reported a prior suicide attempt, only about 12 percent of their caregivers reported knowing about the attempt.

School settings can also help to make suicide prevention much more accessible by providing programing and services in the place where kids spend most of their time, and often there are reduced logistical barriers like transportation and childcare demands and reduced stigma. That said, there are some big gaps in what we know about how to effectively prevent suicide in schools, and research is really desperately needed, particularly now when schools are potentially able to invest more in suicide prevention due to increases in funding.

However, suicide prevention can seem daunting to schools. And schools, as we all know, are dealing with a lot, a whole range of stressful and overwhelming decisions and capacity issues even more than usual. Also, forever, collecting data in schools has been challenging, and that may be even more the case now that schools are toggling between remote and in-person and hybrid.

For today's webinar, we hope to just bring folks together to discuss some of these issues and think about ways to improve the science and our overall knowledge about how to prevent youth suicide. We'll hear about three very interesting and innovative examples of how schools have developed suicide prevention efforts, and also how they've integrated data collection and analysis in creative ways, as well as some of the challenges they face in all of these tasks.

We recognize that today we will probably only scratch the surface of this topic in the time we have, but we hope that through the discussion, we will spur some new ideas and collaborations that can help to advance the science on suicide prevention in schools. With that, I will turn it over to Natalie and her Michigan team.

NATALIE RODRIGUEZ-QUINTANA: Hi, everyone. Thank you so much for joining us today. My name is Natalie. I'm part of the TRAILS team. I'm going to be joined today by my co-presenters Jill Paladino and Dr. Robin Jacob from the Youth Policy Lab of the University of Michigan. Jill works with me at TRAILS. We're here to discuss the suicide risk referral and communication tool that we have developed at TRAILS.

The first thing we really wanted to do was to make sure that we show gratitude to our partners because without them, our work would not be possible. This is going to be the agenda for today. My hope is to be able to provide a brief introduction to TRAILS, and then Jill will discuss the tool and its use with one of our partner districts. Then, Robin will talk about the program evaluation pieces and some next steps.

Everything at TRAILS is grounded in evidence-based practice. TRAILS is an acronym. It stands for transforming research into action to improve the lives of students. Our goal is that what we learn from empirical research and best practices, we take it and then we embed that to the school environment where students can access it.

Our two main theoretical approaches are cognitive behavioral therapy and mindfulness. And probably, as some of you know, these have strong empirical support for most common mental health concerns like depression and anxiety, and they're also appropriate for the school setting. We have found that typically, school administrative staff like it because it's about teaching skills and strategies. Students can learn them, pick them up, and use them independently. They also have an impact on the outcomes that are really important for the educational environment.

Our model is very much guided by implementation science and what the literature tells us about dissemination and how to implement new practices in new settings. We know that it doesn't work to have a one-time professional development training and expect professionals to use these skills that they learned. We know that most people, especially in schools and especially during the period of COVID, they're overwhelmed or underpaid. They're so busy. We have social workers, counselors, teachers, psychologists, etc. And we really can't expect that of them to go to a training, give them a binder, and tell them to go do this work.

So, our model at TRAILS is that we provide that training in evidence-based practices, which comes with an extensive library of different kinds of resources that are hosted up on our website, trailstowellness.org. We also have a coaching and consultation model, as well as some other implementation supports to help them through this implementation process, all with the goal of sustainment in mind. Really, our model is designed to build local capacity and strengthen partnerships.

Our program has been built over the past eight to ten years and has tried to be really responsive to the needs of the schools. We divide our programming into three tiers. Tier 1 is all about promoting wellness for students and staff - including educators - and creating a climate of wellbeing. Tier 2 is our early intervention tier. That's where we have training for school mental health professionals within schools to support students impacted by mental health concerns.

Then Tier 3, which is the most relevant for today, offers training to mental health professionals that interact most with the students who are at high risk of serious self-harm or suicide. And the focus is really on developing systems in place to be able to identify at-risk students and facilitate either intervention or coordination of care, and we're going to be focusing on a tool today.

I'll let Jill take over now so that she can make sure to talk about that referral and communication tool that we've developed. Thank you.

JILL PALADINO: Thanks so much, Natalie. I'd like to spend a moment just kind of contextualizing some of the challenges that we do see around suicide risk management for students before we talk about what we've been up to in Washtenaw County. And I think Lynsay's comments at the top of the call today also help to echo some of the things I'm going to reinforce here.

We see that schools and districts may have protocols for students experiencing suicidal thoughts, but the consistency with those protocols and utilization of those could be low and vary by school to school. There can also be limited use of screening tools and that can result in sometimes unnecessary referrals to the emergency department and poor communication and care coordination between providers.

If we kind of dive into Ann Arbor Public Schools in Washtenaw, Michigan, they had a school year in 2018 where nine students actually died in that community. On top of that, they were experiencing all of the challenges we're mentioning today.

So, it really left school professionals kind of stuck in this space of referring students for emergency care and getting frustrated by the lack of communication and coordination while they were trying to really advocate and protect those students that they cared so much about. This really frustrating situation actually ended up culminating in a work group of a number of different stakeholders - including the intermediate school district, the school district of Ann Arbor Public Schools, our psychiatric emergency services department, our county mental health agency, and TRAILS - all coming together to discuss this issue further.

The work of that work group kind of culminated in this referral and risk management tool, the goals of which are to foster bi-directional communication between school professionals and emergency department staff, to support care coordination for students, also to improve student mental health treatment, and then try to really drill down into ensuring that the students being referred to that emergency department really do need emergency care. And for those other students, they're hopefully getting connected to more appropriate care. They're not just being boarded at the ED where we already know there are really long wait times and it's a really difficult space to be in.

In this process, TRAILS provides training and access to relevant materials. I'm not going to dive into our training content today too much, but I will say that we do utilize an evidence-based suicide risk screener, the Columbia-Suicide Severity Rating Scale. We chose that tool both because it's available at no cost, it's fairly simple to administer, and also it aligned with what our emergency department in our local community was already utilizing. So, we felt like it was a good tool to give to school professionals for screening students at risk of suicide that also aligned with community support.

On this screen, you see a very small version of our protocol. On the left is the cover sheet that walks through all the steps that a school professional should take. I'll show you close up of that in just a moment. On the right, you can see that there is the referral worksheet, and that has some information we'll dive into a little bit deeper.

Looking at the cover page of the protocol, really what we're asking school professionals to do is to, if they need it, contact our community mental health crisis team. It's a 24-hour line where they can seek consultation support on the case should they need additional support.

We're asking them to do a couple of other steps. We've modified this so it can be appropriate for virtual delivery as well, with students going in and out of in-person learning over the past couple of school years.  And then complete the Columbia. If warranted, then they would go on to complete the Psychiatric Emergency Services Referral Worksheet to actually refer that student to PES, contact the family as needed and establish transportation plans, and then make sure that they are communicating that referral form to our psychiatric emergency services department.

Then I'd like to show you a little bit of a close up of what the actual referral form looks like. First, you'll notice at the top that we are, again, reinforcing for school mental health partners that this referral should be coupled with a completed suicide screening assessment tool. Then you'll see that the school professional completing this form would fill out some important information about the student.

But primarily, they're sharing what is really driving this referral today. What is the context that an emergency department staff member would need to know when they are then meeting with and evaluating that student later? Then, the school professional would include their contact information for additional follow up and conversation as needed.

There is a space on the form for parent authorization of this information. Then, this is the communication that schools can expect to receive back from the emergency department. Really, we're outlining what is the plan of care and what are the next steps for this student? Are they being admitted? Are they being connected with a community-based provider? When should we expect to see them in class, and so forth? Again, the contact information for that staff member in our emergency department is also provided should additional conversation be required.

Ultimately, the goal of this process is to, instead of referring all of those students to the emergency department that may be experiencing suicidal ideation and may need additional supports, pausing to help screen and then use that protocol so that there are referrals to the emergency department, but those referrals are likely fewer, they're more appropriate, and hopefully resulting with better coordination. Then, a secondary outcome being that other referrals are going to other, more appropriate local supports.

Through this process, we have trained 183 school mental health professionals. We've received some really positive feedback so far. I have to say, this work really came out of an organic community need when school staff were kind of just raising their hand in the community and saying, we need some help on this front. So, evaluation did take a little bit of a backseat to really making sure that we were being a responsive community partner.

We have conducted some surveys with trained staff members so far, and that pilot data has shown that 85 percent of staff have utilized the Columbia and the referral tool in their most recent referral to the emergency department. Then for students who have been referred to the emergency department, we've seen the admission rate increase from about 5 percent before this tool was utilized to about 29 percent, and we're using that admission rate as a proxy for the appropriateness of the referral. So, we're interpreting that information to say that the referrals have gotten more appropriate over time.

Also of note, 36 percent of staff have indicated that they've received a follow up response from our emergency department in this process. Next, I will turn it over to Robin Jacob to discuss evaluation.

ROBIN JACOB: Great. Thank you so much, Jill. I'm just going to wrap up here by talking a little bit about our plans for evaluation moving forward. We have a unique opportunity right now in Michigan because TRAILS is being expanded throughout the state over the next several years. So, we have plans to do rigorous evaluation of the TRAILS program as that roll out happens.

We are planning to randomly assign approximately 60 schools as this roll out starts to either implement the TRAILS programming or to an as-is control condition. The as-is control condition will later be able to implement TRAILS programming as the roll out increases, but during that first year, we'll have 30 schools that are implementing TRAILS programming and 30 that are not, and we're going to compare outcomes in those two sets of schools.

One of the primary ways in which we're going to collect data is through surveys that we will be conducting in the schools in both the fall and the spring of that implementation year. We're planning to collect surveys from students, from teachers, and from school mental health professionals.

To evaluate this Tier 3 intervention, we plan to add items to the school mental health professionals survey and ask specifically about the use of the PES communication tool. So, we were planning, hoping to ask things like the school year, how many students you referred to an outside provider or to a hospital ED because of thoughts of suicide? How many students were referred to an outside mental health provider? Was there follow up from the receiving provider after the referral? And how satisfied were you with the treatment that these students received? Then we plan to ask a similar set of questions about how many students were referred to a hospital ED.

We will then ask them to indicate how many known non-fatal suicide attempts there were among students during the school year and how many students at their school died of suicide this year. This will allow us to look at the overall impact of the tool and make refinements to it as we move forward.

I'm going to wrap up. There's a lot still left to do in this work. We're still at the very beginning stages of this. Our first step is to expand the tool across Washtenaw County middle and high schools, not just to Ann Arbor. We're planning to modify and implement a similar version of this in the Detroit Public Schools in the coming school year.

Then, we really want to think about ways to make this a scalable program. And the way we're going to do that is by developing a program implementation guide that can be readily used to foster replication throughout communities throughout the state. We are going to be partnering with colleagues at U of M to - that's me - to develop a more robust evaluation of this, as I just talked about. Then, we're super excited because there is a new Tier 3 lead that will be joining the TRAILS program that is really going to help lead the way in expanding suicide prevention and intervention across the state.

I will stop there. Thank you all so much, and we'll turn it over to the next set of speakers.

ANDRIA AMADOR: Hello, my name is Andria Amador. I'm the senior director of behavioral health for the Boston Public Schools, and I'm joined by my colleague, Dr. Shella Dennery, from Boston Children's Hospital. We're here to talk to you about the suicide prevention that we've integrated into our comprehensive behavioral health model. We have our contact information. We welcome anyone to reach out to us, and we enjoy collaborating with colleagues around the country.

We'll be talking a little bit about the programs that we run together and our individual departments to set the context for how we've collaborated effectively to ensure that we're addressing suicide prevention and intervention. I'll first start talking about the Boston Public Schools Behavioral Health Services Department, which is the department I have the pleasure of leading.

Our role is to provide a continuum of behavioral health services across all tiers from Tier 1, universally, how do school psychologists work to develop safe and supportive schools that meet the needs of all students, including the behavioral health needs. At Tier 2, how are the school psychologist and our community partners providing at-risk support for targeted students that are showing early indication of need? And at Tier 3, we're doing that traditional work many folks are familiar with of the school psychologist doing crisis support, individual counseling, and assessments.

Our team is comprised of 2 administrators, 2 secretarial staff, and 83 school psychologists who cover 124 schools with a little over 50,000 students. It's a very dedicated but hard working and busy group of folks.

We firmly believe in Boston Public Schools and in our Behavioral Health Services team that every student deserves a safe and supportive school. And the reason we built the model that we're here to talk to you about is because we realize that in traditional roles of mental health where you're only helping kids that come with high levels of need, we weren't helping the majority of students to ensure that they had an array of interventions to meet their needs, that their needs were being identified and addressed early.

So, we developed a comprehensive behavioral health model in partnership with Dr. Dennery at Children's Hospital, as well as Dr. Melissa Pearrow in UMass Boston. And together, these three organizations created the comprehensive behavioral health model, which looks at providing an array of interventions. We call it multi-tier system support, but public health models are quite used to that tiered intervention.

One of our key components is a universal screener. We'll talk a little bit more about that, but that's one of our big tools in our suicide prevention because it specifically has questions that are screening children for suicidal ideation and risk. We have been working together for a long time. We started developing the model in 2010, and we're currently in over 78 of the 124 BPS schools serving 33,000 of our students.

One of the aspects of the model that we think is important to tell this group is that we have a research team. It's not typical to have an in-district research team, but we have done that with the financial support of Children's Hospital, and the research team is comprised of behavioral health services staff: doctoral level school psychologists, staff from Dr. Dennery's Department - and she'll talk also about that - faculty from UMass Boston, faculty from Boston College, and faculty from Northeastern. Together we have been effective at bridging that research to practice gap by really making a goal of having student-centered research that's integrated and mutually beneficial.

One of the challenges that we're all aware of is, universities having research goals that aren't aligned to what the district's trying to accomplish. So, we made a commitment from the beginning of this research team that we would do things that kept students first. That has proven very effective and has resulted in several peer reviewed publications. It's led to several presentations, but most importantly, it's really led to changes in how we service kids because we're using that student-driven research.

Another thing we wanted to tell you about is our Boston School-based Behavioral Health Collaborative. We know that addressing the behavioral health needs can't be done just by a school district. Much like our colleagues that just spoke said, it really needs to be a community commitment to the behavioral health of our kids. So, we have a collaborative that's comprised of more than 20 community agencies that work in our schools, as well as allied city and state agencies.

We come together once a month and we talk about an array of things, such as evidence-based practices for mental health, standards of practice of how clinicians can work in schools effectively with school-based staff, emergency response protocols. We do joint training with the staff of Boston Public Schools, as well as the staff of our community partners. That training happens all year long, but we have a big conference that pulls them together. That's to ensure that everyone has quality, evidence-based practices to use for their kids, but also that community partners and school-based people are forming collaborations in the best interest of kids.

I mentioned our universal screening because it is really key to what we're doing. When we first started, only 3 percent of schools in the nation did a universal behavioral health screening. Now it's around 12 percent of schools in the nation are doing universal behavioral health screening. A lot of COVID recovery plans at different states are asking for a universal screening. We have believed in it for years because we find it as the best way to identify behavioral health needs early instead of waiting for a behavioral health need to become a high level of need, high levels of care.

How do we find kids in need earlier? We use the BIMAS-2, the Behavioral Intervention Monitoring Assessment System. It allows us to look at behavioral concern scales or things that are negative and need to be improved. It also allows us to look at adaptive scales, skills that students have that we want to keep improving and growing, such as social skills.

And an important thing for suicide prevention for this screening tool is it does have a question that talks specifically about suicidal ideation. When a student positively endorses that question, it immediately sends an alert to both the school psychologist and the administrator so they can follow up.

When we do the screening in our schools in the fall and spring, we have a student support team ready to respond to all the positive endorsements of ideation, and we do a risk assessment that day with students. It's always really remarkable to find students that are struggling, that teachers are shocked by, and that teachers would not have realized there was a student in need or suicidal if it weren't for the screening. So, we found it a very powerful suicide prevention intervention.

In addition to the screening, we do lots of direct support, such as we use Signs of Suicide, which is a training used in middle and high schools to teach students about the science of suicide and where they can go get help for themselves or their peers. We use that in conjunction with Break Free from Depression, which is developed by Children's Hospital, and Shella will also talk about that. Our staff have been trained in CBITS, so cognitive behavioral intervention for trauma. Coping Cat, which is another CBT intervention. Check in, check out, check and connect, and counseling.

What we're really trying to do is not just identify a need and get kids into counseling, but make sure that when they're getting counseling, they're actually getting an evidence-based intervention to meet their mental health needs. I'm going to pass the mic to Dr. Shella Dennery from Boston Children's Hospital to talk about their work.

SHELLA DENNERY: Hello, everybody. It's great to be here this afternoon. We know it's an extraordinary time in the world, but specifically also in schools. And it is a Friday afternoon before a long weekend, so I'm so impressed that you are all here. I am Dr. Shella Dennery and I'm the director of BCHNP.

We are a school-based program that partners with the Boston Public Schools. We have been partners for 19 years. We have a team of 16 social workers and psychologists. We also are really fortunate to have a research and evaluation team. We have a number of initiatives going on, but because of time, today I'm only going to speak with you about two of them.

The first one is called Break Free from Depression, and I'm going to put the link to the curriculum in the chat [https://www.childrenshospital.org/breakfree]. It is a curriculum that we have been working on for well over a decade. I'd first like to acknowledge the colleagues that worked with me on this project: Nadja Reilly, Vanessa Prosper, Molly Jordan and Karen Caparo. We know that this curriculum has reached at least 20,000 youth. About five years ago we sort of stopped tracking that, and we did put all of our training online.

We just checked this morning, and 1,300 school staff have participated in our online training. We do a lot of program evaluation to sort of ensure that we have created something that is important and is working in schools. We are working on a publication as well and it is almost ready to be submitted.

But we developed this curriculum, and there's three goals. The first was to increase adolescents' awareness about depression and suicide, to teach them how to recognize it in themselves and in others, and in their friends and in their family. Then, also to give them strategies for finding help. It's a free curriculum. It is online. It's four sessions. It's for high school students.

My favorite part about it is there's a compelling documentary with it that really is adolescents talking about their real-life struggles. They're talking about their symptoms, their course of illness, and ways that they have coped with sort of managing depression through their own words, which we find incredibly compelling when working with high school students. They don't necessarily want to be hearing from adults, and so, really having the peer-to-peer delivery has been really important for us.

One of the things that makes this curriculum a little bit different - it is designed to be delivered by school staff with training, with supports. We went back and forth about this for a really long time. We know and we've looked at a lot of other prevention curriculum in schools that when you're talking about depression and suicide specifically, teens want to talk to someone that they know - a trusted adult, someone that they're familiar with - and not necessarily someone from the outside. That was something that we felt really strongly about.

Our online platform has the student curriculum. It also has the teacher training. It has parents' resources as well, and then we also have the program evaluation supports and supplemental resources. We usually recommend that a school does the entire 9th grade, for example, or the entire 10th grade in a school year. So, that's really sort of the heart of it. The documentary, which I already said is, I think, sort of the core of it, is 33 minutes long and there really, truly is nothing like having kids talk about depression to their peers and their experiences.

I think the main goal is really to normalize help seeking behaviors for kids as well. So, at the end of each session, each of the four sessions, there is a student request form and kids fill it out. And the three things on it say, I would like to talk to someone right now, I would like to set up an appointment to talk to someone, or I'm doing just fine. That's the kids' exit ticket when they finish that session of the curriculum. So, we monitor kids throughout. We also have lots of areas for discussion and breakout groups, and so the kids have abilities to sort of ask questions and sort of voice their wonderings as well.

Obviously, a curriculum like this has to be really planful and done with the mental health team within a school - the student support team - and really having everyone sort of know that it's happening, including parents as well, because sometimes it can be triggering. And like Andria just shared, there are often kids that are not on the radar that we end up finding are asking for help or are wanting to talk to someone soon after as well.

We have done a lot of culturally responsive adaptations to this curriculum. We work in Boston Public Schools, which is 85 percent youth of color. So really, we've been thoughtful about the approach. We've definitely taken the documentary and put it into more bite sized clips for students depending on their developmental stage and age. Also, we've done a great deal of work trying to ensure that we can do this remotely as well. So, we've taught Break Free from Depression over Zoom and learned a great deal about that as well.

The only other thing I'm going to say about it is in the sort of evaluation of it, some of the most beautiful things are always the qualitative things that kids say. They say that they've never talked about depression and suicide in this way before. But more importantly, in the quantitative aspects, we're always asking kids, would you recommend this to a peer? And you know that teenagers often are really tough critics. 83 percent of the kids that we've surveyed over time have said they would recommend this to a friend. So for us, that feels like the biggest win, even though we also have some other data that shows that this is effective and responsive as well.

That's Break Free from Depression. I think the most important thing is really thinking about how you integrate suicide prevention curriculum into other initiatives within schools. As we know, especially right now, there isn't space and room to do something additional or something more. So, we're always thinking about health and wellness. We're thinking about school nurses. We're thinking about health teachers. How do we also think about the broader curriculum and how these topics can be integrated and added as well?

The only thing I would say about lessons learned, really working with a large district and a large hospital, like the IRB process, for us to do a study of this and all of those other things that we'll hopefully have a little bit of time to talk about in just a moment as well. But the last thing I'm going to share is, we did Break Free from Depression over a decade, and we heard from educators, this was great. This was fantastic. Some of them were really hesitant to teach it. We would give them the support.

But simultaneously we started surveying teachers about their mental health knowledge and we learned from 500 educators in Boston that they don't feel comfortable or confident even in their knowledge in SEL - social emotional learning - and behavioral health, and that they've had little training. So, we developed a project to sort of really focus on professional development for teachers that's more foundational, and that would allow them to feel more comfortable and then doing a curriculum on depression and suicide as well.

Up here on the screen now are the topics. There's the link there - I'll put it in the chat in just a moment - but these are also available on the web [childrenshospital.org/TAPonline].  They are open source. They're free. We have over 5,000 educators around the nation sort of taking this course. We don't want educators to become mental health clinicians or counselors, but we know that kids are coming into school with way more in their backpack than just their academic work. So, how do they feel more comfortable and confident addressing what kids are coming in with as well?

I wanted to give credit to my team and Molly Jordan, and we have TAP consultants working on this every day. We also have anxiety webinars. We have a documentary about anxiety for families that we are really happy to share, and we're working on a podcast about educator resilience and sort of, how do we keep teachers in the field? Because we know we're also seeing a lot of educators are unfortunately struggling staying in the field of education at this time. That's TAP. Then, I'll turn over to Andria. I think we have one or two more minutes to talk about lessons learned.

ANDRIA AMADOR: Thank you, Shella. I think we have just one minute to say, we've learned many things in this long partnership between these three organizations. One is that we developed goals and outcomes together jointly and they guide the work we do. We don't do something because someone's interested, it's their research pet project, it's something their grad students want to do.

We do things that our students need, and our educators need, and we do a lot of stakeholder feedback to understand that. Then, once we understand that we make sure that we are designing projects that are culturally and linguistically sustaining, that represent our children and our community. We develop joint IRB processes and have that joint research team. We also have a plan to continue funding, and we meet monthly as a leadership team to make sure this work continues. We've had great outcomes because of that. We've had improved academic outcomes in our CBHM schools, increased positive behavior, decreases in negative behaviors and fragmented support, and less time on Tier 3 when we're spending more time on Tier 1 and Tier 2.

Again, we're sharing our information. Both of us have websites that capture this work. We'd love you to check out our resources and ask us any questions you have, and we're honored to participate today. Thank you very much, and I'm going to pass the baton to our peers from the Baltimore Public Schools. Stacey, take it away.

STACEY DAVIS: Thank you so much. Thanks everyone for being here this afternoon. We're going to talk a little bit about a tool that we're using in Baltimore City to try and identify kids that are at risk.

I'm going to skip through the agenda really quickly, but you'll see the team. Unfortunately, Chief Hamm was unable to be here today, nor was Dr. Wilcox, but we will make sure that we are covering their information for them.

Just a little bit of background and history for how we became involved in using GoGuardian in Baltimore City. GoGuardian is a classroom management software with student safety features. We originally brought it in - and I'll talk more about that in a second - to help with classroom management as we bought Chromebooks.

The tool itself operates at the browser level and kids log into their city school's account with their username and password. Then, there's an invisible extension that runs in the Chrome browser that helps to capture some key words and phrases that might indicate that they are thinking about suicide and actively planning something in which they might hurt themselves. It does flag searches for threats of violence and other things, but we really don't use those tools right now. We're focused on the self-harm. Then, the other thing it provides that we are not using right now is 24-7 phone escalation, where there's someone there that can talk with students in a crisis and then, if need be, refer to other folks.

There's no recommended workflow for GoGuardian. The company doesn't do that. So, we really kind of had to work that out, and part of what we'll talk to you about today is that workflow and how we got to the point of where we are now, because it's been a lot of different iterations and processes.

Again, this whole process started in around 2017 when we started bringing in a large number of Chromebooks to the district and we needed a way to help our teachers manage those Chromebooks in their classroom. Just so that you all know, my role is instructional technology and library media, not mental health. So, for me, I'm going to talk around some of the technical and software sides. Then, Becky and Patricia will talk some of the mental health pieces.

For this particular tool, we needed something that would allow teachers to help manage these Chromebooks in their classrooms, and GoGuardian came with those tools, along with something that would give us alerts for explicit searches and searches related to self-harm. So, teachers were using it for things like real-time view of screens, pushing documents out to kids, whitelisting and blacklisting sites, chatting with students, locking screens, closing tabs, taking screenshots of things that kids were doing, and being able to share those with families.

During the pandemic, this became incredibly helpful as kids were working virtually because teachers could still use this tool remotely, and it really helped them to manage what kids were doing and make sure that kids could easily log in to different sites. So, it's been a really great tool in terms of the classroom management piece.

What we discovered shortly after we started using it was that the flags started coming pretty regularly for self-harm and explicit content, really starting with the explicit content, which was not a surprise. When students would search for or type anything that had any kind of inappropriate language or reference, we would get an alert, and those started coming in fast and furious, especially from our middle and high school kids. So, what we decided to do with that particular tool was instead of our team trying to manage that and reach out to every single teacher every time that came in, we were able to put a message across the screen that reminds kids about appropriate sites and then sends them back to whatever they were doing.

For self-harm, it was a little bit different because we know that was a little bit more serious and we need to make sure that we were sharing that information. So, when a student searched for something related to self-harm, my team at first would be the only ones who got an alert, and then we would send that out to the principals, our director of related services, and our chief of school police. That was okay, except the numbers started to grow and grow, and it became a little bit unmanageable for my team of four.

The other problems that we had is there was no real district tracking for that. It was managed centrally, rather than at the schools where it belonged, and we really didn't know what the follow up with was at each school level because it was so much at the district level and the communication wasn't what we really wanted it to be.

So, we talked to GoGuardian about that, and they offered us the free version of their tool called Beacon, which is really just an add-on to GoGuardian. This allows us to set up school notification lists and put in the principal, the assistant principal, and any other designated staff who would then be the ones to get those alerts directly. That allows us to then put those alerts in the hands of the schools where they really belong because they know those kids best. We can also get alert follow up. If it's not resolved within 15 minutes, we get another alert. Then, it also allows us to set up some reminder windows as well.

We don't have the paid version, which would give us a more robust escalation list, which would have one person being contacted at first and then 15 minutes later or whatever time you said, it would contact the next person on the list. We also don't pay for at this point the 24-7 monitoring, which is Beacon's opportunity for a mental health clinician to accept those flags and get those flags and reach out to families, especially after hours, and then refer them to mental health clinicians as needed based on those conversations. Unfortunately, we don't have those, so we're managing all of this internally within the district, which can be a bit of a challenge, which we'll talk about.

As I said, the company doesn't have a workflow that they recommended, and so because of our district and the fact that we're trying to manage all of this kind of internally, we did come up with our own workflow. During school hours, what happens is the student logs in, they sign into their Chrome browser, and they search for something, or they type into an online document.

Whatever it is that they're searching, Beacon has a machine learning algorithm that identifies the search as active planning for suicide or self-harm. There's no keyword or list. It's actually looking at the entire page and looking at all of the content and the context of that content to make a determination as to whether or not that page or that site is a risk for students.

Once it designates that it is a risk, the principal and the staff that they've identified will get an alert in their email, and it has a big red circle, so it's really difficult to miss. The principal and staff will then send that email to their school clinician, psychologists, social worker, whoever it is that they have working on mental health issues at their building.

That person will then reach out to the student. They'll do an assessment to see whether or not the student really is at risk. Then if the student is at risk, they will work with the family and other folks to make sure that that student gets help and then they're going to document that action. So, it gives us a lot more documentation and it gives us a lot more control over who is seeing what pieces of all of this. I'm going to let Patricia speak to this workflow from the psychologists' end and the social workers.

PATRICIA ROBERTS-ROSE: Thanks, Stacey. Once we get the alert – both the supervisor of school psychology, the coordinator of school psychology, and the two coordinators of school social work - we all get the alerts, and we create a calendar for ourselves where we actually send the alerts out to all the mental health clinicians at each school.

We do get the alert and we advise them to fill out the tracking form that we created to determine the next step and follow up with the students. We can also go in and see which students are repeaters, that they have received multiple alerts, if they are referring them to outside sources or they are referring them to their own clinicians that they have in the community. Like Stacey said, we get the alerts. We continue to get the alerts every 15 minutes if they're not closed by the principal. In most cases, we are seeing that we can handle the alerts either in-house or connect them with community resources beyond the school.

STACEY DAVIS: The workflow outside of school hours is a little bit different because we have some contractual employees who aren't allowed to work outside of their contracted hours. Some might want to, but it does cause a precedent which can get us into trouble from our unions. So, we did kind of come up with a different workflow for after hours that works. We're still kind of refining it, but we'll go through what that is for right now.

A lot of it's still the same. The student is still logging in. They have to log into the Chrome browser, and they search for content which would flag for suicidal ideation or active planning. Instead of it going directly to the school - and keep in mind, the school will still get all of those notifications - but after school hours, so from probably 5:00 on and through the night, our school police dispatch team becomes the first line of action.

So, the dispatch team will get that alert and they'll go into our student information system, and they'll pull up the student's phone number and they'll call the house. They'll call the family. At that point, if they get in touch with the family, they'll have a conversation about what the student was searching and they will make recommendations if need be in collaboration with the family around whether the student needs emergency services or whether or not they should go to a pediatrician the next day, or if they need to call one of our crisis response teams in the state of Maryland, like Maryland 2-1-1.

At that point then, the officers put a note into Beacon. If they don't get in touch with the family by phone, they will have an officer dispatched to the home address to do a wellness check. That's an important piece because that's one of the pieces of controversy that we've been dealing with in the district, and we'll talk about that in just a second. But either way, the police will contact the family and they'll make a note in Beacon because the next day, when the school staff come in, they'll want to see that alert. They'll want to see what action has already been taken so that they can do whatever follow up is necessary from there.

This would have been Chief Hamm's slide. I'm going to take that for him, but what the school police team does is they really are providing that after hour support. They're the only team in the district that works 24-7, so it made sense for them to be the ones after hours, to be the first line of support.

The officers are dispatched for wellness checks. They don't go into the home. They don't look in the home. We get questions about what happens if an officer is there, and they see a crime happening in the house. That's not why they're there, and the chief has been very clear with community groups that he asks the family to step outside, or the officers will ask the family to step outside, and they'll have the conversation in a place that's safe and where they don't have to worry about seeing any illegal activity. So, they will have that conversation.

If need be, the officers will provide transportation. I think it's really clear and I think that the chief would want to be really clear here. They're not transporting students because they're in trouble. They're transporting students because the family has said to the officer, I think that my child needs to be hospitalized, but I don't have transportation. So, the officer will take the child and the parents to the nearest facility for them to get some help.

Those misconceptions that we have - that the officer enters the home, or the visit is punitive - those are the two biggest kind of battles that we're fighting right now in our process because the community worries about that. There is nationally some disconnect between especially, high poverty communities and the police departments. So, we want to just make sure for our communities to feel comfortable with this process, that this is not punitive and school police is not going into the home.

Just taking a look at a little bit of the data that we have, I will say that one of the things that we kind of talked with GoGuardian and Beacon about regularly is the lack of self-service reporting. So, this is data that we've gotten from the vendor. Prior to 2021, when we started to implement Beacon, we had about 291 self-harm alerts through the original GoGuardian tool. You can see, however - the GoGuardian older tool is the yellow and the Beacon tool is blue - you can see the number of alerts that have been flagged since the implementation of that tool has grown tremendously.

And throughout the pandemic, we've seen just a general increase in numbers of students who are searching for content related to suicide and self-harm. I think the numbers are fairly impressive in terms of us continuing down a path to make sure that we have the right tools in place to capture these searches and to get kids the help that they need when they need it.

Then, just kind of breaking that data down a little bit more. Since March of 2020, we've gotten a total of 786 self-harm alerts. Some of them turned out to be not anything that we really need to worry about. It could be they're searching for something about how to die in a video game, they want to know how to how to get a character to die and then come back without losing any lives. But for the ones where we have had clinicians have to respond, that's for 401 different times. The clinicians at the schools do respond and work with kids.

We've sent school police to homes 12 times. Nine of our students have been referred to an emergency room, and I can tell you that our emergency room partners are saying that, just as a note of interest, that three of the students who were referred to the emergency room were in severe jeopardy and had never had any mental health treatment or history prior. So, it's good that the tool caught them. Then, 12 students were referred to a crisis response center. I'm going to turn it over to Rebecca.

REBECCA MILBURN: Hi, everyone. We've already touched a little bit about the tracking tool for clinicians. We really wanted to get information from our site-based clinicians, and that would be the school social workers, school psychologist, school counselors. We have some outside mental health providers that work in our schools, so we wanted to hear from them, what they were doing, and how they were tracking things.

So, we created a tool where we could basically have some demographic information for the student, the school, the clinician. We also want to look at, is it an initial contact with the student, or is this a follow up? We also ask them, who did you contact? Were you able to contact the student when we were virtual? That could have been tricky getting the student on the phone or virtually, but did you talk to the parent? Did you talk to anybody else, or did you have no contact with anyone?

Also, what resources have you shared? Patricia talked a little bit about, we have 2-1-1. We have other agencies that we can send resources to parents to have them follow up with their child. It could be that it's not an imminent threat at the time to where the student doesn't need to go to the ER right in that moment, but it could be they need some kind of mental health follow up to help prevent any situations worsening to the point that they would need that immediate follow up.

Then, we also wanted to know, were they sent to the ER? Did they do what we call an emergency petition, or did they have school police involvement? That number is a little bit higher than what Stacey gave, because there are many times that we as a clinician, work with the school police officers to either have the student go to the ER from the school, especially now that we're back in person. Or we just have the police officers go with us to do a wellness check or go on their own to do a wellness check during the day.

So, that number is a little higher and our dates are a little different than what Stacey was talking about, but we looked at March of 2021 to December 2021, and we had 528 family outreach. Like I said, some of them could have been imminent danger that students needed immediate assistance, but a lot of that is just connecting with families, giving them whatever the family needs for resources. It might not just be around suicide or mental health. It could be other things that are happening in the home, but since we're reaching out and talking to families, it increases our ability to give them other resources also.

Then, you can see that just sharing of mental health resources with family was 315. Then, again, we've had either an emergency petition or a school police involvement in some capacity from us, meaning the mental health clinicians, 41 times. Thank you, Stacey.

STACEY DAVIS: Sure, thank you. Some of the concerns that we've faced so far and we're working through is privacy, and there has been information kind of published erroneously that we are surveilling students and that we're monitoring keystrokes and all web activity, and we're not. We're using a tool that has machine learning that's looking at the context of the page. So, we have been working very hard to clarify this process with our communities so that we don't have those concerns.

We do have a tremendous focus on equity in Baltimore City schools, and so we do kind of keep that in the forefront. We do also think about the difference of a family with a home computer versus a family without a home computer that's using just a school computer and who is monitored more, and more carefully. Are we going to miss somebody because they're not using their school or district laptop?

Then, that whole idea of the workflow for off-hour alerts. We obviously haven't kind of figured that all the way out, and we're still working through what that process can be. It's really hard to ask people in schools that are already working incredibly hard to work longer hours. So, we're happy right now with the process that school police are following in the evenings. But again, it's gotten a little bit of community backlash that we're trying to kind of smooth over and make sure that folks understand what it is that we're really doing, how we're doing it, and why we're doing it.

Then lastly, obviously this is relatively new for us, and we haven't done a ton of research. So, we're starting to work with Dr. Wilcox and Hopkins to talk about the use of GoGuardian Beacon, and are we really getting all of our kids, or is there a proportion of students who are being missed? Are we looking at numbers of kids that are connected to appropriate care before and after the implementation of the tool?

We can look at how different suicide alert response protocols lead to different outcomes. Again, I'm a former classroom teacher. I'm not a mental health clinician or a researcher, so we're going to rely heavily on our partnership to kind of get to some of these answers.

Then lastly is just conducting research to really understand who this is working for and what we need to do to make this a much smoother process because we want to make sure that at the end of the day, we are protecting all of our students and making sure that they are getting the help that they need if they are in crisis. I'm going to turn it back over to Belinda, our moderator.

BELINDA SIMS: Thank you all for those wonderful presentations for this afternoon's webinar. For those of you attending, we are now going to take some of your questions. We've received a lot of questions. We may not be able to get to all of them in the time that we have, but we'll do the best that we can. Please remember, if you do have a question, please submit your question via the Q&A box and be sure to address your questions to the speaker you'd like to respond.

All speakers can turn on their cameras at this point for the Q&A portion. To get us started, I have a question for each of our program teams to respond to. A couple of you already touched on the ways you engage parents and caregivers throughout your program process, but let's drill down on this a little more. What have been some challenges and or successes in engaging parents and caregivers? We're interested in everything from the development of the program through handling of a potential crisis situation. Why don't we hear from each group in the same order of the presentations, starting with the Michigan team?

JILL PALADINO: I can kick us off, and Robin and Natalie, if you have anything else to say, please jump in. For our process with our referral tool to the emergency departments, school mental health professionals are engaging with parents when they identify the student as experiencing suicidal ideation. They're having a conversation with that parent and alerting them that they are going to make that referral. In that process, there's going to be a conversation that's discussed.

We haven't dived too deeply into what that conversation looks like or guiding that conversation on our end at this point, but I do think that's an area that we'd like to explore as we continue to build out this programmatic model with providing some best practices and recommendations that school mental health professionals could then share and utilize with families. They are the experts of the communities that they work with, they are members of those communities, and so we want to honor those relationships while also providing access to those evidence-based resources that exist.

One gap that I do see in that space is that oftentimes there can be a misalignment between what people envision will happen when a student is referred to the emergency department and then what that referral actually looks like. So, I think even helping to set expectations around that process could be really, really helpful to keep everyone that is supporting that student aligned with the process and what would happen, what some potential outcomes are.

ANDRIA AMADOR: Hi. I'll start and then Shella can jump in. I would say that the first contact with parents shouldn't be in a crisis, so we should be establishing strong relationships with parents as we get to know their students.

School psychologists, for example, help to lead Tier 1 climate teams, and they're creating positive school climates. One of the ways to do that is create norms or values for a school and getting parent involvement right from the very start of saying, what do you value as a parent and how do we bring that into our school? Then, many of our school psychologists are helping support the social emotional learning instruction.

Another place to involve parents is to send them the weekly values or social emotional learning skills that your school is working on, so the parents can be aware of that. Many parents have asked to know about that so they can work on it at home. We also have parents do walk throughs of schools and give us their feedback and what do you think is important? We've made a big commitment to honoring the linguistic and cultural heritage of our families and making sure those are seen throughout the school, including signage in multiple languages.

We've also made it a very huge commitment, as has Shella and our partners, to hiring linguistically and racially diverse mental health providers so that we have people who can reach out to parents in their native language. Then unfortunately, when something happens and we do have to talk to parents, we hope to have had a partnership with them before that conversation. Then, we talk to them about their student's need and have a lot of resources.

I will say, during closure and the pandemic, many parents have reached out to us asking for support for a range of topics, like how to set a schedule when things are remote. Then once we came back, how do we get kids back into a school schedule? One of the things we've done is hold what we call family coffee hours, and we have a topic that a parent has brought up and asked about. We present on that topic and then we record it and put it on our website, and we do that in the languages that our staff speak. Those are just some of the ways that we think about parent engagement, but we consider parents to be partners in our work. Shella, did you want to add anything?

SHELL DENNERY: I would echo everything you said. I think the only thing I would add related to this topic that we're talking about today around suicide prevention and programming, we're doing universal programming. So, we're notifying parents that this is happening, and they can opt out.

But I think it's about printing it and putting it in backpacks. I think it's about emailing. We always do a parent night. We always do a parent coffee. We always do a parent training. We always are thinking about workshops as we sort of do and roll these things out, because there are going to be kids who we end up having to call parents about because of the curriculum or because of the information shared or because of the screening that we did at the end of each session, and we don't want it to be a surprise.

So, I think we're putting it in school newsletters. We're trying to make sure all the teachers are aware exactly when these things are being taught. So, it is that all eyes of the adults around kids are aware and they're on deck. I think, Andria, you said it best about parents being partners in a really collaborative process. That's the only thing I would add.

BELINDA SIMS: Anyone from the Baltimore team want to respond?

PATRICIA ROBERTS-ROSE: I'll respond for Baltimore. Thank you. A huge part of what we do in Baltimore City is engage parents, particularly during the pandemic, when we had an increase in the GoGuardian and Beacon alerts. So, what we've done in Baltimore City, we created an initiative called the Student Wellness Support Team, which is cross collaboration with all our mental health providers - both social workers, school counselors - as well as our expanded school behavioral mental health providers. We work to do a lot of preventive work, which is including the families.

Particularly around resources, we are an urban school district and 50 percent of our population is viewed as below the poverty line. So, we definitely look at resources, engaging parents, inviting them in for preventative groups and chats and things like that. Most of our schools use social media a lot to engage parents, which we found to be very helpful.

The main thing that we've seen pushback on - not necessarily from the parents, but from community and other stakeholders - is about engaging school police. We have our own school police force, so it's not the Baltimore City police. It's Baltimore City Schools police, so they're already very well aware and well-versed in how to deal with and engage our students.

And we use them, of course, throughout the day. It's a 24-hour police force, so they are not viewed as punitive. We're not using them in a punitive way. They are also very supportive and engaging with our students, families, and school communities. So, we are trying to shift that narrative around how they're used and definitely trying to work more towards engaging our families in being more supportive versus adversarial with their families.

BELINDA SIMS: Thank you. I have a question here that came in early in the webinar and it was directed to the Michigan team, but if others also want to weigh in as it relates to your own programs, please do so. The question was, for the Michigan team, do you have a memorandum of understanding or other written agreements for communication between schools and hospitals or emergency departments or other mental health providers? Then I guess a related question is, how does HIPAA work within this program?

ROBIN JACOB: Jill, do you want to start with the referral tool, and then I can talk about our other memorandums of understanding that we have?

JILL PALADINO: Sure. For this process, we have some MOUs with some school partners but not others for the PES tool that we utilize that capitalizes on this situation. So, school professionals can share that emergency health information in making the referral to the emergency department because it's an emergency, without authorization from the parent. So, they don't need a release of information to make that one direction of communication, although I would encourage it as long as there's no contraindication to that.

For referral back from the emergency department to the school professionals, a release of information is required, and so we need parental consent to sign off on that from the hospital system. The parent is going to have to sign off on an authorization form.

We're diving into that a little bit more right now. That's one of the reasons that we actually think our communication back from our emergency department is a little bit lower than where we'd like it to be, because we need to work on that communication component and on probably building some trust in the system and why that information sharing is so important. So, that's something we were looking forward to exploring more as a collaborative work group.

ROBIN JACOB: I'll just add quickly that our contractual relationships and our legal relationships with districts differ depending on the needs of those districts, depending on TRAIL's involvement in those districts, and we do it on a case-by-case basis. So, working with the district administrators to understand what TRAIL's involvement is going to be, and then from their perspective, what is required legally and contractually on their end. It really does differ as we go into various districts across the state what those contractual relationships look like.

BELINDA SIMS: Thank you. Any of the other teams want to add anything related to using MOUs?

ANDRIA AMADOR: I'll say slightly different, but I see a lot of questions about partnerships in the chat, so we do have MOUs with all of our community partners. There's blended funding for how they're paid. Some are paid directly from insurance, and some are paid with some insurance money or Medicaid money and some school money. But there are MOUs that spell out the relationship that we'll have with them.

SHELLA DENNERY: I would just add that also in Boston, we've worked hard to do standards of practice so that 20 organizations doing mental health and behavioral health work - I mean, this is a really resourced town and we still don't have enough, to be honest - just to sort of meet the needs. But I think having everyone at the table once a month, having open conversations - all of the directors of these programs know each other. I think it's about the paper contracts, but I also think it's about the relationship, having those collaboratives, having those spaces to work with each other, especially when this work is hard. It is challenging.

I think Andria has led that group for many years and really put practices in place. It's just not about the contract. I think a lot of times it's about the communication, the trust, and also allowing us to work together. Because we were, in the past, sort of doing more siloed work, and I think in this space, this is where bringing people to the table is essential.

BELINDA SIMS: Okay. Great. Let's go on to another question here. Let's see. This is for Boston or Michigan. This question was related to the data, just wanting to know a little bit more about the challenges during data collection in the schools. Can you speak to some specific challenges that you've been able to overcome?

ANDRIA AMADOR: Sure. I don't mind jumping in. When we started this project with Shella, our district did not have a collaboration like the one we developed. They didn't have an internal, external research team of practitioners doing research, so we had to work with our Office of Data and Accountability just to make them feel comfortable that this team was skilled enough to do the work we set out to do.

We had to do non-disclosures with everyone on the teams, and we do that annually. We had to work with our ODA, both ours and Shella's and UMass Boston. Every organization has to think about their IRB process and their data collection and sharing and what can be shared and what cannot be shared. So, that took a lot of time to set up the logistics of it.

Then, we meet monthly and have since 2010. Yes, there is a lot of data coming in because we are screening 33,000 kids twice a year as well as doing other interventions, but we use our logic model that guides our partnership in deciding where to focus on the data, because you could go in a million directions with data. So, we're really focused on student level outcomes and our students getting better, and that's what guides us as to what we prioritize in looking at the data. Did you want to add to that, Shella? Oh, our interim reports, maybe to talk about those?

SHELLA DENNERY: I think I'll just add really briefly that also within our contracts and MOUs, for us as a hospital, it says that we are going to be evaluating every single service type we do, whether it's therapy, a group, or if it's classroom intervention. So, we're doing pre-, post-[tests] on those things, we're doing satisfaction surveys, and it's part of the work that we do. I think in the field of school mental health, these are all areas that need to be strengthened, and there are some rumors out there that this isn't possible, that it is impossible to really, truly evaluate or do research.

So, we're definitely in the thick of it and trying to make sure that there are pathways to make this happen and these partnerships happen. It doesn't sound as easy as we may make it sound today, but I think putting the relationship at the heart and working together has made a huge difference for us.

BELINDA SIMS: Thank you. This is for Michigan or any other teams that are doing suicide risk assessment in the schools. Where do you advise schools to keep records of risk assessments, safety planning, and referral information? All of these programs generate a lot of information.

JILL PALADINO: I don't know that I have good recommendations to lean on or to look to. So, I would ask the other panelists in the group if they have recommendations. But in terms of protocol storage, one thing that we found is that it's lovely if there's a standardized protocol, but what often happens is that it ends up in somebody's desk drawer somewhere.

So, in lieu of that, it needs to be in a place that's accessible to everyone where they're routinely going to access that information. If there are updates, there's one central kind of file version so that that update is the one that everyone is accessing, instead of the one from three years ago when it was two administrators previously facilitating it. That's the one recommendation I have.

REBECCA MILBURN: I would also add, what we recommend for our staff is that any time that they complete the suicide risk assessment - we also use the Columbia, or we use the abbreviated version of it - that they need to keep that locked in their clinical office. They should have a file cabinet that's locked, that's confidential, and they keep records of all of that. So, if they need to look back on it or find trend information - if I were to leave that school and a new school psychologist would come in, then they would have access to that, so they could see historically the students in that building, who had that.

You don't label it suicide assessments, but you have it in a locked file cabinet. We say that you have to keep all records seven years post-graduation, so, it would stay in there. I know when I was in the schools, I would frequently go through mine to see if there's any of that I could shred because they were old. I don't know if anybody else has anything to add to that.

BELINDA SIMS: Okay. Well, thank you for that. I have a question. I just want to go back to the Baltimore team, and I believe you addressed this in your presentation, but we did receive questions about this. So if you could just remind us or reiterate about whether the parents and the kids consent, opt in or out, and are aware of the monitoring with GoGuardian.

STACEY DAVIS: Yeah, I can answer that. Our use of GoGuardian has really kind of grown organically, and when we first implemented the tool, our plan was not to use it for suicide and self-harm assessments and risks. It was really to use it for lab management software. The use of the alerts has really kind of just very spontaneously grown, and we've noticed that it's gotten better and better at capturing students that are at-risk.

So, we are working with our chief legal counsel now to make sure that all of the correct agreements are fully executed because it has grown organically, and we have a new chief legal counsel. We're kind of going back through all of that process. But yes, parents have been made aware and it's also a part of our acceptable use policy that when you are working on district tools with district resources, that nothing that you do is private. So, we have made that very clear, and students and families have to sign that agreement every year.

We do still get those questions and we do still remind teachers that when they're using the management piece of the software, to end their session when they're done their class, so that they are not able to mistakenly see someone's computer when it's not time for them to do that if it's outside of school hours. We've not had a complaint about that, so I think our teachers are doing a good job of that.

Then, we do have students who have who have realized that as soon as they log out of the browser, they're no longer monitored. That's good and bad. It's bad because it does keep us, in some cases, from being able to monitor what they're doing, and I've worked with a lot of parents to try and explain that to them because they're complaining that their kids are doing things that are unfiltered. Because I'll say that the other part of GoGuardian, it also filters web content so that kids are not getting to inappropriate sites outside of the district or to sites that are dangerous during the school day.

So, we have to work with parents about what we block and how we block it, and then how they make sure that their students are filtered while they're at home. All they really have to do is log into that browser with their district account and they're filtered. So, we have done outreach and we have communicated. We obviously haven't hit every single parent and family because we do still get a little bit of pushback around the police part of it, but we are really working with our community groups and youth groups to make sure that everybody is fully aware of what we're doing. Thank you.

BELINDA SIMS: Thank you.

STACEY DAVIS: Sure.

BELINDA SIMS: This is for any team. How can you help teachers have more buy in on this issue related to bringing in suicide prevention into the school context? Teachers can be scared or afraid to talk about these issues.

PATRICIA ROBERTS-ROSE: I can take that for Baltimore City. For Baltimore City, all of our schools, first, have full-time social workers in every school in the district. They also have school psychologists based on enrollment, and they also may have other mental health providers in the schools as well. Every year we train all of our staff. It's mandated that all of our staff have to be trained on suicide prevention. That's everyone in the school, the custodian, the cafeteria staff.

Everyone in the building has to be trained on suicide prevention, child abuse, and bullying. That's something that happens every single year. Then, we're looking at doing it multiple times a year to have a refresher for everyone midyear as well.

BELINDA SIMS: Okay. Thank you. We got a couple questions where there's interest in these types of programs, but the communities or the districts have very few resources. So, some of our attendees would like to hear more about how programs like these could be implemented in smaller or more rural school districts with fewer in-school and community resources. Any tips on where they could start?

ANDRIA AMADOR: We get this question a lot, and one thing I like to tell people is when we first started this journey with Children's and UMass, we had only 48 school psychologists for, then, about 60,000 students. So, our ratio was very bad. We had six social workers, so we were in the place a lot of rural districts were in terms of ratio.

But one thing we did was, one, we had to change the role of existing staff. Whatever staff you have, make sure that role is expanded to meet students' mental health needs and not restricted to special education eligibility work. The other thing that made us be able to do what we do is finding partners. One of my first partners was a sergeant detective police officer who said, I'm getting calls for kids that need mental health. I need to be your partner because I need the kids to get mental health support.

So, it's kind of turning to whatever partner you have in your area. It could be a local clinic or a hospital or university. Anyone who has a building, we'll partner with. So, just kind of reaching out. I think once you talk about students and student needs, it's amazing the number of people that will step forward. I know Shella has been really effective at building strategies, partnerships, and getting more services by telling the story of what's happening with kids. I don't know if you want to elaborate, Shella.

SHELLA DENNERY: I mean, I think I said this earlier, so I don't want to take up time. I know we're almost out of time, but I think the importance of having sort of resources and training online. There was years ago where I was not a huge fan of online training and now, because of COVID and the pandemic, we didn't have a choice. But I think that there's so many resources that you can sort of provide to the educators and the school staff to think a little bit differently and be creative.

Sometimes it is about partnerships, but sometimes it really is about teaming with the existing staff within the school. Teacher buy in, to me, I don't think this is an issue that we can ignore anymore. I mean, before the pandemic, it was one in five kids with a behavioral health concern, and now we know the numbers are so much higher. It's really thinking about the whole child, and I think teachers know that.

So, it's more about helping them, giving them the support, providing some training, and thinking really innovatively. That's what I love about webinars like this. Different ideas, different ways we can think about, talk about, bringing in, and normalizing mental health in schools as well. Just online resources, free things - I think schools are all excited about that as well.

BELINDA SIMS: For the programs that are being evaluated, are you including specific indicators to understand whether or not the program is having an impact on youth suicidal thoughts, attempts, self-harm and even suicide death?

ROBIN JACOB: Just saying quickly from the TRAIL's perspective, yes. We are attempting to measure all of those things, both through survey collected data and through administrative data that we collect from the schools. All really important things, and that's really what - at the end of the day - what we want to see moving.

ANDRIA AMADOR: I'll say for Boston, we are, as well, using our behavioral intervention monitoring assessment universal screener. We're tracking the students that endorse suicidality. We're tracking that and tracking that they're getting support.

STACEY DAVIS: We're tracking it in Baltimore as well and working with our partners at Hopkins, both in the university in their suicide research group, and then with the hospital as well. We're really in our kind of infancy stages in all of this, and so the tracking part of it is kind of new this year. Then, the partnership with Hopkins is new as well, but I think it's going to turn into something that's going to be incredibly helpful for what we're trying to do.

BELINDA SIMS: I just want to say thanks to each and every team, each and every speaker, for participating in the webinar and participating in this Q&A discussion. We do just have one minute left, so I think we're going to try to wrap up here. I want to take a moment to thank all of our attendees for attending the webinar this afternoon, and I think we have one more slide to share with you.

Again, we just want to reiterate that if anyone on this webinar knows someone in crisis, you can call the National Suicide Prevention Lifeline. 800-273-8255 or 800-273-TALK, or you can text hello to 741741. Also, we have recorded this webinar and it will be posted on the NIMH website in the coming weeks, and all webinar registrants will receive notification when the recording is posted.

Again, thank you for attending and we look forward to sharing this information with you once the video is available. Take care.