NIMH Director’s Innovation Speaker Series - Dr. Rosalind Picard, PhD

Transcript

David Armstrong:
I think just for the sake of time, we'll begin as people continue to come in, but I know we have a very full day here and we want to give our speaker all the time that she deserves. I'm David Armstrong, and on behalf of Bruce Cuthbert, our acting director of NIMH and the organizers of this series, including Don Smith [spelled phonetically], who does a wonderful job with all the logistics, I welcome you. We're in our 10th year, for those who aren't accustomed to coming here, and through those 10 years we've seen some really incredible lectures from this podium. Marty Seligman, Karl Deisseroth, Bob Sapolski, Hugh Herr -- who I'm sure you're friendly with -- David Brooks, John Carrol, Alan Alban, his good friend Roger Rosenblatt, but as I reflect back, there's really one presentation and particularly one presenter that really stands on top for me, and this was a presentation that occurred here shortly before Christmas in 2008, and that's when we were all enchanted with and enthralled with our -- today's speaker, Rosalind Picard, as she provided a very innovative talk about her work with autistic children, so it's with great pleasure that I am able to say, welcome back, Roz.

So, a little about Roz, who doesn't know her. She was educated at Georgia Tech and MIT and found herself in a very male-dominated world of electrical engineers and computer scientists, but Roz, like most of her outside male counterparts, believed that the quest to build true artificial intelligence could be achieved by reproducing the higher functions of the human brain, that is, the visual cortex, auditory cortex, and those regions of the brain involved in most perceptual experiences. That was until in 19 -- until she discovered a book written in 1993 by Richard
Cytowic the book being "The Man Who Tasted Shapes." In that book, the neurologist Cytowic describes an exceedingly rare perceptual disorder, synesthesia, in which the senses become intermingled -- an artist, for example, whose sense of taste elicits a sense of touch. In his book, Cytowic offers very convincing evidence demonstrating that humans are irrational by design, and our emotion, not our logic, is really, quote, "in charge." I'm competing with someone.

This work led Cytowic and others to kind of a new understanding of the relationships between the neocortex and the limbic system, with the latter, that is, the limbic system, being dominant over the neocortex as to what will be thought, said, or done relative to any given situation. With more reading, it became clear to Roz that any attempt to build a perceiving computer that did not take into account would fail, and I quote: "I realized we're not going to build intelligent machines until we build, if not something we call emotion, then something that functions like our emotion systems," but Roz resisted, and I go back to a 2012 article in "Wired" magazine that begins, "A woman, and worse than that, a blonde, Rosalind Picard was determined to have nothing to do with the study of emotion."

Determined, though, that she was on the right course, she did what any bright, intelligent, and motivated person would do. She tried to get her colleagues to work on the subject of emotion.

[laughter]

But that failed, so, reluctantly, she took it on herself, and in1995 circulated -- and to put it in perspective -- 1995, if I'm correct, is four years post doctorate?

Rosalind Picard:
Yes.

David Armstrong:
Four years post doctorate, so we're talking kind of a young, feisty individual at this time. So, in 1995, she circulated a tech report she called "Affective Computer" arguing the importance of integrating emotions into the machine environment, but as she feared, her manifesto was repeatedly rejected from peer review publication, with one editor suggesting -- I hope this doesn't get her to rise -- it was better suited to an in-flight magazine. About that time, too, she was at an international meeting, and she overheard some of her quote colleagues say, "That's Roz Picard. She used to do respectable work."

But fortunate for us and all the other thousands that her work has affected, Roz stayed with it, carried on, and her tech note and her book that followed in 1997 by the title the same, "Affective Computer," is credited with creating an entire new field of study, which today has its own general, international conference, and professional society. Today, Roz is the director and founder of the Affective Computing Research Group at the MIT media lab. She is also the co-director of the media lab's Advancing Well-Being initiative and faculty chair of MIT's Mind Hand Heart initiative. She is cofounder of Empatica, Inc., which works on creating wearable sensors and analytics to improve health, and Affective, Inc., delivering technology to help measure and communicate emotion. Roz is the author of many hundreds of scientific publications and chapters, active inventor with multiple patents. She serves on countless international and national committees and boards. She consults with many companies, and if that's not enough to fill her time, she's also the mother of three sons. So, it's with great pleasure to welcome, once again, Roz to NIMH and to this podium. Okay.

Rosalind Picard:
Thank you, David. I have relived multiple embarrassing moments there [laughs]. Hope the talk will be one half as good as the introduction. It's a real pleasure to be back here with all of you. I'm always amazed by the brain power in this room with the expertise in neurology, psychiatry, and so many things that I'm just kind of bumping into trained as an electrical engineer. I want to give a couple disclosures to get these out of the way, and then we'll dive into some fun stories.

While I'm full time at MIT, we have a lot of corporations sponsoring our work, including lots of the high tech companies and pharma companies and an amazing number of companies making wearable sensors these days, and I talked to just about all of them, even the ones not listed here, and there's a full list of those at this link here, and, as mentioned, I've cofounded two companies. The one that's relevant to today's presentation because I'm wearing sensors from them and I'll be mentioning some findings they're taking to market is Empatica, and I'm a cofounder and a shareholder there and somebody who's invested in them, but mostly today's going to be about the stories and the science.

And about that time of that paper you mentioned [laughs], I thought, "Okay, how do you like teach a computer to understand emotion?" and as an engineer, I was not happy with questionnaires and self-report. It struck me not just that people might not tell the truth, but that some people might not actually know what they're feeling, a kind of a common phenomenon among my computer science colleagues, who would corner me and say things like, "Roz, what do you mean emotion? Like, I mean, I know if somebody's really upset at me, but like this feeling stuff, you know, I don't really get it," and they -- a lot of them were not kidding. They really didn't get it. I remember one colleague -- I think I can walk away from this microphone. Oh, no. Turn this on. Now you can hear me, right?

So, he walks up to me, and he's looking at my feet, and he's telling me, "Roz, you're wasting your time measuring emotion. It's irrelevant. It's the noise. It's not the signal," and he never looked at my face once, and I thought, "Okay, well, he's at least -- they say you're extroverted if you're looking at the other person's shoes instead of their own," right?

[laughter]

So, I thought, "I've got to prove there's something here with this emotion. I've got to find things that I can measure that are objective, reliable, repeatable," and we were starting to get these wearable computers that weighed under 50 pounds that we could start to have on our bodies and take with us bicycling, driving, put sensors into glasses that could read your corrugator, your brow furrowing. These were used in classroom studies, where for some reason when I would present, you know, for an analysis or something to my MIT students, and I'd look at them, and they'd all look like they perfectly understood everything, but they weren't necessarily showing perfect understanding on the tests. So, we enabled them to hide their facial expressions and furrow their brow in confusion, which many of them did but not while looking at them. They furrowed while I was writing on the board, I found, and every time I turned around and looked at them, all the furrowing stopped, so once I could measure it, I could understand that social dynamic a little bit more.

There's my student, Steve Mann, who's gone on to become a very famous pioneer. He had been doing wearables before I had. He and Dodd Starter [spelled phonetically] really pioneered this at the media lab. One of our more heavy computers there with an antenna on its head, I could tell you some fun stories about taking him in my car at night. We had the -- good thing I had a sunroof. Stick that antenna out -- and then we went on to try to do things like measure people during live performances and stuff like that. This is Keith Lockhart at the Boston Pops, where we carefully worked to hide all of the wearables inside his Armani suite, sewing it in so he could change clothes quickly and all kinds of flexible ways of assembling the electronics, taking it to him, finally getting him to agree he would wear it, so excited, and right before we left, he said, "I have one request. Could you put all the sensors on the outside so everybody could see them?" [laughs] after we had gone to all this trouble, so we had to hire a costume designer to make him look more like Buck Rogers. There.

So, then we -- things got smaller. We were able to embed a lot of stuff in wristbands. I haven't looked, but maybe we did the first wearable that had heartrate and skin conductance in it that you now find in a lot of off-the-shelf stuff.

I'm going to talk a bunch today about a particular signal that's not done well in most of the devices out there yet, but I think it has turned out to be very interesting. Many of you may have heard of galvanic skin response. I don't -- the psychophysiologists don't like to use that term. They like to use electrodermal activity because under the name GSR it's -- a lot of bad work has been done and including very nonscientific work, but there's also been some very nice scientific work, and if you look at this signal in the traditional sense, it's measured on the palm or the sole of the foot with two electrodes and a box, and we would duct tape that box to us and walk around, but it was a little cumbersome for your average non-MIT student to wear 24/7.

So, we started innovating a lot of versions of this, such as giving an audience like you a version on your palm that -- with a very simple circuit that would map to the brightness of an LED, so in an auditorium when the lights went dim, we could -- how many of you have ever had the experience where you're speaking, and they have a bright spot on you and you can't see any of your audience? So, we'd had this experience in Kresge regularly, and so I thought, "What if we could see if our audience was excited or not, and make them glow if they're excited and go dim if they're not?" But we did that, and we saw, and we saw that every time a new speaker came onstage or there was a live demonstration the audience glowed, and every time there was live Q and A, the audience glowed, and every time a demo failed, the audience glowed, so do the live demo even if it fails.

[laughter]

And unfortunately, every time there was a long line -- long bit of PowerPoint, there was a decaying exponential in the brightness of the audience, so you are hereby invited -- I was told this is not the norm here, but you are invited if you feel a question midstream, especially when I'm not clear about something grab a -- one of the mics. They just ask that you use the mics so that people can hear you, and pop up -- raise your hand, run to the mic, and feel free to interrupt me as you go because that will help the audience autonomic arousal keep going on.

And now there have been a series of efforts to bring this to market, and I'll tell this story, that has led to Empatica taking some of this technology out of my lab.

Now, as David mentioned, I started by trying to help understand the stress in people with autism, and here's an example of a child, Lucy Jane Miller Center, who was on the autism spectrum. I'm going to turn of the sound here because you don't really need that. There's other sounds in the room where she is. The stuff on her head has nothing to do with measuring the physiology. She's wearing the sensors on her ankles, and what we see in this blue window is what's happening now live in the video with her skin conductance. Higher is more aroused. Here, she's having a meltdown, and it's peaking. This blue window is this blue strip down here. This is one minute's -- one minute of time in an approximately 45-minute occupational therapy session.

So, this was the first time we had built the sensors so they could leave the lab and measure autonomic activation in particular sympathetic nervous system activation through the skin in a child who if you brought her into the lab, she might have a very different experience than if you measure her in her natural setting. Many people with autism told me of overwhelming experiences going to a new setting where in some very extreme cases the whole world would go white on them, and they'd lose all kinds of perceptual information.

So, we wanted to build the lab that went on the person where they were, and here you just saw an example of it growing before a meltdown, peaking at that meltdown. Here's a very different example, same person, a little bit further on -- I'm getting some message here -- and here she is getting on a swing. There's another peak. I put in this example because it's important to let you know that many things map to this signal. It's not just -- a lot of people want to uniquely associate it with stress, whatever that is. Here, the physical effort of getting on the swing, the cognitive effort of motor planning for a person for whom that takes conscious cognitive effort, can be very hard to do. I've seen somebody make this signal go up just trying to wiggle their toe, and the emotional load all can make this go up, and we can't necessarily say which of those was most activated in a condition where all three may have been present, but she peaked, and then, beautifully, while she engages in this repetitive motion, it goes down, decaying exponential. Again, I usually only see that during lengthy PowerPoint.

[laughter]

We have also seen that during other repetitive movement in autism, somebody rocking, it can just go down like a slide on a playground sometimes. Not always. Stimming repetitive movements can make some people more alert, too.

Right. The first time one of my students had built a version of this that could log the data and 24/7 get us enough resolution that we could reliably see what was going on in a whole day without having to strap a box to your arm all in a nice wristband, this is what I saw, and here let me explain what you're look at here. Higher is more skin conductance. This is 24 hours, seven days, MIT student, and we see a bunch of activation at the end of this lab session, chilling watching TV, biggest peak of the day during sleep, very surprising. Okay, if you've been paying attention, autonomic arousal, activation going up with cognitive, affective, or physical load, like it just doesn't seem like these should be there during sleep, right? And yet, consistently, it's one of the biggest peaks of the day. I'll say more about that later. Chilling with relaxing, and activated for a lab, and consistently for MIT students, exams, studying, homeworks, labs, big peaking, okay, and consistently to the embarrassment of we professors, the low point each day --

[laughter]

-- classroom activity.

All right. So, we can get this kind of temporal resolution, and we can start to ask for an individual for whom there could be huge heterogeneity, right, even if they have the same diagnosis. One of the reasons I love the R docks [spelled phonetically] is that they -- you can see if this person is usually very aroused or not and during which kinds of tasks, so here was the first data I saw, and I'm giving you just the simplest version of it here. Here are five children who have the same autism diagnosis. They're all at the same school, same weather, same heat, same humidity, same days that they're wearing the sensor. They come in in the morning. Here, they're wearing it on the ankle again because a lot of kids are distracted or bothered by something on their wrists, and they're taking it off before they go home here. What do we see? What is plotted here is the number of days that they have this average value for their electro-dermal activity measured as skin conductance here. So, we see, for example, that this child, and this is 60 days of data for each child -- in 60 days, his or her average never goes above one, all right, and this other child looks kind of like that. In 60 days, it looks like about two days, his average went above one, but generally speaking, these kids on average stayed pretty low.

Now, look at these two kids. These kids' average, right, is higher than this guy ever got in 60 days, okay? They have the same diagnosis, same, you know, results from the DSM, but they have very different physiological wiring here, and that's just the start of it, right? What we're able to do now is start to see not just this as a bulk average but this contingent on social or this contingent on a good night's sleep or a bad night's sleep or contingent on the med versus removing the med and the interactions there, so it becomes -- we are now able to make outwardly observable something that previously was only inwardly happening. We've even seen it in kids who are identical twins with the same RETT syndrome, and they have very different autonomic profiles.

Now, one day, it was right before Christmas break, and one of the undergrads in my lab came to me and said, "Professor Picard, could I borrow one of your wristbands? My little brother can't speak, and I want to see what's stressing him out over the break. He's got autism," and I said, "Sure. Take one. In fact, don't just take one. Take two," because they were hand built and they often broke back then. We were always fixing them. I said, "Do you need a soldering iron?" He said, "No, I have one of those." [laughs] It was like, okay, you'll be all set for the lengthy vacation.

I'm back at MIT looking at his little brother's data on my computer, and, you know, I'm looking at, you know, this day, and it looks kind of normal. Actually not very activated for a kid with really severe autism, and this next day looks kind of normal. Next day, kind of normal. I'm like, gee, you know, not going to be anything interesting here. Next day, my jaw drops. First of all, he had put the sensors on both sides at the same time. Why would he do that? I -- you know like you should do one and then when it breaks, use the other, right? Well, I'm so glad he didn't follow my instructions. One wrist went so high that I thought that it must be broken because we have stressed people out at MIT in every way imaginable -- qualifying exams, balloon popping in your ear, you know, all kinds of -- you know, suddenly inviting the guy who wrote -- you know, who created the field you're in to quiz you in front of everybody -- and we had never seen a signal this high, and I've measured Boston driver stress and all kinds of things, and furthermore, the other side was not responding, so I thought that side must be broken except that right before this weird moment and right after this weird moment, the data looked normal. It was doing the same thing on both sides, and after there was this very clear sleep signature.

Now, I wasn't zooming into those sleep peaks there, but when you do that, you see a very clear signature during non-REM sleep, and that was present on both sides afterwards. So, I thought, "Boy, this is a weird kind of way for sensors to break," and I tried to replicate it in lots of ways that I won't go into here. Long story short, I failed. I could not figure out what was going on, and I did something I've never done before. I called the student at home on vacation. "Hi, this is -- this is Roz. How's your little brother doing? How's your Christmas? Hey, any idea what happened to him --" I gave him the exact date and time of the funniness in the data, and he says, "I don't know, I'll check the diary." Diary? MIT student keeps a diary [laughs]? Like how lucky is this? And he disappears for a little while. He comes back. He has the exact date and time. I'm holding my breath, and he says, "That was right --" he said at the time, he said, "That is 20 minutes before he had a grand mal seizure."

Now, I'm not a neurologist. I didn't know hardly anything about seizures. Start searching on this, and I -- next thing I know, I realize another student's dad is Joe Madsen at Children's Hospital Boston chief of neurosurgery. So, I call him up. "Hi, Dr. Madsen. My name is Rosalind Picard. Is it possible that somebody could have a huge sympathetic nervous system surge 20 minutes before a grand mal seizure?" and he said, "Probably not. But, you know," he says, "it's kind of interesting. We've had patients whose hair stands on end on one arm before a seizure. You know, or goosebumps on one side," and I thought, "One side," and then I fessed up to him that it was on just one side. I was sure before this that one of the sensors must be broken. I mean, this was arousal stress, and sweat makes it go up, right, like who's ever heard of you being stressed on one side and not the other [laughs], you know? I thought, "This is pretty ridiculous."

He got extremely interested and said, "Let's --" you know, we talked more. I showed him the data. We got -- we built more sensors, we got IOB approval, enrolled 90 families who were already coming in to have monitoring around the clock video EEG, ECG, and now EDA on electrodermal activity on both sides to see if this was a fluke or if it was something interesting, and we found that 100 percent of the grand mal seizures that we recorded, and here are three of them, had significant skin conductance peaks with the event.

Now, this is 24 hours of data here. This is data from a 17-year-old boy. Remember, I told you the MIT student's biggest peak of the day was sleep? Well, this is this boy's sleep, so these seizures are very big compared to what's usually the biggest peak of the day. You know, I felt like most of my emotion analysis work, which I'm not getting into that much here today but still goes on -- most of it goes on down here in the weeds, okay, and this is like I'm used to these being like the little baby maple trees, and I'm running around pulling those up, and all of a sudden I get over here and there's a redwood tree in New England, right [laughs]? Like wait a minute. This just doesn't fit, but these are very reliable. We found 100 percent of the grand mal seizures -- they're not always this big, but they're always two -- more than two standard deviations above the pre-seizure period, so we were able to build a seizure detector that not only used what you see here, the accelerometer data that is definitely getting big because these are convulsive seizures, but the autonomic data, so we get something that's not only sensitive but it's also more specific to these, and at the time we published a 94 percent accurate rate. Empatica now is commercializing this, and they've already improved on that rate with a low false alarm rate.

Now, there's another thing on here, and that is that the seizure's actually only a couple of minutes long, but this is an hour, here, right, 20 to 21, 8:00 to 9:00. Why is this autonomic response so big and so enduring? You know, did the person convulse so much they got enormously sweaty, and the sweat lasted for 45 minutes? It hardly seemed true, but we could measure all that, right? We had the data, so we went through. Yes, ask a question, and it's not related is the answer.

Male Speaker:
Do you know, did it remain unilateral throughout the seizure [inaudible]?

Rosalind Picard:
Here? No. Most of the patients in the video EEG monitoring EMU situation it went bilateral. Almost all of our generalized tonic clinics -- they generalize. They show up on both sides. It turns out to be very unusual that it stays on one side. Let me also say it's very unusual that we see it 20 minutes in advance. We now think the diary time was wrong. What we're usually seeing is it's starting electrograph -- when the electrographic seizure is starting, so it's not -- we are not claiming, if you're now just waking up and you heard that earlier part, pay attention [laughs]. We are not claiming we can predict it 20 minutes in advance, okay? That would be amazing. We are looking at other ways we might be able to forecast seizures, but there is no claim of prediction here. There is only a lot of data supporting a claim of detection.

Now, I'm curious because this is such a bright audience full of so many people who like know so much more than I do. When you look at neurological diseases, and I know this is NIMH, but I understand there are millions of people here, too, and you look at not the number of deaths but the years of potential life lost, number of deaths times sort of, you know, it gets bigger if you're young, then stroke is still number one here, and then we see Lou Gehrig's disease, multiple sclerosis, Alzheimer's, Parkinson's, meningitis, encephalitis, but does anybody know what number two is?

Say it loud because --

Male Speaker:
[inaudible]

Rosalind Picard:
Actually, it's not epilepsy, but it's related to epilepsy.

Male Speaker:
[inaudible]

Rosalind Picard:
SUDEP, yes. So, epilepsy does kill a lot of people, but SUDEP kills even more. Now, what's the difference? SUDEP is sudden unexpected death in epilepsy. If you have a seizure while you're crossing the street in Boston and you are struck by a car and that was your last event and that death is recorded, it is not called a SUDEP. It's, you know, it's a death from the seizure, but if you happen to have a diagnosis of epilepsy and they find you dead in bed in the morning, then it's a SUDEP, and it's -- SUDEP is when they can't explain, you know, when it's not clearly a drowning, and it's not status epilepticus that went longer than five minutes, and it's not a whole list of things. When it's none of those things where they know what it is, it's a SUDEP, and this is number two, just the SUDEPs, which they also now know are dramatically underreported because they've run around to see how many coroners know of this, and there are [laughs] something like half of them don't even know of this, and so they're probably chalking up the deaths to heart disease, some other default.

SUDEP takes more lives every year than house fires, SIDS, and AIDS in the United States, and most people have never heard of it. I'm just curious if you raise your hand if you've never heard of it before now. Most of the hands are up. Okay, so now you've heard of it, and hopefully you will talk about it after leaving here today because we now think it's preventable, and so we could together start to drive this number from 101 down to something like one.

All right, so what happens? Why does SUDEP happen? Well, of course we don't know yet, but there have been a lot of cool findings lately, and one of them is that in 100 percent of cases where they found a patient who has a seizure and then they were alive and they happened to be brain monitored and then later they're found dead in bed or dead on the kitchen floor, they went back and looked at the EEG, and here's where the seizure ends across these generalized EEG traces here. This is seizure. Doesn't look good. Here, we'd like to see normal brain activity, and instead it's all below 10 microvolts, so this is called postictal, which is post seizure -- generalized because it's all the channels, EEG suppression. This suppression is observed in a lot of people after a grand mal seizure, but not all, and it's not usually that long, but in case of death it becomes terminal, but it was the only biomarker found so far that occurs in 100 percent of SUDEPs.

Now, two are a surprise. Back to me saying, "What is the size of this response? Like what does it have to do with?" Well, it doesn't have anything to do with motion or, you know, convulsive activity. We've actually even observed it in some partial seizures where there's no convulsion, and what we have found, though, is that the bigger the response on the wrist, the longer this period of time that the brain waves are suppressed. Now, in all of our patients, the brain waves did eventually go back to normal. Yay, they all made it. People are much more likely to survive this period of time after a seizure if somebody is there. SUDEP is much more likely to happen if nobody is there, so if you have a friend who has grand mal seizures and has more than one of a year, make sure they've heard of SUDEP and they talk to their doctor about what's best to do, but also they should know that they are at heightened risk of SUDEP and that they should not sleep alone. They should have somebody check on them or have some kind of alert system or something. There's lots of choices. By the way, this has been replicated in -- this was originally the pediatric population. It's been replicated in adult populations with a lot more dots, also, and it's still statistically significant.

All right, now another cool thing we learned about the brain -- I'm going to relate back -- bring back to the psychiatry stuff here. It had been shown in the 50s and 70s that stimulating the human amygdala could turn off your breathing, but I guess nobody really believes things from the 50s and 70s anymore, so they have to do it again [laughs], so just last summer they published this really fabulous work where they went into a patient with epilepsy, and they watched -- they're monitoring the amygdala, and when the seizure spread to the amygdala, the person stopped breathing, but fortunately the person was not alone, right? The researchers were there, so they can talk to him: "Hey, John, how's it going?" John's able to say, "Oh, you know, I'm fine" after the seizure.

They also found when he was not having a seizure -- no sign of seizure activity -- they could just go in and stimulate that region directly, and John would be sitting there typing, showing no signs of distress, but not breathing. Twenty seconds, 30 seconds, 40 seconds, no signs of distress. "John?" "Yeah?" Takes his first breath. Replicated this in the other amygdala, replicated this in other patients, including one without epilepsy. Very interesting.

Now, separate set of findings. I'm going to piece together some things here to make -- tell you why I'm excited about this. You go in, you stimulate the left amygdala, a separate set of patients also with epilepsy because so far they're the only ones who are kind of volunteering to have a craniotomy and have electrodes stuck in deep. Brave, amazing, courageous people, and here they go in and they stimulate left amygdala repeatedly. These are patients off their AEDs, not having seizures, off their anti-epileptic drugs, not having seizures, and they get a big left skin conductance response and very little right one, and they go in and they stimulate the right amygdala. Big right skin conductance response and very little on the left. Very interesting, so this suggests that if a seizure is spreading to your amygdala, possibly turning of your breathing, which they think is not turning off your ability to breath because your brain stem certainly has that ability and they start breathing fine when you talk to them, but it's more like it's turning off your drive or your desire to breathe or something. If that gets activated in a, say, typical way and turns that off so that you're not breathing, we would get -- we would expect to see a big skin conductance response, so this led us to learn a lot more about this. We've learned, for those of your who aren't familiar, seizures can be thought of as this atypical electrical activity, kind of like a little electrical fire somewhere in your brain that can stay localized or can spread, so it can be focal. It can be generalized. When it's generalized, we see it on both sides. If it stays, say, in just the right amygdala, we would expect to see it on only one side, and because these deaths kill more people every year than house fires and now it appears to be something that if people were there simply to talk to them or stimulate them, in many cases afterwards they might start breathing again, and that might be one of the biggest contributors to SUDEP. We think that some kind of alert may prevent most seizure deaths.

So, while I set out to just build technology to understand emotion, I got a little bit derailed here, and it was working with kids with autism when one of our kids with autism had a seizure, and then we started following up to see if there was something there, and there definitely was, so we have done a tone of engineering and design work and taken what was before thousands of dollars of researcher equipment and shrunk it down into a device called Embrace that is now available to run algorithms onboard, which we're -- we've designed to run algorithms not only to alert people if they have an unusual autonomic and movement event. Let me jump to this -- and provide alerts to that to bring somebody to check on you.

We are applying for FDA, so I can't tell you what we actually think it might be detecting, but I bet you could guess. No medical claims here with product on the picture here, and measuring the activity, physical activity, temperature -- oh, it doesn't say temperature, but it's measuring temperature. Should turn out to be interesting, too. Sleep and letting people put in information. Oh, very important. I can now finally tell the time, which is good. Keep me on time here, and an API because a lot of researchers want to hack this and use it in interesting ways.

Also, finally, we are getting close to what I've been wanting to do in autism, and here's just data from a person on the autism spectrum who had asked to borrow a sensor to see if her levels would go high during a talk that she was to give that night, and indeed they did, but the interesting part here is not that she peaked when we thought she'd peak. It was that the event was supposed to start at 8:00, and I don't know about you, but, you know, uncertainty is stressful, right? It usually is for just about everybody. It's particularly stressful for a person on the autism spectrum. They really -- it really can make their life better if you give them a schedule and you stick to it, and here the schedule changed at the last minute, and she responded by pacing, so she's walking back and forth, pacing. Now, she had a male friend sitting nearby, and he looked at her, and he said, "Stop pacing. That doesn't help." Okay, so he was a little stressed by her pacing. So, she stopped pacing, and she started stimming, and I asked her later, "You know, what do you mean, stimming?" and she said, "I was kind of rocking and flapping, all right, to -- you know, to calm myself."

So, that's the stimming. Now, at the time, we were not streaming the slide. We were just logging it, and right after the evening event -- oh, and by the way, after her talk, she's kind of going downhill here and looks more relaxed. Oops. I lost my marker here. So, there's this nice decaying exponential punctuated by these big pieces of what look like noise, but they're actually signal. She explained that there was microphone feedback, loud auditory feedback sounds. Aaah -- freaked her out, and that explains these other peaks on the way down, so afterwards, she and her male friend both wanted to look at the data. She wanted to share it with him, and they looked at the timestamps, and the first words spoken were by him, and he said, "I'm not going to tell you to stop pacing anymore."

[laughter]

And the next morning I saw the two of them before another event, and this time she was pacing, and this time he's typing happily and letting her pace.

So, we're doing a lot of work on behavior change now, and one of the hard things in behavior change is, you know, emotion, right, understanding it, understanding its role, helping people understand it individually, whether they're eating because they're stressed, whether some activity is increasing or decreasing their stress, whether some person is increasing or decreasing their stress. A little awkward sometimes to look at this data with some people, and -- but it's objective, and what we find is that when people lay out the objective data, it changes the conversation, and people are sometimes able to move past the arguments and get onto okay, what do we do about this?

All right, so I mentioned the findings. I mentioned [unintelligible] between amygdala giving the largest left palm EDA, and it was lateralized, it's ipsilateral. They also showed this for a lot of other regions of the brain. I mentioned hippocampus, cingulate, not true for the frontal cortex or the BID T2 regions. They're very symmetric and very small compared to these temporal lobe regions and cingulate. Quite interesting because these are emotion, memory, and pain and a whole lot of other interesting things, and they're showing this differential information on the two sides of the body.

Now, I don't know about you. Maybe this is obvious to you, but this was not at all expected to me, and I was quite astonished to see these data and to bump into these findings, and it was remarking on this to an audience, and afterwards one of the doctors came up to me, and she said, "Oh, Roz, it's easy." "Easy, why is this easy? I don't know what this is." She said, "It's Medicine 101." Well, probably like 80 percent of you had Medicine 101, but I never did. I'm an engineer, and she says, "In the embryo, you know, when we were all embryos, we had three kinds of tissue: the ectoderm, the endoderm, the mesoderm. Our skin and neural tissue were bound together intricately and formed together the whole time from our embryonic stage, so why should we be so surprised that you stimulate an amygdala and maybe there's -- maybe EEG on the scalp is flat, but you get a big signal here, right?" I mean, I thought this was weird, right? It looked like the whole brain was shutting down and postictal generalized EEG suppression, but it's not. There's still activity going deeper, and of all wacky things, it's showing up on the wrist.

So, then we started saying, "Well, gee, what if it's showing up in other interesting things, too?" and this started to enable us to make hypotheses, and I'm going back and looking at these peaks during sleep, and we still don't have a full answer of these. I have a lot of hypotheses to tell you about, but let me show you one tantalizing set of findings I can -- also, for each one I'm showing, I can tell you lots of other things that haven't worked.

So, we were looking at what do these peaks during sleep mean. We had noticed these peaks happening when people were alert but very concentrated, attentive on certain things, and they looked kind of like these peaks during sleep. You zoom in on them. These are high-frequency peaks, high frequency for electrodermal activity. Here, the arrows indicate segments of non-REM. Red is REM here in the hypnogram, and usually it's the latter stage of non-REM, and the peaking ends right when you go into REM, and this is very consistent across lots of people. One of -- data from one of our published studies with 15 participants, purple is slow wave, blue is non-REM two, so most of it is slow wave and non-REM two. This person's odd, but they only had five peaks the whole night.

All right, so this is all published. This paper goes through a whole lot of stuff we've done there. It's not explained by temperature or motion completely. There's a small temperature effect, but it just doesn't explain most of what we see.

Now, this is a little busy, but bear with me. Working with Bob Stickgold, who's an expert on memory consolidation, dreaming, and all these really cool things, he took his usual protocol where he teaches people this visual discrimination task, has them sleep, and then measures how well they do in this very carefully studied way after the sleep to see if they have learned more or learned less, if they've gotten worse or haven't gotten a whole lot better versus they've gotten hugely better. So, we just did a binary split, top -- in this case, top 20 percent and top -- bottom 20 percent in this slide, but we -- the paper goes through lots of other splits, also, and it's consistent across the splits.

We asked if any of these features were better than random at telling if the person was learning more after sleep or less. We looked at a whole bunch of EEG-based features, polysomnography features related to sleep staging, accelerometers that are in like the Fitbits, combinations of that with electrodermal activity, and all of the above. Tons of features, gave it all to a bunch of machine learning algorithms, let them -- six machine learning algorithms. Don't ever just trust one -- and here's six different machine learning algorithms. How did they all do discriminating against two groups? We see some pattern here where the red is almost always the highest. In one case, the purple is the highest, but consistently the red is outperforming the others. The red is just the electrodermal activity. Quite interesting.

Why might that be? Well, let's got back to what's involved in memory formation during sleep. Well, it's lots of stuff, as you guys know better than I do, but pieces of it that are quite involved involve amygdala and hippocampus, and we expect these to give big skin conductance responses, so maybe it has something to do with that, and if you're really interested in this, catch me later, and I can tell you a whole lot more stuff. We're doing this, but I need to keep moving here.

All right, now this study where they went in the brains of epilepsy patients while not having seizures while off of their anti-epileptic drugs and stimulated these regions and measured electrodermal activity, they weren't thinking about SUDEP. They were thinking about the fact that they had noticed kids with learning disabilities were having asymmetric responses, and they were trying to figure which regions of the brain might be involved in that, and I thought this was really interesting because many of my friends with autism had explained that when they go into be evaluated and tested by scientists, they go -- they often go with a state of mind that you're just going to show me something's wrong with me. You're just going to give me something I can fail on. You're just going to make me look bad, and they feel a bit threatened, socially threatened. I mean, you can imagine how awful that would feel, and so it's a very biased affective state going into these studies potentially. I'm sure it's not true of everybody, but one can expect that what they've told us might be worth paying a lot of attention to.

So, we have started to suggest that maybe people should not follow the gold standard recommendations for the last 30 years of only measure on the non-dominant hand but take a look at both sides, and -- because there's a lot of brain imaging studies that suggest that the two sides are going to act differently. For example, studies -- there's just one -- I know there's a lot, and it gets more interesting than this, but often in a right hander, the right amygdala's associated with more threat, negative stimuli, and anxiety, and a lot of studies the left gets a mix of these. It's not a simple negative/positive thing. It really seems to be more of kind of a threat type thing.

So, I said, "Gee, I wonder if we've seen that in any of our MIT students" in this particular task that's very well-known where I certainly felt a little bit of threat when we were debugging the task, and I usually subject myself to these things early on to make sure we're thinking of everything before we bring in the real subjects. So, here's 25 real subjects who have come in for this classic task where they have to count from 3,000 backwards by sevens, all right? Now, we make it a little more stressful because we put a person behind them watching them, scoring them, telling them they've got to go fast, and they've got to get it right, and if you get one wrong, we hit this really obnoxious-sounding buzzer, and we let you know what it sounds like first, so I don't know about you, but I sort of feel like with -- you know, you kind of like have a PhD. You sort of feel like you should be able to subtract. You feel maybe a little social threat like, "I really shouldn't screw this up," right, and they're sitting behind you judging you.

[laughter]

Now -- so, I thought this might be a good threat task. Well, it turns out we had only been looking at the blue data because the literature says, "Just measure on the non-dominant hand," and if you just look -- so, let me explain this here. This is the average skin conductance response during this task for each person, so person number 17 here, they're two sides are about the same, and just visualize the blue here. You know, and it kind of looked like everybody else's paper, you know. A bunch of people are responding a bunch. These guys are probably not stressed based on their blue data, and these guys are not really stressed based on their blue data.

I said, "Let's just look at the red data, too -- the right side, too, just for fun and frolic," and then I lined them up based on the right. It's exactly what the brain imaging would predict. If there's more of a threat present, which we can't positively confirm here because we didn't measure that, but we have now replicated this many, many times, and we see this huge bias toward the right and not a huge bias toward the left.

So, if you decide to measure electrodermal activity and you do your homework and read the literature and the rules about how to do it, you will miss all of the red data [laughs], and you will see only the blue data, and I now suspect that a lot of the stuff in the literature that only published the blue data may need revisiting because a lot of the conclusions, I think, are maybe partly incomplete. Yes?

Male Speaker:
Are the responses in any way correlated with performance?

Rosalind Picard:
I have asked my students to look at that, and I don't have good answers yet, but we will --

Male Speaker:
[inaudible]

Rosalind Picard:
Yeah, are the responses correlated with performance, and we are looking at that. Yeah, don't have the answer yet. It also may not just be performance. I'm expecting more it may be -- because I've seen great performance when people have huge asymmetry. I think it may be more how you feel about somebody judging you and how badly you would feel if you screw up. It's more the anticipatory state than the actual state, but we are looking at that, so ask me later, and we should know because we see for a person who performs well in a high-stakes situation that the right side can go way above the left at the start of the meeting. When the meeting ends, it goes back to [laughs] the two being together, okay? So -- because now we don't get snapshot data of a pre and post in the lab or something. We get all-day data. You can see what's normal for a person. You could see if they walk around kind of permanently shifted or if they only shift when they have these meetings with their boss.

By the way, this is all published now recently in "Emotion Review" with commentary because the editor said, "You know this is going to be a little controversial [laughs]. It's kind of going against 100 years of how it's been measured," but the commentary has been very friendly. Now, we can make lots of predictions, and I know one should be careful doing this because there are a lot of studies. A lot of the studies on left/right hemisphere cortex are mixed. I am not making predictions based on that. I would -- don't expect to see huge asymmetry there, but when there's huge asymmetry with these regions where anatomical studies, direct stimulation studies, the epilepsy studies, and others seem to show this pathway. There we can make a lot of predictions, and we expect connection with anxiety and depression, as well.

Now, the next data I'm going to show you is very preliminary. We're getting a whole bunch more. I may tell you later to forget this, but right now, the first two subjects, each monitored for many sessions of transcranial magnetic stimulation, collaboration with John Kimperdon [spelled phonetically] at Mass General Hospital. These patients have severe depression. Other treatments have failed for them. They are coming for TMS. The HAM-D-17, the HAM-D-28, the QIDS-SR-16 are all saying this person's not doing well. Now, they're doing even worse. They're doing even worse, and this is 28 of 37 sessions here. We are measuring in each session their asymmetry with this electrodermal activity, and here it's more right over left, which is what we'd predict with this right-handed person and their possibly greater right activation, and it got much worse right here, which is interesting because that's also interesting because that's right before it got much worse here and here, and this person's just not doing well the whole time.

This person starts off looking pretty bad, a little better, a little worse, a little better, a little worse, better, better, better, better, a little worse, and here we see them getting better. Still need a lot more data, probably nothing will work for everybody, but this is really interesting because this is what we predict, and we're seeing an objective biomarker that is associated with these other measures.

I'm going to zip ahead because I want to leave more time for Q and A here. People often ask me about -- you know, I've been talking about sympathetic nervous system. I haven't talked about parasympathetic. Do we measure that? Yes, we measure that. I'm wearing another device that we use to measure that. We get continuous with multiple kinds of light, heartrate, heartrate variability, skin -- so, if you want to come up and play later after the talk for a few minutes, you can try this out. We can stream your data, and you can see that we can get that. However, the heartrate variability is not good when people are moving yet, okay, so -- the electrodermal activity is usually good when you're moving. It's really good when you're blood volume, pulse, and heartrate variable are really good when you're not moving, like during sleep they're beautiful, but when I'm up here gesturing, that's still garbage, and these things have been compared to gold standard systems. If you're interested in this, we can show you more of that, and the overall error comparison is about two percent compared to many thousands of dollars of equipment technology across lots of measures that are used for heartrate variability.

All right, I want to end with a little bit of vision here. I've just spent that whole time talking about some physiology that can be picked up from the wrist. There's a ton of other things we're measuring that are exciting, too, and all of these things matter more in context, right? If you know a person's just come back from exercising or you know they've just gone to visit a friend, or, you know, on their calendar that it's been a busy day, so we get all that stuff, too, and we're looking at that, and we're measuring a ton of stuff, and our -- this is where NIH -- thank you, thank you, thank you -- has helped fund, with Chuck Czeisler at Harvard Medical School and Brigham and Women's Hospital, and we are trying to characterize social interaction, a whole lot of things, but one of the things we really want to do, that I dream of and would love your help and advice on, is this word here, which is a placeholder for a very complicated, high-dimensional space of measures tentatively called well-being.

So, when NIH or your employer hires you, you're hopefully positive. You're doing great. You've got great well-being. You come across super in your interview. You're their first choice out of this huge number of candidates, and over time, you know, hopefully you do even better, right, but then everybody hits major stressors, right, and almost everybody takes a dive, and hopefully you bounce back, right, and you're resilient and you come back up, but at MIT each year, where we admit who we think are just the most amazing people, the unfortunately large percentage of them take this path, and somewhere down this path, hopefully, they get good medical care, and a doctor gives them a diagnosis for something like major depressive disorder or something, and they get treatment, and pharma companies are always coming to us, and many of them having come to me and said, "Can you measure the difference between here and here? Can you help us see if a person is moving up this graph?" and we're working on that, and that's interesting, and you saw an example of an attempt to do that with some objective biomarkers and combine those with questionnaires and social interaction and GPS coordinates and a lot of other things we're looking at, and the medical profession and pharma are really happy to work over here and to fund people over here where people have diagnoses. But why aren't we trying to figure out when these students come in or when these employees come in what was happening over here? Because I don't think you just wake up and there's a step function and you were here. I -- in fact, we've been measuring this, and there are a lot of things that are changing, and we can go into those in a longer conversation.

There will be a meeting Monday going into some of this, but I would just want to challenge you, you know. What could we be doing much greater than what we've been doing? To not just keep characterizing people over here but to figure out if you're on the red line or the blue line, and as I asked one person who we retrieved from the mental hospital, and fortunately, she's fine. She's come back to school, and her life is on course again. You know, hopefully this will never happen to you again, but if this -- if in the future -- again, hopefully this never happens -- but if in the future, your data looks like it did back here, again, would you like to know, and she's like, "Oh, yes [laughs]. You know, please let me know. I never want to go through this again. I'd love to know, you know, if I ever again look in the future like I did back here on this red line." So, I just want to challenge all of you with your amazing brilliance and expertise to let's work together to think about how we can do more up on this left side, and I'll leave on that.

It's been a journey here through autism, MIT problem sets and sleep, and learning humble things as a professor and mapping different signals to predict memory and learning and sleep, trying to get some of our stuff out of the lab, and learning fascinating connections between central neurological mechanisms and peripheral mechanisms, and I think we're just at the beginning. There's a ton more to learn, and I want to thank some of these wonderful folks listed here for supporting our work. Thank you very much.

[applause]

[end of transcript]