Director’s Innovation Speaker Series: Beyond FDA Approval: Assessing the Value of New Health Technologies

Transcript

JOSHUA GORDON: Welcome everyone. We are going to get started in just a few moments, as we allow people in from the waiting room. Once again, welcome, everyone to the National Institute of Mental Health Director’s Innovation Speaker Series. I am Dr. Joshua Gordon, director of the National Institute of Mental Health. It is my pleasure to welcome all of you who are joining us virtually for this outstanding lecture to be given by Dr. Foluso Agboola of the Institute for Clinical and Economic Review. I am going to introduce Dr. Agboola and the topic in just a moment, but begore I do let me just go over some housekeeping measures.

Those of you who would like to ask questions, please use the Q&A function on the bottom of your screen.  You may enter the questions at any time during the talk and I will ask them for you towards the end of the talk. If you require technical assistance, you can also use that Q&A box to query the event production staff.

I would like to remind everyone that the webinar is being recorded. That means that the recording will be made available in the coming weeks on the NIMH website, as indicated here. I encourage any of you who know colleagues or friends who would like to be able to watch the lecture but couldn’t make it today, to point them to this website and invite to hear it at that point.

With no further ado, let me again introduce Dr. Agboola, who is the vice president of research at the Institute for Clinical and Economic Review. ICER, as it is known, is an independent non-profit research institute that generates evidence on the effectiveness and value of drugs and other medical services.

Dr. Agboola is a leader in health technology assessment and an expert in the field of evidence synthesis. In her role she conducts and oversees systematic reviews and synthesis of evidence on the comparative clinical effectiveness of new and emerging treatments. She facilitates the use of these reviews in order to help decisionmakers, like policymakers, stakeholders, health care organizations, decide how to value certain treatments and indeed, how to negotiate pricing of those treatments in the marketplace.

Dr. Agboola has an extensive background in evaluation research and quantitative methods and a strong track record working in health care, academia, and public health settings. Before joining ICER, she worked at the Massachusetts General Hospital Center for Drug Policy and the Harvard T.H. Chan School of Public Health. She has offered numerous peer reviewed manuscripts, as well as serving as an author on 18 of ICER’s health technology assessments. She also currently serves as on the Editorial Advisory Board of the Journal of Managed Care Pharmacy and she was named one of 13 up and coming health leaders, featured in the 2021 Annual Managed Healthcare Executive Series.

Dr. Agboola received her medical degree from the University of Ilorin, Nigeria and her MPH in Quantitative Methods from Harvard. Dr. Agboola, thank you for joining us today. We are really looking forward to your talk.

FOLUSO AGBOOLA: Thank you, Dr. Gordon. I really do appreciate the opportunity to be here to talk to you all about what we do at ICER. I am excited to share all of that with you. I am going to be speaking today on a topic that I have entitled, Beyond FDA Approval: Assessing the Value of New Health Technologies.

Here is a roadmap for where I am hoping we are headed today. First, I am going to be providing a background on the Institute for Clinical and Economic Review, ICER. I will also be describing a value assessment framework. Lastly, I will list some ways in which ICER’s work has been used in the real world.

So ICER is an independent health technology assessment organization. Just so that we are all starting from the same background in terms of thinking about HTA, I have provided a definition here by the World Health organization. They describe HTA as the systematic evaluation of the properties, effects, and/or impacts of health technology. That is a medical treatment or healthcare delivery mechanism. They go onto describe it as a multidisciplinary process to evaluate the social, economic, organizational, and ethical issues of a health technology in order to guide policy.

HTA is an offshoot of what you can consider as a continuous ingrained innovation in healthcare system and the very real budgetary constraints within the health systems in societies. HTA in a sense, seeks to determine if the benefit provided by the new innovation is worth the extra cost. HTA is not a new field, it exists in essentially every developed nation in the form of a government and quasi government organization, of course with some variation across countries in their methods.

The U.S. is an outlier in that we do not have a government body that does this and really that is where ICER comes in.

What we do is develop publicly available value assessment reports that examines both the comparative effectiveness and cost-effectiveness of various health care interventions. Our HTA activities are funded by non-profit foundations and it is really what funds the majority of our funding at ICER. Using this exclusively this non-profit funding for HTA activities, allows us to be independent in our assessment.

Our goal we stated simply as fair price and fair access and future innovation. That is because we believe that we can use evidence in a transparent way to align prices with the benefits it provides for patients and families. This would allow us to be able to improve both access and affordability while retaining the incentives necessary for future innovation.  

We were founded in 2006 as a research program at Harvard Medical School. At that time we were focused on health technologies but also health services. In 2014 we shifted to drugs more due to the rising concerns related to the potential budget impact of a new treatment for hepatitis C at that time.

Right around that time we received a large grant that allowed us to be able to start our Emerging Therapy Assessment and Pricing Program. We also released what we described as our newly formulated value assessment framework. Which really is a uniform approach we take in evaluating drugs.

In 2017, we released a revised version of this Value Assessment Framework, and in 2020, released a third revised Value Assessment Framework. Currently we review about 10 to 12 topics per year.

To give you a general sense of our process from topic selection to final report takes approximately eight months. As part of our process we include regular engagement with patient groups, clinical experts, life science companies and other key stakeholders. We take a iterative based approach in developing our report. That is we first develop a draft version of our report and we seek public and stakeholder comments to our methods and our conclusion, and we revise based on those comments.

We time our reports to coincide with regulatory approval because right around the time a FDA drug is being approved, we believe it is important to have our report. That is the time it is going to be most useful because it is right around that time that decisionmakers are making the decision around the price and negotiating for prices of new drugs but also coverage decisions.

That means we would actually choose a topic selection so when the drug is still in the pipeline of the FDA approval, and we have a process in which we track the drugs that come down the pipeline and based on a set of criteria, determine which ones will be most appropriate for an ICER assessment.

All of our reviews are evidence based, transparent, and that is we detail lots of our methods and that tends to be thick, and available for free on our website.

To give you a sense of the various teams we have on ICER, one of the things I love the most about ICER is really the collaborative approach in which we do our work. ICER employs about 40 people really, but on various teams for instance, the Pharmaceutical Intelligence Team that helps us track down the drugs that are coming down the pipeline, the Evidence Synthesis Team, which I lead, and the Economic Modeling Team, they do a lot of the core assessment. On that team we have health economists, we have researchers with a background in (indiscernible), and we have clinicians.

In addition, we also collaborate with clinicians that are trained in evidence-based methods that work in various academic institutions across the country, and also health economists from various academic institutions across the country that also work with us in building our models.

We also have members of our team that work on the Program Team. They keep us honest in terms of thinking about the timeline, but also helping us to facilitate conversation and making sure we are speaking with the right stakeholders as we go along in our process. They work hand-in-hand with Patient Engagement Team, making sure we are connecting with the right patients and patient advocacy organizations during our review.

I am going to do a more in depth look at what our Value Assessment Framework looks like, but I am also going to be using our Review of Esketamine for Treatment Resistant Depression as an example to walk along to describe this as I go along.  

For those who don’t know esketamine, esketamine is available as a nasal spray. It is a non-selective, non-competitive antagonist of the NMDA receptor. It was approved on March 5, 2019 by the FDA for patients with treatment-resistant depression.

Our Value Assessment Framework is really intended to make transparent how “value” is conceived of and evaluated in ICER reports. Starting from this framework of what our goal of fair price and fair access and future innovation is, our Value Assessment Framework is anchored in a long-term perspective on value. We think about value as a complex comprehensive process, the foundation as comparative clinical effectiveness where we are really asking questions on the balance of health benefits, really assessing thinking about the benefit of the drug versus the side effects.

When we add cost concentration to it, we are asking questions around the cost effectiveness of the drug. Beyond these core measures, there are other important competence of value which we call potential benefits beyond health. This really is a recognition that what matters to patients is not limited to measured clinical outcomes.

In addition, there are special concentrations we may need to consider in different areas. For example, we may want to give additional concentration to diseases that are particularly severe or perhaps one in which a treatment may help a society reduce health disparities.

Finally, as part of a concentration for value assessment framework, we also consider short-term affordability which we employ using potential budget impact. As I go along in the presentation, I am going to talk about the various aspects of this Value Assessment Framework.

First I wanted to show you these fun pictures of my daughter. We are New Englanders’ and apple picking invokes everything we love about fall. As a family it is usually one of the last outdoor activities we do before the dreaded winter months – at least for me. When we go apple picking we get to enjoy more than just picking apples off the tree. It is usually full of adventures where we want to do it with friends, leisurely walk around the farm, enjoy donuts and ciders. And for my daughter, as you can see in this picture, also climbing the apple tree.

In that first picture in 2019, she was eight years old and here she was nine. I took this picture during our last apple picking. This one in 2021, during our last apple picking experience. What you don’t see in that picture is my daughter on the apple trees and that is because those trees are not climbable. What you also don’t see is how unhappy she looked beyond the camera. Here is the mom I heard rave about this beautiful apple farm and I thought this would be a great apple picking place to go. But I forgot that for my daughter, even though she loved picking apples and she loved walking around lazily enjoying the beautiful view, enjoying the donuts, for her what raised the value of the experience was also the fact that she gets to climb the trees.

Here is an important illustration of how we think about our work at ICER. We don’t want to end up in a different place from patients in defining value. Patient engagement is at the core of our work. Our Value Assessment Framework, although forms the backbone of our assessments, we use it within a broader method of stakeholder and public engagements.

At the beginning of every project that we review, we seek input from patients and their families. We also see input from clinical experts and manufacturers and payers. What are we seeking to get from the stakeholders? We want to gain insight into patient perspectives on value. We also want to learn about the diversity of experiences individuals have living with and caring for the condition.

We also want to understand which outcomes are most meaningful to patients. Putting all of this together really helps us to properly frame our research question as we look back on the clinical review and also the cost effectiveness evaluation.

So turning back to our Esketamine Review and what we engage with patients and also clinical experts, here are some of the things we heard. We heard about the impact of treatment-resistant depression on patient’s quality of life. Patients describe different personal stories but all have the common things that emphasize that major depressive disorder is a chronic disease that has profoundly effected all aspects of their lives and the lives of those closest to them.

Medications, also describe how they don’t derive long-term benefits from their current treatments either because they stopped working or they developed intolerable side effects.

We also heard about some of the patients that in an effort to get something to work, have turned to non-medication therapies such as TMS and ECT. Although these therapies have shown to be effective, patients also experience high relapse rates and oftentimes are quite time consuming and inconvenient for patients to be able to go into the hospital to actually receive this non-medication therapies. Patients also talked about some of the intolerable side effects associated with using ECT.

We also heard about how some patients really are turning to off label therapies such as Ketamine. And for patients who reported benefits with Ketamine, many of them expressed interest in a possibility of a FDA approved drug that is expected to work in a similar manner. We put all of this together and it helps us to properly framework what our evaluation should look like.

Starting from that place of the foundation for valuing our assessment and thinking about the clinical effectiveness, first I want to emphasize that our assessment is grounded in systematic review of evidence. For every review we are looking for the highest quality evidence available. Of course, we are interested in finding trials that have direct comparison of the intervention or outcome of interest, but in situations where we don’t have head-to-head trials that are supposed to be able to answer that question, we use in-direct comparisons through network meta-analysis when we identify trials that are similar enough for us to be able to do that.

We are often interested in looking at subgroups and heterogeneity of treatment effects. We present our ratings in what is called the ICER EBM Matrix. Really this is the way in which we reflect our judgements on the magnitude of the difference between a therapeutic agent and its comparator, that is its “net health benefit” - the balance between the benefits risks and/or adverse effects and the level of certainty in the best point estimates of that net health benefit.

For example, when we give an A rating, that indicates a high certainty of substantial comparator net benefit. As the magnitude of the comparator net health benefit decreases the rating moves accordingly to B, which would be incremental, C which would be comparable, and finally D indicated an inferior or negative net health benefit.

When we give an evidence rating of something like a B+, we are really saying that the new drug offers a distinct advantage over the comparator but we are not sure about the true level of the incremental benefit. It could be a small net benefit or a substantial net benefit. That is how we describe our ICER Evidence Rating.

In addition to giving this letter grade, we typically would include a textual description of why we landed in that rating.

So applying this to our esketamine review, we are interested in reviewing the net health benefit of esketamine nasal spray plus the background of using an antidepressant versus using a background anti-depressant alone, which really is represented by the placebo arms of the clinical trials.

We are also interested in learning the net health benefit of esketamine versus all the other treatment options we heard about like ketamine ECT and TMS, also in the background of antidepressants.

So when we did our systematic review, I am just going to give an overview of some of the things we found. We identified three randomized control trials that compare esketamine to placebo, all four weeks long. This trials enrolled patients with severe depression. Patients have failed two or more antidepressants that had been taken in adequate dose and adequate duration. In fact, about 40 percent of the patients enrolled failed four or more antidepressants.

We meta-analyzed two of the trials that we thought were similar enough that it enrolled patients 18 to 65 years. We found that esketamine showed greater improvement on the Depression Rating Scale compared to placebo. We also found that patients on esketamine were more likely to achieve clinical response. Patients on esketamine were also more likely to achieve clinical remission, although this was not statistically significant.

We found that similar improvement were really observed in the third randomized control trial that enrolled patients 65 years and older, although some of the results were also not statistically significant. There was a fourth trial, which was really a withdraw trial that was designed to assess relapse prevention in patients that had achieved stable remission or stable response. This trial showed that maintenance esketamine reduced risk of relapse by about 51 percent.

In other words, patients that achieved clinical remission on esketamine are expected to continue on esketamine long-term in order to avoid relapse.

Just a quick overview of the some of the things we found in overview of the harms of esketamine. It was generally well tolerated in the short-term RCTs. Some of the common side effects that were observed which include; nausea, dizziness, dissociation, sedation, and increases in blood pressure were generally mild to moderate and generally resolved on dosing days.

We also found that there was no evidence of abuse and misuse during all the esketamine trials. However the safety concern remains just given esketamine’s pharmacological similarity to ketamine. In fact the FDA label includes a boxed warning for abuse and misuse and requires this drug to be given in the REMS setting.

We also emphasize the fact that there is really limited data on the long-term safety of esketamine. The longest available trial that we identified was 48-week open label trial. In this trial we observed that there were some serious adverse events, including suicidal ideation and attempt, which were reported in approximately six percent of patients.

There were also a total of six patients who died in phase III trials of esketamine. However because this open label trial was an uncontrolled one, it was typical for us to be able to judge if it was really esketamine that was causing this serious adverse event or the underlying condition.  So we felt like there was need for more long-term data on the use of esketamine.

A couple of areas of uncertainty that we emphasized in our report, I already talked about one, the long-term safety, but also the long-term benefits and just thinking about the durability of the effect of this drug. We felt that we would need more data to be able to say something about that. Like I described earlier, we were interested in also looking at the net health benefit of this drug versus some of the other treatment options.

However, we were not able to do this because we did not find one. There was no head-to-head trial that actually looked at esketamine compared to some of the other treatment options. But also there were no good trials that were similar that would allow us to be able to do an indirect comparison. There was some differences in the inclusion criteria of these trials and some of the baseline characteristics.

We also emphasized some of the potential limitations of the clinical trial. For example, the use of esketamine resulted in immediate side effect such as dissociation and sedation. So it was unclear how the trial maintained blinding given that patients of esketamine would likely have known that they were only active agents.

We also emphasized the area of uncertainty in terms of the effectiveness of the broader TRD population because some patients that had associated psychotic features with their depression were excluded from the trial.

So putting all of this together we give an evidence rating of promising but inconclusive when comparing esketamine to not using esketamine in the background of antidepressants. Really what we mean by this, again looking at the ICER evidence rating, is that this could extend up to the substantial net benefit, but also on the other hand, potentially there could be a negative net benefit.

Like we typically do in our report, we give a textual description of why we landed here and we explain that even though that it looks like this drug is promising in terms of clinical efficacy, we cannot definitely rule out the possibility of a small net hard in the absence of a long-term safety data.

In comparing esketamine to the other treatment options like I described earlier, we did not have good data and good trials for us to be able to do an indirect comparison. So we gave it an I rating, really to say that we have insufficient evidence to be able to answer this question.

Putting all of this together and turning to the cost-effectiveness, again, putting the cost concentration on top of this foundation of the clinical effectiveness – and just want to give a general background in terms of what we are doing when we are doing a cost-effectiveness.

Really we are comparing the clinical and the cost outcomes of two or more care options. I have provided a simple formula here which is really the difference in the cost over the difference in the effectiveness of the two treatment options. Typically comparisons are done through a simulated computer model of patient and cost outcomes of difference care pathways.

Some of the measures of gain in health that typically measure when we do this include; functional improvement, how the drug may have prevented negative outcomes. Additonal life-years gained. Improvement in quality of life, and when we sum all of these for comparison across treatments we have what is called quality adjusted life years (QALYs) and also while we typically reported at ICER, equal value of life years gained (evLYGs). The evLYGs captures the quality of life changes, just as the QALY does, but all gains in lifetime are weighed exactly the same at full health.

So when health economist do all of these and they put all of these numbers together, what does it really mean?  Health economist like to report those results on something called a cost-effectiveness frontier. If you are  able to think about this frontier in full quadrants. If the results of the new intervention falls into either of these two quadrants, that is the upper left quadrant or the lower right quadrant, then really no further analysis may be required.

In the upper left quadrant is a new treatment that is more costly and less effective. I think we would all universally want to reject that treatment. If a new treatment falls into the lower right quadrant where the new treatment is less costly and more effective, we would all universally want to adopt that treatment. Where the decision gets tougher are the other two quadrants.

That is not typically the case that a new intervention falls in the lower left quadrant, where a new treatment is less costly and less effective. What is more typical for most treatments that come out is that they are more costly and more effective. When a drug falls in this quadrant we are faced with a decision which really is what will be threshold in terms of thinking about a new treatment.

The ideal threshold will be some form of society’s willingness to pay for a single increment in health improvements. Many academics use $50,000 per QALY gained as this threshold. At ICER we typically use R100,000 to R150,000 per QALY gain and for evLYG gained. That is because recent research has shown that opportunity to cost for people in and outside the health system, that is how much we should pay for new health without doing harm for people both in and out of the evLYG system is around $100,000 per QALY gained.

So we are going to have an upper bound to that, we have a range of $100,000 to $150,000 per QALY gain, where we typically report our health benefit price benchmark.

So putting all of this concentration and going back to our esketamine review for a minute, and thinking about the question that we are asking here. We are really interested in understanding what the cost-effectiveness of using esketamine versus not using esketamine, again in the background of antidepressants, looks like. If we find it not to be within that range of threshold I just described, what price would it fall in that threshold?

For this model we worked with our academic colleague from the University of Illinois, Dr. Dan DeShay(ph.), who took the lead in developing this lifetime model from a U.S. healthcare perspective. We also did a modified societal perspective as part of our typical methods at ICER. We used the meta-analysis I shared previously, to derive short-term efficacy input for the model, and long-term estimates for esketamine were derived from the open label trial.

We also used the information from a pragmatic trials STAR*D, to derive long-term clinical input related to alternative antidepressant treatments.

For cost, we used esketamine what priorities and information on non-drug related health care cost was obtained from research on in patient and out-patient costs by number of depression medication changes.

When we put all of that together into the model the result of our cost-effectiveness analysis shows that esketamine had an incremental cost-effectiveness ratio of about $200,000 per QALY gained. Which exceeds the commonly accepted threshold of $100,000 to $150,000 per QALY gained.

As part of our process typically, we would report the result but also uncertainty around our estimates and sensitivity analysis. Like I mentioned previously, we also did the modified societal perspective and the result also showed it was in the $180,000 per QALYs gained, which was still outside that threshold.

We estimated that in order to fall within that health benefit price benchmark of $100,000 to $150,000 per QALY gain, it would require a discount of about 25 to 50 percent from the list price.

Again, thinking about some of the benefits beyond health and social and ethical priorities, which I described we always believed it is an important part of thinking about the long-term value. Really, like I described, it is in recognition that there may be some aspects of either some additional outcomes that is important to patients, that may not truly be captured in the clinical evolution. But also some special concentration like the severity of the disease that we want to emphasize to be considered as we think about the long-term value of the new treatments.

So for example, in the esketamine review, one of the things that we stress was really the burden of this condition to resistant depression. We know that it increases the risk of suicide and result in a profound impact on the quality of life. Which includes even relationships with family and friends, and ability to participate in educational and work activities, and even perform activities of daily living.

Some of these additional context, we think it is important to emphasize, and you may think, so what do we do with this?  We like this because we want those negotiating for the price of new interventions to also consider these factors and think of ways to factor in considerations of health benefit price benchmark we have provided. Of course, it may also lead beyond the boundaries in some cases, but this is the conceptual approach to using these considerations and our value assessment framework for pricing.

So talking about the short-term affordability. Again this is typically estimated with potential budget impact. The potential budget impact is the total cost impact of implementing a health technology in a specific population over a period. When we do this we then compare our results to what is called a budget impact threshold, which we calculate and annually. The basic idea of the threshold really is that we don’t want the growth in drugs spending to grow faster than our whole economy by more than one percent.

ICER calculated potential budget impact threshold as ranged from presently a $1 billion to $800 million per year in net new spending.  

First I want to emphasize that our comparing to the threshold is not a budget cap. The goal of this approach is to be able to signal to stakeholders and policy makers when the amount of added health care costs associated with a new service – even one with good long-term value – may be difficult for the health system to absorb over the short term without displacing other needed services or contributing to rapid growth in health care insurance costs that may threaten access and affordability.

If utilization exceeds the budget impact threshold for that year, we use something that is called access and affordability a lot in our report, and it is really to point out these cautions of the possible policy steps that will be needed in absorbing this technology into the health care system.

You may wonder why this is important. I think a good way to think about this is here is an example of a review we did a few years ago, which is Solvaldi for hepatitis C, which was approved in 2013. This drug had a cure rate of 90 percent and a favorable side effect compared to existing therapies.

The cost-effectiveness of Solvaldi was less than $50,000 per QALY gain at the original launching price of about $80,000 for a 12-week treatment course. That cost-effectiveness ratio less than $50,000 per QALY was really good. However, to provide the drug to just half of the four million people in the U.S. with hepatitis C at launch would cost in excess of $200 billion.

To put that in perspective, in 2018 the U.S. spent $500 billion on all prescription drugs. That is why we think it is important for it to be part of the conversation on value.

So turning back to the esketamine and the potential budget impact, we actually issues access and affordability alert for these drugs because we estimated that only 16 percent of eligible patients with TRD, Treatment Resistant Depression, could be tested with esketamine at its least price before it exceeds that threshold of $819 million, which was the threshold that year.

In fact, if priced within the ICER health benefit price range, only about 30 percent of eligible patients could be treated.

Some of the discussions that we had at our public meeting, really suggested that uptake could approach or exceed this level given the unmeet need for better management of Treatment Resistant Depression.

We do all of this and build various aspects of our Value Assessment Framework and do our work and put it in our report, but we don’t just end there. As part of our process we do have a public meeting. In the U.S., consequential decisions around prescription drug prices and patient access are generally made based on oftentimes limited evidence and without patients in the room.

Our approach is to be able to bring this conversation to the public where all the various stakeholders, including patients, are present and where we can talk about pricing and coverage decisions more in the open. What we can all wrestle with – how to understand the benefits with drugs and other health care services bring to patients and their families. And how we can try to align our spending to make sure we are getting the most help we can get out of the limited dollars available.

For a public meeting we do have an Appraisal Committee that are made up usually for public meetings about 15 members – about 15 members that are part of the Appraisal Committee, and these are made up of practicing physicians, methodological experts, leaders in patient advocacy and engagement, that provide objective independent guidance on the application of medical evidence.

So ahead of the public meeting we also send out the report and they are able to review it entirely before the public meeting.

In addition, the members of the public that are there, the Appraisal Committee also vote on our evidence and some of the decisions that we have made in the report. So at a public meeting we will have the evidence presentation first, which is usually a summary of what we have in our report. But we also have something that we call Expert Perspectives, which is having clinical experts and patient representatives in the room. This allows the members of the Appraisal Committee to be able to ask these experts specific questions about the condition and about how patients are being treated.

As part of the public meeting we also have comments from the general public and manufacturers and patients, where they can talk about methods and the conclusions of the report. We often get an opportunity to also respond to their comments.

At the end of all of this the members of the Appraisal Committee vote on both the clinical evidence and the cost effectiveness and some of the potential other benefits. Sometimes they agree with us and sometimes they disagree, but it is always a good opportunity to be able to wrestle with the evidence together.

Finally, as part of our public meeting, I think a key part of it is the policy roundtable, which is typically facilitated by our president, Dr. Steve Pearson, where we are able to bring the key stakeholders together, including patient representatives, clinical experts, manufacturers and payors. To then be able to ask the question, now the drug is approved – a drug has probably been set at a time, now what? We have to think about ways to make sure that we design coverage criteria in a way that patients would have good access.

It is also an opportunity to be able to come up with some recommendations that have helped to set the agenda for future research.

So in our final report, in addition to all of the evidence we have, we also have some of the information and the public meeting deliberation and the result of the votes, but also some of the recommendations that come from the Policy Roundtable.

Just a few recommendations from the Esketamine Review, again knowing that I probably have lots of researchers on the line, I just wanted to emphasize that we one, it would be good to read the actual report because we have like four or five pages of recommendations. I just wanted to highlight these to manufacturers and researchers. We should conduct studies that compare esketamine with other treatment options because this would really allow real-world, long-term assessment of the comparative effectiveness of this drug.

And also to payers, one of the things that we emphasize was just that given the considerable uncertainty that remains regarding the longer-term benefits and risks of esketamine, it is in fact reasonable for them to develop prior authorization that will ensure the patients are carefully selected and managed by clinicians with the necessary expertise to ensure appropriate care.

In the last few minutes I am just going to highlight some of the ways in which ICER’s work has been applied in the real-world. First to provide a general overview of what ICER reports find regarding pricing in general. Approximately 25 percent of the treatment we assess have found to be priced in alignment with the typical cost-effectiveness standards.

These drugs range from drugs that have large impact on population health to gene therapy and CAR-T and even orphan drugs.

On average, specialty drugs would need to be discounted by about 55 percent to reach standard levels of cost effectiveness.

Here are some of examples of drugs that we have found to align with cost-effective standards. One of the things I want to emphasize is really to see that for many of these drugs the ICER evidence rating is on the high side, where we have found them to provide some form of benefit compared to the standard treatment.

Here are some of the examples of drugs that we have found not to align with cost-effectiveness standards. I just want to emphasize the evidence rating here range from having insufficient to a, also proving substantial health net benefits. That is to say that for some of these drugs again, they are really great drugs. The fact that we have said that we don’t think they are cost-effective at the price doesn’t mean they aren’t great drugs. We really tried to emphasize this and tease apart this in the report so that it is clear even when we think a drug really provides the substantial net health benefit.

Again, just highlighting some of the people that use ICER Assessments. One thing I want to say here is that unlike governed HTA bodies in other countries like Nice in the UK, we do not have regulatory authorities. What we are providing is more advisory and recommendations that can be used by decisionmakers.

Some of the people that use our assessments include drug makers and payers. This helps theme to negotiate prices in conjunction with fair access. Payers and employer groups, and also use our assessments to be able to guide coverage decisions and pricing negotiations. And policymakers also use our report as a source of independent evaluation of value and suggested value-based prices.

So just to highlight some drugs in which I think our work is really leading to fair pricing and fair access, again, just zoning in quickly in some of the examples. Here is one drug that we reviewed a few years ago, Zolgensma, which is a one-time gene therapy for fatal childhood condition. When we reviewed this drug we found that it had adequate evidence to show significant net health benefit I patients, and we found it to be fairly priced. We said a fair price would range from $1.1 million to $2.1 million. Novartis actually cited our analysis in its discussion of price.

Here is another example, Praluent, which is a PCSK9 inhibitor. When we did the evaluation of this drug we found it not to be cost-effective. There was new data that came out from a new trial and they actually shared with us. We updated our findings and still found it not to be cost-effective using the standard thresholds. The companies, Sanofi and Regeneron actually reduced the price of their drug by over 50 percent to be able to reach what we described as a value-based price. As a result, Express Scripts, among other payers, expanded access to these treatments and lowered the out of pocket expenses.

Finally, I want to highlight the Aduhelm Case Study, which many of you may know about, which is a controversial new treatment for Alzheimer’s disease. I think this drug really highlights many of the tensions in the U.S. health system. Which includes patients’ unmet need, uncertainty about evidence, pricing that doesn’t align with effectiveness, equitable access, and overwhelming cost.

We reviewed this drug and actually we independently evaluated the clinical evidence when the FDA’s internal disagreements raised some doubts. We were able to set the benchmark for how we thought this drug could be fairly priced. As part of our process also convened a transparent, public discussion including the manufacturer, the patients, the clinical experts and key stakeholders. We established guidelines for coverage and future research that we believe that policymakers and researchers can use.

At this point, I am going to stop so that I can answer any questions. Thanks again for all listening to me.

Agenda Item: Q&A

JOSHUA GORDON: Thank you so much, Dr. Agboola, for a really wonderful and thought-provoking talk. There are a few questions from the audience, but I just wanted to remind our audience members please do enter questions into the Q&A function so that I can ask them. Asking the questions that you and I will probably throw in a few from myself as well.

Here is one question from the audience, it relates to abuse potential. There have been known cases where people starting seeking esketamine or ketamine from different illicit sources after experiencing the clinic. Dr. Agboola, I know that you are not an expert on esketamine and its use in the clinic, so I am going to ask the question slightly differently. Which is how do you deal with off-label uses or uses of a drug or potential uses of a drug, that are different from the ones that were studied in the publications that you used for your studies?

So for example, esketamine is specifically targeted at treat resistant depression, but of course once on the market maybe people will use it for first-line treatment, which has a very, very different economic potential compared to other drugs which are cheaper and might work for those folks who haven’t yet been demonstrated to be treatment resistant.

Do you include that kind of work into your models or do your models really hue to intended use or intended let’s say marketing use, of these drugs?

FOLUSO AGBOOLA: So that is a very good question. Typically we target our model in terms of to the intended use when we think one, FDA label would consider that, but again it is also part of the reason why we engage with clinical experts. Because something might not be on the label but it is generally accepted in the clinical community to be used that way.

Oftentimes, if we hear that, we build that into our model and can actually model it that way. So again, it depends on a combination of how it is approved in combination of what it is generally being done in the clinical world.

JOSHUA GORDON: Gotcha, so you can look at both the intended use and other uses and factor that into the model.

FOLUSO AGBOOLA: Yes.

JOSHUA GORDON: Thanks.

FOLUSO AGBOOLA: And oftentimes these would also guide our recommendations, which we generate from it depending on what we find.

JOSHUA GORDON: Have you or ICER done any studies that looked at treatments that might be underutilized – identified treatments that are particularly effective that are not being used for perhaps a reason related to social stigma? I am asking this because of a question who is asking about for example, ECT, which is very, very effective, relatively inexpensive, and many in the psychiatry community feel is underutilized.

FOLUSO AGBOOLA: Also a very great question. Generally we would want to include – in fact, when we were developing our scope throughout those projects, it is part of the reason why we – we really wanted ECT to be in our scope. We included it in our scope with the hope – because again, we heard just how effective it is, with the hope that we would actually be able to say much more about it. Unfortunately because of the differences in trials, it was difficult for us beyond highlighting the evidence, which we have in our report, even though I didn’t share as part of this presentation, on its own.

But generally, even in other reviews where we know of drugs where we think are underutilized but are actually good. Oftentimes we want to spotlight that in our report. We want to make sure it is part of the evaluation.

The problem with some of this is there may not be good studies and good research for us to be able to build the evidence on. But again, this is something we try to highlight in our report.

JOSHUA GORDON: I imagine you must be very supportive of efforts by organizations like PCORI and HRSA to look at real-world effectiveness and particularly comparative effectiveness between drugs?

FOLUSO AGBOOLA: Yes.

JOSHUA GORDON: That are already on the market.

FOLUSO AGBOOLA: Yes. So that is generally a pull for us. Oftentimes when we are looking at our scope of a new treatment we want to know – which is why we end of speaking – we try to speak with different clinical experts because from them we get insight about some of the things that we typically would not have thought about as good comparators. But with them physically telling us how great a particular drug may be, this just gets us interested in including it as part of our scope.

JOSHUA GORDON: Okay. There is another question pertaining to whether and how you can expand the influence of these reports. Particularly asking about CMS and Medicare. What do you do in terms of getting your recommendations to the health care policymakers and payers who can use that information, and do you see organizations like CMS actually acting on your recommendations?

FOLUSO AGBOOLA: That is great. One of our reports publicly available on the website – and I shared how some of the people that we know are using our report. I think a recent survey, we assume that about 75 percent of payers actually use ICER reports. We also are aware of some state Medicaid that actually use our reports in thinking about their coverage decisions.

Again, just thinking about the Aduhelm case study that I shared, some of the things we shared in our recommendation in how we should thinking about what esketamine, for example, to think about the coverage in evidence and how the coverage decision for Medicare that we put in our report, it sounds like some of those things are also showing up in their concentration. Although I am not sure our report was exactly cited. So whether they used it or not, oftentimes it is hard to track whether are reports are used in such decisions but we know that a lot of payers are actually using our reports.

JOSHUA GORDON: Another question pertains to your costing estimates. You mentioned some factors that go into the cost estimates beyond just the pricing of the drug like health care acquisition costs, et cetera, what about training costs? So esketamine has to be given intranasally, there is probably training costs associated with that. It is also a drug that has to be administered I believe, in clinics so there are patient costs related to that – traveling to the clinics, co-pays or deductibles. What other costs besides just the price of the drug do you put into your models?

FOLUSO AGBOOLA: So we put some other – I think for sure in terms of thinking about travel cost locations, it doesn’t go into the models. Especially again, this is from the health care perspective. A drug that is administered in a hospital and thinking about the administration costs and some other non-drug related costs, typically will be included in our model.

JOSHUA GORDON: Thank you so much. We really enjoyed the talk, Dr. Agboola. I know there was a lot of enthusiasm from hearing from you. Really wonderful that you chose esketamine as an example. I will just point out that despite – I received many, many questions about esketamine, and whenever I say that this was a major advance that was supported over the years by lots and lots of research conducted at NIMH and by NIMH funded investigators, every time I say this is an excellent example of basic science leading to novel therapies everyone says, but it is so expensive.

So now I have a counter and I can say actually compared to other drugs that recently marketed, the cost is pretty good compared to the cost-effectiveness. Although obviously, we have some challenges at that cost of meeting the need without bankrupting the system. I assume the payers are negotiating based upon that need.

Thank you so much again for coming to us today, for fielding our questions. Everyone else I look forward to seeing you all at the next Director’s Innovation Speakers Series. A reminder, that this lecture like most from the series, will be available online within a couple of weeks.

Bye-bye all.