The Briefing: Financial Services

0:44 Please welcome head of financial services at Anthropic, Jonathan Pelosi. Thank you. Thank you for being here. On behalf of Daario and the entire team, we really appreciate it. And for those joining us on the live stream, thank you for dialing in. In the room, we have the CEOs and CTOs and CIOS of the world's most influential financial services firms. and carving out your morning to be with us is not lost on us. So, we promise to make it worth your while.

1:24 A year ago today, when we launched Claw for Financial Services, we made a promise. We said, "This technology is going to help change the way your people work, your business operates, the products and services your organization be capable of building." And today, that promise has arrived. And what changed isn't just new models or new features of which there have been a lot. But it's the answer to the question every your boards has been asking. Is this hype or is it real?

2:01 And we're thrilled to announce that today it is real. 12 months ago, a demo has been brought to life today by helping reconcile the books, process claims, build new models, transform the front and back office, and that's what we're here to talk about. So this morning, you're going to have a chance to hear from our product leaders, our research leaders on where this industry is today and where it's going.

2:39 And you're going to hear from your peers at Goldman Sachs, JP Morgan, and AIG on how they've driven this transformation within their own organizations in a safe and responsible way. The intention of this morning is not to do a pitch or provide a laundry list of all the new things coming out. It's to have a candid conversation about where the industry is and where it's going and how we Anthropic want to help partner with you to get there.

3:12 And there's no better pair to kick us off than our very own CEO Dario Amode and the CEO of JP Morgan Chase Jamie Diamond in conversation with Andrew Ross Orcin. So please join me and welcome them on stage.

3:44 Good morning everybody. We have so much to talk about this morning about where AI really lives, how it lives within the world of finance, the issues around cyber security, around compute, about the role of government and regulation in all of this and how it's going to affect so many of us. So here's where I want to start the conversation. And Dario, I think we spoke at Dealbook a couple months ago, but even a year ago, we were talking about just the acceleration of all of this and how fast it would move. It has moved fast, but if you were to sort of think about what it looks like 6 to 12 months out, even from now, in a very real world sense, what does it feel like to you?

4:30 Yeah, I mean there there is this dynamic where you know you can write down a line on a curve like the the the kind of exponential and and we've been right every time, right? We've been right about the way the business grows. We've been right about the how the capabilities grow as measured by various benchmarks. But there's this inherent unpredictability to what does it look like when you actually make a model of a particular capability? What products do you actually build on top of it? like what what what you know what is the texture of what it plays out in reality.

5:00 So that's my first caveat that there is a fundamental unpredictability to this. But but you know I I I will say that I think we're going to see few a few things that you know are are are kind of predictable from the existing trends. anything anything that you do today, there's going to be more accuracy, more reliability, right? As as as the models get better, they do new things, but also things that they do now where we have worries. Hey, did it actually get that right? It wrote this research report, but can I can I rely on it? Right? We were just talking about, you know, both of you kind of using, you know, using using quad for, you know, basically to do to basically to do research on something and like it's really plausible. It's right most of the time, but how can I be sure? Those those error rates are going to go down. We're going to see also more agency, more autonomy, ability to do things end to end, right?

5:56 Ability to instead of, you know, here's this thing I would farm out for an hour, here's this thing that I would give to a team for a week, two weeks. so, so I think I think I think those are the things we're going to see. maybe one more would be pulling together different pieces right searching the web searching different financial data ability to do various other things. So so I think autonomy reliability and accuracy and doing things end to end.

6:24 >> Jamie Diamond how do you use it yourself personally and how are you using it in the bank in a material way? >> Yeah. So first of all welcome everybody Dario thanks for having us here. personally this weekend I actually went on Claude code and I asked the following questions of your financial services. I I'm the CEO of a bank kind of give it context. I want to look at five years. >> It doesn't know who you are yet.

6:47 >> Well it actually keeps records of who you are. So it actually probably did but I want to know about asset swaps and treasury bid ass spreads and liquidity in the markets and investment grade and you know risk and that and just you know in 20 minutes it created this huge dashboard with all the backup and all the research and it was very accurate about what I wanted and stuff like that. So the bank first used it in 2012 I believe when we have having some of these mortgage problems and back then I call it more advanced math. It was like machine learning where we notice patterns and run through lots of data that a human being couldn't do and notice those patterns. That was early on and we once I saw it for the first time I said holy I want to do that ourselves. So we built our own department. Obviously we can't do it he does but so we you know we have a lot of people deployed doing it. hundreds of use cases risk fraud marketing design note takingaking document review.

7:39 and it and it's just starting. I mean, I it'll be customer errors and and as you get better at, there'll be second order effects we haven't thought of yet. There'll be databases we can use in completely different ways. And it'll also, like you already mentioned, you know, we have we have internal data, we have external data, we've got data we buy, there's data in the web. It'll be able to do all of that and, you know, just get to a different level of managing risk or, you know, managing customer p personalization, etc. So, >> one of the things that you announced just yesterday is a deal with some other firms including a competitor of yours as well as Blackstone to effectively put AI into portfolio companies in private equity all across the country.

8:24 What does that look like? And and one of the questions I had for you actually is why do that? meaning do you not think that there's enough other companies that could go do it on their own or is this about trying to get this at at to accelerate how quickly it happens? >> Yeah. So I think the dynamic I would describe this is this is an example of a familiar dynamic that we've seen with the technology which is that the technology itself as I said is on this exponential and kind of moves at lightning pace but large enterprises aren't like that right even if they have CEOs who really get it who are who are who are visionaries as as Jamie as Jamie does you know even even the ones who are most visionary you have to control this company of thousands tens of thousands hundreds of thousands of people it's like You know, you just have to educate everyone on where the technology is, how to use it. And by the time you're done educating them on a certain level of technology, the next model has come out and made new use cases possible. Some pitfalls that were present are not are not are not there before. Some things are are more accurate. So, this is an incredible pace of change and we're held back. our our ability to commercialize this technology to gain revenue for this technology is is held back not by the power of the models and the economic value can generate in principles but that sort of diffusion through the world. It is both the the case that that diffusion is faster than we've seen it for any technology previously in the world but it is still much slower than what the technology itself is capable of and how the technology is moving fast. Now we have our own go-to market team for doing this but think about it and Enthropic is you know roughly a 3500 person company you know our our our go to market team is half a thousand going on a thousand you know some of the companies that have you know revenue of the same order of magnitude as us have 50,000 person sales teams. You can't hire a 50,000 person sales team overnight. even if you could, we need help. And and so this this idea that, you know, there are parts of the economy that we can leverage that we can use to say, hey, there are some common lessons here in how to distribute this technology to your portfolio companies and make you more productive. this this kind of company that we're putting together you know it can have a kind of you know it can have it can have a lot of staff that specialize in this that don't have to work directly at anthropic that can focus on the diffusion through society rather than on the inherent technology itself.

10:59 >> There is likely to be a trillion dollars spent in this space over the next year which is astonishing if true. Do you say as as somebody in your business this makes a lot of sense to you? Do you think this is crazy? What do you think? >> So I think you're referring to data centers this year is 450 billion. Next year it's 150 billion. >> The way I look at it is that in total it will make sense. I if you want to try to pick the winners and losers, you will have a hard time. So there will be losers and they have to be winners.

11:33 People said I told you so and stuff like that. But the technology itself is so powerful, it's worth a trillion dollars of investment. And the investment is way beyond that, by the way. It's in chips, it's in wires, and it's in hardware and all these other various things. So, but you know, technology tends to pay for itself, just not in a straight line. >> You've long talked about it as a cone because we've talked about your spending and people say, "Is this going to make sense?" And you've said it could go >> perfectly or it could go terribly.

12:02 Things have changed though even in the last three months. Yeah, I mean you know I I I I wouldn't so much say that things have things have you know that framework has changed so much as it's it's kind of illustrated this framework right you know I think I think what I said when we talked four or five months ago was you know we've had this kind of 10x you know rate of revenue growth in the past and we don't know whether that'll continue or whether that'll slow down and so you know you have this 10x uncertainty if you look back at the first quarter of the year the annualized rate of the annualized rate of growth if you if you annualized it, which is not going to continue annualized, but just if you annualized over that quarter, we didn't get 10x, which I thought was the biggest we could get. We got 80x. So like the cone is even wider than I thought. So I so I I I I think I think focusing on like whether you know does does kind of do these investments make sense. It it makes more sense to focus on wow this is a situation of actually radical uncertainty and it is actually very hard to predict where you know we were prepared for 10x growth and then and then we saw for one quarter and you know we don't know what's going to happen in the quarters after this annualized 80x growth. If you're as successful as you want to be, and this I'm curious where you land on this, Jamie, too. There are so many companies in this room that rely on other software software companies.

13:24 And there's a real question right now about the future of software, whether there's any moat around software. Do you say to yourself that all that software that you're using at the bank right now or even you may be using inside Anthropic ultimately will not be that way? First, I'll just go back to this this growth he has. I would separate a little bit consumer from enterprise because it's not clear to me how consumer is going to play out. A lot of you probably use you can use Gemini, you can use stuff, you can use it for free that may be completely sufficient for your requirements. enterpris you look at always a little bit more like you know I'm making an investment and what I'm getting for it and if it makes you better off you're going to make the investment you know whether it's hardware software or machine learning or adding salespeople and and your question was the other question is what >> well I think the real question the other big question is what happens to all the other software companies in the world >> it'll be both you know a lot of these software companies and this creates another little bit of a risk they're already adding agents to what they do so it may be easier to take their software add the agent use that than try to redo it. But I could imagine like I just mentioned what I was talking about doing, you know, there a lot of data sets out there that I can go do directly. I don't have to go through third parties. So I think it'll be a little bit of both and it'll be what their product is, who they're servicing.

14:40 And the other thing is a lot of smaller companies. They're going to buy those services from the vendor. Like there are a lot of small banks all use the same three vendors for their main statement system, stuff like that. Those big vendors will probably deploy, my guess is Claude into those banking systems and they'll buy it from them. they just can't afford to do it themselves. >> Where do you land on that? >> Yeah. So you know the framework I would use to think about it think about SAS in general but I think it also you know it also applies to SAS as as applied to financial services services in particular is is look there's you know there's this concept in engineering called AMD doll's law which is that if you you know let's say you have a complex process with many inputs and you speed up you know many of the inputs like 50% of the inputs 90% of the inputs you're then kind of held back by the ones that you're you're kind of not quite able to speed up and so I I I think In terms of moes that software companies have, it's going to be a little bit the same. you know, I am a I'm a maximalist in terms of the model's ability to write code and write software. I think if if your moat is our software is complex and difficult to write and we can write it and others can't match it, I think that's going away. but if you look at a company, any any company of any significant size is complex, right? They have a lot of different sources of modes. So what we're about to see is a rapid change where many of the modes go away but some stay around and there are new modes that kind of arise in a world of AI that we can't kind of conceive of yet. And so when I think about it my view is look you know software as an as an industry because it's getting cheaper to produce software because there's going to be so much more of it like in aggregate that's going to go up. It may not come from incumbents necessarily, right? I don't know what will happen to the group of today's SAS incumbents as a as a as a group. That's more indeterminant. I think individual SAS companies, it's very possible for them to lose market value, go bankrupt, completely go bust.

16:39 But but it it it depends on on the response, right? I think there are incumbents today that are going to see very clearly we have a lot of moat here. the moes here are going away. we're really going to pivot and we'll do better than we did before. And there are others who are not going to pay attention, who are going to be blindsided and you know they're going to have a really bad time. >> Okay, I want to get to a big headline because it is the issue of cyber security and Mythos in particular.

17:07 the White House has been anxious about it. There's meetings between the bankers and the Treasury about the possible cyber security risks. When you think about this, I mean, there is a bit of a freakout going on. Is it warranted? You >> want to start? >> You know, I I wrote my chairman's letter this year and I had a little section, cyber is our biggest risk. It's been our biggest risk for years. And I said, and it'll be made worse by AI and like a week later, Mitos comes out. Yes, it very heightened risk. They I think Mitos did exactly the right thing. you know Dario and anthropic to to lay it out give people a chance to study it understand the vulnerabilities come up with plans so we can handle it it's very important I mean I think that in big picture AI is going to cure some cancers folks it's going to save lives it's going to make life much better this is a temporary thing because this these code also new code you're going to write will have all the good stuff already built into it so there's but there's a transitory period but we have to make sure we understand and again it's not just for us so think of I'll split a couple categories JP Morgan own applications probably can handle it open source we don't all need to burn the same tokens fixing the same open source we can all fix that for everybody >> saying the open source that's inside JP Morgan >> no the open source that JP uses but the same open source you use or the same open source Goldman uses etc and then there's third parties there's critical to non-critical and how do we get that fixed and we should use that to help everybody this not about the big banks or or all it's all critical infrastructure by the way it's just to make sure we understand the risk that this new technology >> onetime transition or do you see this as something that is just an ongoing arms race?

18:51 >> I I I would I don't know you may want to I say one big on time which may take a year or so. but then after that well you'll always have to be adjusting to the new models. >> Well you're estimating this is a year process. >> I don't know yet. Yeah. >> So a few points. One one I would say you know I I completely agree with Jamie the risks are very real. This is why we took the actions we did. but but they're also in some sense not that surprising. If I go back to my whole way of viewing the world of AI, which is the exponential, we've been seeing warnings of this for a while, right? you know, a year ago, we we first started, you know, looking at, you know, using claude to review code for security vulnerabilities, right? It wasn't that good back then, but you know, it it provided some value for engineers who didn't know anything about security. You know, six months ago, we f we found the first attempts by Chinese state linked actors to conduct attacks on US tech companies with Claude. We sent someone to testify in front of Congress about this. We said, "Look, this is a you know, this is start this is the merging problem now. It's going to be a big problem soon." Three months ago with a model before Mythos, we found a bunch of vulnerabilities in Firefox. That's that's an example of the open source Jamie was talking about. We disclosed it to Firefox. We announced it together.

20:04 it was something like 20 vulnerabilities. Now with Mythos, we found almost 300 vulnerabilities in Firefox and thousands probably by now tens of thousands beyond closed doors. So I think I think the risks are the risks are very real. you know in terms of the government response couple points. One is you know I I kind of actually commend them for apprehending fairly quickly that that there is really a you know that that there is a serious risk here. you know, as was reported, I had a very good meeting with Secretary Besson.

20:39 where, you know, we we we we talked about the the the cyber risks. you know, the secretary is getting getting very good advice from, you know, PE people across the banks and financial services. And so, I think they're taking the risks I think they're taking the risk seriously and I really commend them for what they're doing. But because this is emerging so quickly, you know, there's I think I think there's inherently a scramble to figure it out.

21:03 And and I you know, I think I think what we have now is a sort of you know, a bit of a bit of a bit of an ad hoc process, right? Something that almost mimics regulation but lacks the the consistency. >> What do you see the biggest danger being right now? Well, the the what is the true threat that just everything can't get patched fast enough and that simply by identifying the problems it it effectively allows allows the bad guys in.

21:28 >> Yeah. So, you know, we have other companies and eventually open- source models catching up, right? So, you know, I would guess the the you know, other companies are maybe one to three months behind. you know the the kind of major other credible US labs are maybe one to three months behind and you know the Chinese models are maybe six to 12 months behind so I think we have roughly that amount of time to fix all these vulnerabilities right we've identified these tens of thousands of vulnerabilities the reason we haven't announced many of them is of course you know only a small fraction have been fixed right we fixed 300 with Firefox like we're going to do periodic updates on what has been fixed but you know not everything has been fixed because if we announce something without it being fixed then the bad guys will exploit it and and so I think I think the danger is just you know some enormous increase in in the amount of vulnerabilities in the you know in the amount of of of breaches in the financial damage that's done from you know ransomware on schools hospitals not to mention banks financial accounts you know pharma various other things now to answer the question Jamie asked I actually think if we handle this right in 6 to 12 months, which is the time on which we have to handle it because the you know the the the Chinese models will catch up by that time. If we handle this right, we could be in a better position than we started in because we fixed all these bugs.

22:55 There are only so many bugs to find, right? The depth doesn't go forever. And if we're rewriting all of our mo all of our code with models like mythos, we can use mythos to write code that's inherently more secure by design. So I think there's a better world on the other side of this. It's not it's this isn't just about fear. This is about a moment of danger where if we respond to it correctly and I think we've started to take the first steps then we can have a better world on the other side of it.

23:26 One of the worries that the White House has expressed is about compute. That there's not enough compute in this moment for the government to be using this software and be able to find all of its bugs and everybody on the commercial side to be able to do that as well. >> That that so so that is actually a straight that's actually a straight mis that's actually a straight misconception. It may even be a misconception that the government thinks that. I don't know. But it's it's definitely a misconception that it's true. mythos mythos consumes you know a a very tiny fraction of enthropics compute. It's it's no issue whatsoever to to to you know to to increase increase the amount of compute by 3x or 10x. There's there's no contention between tokens for the government and tokens for the private sector. Now, where I think they do have legitimate concerns is the idea that if we open up mythos to too many actors, you know, again, you know, I I you know, JP Morgan was a was a you know, was a was a kind of, you know, one of the one of the prime headline partners, but you know, I think I think you know, Jaime is aware that like there are a lot of people within the bank. He has to be careful who he gives mythos to. If we open up mythos to hundreds of organizations, then there's this there's this danger. On the other hand, we need to be fair to everyone. We need to fix things as quickly as possible. We're hoping eventually to get to the point where we have safeguards where we can release mythos to everyone and block the cyber problems. So the question of how slow versus how fast to go through that gradual release process. It can't be infinitely slow because you know again we have the the Chinese models are our pacing. They're 6 to 12 months behind.

25:07 that is I think the thing that the government is grappling with that we're trying to work with them and I think that's the area where you know we're all working with this we're all grappling with it for the first time and you know I was I was grateful to see that the again sounds like the government is starting to think about a more streamlined process that would apply equally to all of the different all of the different companies in the space right I think what we need here is regularity and predictability Because the last thing we want is an arbitrary process that is unpredictable for the private sector. Like it's good to do these responsible releases. Mythos was kind of one of a kind, but we have several different companies in the space. They all need to be able to make ordinary commercial releases every month, right? There's an ordinary commercial race and that race needs to needs to continue but just subject to some some guard rails you know to make sure we don't cause cyber havoc. Jamie, you've been highly regulated >> perspective a little bit. So the banks, the large banks have already gone together. They have a lot of working teams. You're going to hear it from Lori Beer who's our co next triaging the issues look at them looking at all the critical paths of failure all the infrastructure we rely on including Fed wire and all the things like that. It's got to be done by industry. It should be done eventually to help all the banks.

26:28 So it's not about the big banks about protecting the system. So we're already looking at what we need to do to help smaller banks etc. Every industry needs to do it. The government can't do all that. The government can help us by there are certain laws we have to deal with. There are certain requirements they can help us by not overburing us when we have, you know, eight regulators asking us the same damn questions every day, you know. So, there are a lot of things they can do, but I I agree. I applaud the fact they've they're they know it's serious. They've taken it. I I know they've spoken to other industry groups. And at one point, all the industries should get together, you know, of course, look at all each other's notes. What did we do? What do we learn? And we don't all need to do the open source code. So we there there's just a lot of things to improve the process. And you mentioned the thing about patches. In the old days, you put out a patch, people had a week or two to fix it. And now I say it's got to be like minutes. Okay. It's something like that. So you got to just change the process by which we go about these things, which is doable. Dario, just speak to this though because there is a headline just the past 24 hours about a potential executive order to create either some kind of working group or some people have described it as an FDA that would approve >> you described as an FDA >> that would that would approve each of these new model iterations and whether you think that's a good or bad thing.

27:43 >> Yeah. So, you know, I think I think this is an area, you know, whenever there's an emerging technology, we got to get the balance just right. and you know I think there's some you know there's some bad examples on both sides and I think there's a good example in the middle. Like I think you know what I've been saying for a while is you know we we don't we don't want a wild west where where you can just do anything right. We chose to do what we did with with mythos but there's there was literally no law no requirement you know preventing us from just offering this thing in the wild with no safeguard.

28:12 Should there be that's not yes so that's not that's not going to work. Now, the example you gave of the FDA, I think the FDA, you know, slows down medical progress a lot. So, you know, that's a cautionary tale in the other direction. what I think an an example we could look to where where I think they've gotten the balance pretty right is is the automotive industry. where, you know, we understand that that that automobiles are a technology with immense economic value. We couldn't live without them. if we hampered the automotive industry. It'd be horrible for the economy, horrible for human welfare and human life and the ability ability to live our lives. And yet we understand that, you know, you can't just, you know, you can't just start a car company, make a car, start, you know, without without any hey, you know, are there brakes on this thing? Does it does it does it do that? And and so we need to li little little by little kind of grapple or grope our way to you know to to some some process that that lets the industry operate expeditiously is fair but but puts guard rails on the most serious things.

29:17 >> Before we wrap up, I've got two more questions. One is just if you could give us a state of play of what's going on with the relationship between Anthropic and the Pentagon. I ask because when you start to think about executive orders and approving models in in the future, there's the prospect of politicization. >> Yeah, look, I mean, you know, I I you know, this is an ongoing legal case, so there's not much I can say about it. I mean, you know, we've kind of made our position clear in terms of what our red lines are and that actually 90% of 98% or so of of of use cases, you know, we're we're actually, you know, very comfortable with and we think are good for this country. We were the first to put our models in the classified network, the first to work with the Pentagon and the the Pentagon and the DO and you know it's it's just about these couple use cases where you know I think there's even been work in Congress to kind of try to enact these red these red lines into law and and you know we hope that those that those those succeed if those succeed it simplifies things greatly. But you know in the meantime we're here this is our position. We're still you know excited to serve our country country patriotically. you know, subject to the position positions that we've laid down. So, you know, my my I think anthropics view is the same as it always has been. You know, in in terms of politicization, this is this is the point this is, you know, this is the point I made about being kind of, you know, streamlined and systematic and fair to everyone, right? you know the the purpose of you know the purpose of kind of laws and and not doing things in an ad hoc way is that you know then all all the companies are are at least in principle you know I know it's more complicated in practice treated in the same way and so we should aspire to that even if it's never going to happen perfectly that way. Jamie Dan Schulman, CEO of Verizon, was in the Wall Street Journal just about a week and a half ago now, suggesting that in the next two to five years, he says he thinks there could be somewhere between 20 to 30% unemployment in America as a function of AI.

31:12 >> Yeah. So to go back to context, technologies have made mankind's lives better forever. Starting with agriculture and electricity and internet, all these things were used by bad guys too. by the way all of them they got some kind of regulation proper overregulated sometimes so I think it's a legitimate when you look at this technology it is exponential and if you look at agriculture and steam engine electricity internet it took years to kind of roll those in and the capitalist society is very good at recreating jobs and creating new things and life is better not always if that town loses a you know loses a factory but in general better and we should worry about the people get hurt by something so I think it's a legitimate concern concern. I don't think it's going to be like quite like that. But my view in life is we don't really know how quickly it's going to go. Therefore, we should get prepared that if it was going to go quicker than we thought. You know, we don't want to take two million truck drivers or something like that and all of a sudden they don't have a job or the next job is making $25,000 a year as opposed to 120,000. And can we be prepared for that? Yes. And you know, to me, it's going to be local training. There a lot of great jobs out there that are open today. They're probably better than some of these other jobs. And so we Jay Moore is going to have redeployment. We've been doing it our whole lives. We're going to double down on it. We know specific jobs have been lost. These are great people. So we say to them, "Hey, you've got all this unbelievably talent.

32:32 Would you want to move over there?" You know, income assistance, relocation, sometimes early retirement. So we should get prepared to minimize any negatives there. I don't think the right thing to do is to sit here and say, "My god, it's going to be 30% or 50% or 100%." that it'll make life better, you know, and there will be problems of course as social so and I think we're going to be prepared. I hope that one day I've asked my own folks here to write position papers and what we and government can do to humiliate the negative effects of AI if it has any just like you know we should have done the same thing on trade and there was something in trade called trade adjustment assistance. It was done right after NAFTA to give people income assistance and relocation, reskilling didn't work because it wasn't set up right because it made too hard to get the the benefit. So we it it's solvable but only with collaboration between government and business.

33:25 >> Final word to you Darren. What do you think? >> Yeah, you know you know I think I think what I've always said here is this is an area of very high uncertainty. you know, this is like with many things in AI. Many things in this technology that I've seen over the last few years have this feeling of kind of an unstoppable object hitting an immovable post or something like that and you're not quite sure what's going to happen.

33:45 you know in this case the the you know the immovable post is you know Jaime's right like you know when when technology kind of increases the pie like the economy is very good at kind of again you know it's related to kind of the AMD doll's law Jevans paradox like things are things are flexible things are fungeible they tend to move if you automate 90% of the job then everyone does the 10% of the job and the 10% kind of kind of expands to be 100% of what people do and kind of 10xes their productivity. Now the unstoppable object is AI is moving faster than all these previous technologies. And so when you strain a system more than you know than it's usually strained, it's possible you get these weird behaviors and this big disruption and you know you could get some of these these big numbers which I think no one knows but that you know people have thrown out and that I have I have worried about as well. So I I don't think anyone knows, right? Whenever people clip these things on Twitter, they only clip the part where you say the most salacious thing. But from the beginning, I've expressed uncertainty about all of this. I'm not sure what's going to happen. I agree with Jamie that that therefore, you know, Jaime thinks of himself as managing risk. I think of myself as managing risk with this new technology. Like that's a core part of my job. So we we need to think about the pessimistic case and plan for the pessimistic case and then if it doesn't happen we're very happy.

35:12 I agree it's going to be a collaboration between industry and government. We need to think about doing macroeconomic policy differently. We have like a huge pie. We don't want the pie is going to get much bigger. We don't want to disincentivize it but like that pie gives us an opportunity to do things with it to help the people who might in principle end up worse off. Trade adjustment is one idea. you know, we've talked with economists about ideas like wage reinsurance or or kind of wage adjustment umh programs, things to kind of help people adapt to focus on you know, you know, companies have a choice. They can do the same thing with less resources and that usually leads to things like layoffs or they can do more with the same amount of resources, but that requires creativity. And so there are lots of small decisions in the market where we can push people towards one instead of the other. And we try and push people towards hey why not do more with the same amount of resources.

36:11 That's better for everyone in the end. you know it takes some creativity to reassign people instead of instead of laying them off. It's not always possible. The economy is going to do what the economy is going to do but we can nudge it. And and I think the government also needs to think in that way. The government needs to think about, you know, what are some, you know, what are some things we can do starting small and if this becomes a real and serious problem, you know, just just getting more kind of, you know, getting more robust and radical in radical in term in terms of its response. So I think my my feeling here has been I think there's lots of ways to try and make this go well but we should have a mentality of you know in the work we need to prepare for the worst cases because we need to start thinking about them at least collecting data now which we've done with our with our economic index and even start to set policy because that policy has a lag of two or three years and the technology is moving so quickly.

37:08 >> We got to leave it there. Dario and Jamie, thank you very very much for the conversation. Thank you. >> As usual. >> As usual. Please welcome Chief Commercial Officer at Anthropic, Paul Smith.

37:49 Good morning. Good morning. and once again, welcome to all of you and and thank you so much for for joining us this morning. that's a pretty highowered panel to follow. So that's you know awesome sequencing on this event. I think one of the things that that struck me as I was listening to Dario and to Jaime's remarks is that so much actually tracks back to trust, safety, acting with responsibility. And I think that's something all of you wrestle with every single day. It's something that we wrestle with every single day as well.

38:26 and and I think it's why there is such a close alignment hopefully between what we are doing and and what you are all counting on us to to deliver for you. So financial services is one of our most important strategic industries for all of the reasons I just described. you in this room have always been very very fast adopters of technology through any of the technology shifts we've been through AI is no exception to that and so like be just after technology claude code was most rapidly adopted by all of the folks in this room it's all of you that have driven this this growth so thank you for that but I think the thing that is most interesting to me is really some of the use cases that this audience is driving as well across all of your businesses. If I look at this, this was a snapshot that we took at the end of last week of all of the workflows that are running on claude across the financial services industry as a as a whole. Now what's interesting to me here is there is some of this that we are deliberately leaning into as anthropics.

39:49 So we're working with great partners like Goldman Sachs on KYC and customer on boarding and how can we create like repeatable blueprints there. But a lot of what I'm seeing here as well is just entirely organic, right? This is and and the pattern that we typically see in this financial services is this financial services industry is you have a team and that team very very quickly goes all in on something like claude code and then claude code spreads from a core team to your entire software engineering organization and then that software engineering organization starts becoming more and more confident in terms of how they are attacking different business processes and then they start tackling these business processes one by one by one with this like Cambrian explosion of creativity and productivity across the organization. So that could be loan origination, it could be decision support in terms of M&A and pitchbooks through to yes KYC, customer on boarding, but then also in the insurance space underwriting straight through claims processing an incredible amount of of creativity. One of the things when I hand over in a moment to our product and our research organization. One frame to kind of look at all of this then this is how we think about our product roadmap, our research road map and and how it manifests is really think about everything that anthropic is doing through three lenses. So one of those lenses is really like how do we give all of your employees superpowers?

41:25 Claude Code is a perfect example of that. And then co-work following on from Claude Code, which is effectively co-work for the rest of us that don't want to use a command line interface, but still want the power of something like Claude code. That will 10x our productivity across a whole range of different functions. That's kind of like pillar one of our business. Pillar two of our business is how do you look at just business processes? How do you reimagine those business processes? And then once reimagined, how do you fully automate them end to end using claude as a platform? Second pillar. And then the third pillar of our business is how do you build on top of Claude? How do you create some entirely new revenue streams that did not exist before? I pulled out three examples from this here that kind of each of them reflect those three different pillars. So on the left we've got Aliance. We've got Axel the CIO of Alihance there. They started with claude code. They did that thing I described earlier. They rolled out claude code to their software engineering team. And that is now turning into a roll out of 50,000 individuals using co-work with the intention that if they've seen a 10x productivity increase in claude code, how can I get that same productivity increase now if I put this in the hands of my underwriters, of my claims assessors, claims administrators? How do I do that wallto-wall across my business? And then this second one here, Leanne, at BNY. BNY have one of the broadest based AI programs that I've seen just in terms of like the extens the extensiveness of what they are contemplating. But one area that is particularly innovative here is where they've actually gone now created full digital co-workers inside the organization that you can find in the corporate address book and you can actually delegate entire business processes and tasks to them. So if you're working on something like a financial transaction reconciliation that is a very manual process, how do I give that to my digital co-worker and let that digital coworker go and run that entire process end to end? And then this third pillar like building things that didn't exist before. If you take something like Moody's, the value in Moody's and the value in Moody's data is when it is put to work. When it's at at rest, not so much. So how do you expose all that data? How do you expose it to agents coming in and getting micro decisions on things like credit memos and being able to pull those together in minutes, not hours? And what kind of revenue streams does that unlock?

43:54 Now, when I pull all of this together, what you really are thinking about here is almost like a curve of autonomy, right? And a staircase of autonomy. And you start by looking at what happened with Claude code from launch back at the beginning of last year through to where we are now. And even 18 months ago, coding with Claude or with AI in general was really a matter of like autocomplete and very very very small tasks. And then those tasks became minutes, more minutes, hours, multiple hours, increasing the time horizon to the to the extent now that you have basically fully autonomous software engineers and all of your software engineers are now elevating themselves to be software engineering managers basically guiding a a fleet of of clawed code agents that are 10xing their productivity.

44:50 Okay, so that's where we started first and that's what's happened in code. we are going to see I think exactly the same curve of autonomy in the world of financial services. So again that starts with basically how can I get a piece of research or analysis done faster? Then how can I create this very complex Excel model? How can I get that done faster? And then that becomes okay is this now effectively a junior analyst needing close supervision. And then that junior analyst effectively becomes a more autonomous junior analyst that then effectively becomes a senior analyst that then effectively becomes a VP of an entire research function orchestrating a fleet of digital co-worker analysts and ultimately where do you get on that that that autonomous curve and and right now this is really just a function of the model intelligence which as you heard from Dario is there and is coming and it's also a function of the data that we can expose that model intelligence to and in this particular industry Those two things are coming together. So I think we will see this ramp up this curve of autonomy very very quickly. So I'm going to hand over now I'm going to pass over to Lisa Cro and our product and research team to talk you through some of the some of the R&D that's going behind this curve and how we're going to get there. So thank you and please welcome Lisa.

46:17 >> Hi. I'm Lisa. I'm a research product management leader at Anthropic. So, I spend my days talking to customers about how they're using Claude and working with our research teams to build Claude, deploy models, and improve Claude in the ways that matter most for customers. A question I get asked often is what to expect from upcoming models. I find it helpful to look backwards in order to project forwards. So scaling laws shown on this slide are the observed phenomenon that as we scale models, so in size, in amount of data, in algorithmic efficiencies, the models get better. Scaling laws are what's driving the rapid improvement you've seen in AI over the past two to three years. So just under two years ago, we launched Claude 3.5 Sonnet. It was the first model that was good enough at a capability called tool use to start doing more than just generating answers.

47:14 Claude could do things. Just under one year ago, we launched Claude for Opus. Opus was the first model that showed signs of being able to generate PowerPoint slides and Excel documents. With our most recent models, Opus 4.7 and Mythos preview, we're seeing Claude is able to own outcomes endto end. So what that means is it's applying judgment and completing tasks that have high ambiguity and completing the tasks over longer time horizons.

47:46 So an overall theme here is autonomy. As we scale models, the models are becoming more capable and they are able to complete harder, more ambiguous tasks over longer time horizons. One interesting thing about AI progress is that this isn't happening at the same time across all domains. So, we often describe this in research as the jagged frontier. like Paul mentioned, software engineering was the first domain where Claude was able to do real work. In under two years, we went from autocomplete models to teams of autonomous agents building production software applications.

48:27 Software engineering has changed forever and is still improving rapidly. So finance took off relatively later, but we believe we're at the inflection point right now, about six months to one year behind coding. Less than a year ago, Claude could barely format a table without ref errors. And today, it's doing senior analyst level work, and we expect it to improve rapidly from here. There's two things that are driving these future improvements. First, Claude's finance capabilities are actually supported by its improvements in coding. So, you can think that CL code is actually how claude interacts with its digital environment. It uses Python to run analyses, Office.js to create Excel files. And so, all of the improvements that you're seeing, feeling, and hearing about in code actually also help support finance improvements. Secondly, once Claude is able to complete a task with a reasonable amount of proficiency, we can collect data on it and similar harder tasks and train on those shapes to improve it rapidly. This is what's driving the exponential you're seeing today. So, I want to make some of these more concrete by sharing some examples of progress we've seen in the field.

49:44 We gave Claude an open-ended brief that you'd hand to a team of seasoned analysts. forecast next week's energy prices. We didn't give it complex instructions or a playbook on how to do this. Instead, it had to figure out that it should read the news, find some data sources for inputs, and try multiple approaches, iterating on its own model against the actuals. What's especially interesting here is that we're collapsing several roles into one. So, normally building the model would be a quant job. Looking at the market and forecasting would be an analyst job, but Claude is doing both end to end. From week one, Claude beat the published state-of-the-art benchmarks and kept improving on its own, iterating based on what it observed in its model. One of our researchers who was working on this used to do this exact type of work at a major bank.

50:36 And what took him 3 weeks in the past now takes Claude just 2 hours. Another more visual example is Claude's progress on creating PowerPoint slides. Opus 4.5 could produce slides, but maybe not with the level of quality and polish required for professional work. Opus 4.6 6 was a noticeable improvement and Opus 4.7 is able to handle much higher complexity and information density without formatting errors. In just a few months, Claude got several promotions and is now producing outputs that are usable with minimal human intervention.

51:22 Safety is non-negotiable for us and as we increase capabilities, we are also improving on safety. We make sure that every model we release is comprehensively tested for safety and alignment before we release it. We've driven recent improvements in prompt injection. Prompt injection is a technique where adversarial third parties can poison a model's context window and convince it to do harmful actions like excfiltrating data. Claude is five times more robust to prompt injections than competitors. We also continue to drive improvements on Claude's honesty and hallucinations.

51:55 We want Cloud to tell you if it doesn't know how to complete a task rather than making up an answer. So, I wanted to leave you with a summary on how this all comes together. Our research team drives improvements via scaling laws that make Claude better and give it new capabilities. We then ship those capabilities as products which lets you leverage them to drive improvements in your business but also gives us real world data on how and where Claude is working and not working for customers. The pain points become evaluations and training targets that we can further improve Claude on. We're not training on your data. This is about partnership. So we're partnering with you to decide which problems are worth solving and we will go make Claude better at those things. Our partnership drives this flywheel. So now I'm going to hand it over to Nick, who's going to show you how this all comes together inside of our products.

53:04 Good morning. My name is Nick and I work on our products for financial services. I am also a recovering investment banker and private equity investor. So I remember all of those late nights ticking and tying my Excel models and making sure that my PowerPoint slides are the same exact shade of blue. Now Lisa mentioned that finance is just a few months behind coding on the exponential. My job is to make sure that the exponential actually shows up inside of your organizations.

53:41 Now, what is Claude? We think Claude is one thinking engine. At the foundation are the models, the same ones Lisa just walked through. We're Frontier Research Lab. Nothing matters if our models are not the best. On top of these models sits a platform, APIs, tools, manage agents, an operating system for you to build on.

54:11 And on top of this platform, we build applications together. As Paul mentioned, this means two things. Claude alongside your people, transforming, accelerating how work is done. and clawed inside of your systems fully customizable to your processes. Now, this entire engine comes together to create agents.

54:43 Agents that can do useful important things within your organization like creating a pitch deck, discounted cash flow model, or evaluation review. Now, every one of these agents is built on top of the same foundational Lego blocks. Imagine you're bringing Claude into your organization for the very first time. What do you give it?

55:13 A computer with tools like S&P and Moody's. A training session on how DCFs are done at your firm. a set of instruction manuals on how to calculate the discount rate in the comps analysis. This is what we mean by connectors, skills, and sub agents. The building blocks that really help Claude get your job done. Now, what's new today? We're excited to announce 10 new agent templates built for some of the most valuable but time consuming tasks within financial services. Think pitch and meeting prep, valuation review, discounted cash flow modeling, and many more.

56:03 You get to decide how these templates show up within your organization. Want them working alongside your people? Install them as plugins inside Cloud Co-work and Cloud Code. Ones that living inside of your systems, deploy them as Claude managed agents. Now, let's dive into one of these templates for smarter employees. Now, as Paul mentioned, we're all knowledge workers. So, we do a few things like retrieving data, pulling analyses, and creating outputs.

56:42 What does that actually look like with Claude for say an investment bank? Let's meet Sarah. Sarah is an executive director of the natural resources group at Arcus Bank. She's out at an industry dinner and received an urgent email from her client on a possible take private transaction. Now, every banker in this room knows what happens next. Three analysts just lost their weekends and tonight's going to be a pretty long one.

57:13 Except tonight Sarah has Claude. From her phone, Sarah forwards the email directly to Claude using voice mode and co-work dispatch. Claude is able to get to work immediately. Under the hood, Claude is running plugins, the pitch builder plugin directly inside co-work, one of many inside our library.

57:44 Now, the pitch builder is pre-wired to S&P faxet, some of our longest standing partners. It is also pulling the comps analysis skill which encodes exactly how a banker actually pulls together a comp set and executes in Excel. By the time Sarah's heading home, Claude has already prel prepared a preliminary analysis with an execution plan, detailed recommendation on next steps, and a first draft of the LBO model and the comp set. All ready for Sarah to review.

58:27 Now Sarah is back at her laptop using Claude and Excel. Sarah starts to refine on these outputs together with Claude. Starting with comps, she pulls out a pier that isn't quite right. She lets the analysis and the calculations run, double checks the methodologies and the narratives, and everything looks pretty good. Now comes the hard part, the leverage buyout model. Sarah gets some intel that the sponsor is willing to stretch a bit more and has Claude rebuild the model a bit more aggressively.

59:06 Claude immediately reruns the model with one additional scenario, making sure that the sources are timed and the numbers are always cited across the operating model, the death schedule, as well as the returns analysis. No more manually checking every single one of those cells. Claude is doing the heavy lifting for you. Now, the deck, the part that every banker in this room probably has lost many Saturdays to.

59:40 Instead of building it in PowerPoint, Sarah directs Claude to Claude Design. Now, the beauty of Claw Design is that it doesn't just create static decks. It's meant for interactive prototypes and dashboards. Within minutes, Sarah now has a fully fleshed out templated deck with interactive sliders of key assumptions like entry leverage or exit multiples.

60:12 And she can watch the returns adjust in real time. Now, this is a conversation with her client, not just a deck presentation. The last step, let's get this over to the client. Within Claude for Outlook, Claude prepares a first draft of the email. Sarah takes a look at the email, trims it, iterates on it until it's client ready, and sends it with full confidence.

60:45 Now, let's recap what we just saw. The pitchbuilder agent lived inside every single tool the bankers use every day. As of today, Claude is now generally available across Excel, PowerPoint, and Word. And we're launching Cloud for Outlook for the first time in beta. Now, that means the same cloud is working across your entire cohesive workflow.

61:15 The analysis you just saw is also grounded in the most comprehensive data ecosystem we've ever assembled. As of today, we're live with eight new partnerships with many of you in this room. Varysque, Guidepoint, Dunham, Brass Street, and many more. Now, on to faster processes. We're also launching five new agent templates built for some of the heaviest operational workflows within your organization like accounting and valuation.

61:51 Let's look at what one of these workflows looks like in Claude managed agents. Let's meet Maya. Maya is a senior fund accountant at Aros Fund Services. She has a dozen buy clients, each of them with a portfolio requiring quarterly valuations and quarter end has always been a nightmare for Maya. With Claude managed agents, she can own Quarter End with much more confidence.

62:24 Maya kicks off the valuation review agent. Now, this agent spawns one sub agent per portfolio company, pulling financials, running the valuation in parallel. Maya watches all of these agents execute and run real time inside of our API console. She opens one package, double checks the Evida bridge, makes sure that the numbers tie exactly with the financials, and then she hands it off to the statement auditor agent for the final touch before she sends it over to her client.

63:10 What you just saw is a fleet of cloud manage agents working together. Manage agents are pre-built agent harnesses that we run inside of our managed infrastructure. They're meant for longunning asynchronous processes. They're fully customizable for your workflows and are fully observable within our console as well. Now, we've heard from all of you that processes like these take your teams weeks to months to deploy.

63:46 Our hope is that we're publishing these cookbooks that can be immediately customized and built into your codebase. And all of these agents also run in co-work as well. You get to choose exactly where they run and how they run. Now, this is of course just a starting point. We think our models are the best for finance and getting better every single day and our platform is built for your enterprise.

64:18 But you have to build it your way. We're here to help with a full team of customer success solutions architects and applied AI engineers who sit with you every step along the way. We're also hosting a hands-on session this Thursday to guide you on exactly how to deploy what we just saw into production. Now, everything you saw today is available today and shaped by every one of you.

64:53 We're so grateful to be building this together with you, the best ecosystem of customers and partners we can ask for. And with that, I'm thrilled to welcome to the stage Peter, our chief economist, who will tell you a little bit more about how AI is transforming not only finance, but the rest of the economy. Thank you.

65:26 So, the innovations we just heard about from Lisa and Nick are astounding, but the pace of change can be somewhat disorienting, raising the question, what impact might this rapidly changing technology have on our jobs, on the economy, and what comes next. My name is Peter McCroy. I'm the head of economics here at Anthropic and my team is working very hard to make sense of the economic implications of AI to support Anthropic's mission to have a path to powerful AI that is both safe and beneficial.

66:03 Now, it is clear that AI is poised to have large albeit uncertain implications across the economy. In part, this is because AI is what economists refer to as a general purpose technology like electricity, steam power, and the internet. Every sector and every occupation is set to be changed to some extent. As compared to tech, as compared to past technologies, the rollout has been much faster.

66:33 Businesses and consumers are adopting AI much more quickly. One way that we see this in our data, a year ago, about across all jobs across the US economy, around a third of jobs had at least a quarter of their tasks where Claude was being used. In just the past year, that number, a third of all jobs, has risen to about a half. Now, to get a handle on what this means for the macro economy, we added up the time savings across all the range of tasks that people and businesses bring to Claude. compiling information from reports, checking diagnostic images, or estimating statistical models like I do. We estimate that AI could indeed increase labor productivity growth by 1.8 percentage points each year over the next decade, if that's how long it takes the technology to diffuse throughout.

67:24 This would roughly double recent run rates and bring us back to the productivity boom of the late 90s and early 2000s. But it's not just that AI makes us better at the things that we already do, which is what this number illustrates. It's also that AI represents an innovation in the method of innovation. If you write down your favorite macroeconomic model of long run growth, this automation of innovation can produce an even stronger acceleration of productivity and economic growth with potentially large and consequential effects in the labor market that could materialize very quickly.

68:02 To assess how the labor market is responding to the change, this immense change that's on the horizon, we turn to the anthropic economic index, which is our privacy preserving measure of how people and businesses use Claude around the world mapped to different occupations in the economy. We assess which jobs appear most exposed to AI based on where Claude is being used for in automated ways for work purposes. And what we find is that in the most exposed jobs, so you can think computer programmers, data entry workers or economists, unemployment has not risen unusually faster for those workers as compared to less exposed workers.

68:45 But that's not to say that this immense change that's underway is not felt by workers themselves. The societal impacts team at Anthropic recently ran a large-scale qualitative survey of 81,000 Claude users around the world, asking them how they use AI in their personal and professional lives and what fears and hopes they have for the future. A commonly reported worry was around job loss. And when we look at these jobs that are most exposed to AI, we find that workers in those roles report three times higher likelihood or worry about losing their job in the years ahead.

69:25 Now, when we dig into the details, we find that this worry about job loss tends to be higher for workers who report higher productivity gains. technical and financial professionals report both being productive with AI and perhaps coming into close contact with using these tools greater concern about losing their job. Now, interestingly, this breaks down when we look at managers overseeing this sort of work. While they report large productivity gains, they seem to be much less worried about losing their own job. In other words, the people closest to the tools seem more convinced that change is on the horizon. And in fact, early career workers are twice as likely to express fears about job loss than senior workers. Now, this illustrates a leadership gap of sorts, a challenge that we might need to contend with in the years ahead. How do we equip young workers with tacid expertise, skills, and the knowledge needed to move into leadership positions even when the onboarding jobs and responsibilities that they typically took on are rapidly changing and being taken on by AI?

70:35 So, how do we solve this challenge? In part, we do this by understanding exactly how AI is changing work, not always eliminating it. Now, one thing that surprised us is that when you when you have people describe or when people describe where they get the productivity benefits from AI, they don't just report, I do things faster, they also report, I do more work that I couldn't do before. And interestingly, we find that this increased productivity from greater scope is actually greatest at the lower end of the income distribution. people seem to be stepping into work that had previously be been beyond their beyond their reach.

71:15 Product managers perhaps using claude to pro build prototypes and vice versa software engineers taking on some of that product management responsibility. So the question is not necessarily will my job survive that's relevant in some instances but how might my job change and what expertise do I need for an AI changed economy so what does this mean concretely I'll suggest three things first give people the room to experiment the benefits of expanded scope rely on implicit and explicit permission to work beyond your lane of work this is typical of how technologies have train trans transformed labor markets in the past leading to new tasks, new jobs and a reorganization of work. Second, keep humans in the loop. What we see in the data is that human expertise seems to amplify the impact that the technology has. More experienced word more experienced Claude users not only use Claude in more collaborative ways as a thought partner, they also appear to get more value out of the technology. In fact, when humans are in the loop, we find that Claude is able to tackle harder problems more successfully as shown on this slide.

72:31 Third, invest in the systems and processes that surround the model. When we look at how businesses embed claude through the API for existing and new workflows, we find that for the most complex tasks like automated biological research, they rely on disproportionately more contextual information as compared to simpler tasks. So this implies that the productivity gains of AI may hinge as much on capabilities as having access to the right context at the right time.

73:01 This is a standard story of complimentary investments in data modernization, for example, or in new organizational workflows. If your co-worker has relevant information to a sales strategy that Claude is helping you develop, you need to think about what's the process. What's the process to elicit that information? And so, I'll leave you with this as a final remark, which is the motivation for all of our work on the econ team. The impact of this technology will be shaped as much by the choices that we make as by the underlying capabilities.

73:34 We have an opportunity to shape the future so that the benefits are broad-based and the costs of the transition are not not unequally born. In other words, we have agency over our future. That's why we do this work to equip businesses, policy makers, and the broader public with the tools and insights to navigate this transition. And while I can show you some of our data, the leaders that will be joining us next are making choices daily about how this technology is actually used, giving you insight into what effective AI adoption looks like within the firm.

74:09 And with that, I'll hand it back to JP. Thank you. It's always amazing to hear from our research teams, product, and of course our leaders on what's happening theoretically and practically.

74:40 It's even better to hear straight from your peers on how this is being implemented in the day-to-day work. So it brings me great joy to introduce three folks I personally get the chance to work with. The CIO of JP Morgan, Lori Beer, the CIO of Goldman Sachs, Marco Ornti, and the CEO of AIG, Peter Zafino. So please a warm welcome for our panelists.

75:26 We've assembled the AI financial services dream team right here. So recognizing that you three quite literally touch almost every aspect of the financial services world from pricing risk and underwriting to core banking to asset wealth management and everything between. When you hear the term AI is rewiring the way financial services works, what does it actually look like from your point of view, from where you're sitting?

75:59 >> Peter, you want to start us off? Yeah, thanks for having me JP. I I the answer is everything. and when you think about you know underwriting or where we started with like you at the beginning with claw 2.0 it is data ingestion it's supplementing data it's augmenting it it's computing it and it's creating you know opportunities with that enhanced compute of making better decisions and then also how you actually look at claims. So, you know, for us, we went to the heart of the business, which was underwriting. you know, working closely with you and the impact in terms of the development of Claude's capabilities and and our ability to try and harness that change and and change our own business. We're going from some proof of concept, you know, 18 months ago to trying to do it in enterprise. I think it's changing everything.

76:47 >> Fantastic. Marco, how about you? So our sector is very technology heavy to the point that you know sometimes you can argue whether we're technology companies they're operating finance or financial companies that you know are big users of technology and so that's kind of where I would say the wave one of this revolution actually came about where we essentially like empowered our tech team which in our case is almost like a third of the entire company to really operate at a completely different pace and speed. In fact, you know, jokingly, I mean, we might end up being the fastest you know, like you know, of all sectors in terms of how our developers are developing software, you know, and obviously banks were not generally gen generally associated with speed. But given the impact that it has on our business, we kind of have to do that. We need to step up and that was kind of wave one. I would say wave one is followed by wave two, which is looking at our operational processes.

77:50 That's the other thing that we're very good at. We have big operational processes end to end, front to back. And I think the intuition there was, hey, the same reasoning capabilities that you could apply on coding. At the end of the day, processes and coding are very similar. It's logic to a to a to to an AI. And so expanding that to actually make us more efficient. And I think wave three, which I think in my opinion is kind of the more exciting and in the long term is when AI actually will help us make better decision, better better risk decisions, better investment decisions faster and smarter. And that's really where you're going to see the growth. And so we've been really kind of you know driving ourselves to kind of think beyond each one of these waves because ultimately I think you know this will really redefine you know who are the winners and losers, who are the players and and really have an impact on the entire ecosystem for sure.

78:43 >> Totally agree. And Lori, how about you? >> So being a company and in business for over 225 years, we've had to rewire ourselves for a few times. And for us, I think it always starts on at the end of the day, what is the outcomes and value we can create for our clients, our customers, and communities. because that's really what it's all about. And when you break it down, it's many of the things that Peter and Marco talked about. We think about looking at our business end to end. And there's new value that can be created. There's also existing value and how do we do that more effectively and efficiently and so totally agree with Marco on how you rewire your business processes really look end to end how we rewire the way we build and it is about speed I think all of us though when we hear speed it is not just speed it's speed security scalability efficiency resiliency and so doing all that in that context and then how do you rewire the way you think about the talent you need to help drive the execution of that agenda. And so those are many different things, but again, I think the technology is incredibly powerful. I think some of the harder parts are the things that that we talked about.

80:00 >> Yeah. And this this this comes up a lot. This notion of that this capabilities overhang that we refer to anthropic where the technology can do so much. It's the actual organization's ability to digest and absorb it that tends to be where the gap is. Marco, to you on the topic of speed and technological revolutions, you have the unique purview of having built a startup during the dotcom phase. seen the mobile boom at Nokia, the cloud evolution at AWS. Marco's got a serious CV by the way. and now you know running one of the world's biggest banks as a CIO. How is this transformation any different from the ones you've seen in the past?

80:44 >> Well, first of all, I'm kind of living with the you know, so to 10 years of AWS now in one year in a way from a speed standpoint. It's literally 10x and there is like a a daily cadence of the business. There's no longer like a or sometimes even much shorter than that. I would say the impact on society can actually be much more profound. I think the prior revolutions especially if I look at the cloud was a huge enabler of businesses and then indirectly kind of benefited people but it wasn't to the point that you know it kind of allowed people to in a way think differently about things and I think the power here of you know also like with regards to how we see that this kind of it's the first time that instead of buying infrastructure you can actually by intelligence. It's never happened before. You can actually be smarter. You can actually interact with with someone as a sparring partner that knows almost like you or sometimes more than you. And so it kind of goes more profoundly into the way we operate in the way we think. And it's really at the end of the day, it's not a technology revolution at all. In fact, it's kind of very far from computers. This is almost like, you know, it's a way you need to adapt your behaviors. And every single person in every profession needs to think about how do I actually change my habits, how do I look at things differently and how can I actually work differently than I used to. So I think for us is like a rethinking of you know really like how people make decisions, how people you know think about the next steps and and and I think this is going to be we're kind of the front runners I would say on all industry but I don't think there is going to be any industry that is not touched by this >> and on your topic of you know reimagining processes historically in our industry financial services some of these processes have been tried and true for a very long time. And Peter, I think of you and, you know, you're setting the board level mandates and thinking about the insurance business and underwriting and the role AI can can play in that obviously being transformational. What was the the first catalyst that you kind of paused and the first moment you're like, oh, this aha come, you know, aha moment takes place, you're like, this could actually change the entire way we do business. Do you remember that kind of tipping point for you?

83:13 >> I do. you know I started to chase because I thought that the business just needed a lot more compute and I started to try to learn a lot more of how to apply quantum like in you know that wasn't the right path and then starting to learn a lot more about you know AI gave me a different path for the company and as Marco said it's it's 10 years compressed into one and it's unbelievable how fast things are evolving and so like you need to have I always look at in three categories one is to understand the technology no matter where you're looking Two is is there's pattern recognition of how to apply it. Like if you don't know how to apply it, you know, it's not a tool off the shelf. And then three is like the pace of execution. Like I mean, can you actually do it? We had our first investor day, as you know, because Dario was there. and you have been a big supporter a year ago. first one in 10 years, and we've already had to change what we're outlining for our AI strategies based on how fast things are evolving. And back to that pattern recognition is that there's more capabilities today to be able to make our colleagues smarter. and you know, I say a lot of times if you have consensus around a table, probably means you're not pushing things too hard.

84:26 and I don't vote around my table, but if I did it, it would have been 13 to1 against me in terms of what we should be doing. but we still did it. And and you know, I think what is important is also taking the culture and people with you. and you know to get them to understand what the value is because there's opportunities not only for large language models you and I have experimented with one recently where claude out of the box was 88% as good as an expert on claims and the theory is can it get better yes but that assumes that the claims expert doesn't get better and so I think it's also a mechanism for people to learn and for very good questions you know we'll call it a critic agent but you know how do you ask very good questions for the organization to continue to evolve. So I I think it's all of those sort of patterns and applications and pace of change that are going to remain catalyst for the future. Thank you, Peter. And what you touched on, which we have a chance to chat with most of you in the room about very often, is this double-edged sword of capabilities and what the model can do and the reality of being in the industry we're in where you have to do it safely and you have to do it very responsibly. And Lori, I think about the sheer scale of the operation you're running. How do you ensure you're able to leverage these capabilities in a safe and responsible way in an organization your size? Yeah, I think the the first thing is we are a business of trust and you know I think this is this moment where you see this incredible balance of innovation and risktaking and a moment where there's so much pressure because of the speed that you have to be moving fast on the innovation front but you have to do it in a safe and secure responsible way.

86:12 And so for us, a lot of this has been based on a lot of the modernization work we've done historically, which allows us to leverage a platform strategy to be able to build a lot of the controls that we have in place. We have laws, rules, regulations we're accountable for across the globe. and it really allows us to to get the scale. So taking it forward when you think about things like cyber security because we have a platform that allows us to extend those capabilities across the firm and the talent to do so and then you think about not just using it reacting to new vulnerabilities but how you think proactively. We built things like a cyber threat modeler that allows us to first make our security analysts more effective but then ultimately putting it in the front end as Jamie talked about building secure software. How do we use that and the power of the models to make sure that's the output going forward?

87:07 And so I think think about cyber security, think about our model risk governance and making sure we shifted that from the old AI ML traditional way to adapting to these new models. looking at third parties differently which is a huge thing that the industry leverages because now we have to think about agents and things that might be get u might getting built how we influence the industry and so I think managing re risk there's the things we can do to manage that there's also things where we can drive and influence more broadly across but at the end of the day we have to increase the speed of innovation and we have to be comfortable that we're balancing this risk takingaking and we do that in a way that leverages platforms and capabilities.

87:53 >> Yeah, that's so well put. And you know, as you think about the precursors to deploying this technology, some of the themes we just touched on, Lori spoke to, you know, has to be safe. It has to be responsible, the capabilities are there, and even with all that in place, one of the biggest challenges we see and you touched on as a group is change management. is inevitably you've got folks in your firm that are AI sants and you've got folks that are either afraid of the technology or just haven't had a chance to test it. So as you think about your own organizations and driving AI fluency and to bring your organization up to scale and your associates, what are some of the strategies or approaches that have worked and served you well to drive this kind of AI fluency at scale?

88:39 And Marco, we can start with you. like someone mentioned that already but I think right now the state-of-the-art of technology is probably you know more ahead than what people are actually ready to accept or even to do with it and so really I think the the real work is on the people is on the cultural transformation I think in terms of techniques on how to enable I mean obviously things change all the time and so sometimes you know with regards to AI adoption especially for appetite for that I feel like you know until maybe two or three months ago, I feel like I was kind of pushing a car up a hill and now I'm kind of holding it, you know, as it goes downhill because everybody wants everything. So, right now, I don't think we're in in the in in the in the adoption sort of a stimulation phase. We're more like in try to do it right. Okay. And so things that work first of all the mandate needs to really come from the top. So the company really needs to embrace the strategy from the board to the CEO and and of course you know in our case like in the case of others obviously you know the our CEO is directly like you know setting the strategy and pushing us to do more and I think that's very important for alignment because one of the failure modes of adoption is that you have a gazillion PC's that never end up with something really useful and so f like direction from the top and also kind of defining what are the areas of highest impact and then really doubling down on that because you know what I mean AI is still nent it's still not perfect and I think we need to create success right now it is the most important thing that we have some resounding success is that then we can use as a blueprint to then continue on on the other use case. I would say the last thing in order to really like have people kind of overcome their fears and and really be willing to kind of you know sometimes I use the term you need to kind of resurrect into a new profession because you need to be willing to let your old habits die in a way and so it's almost like a a professional resurrection and in order to do that you can't not just hear it from the top you you have to also lean on your peers and So one thing that we've done is been successful also for other things is using the blueprint of bar raisers which are people that are real AI champions. There are within the organization they have a they are a community that community is well organized. They share best practices and then they become the focal point for others in order to to find help but it comes from a peer. At that point it becomes much more acceptable. And so this combination of strong community of practice and strong mandate from the top I think so far has been quite successful.

91:48 >> Fantastic. And Peter recognizing you you know being the CEO set the tone and as you'd mentioned going very deep with specific use cases that are core to the business. What's worked for you guys to drive you know more broad AI adoption or fluency across the or getting you know sponsorship from the top but also I'm like very involved. I you my team probably wouldn't agree with me but I I'm very close to almost being the project manager where I'm involved in it just about every day to drive you know progress. I I would say some of the areas what I would have done differently if I know you know today what I didn't know then which is like linear rollouts were going to be too slow. and when we went to the heart of the business, we didn't go to the biggest part of the business. And so, like trying to get to that PC and the smaller scale, but by the time we learned everything of that, like the technology had advanced and the LLMs are a lot more you know, effective. And so then we needed to go back and look at it and then where are we going to deploy? You just have to like roll it out simultaneously and make sure that either your staff, do you have a you know, a set of partners that can do discrete work for you to be able to leverage that. And I would say this will sound very elementary but making sure that your team when you're talking about critical strategic initiatives everybody's working with the same definitions. I mean I I I sat with my head of digital, head of technology, head of strategy and head of operations and it took us five hours to define what is orchestration because everybody had a little bit of a different view and it's like and then okay if you have a different view how that actually goes into an organization of how you're orchestrating agents or you know how you're re-engineering the process what processes are we taking something that was already documented and you know looking at agent you know orchestration to re-engineer the process and so I think making sure there's enor enormous clarity and you continuously do that for alignment.

93:44 >> Fantastic. And I, you know, as I think about this clearly sponsorship from the top and I can attest that Peter is one of the most leaned in CEOs I've seen in this in this area having these champions to Marco's point getting peers to serve as these champions to help influence other teams so it's not just being kind of mandated from the top are all great tactics. And now once the fluency is raised, folks are consuming this technology at a high clip.

94:17 Naturally comes the comes the question of ROI. You know, this is starting to become a real line item and consumption's big and we hear great anecdotes about how it's changing the way people work. But this one's for you, Lori, as you think about deploying at the scale you do, what are, and this is a question we get asked virtually by every partner. What are the systems in place you have today to help start to capture some of the ROI that these investments are driving?

94:42 >> So, you gave me the hard question. >> Yes, exactly. >> so going back to and I agree with all the points that Peter and Marco made about driving the adoption. And so, when you get to ROA, I think a couple of things. I think we had again the awareness as this stuff hit the stage back in 2022 at at scale and you started to see the maturity. This was different and you had to think about it as a horizontal capability. Right now it was broadreaching. You could literally have conversations the learning making you smarter. We had to think differently in that moment. You have to say there are capabilities we need build the platforms we need to do agent orchestration protect and secure change every layer of the tech stack how we rewired how we trained our people. We don't measure ROI on those things. They are musto. Where it comes into play is now that it's really embedded in the fabric of how we operate. It's not just AI. It's how you're solving a problem differently.

95:44 we're looking at what are the outcomes. So if you think about the work we've done in our asset wealth management business where we have a mature product called connect coach it is making our our advisors so much more effective to the point that they can handle more customers. They can serve those customers better. We can create think about new products that we can build our bring our customers or clients and it's a revenue generation opportunity and so everybody understands the efficiency oper opportunities but that is another one how it's going to impact markets and how we think about what does trading look like and risktaking look like over time that's another area where we can start looking at things and testing them technology is a big one and we know that the tools are are very advanced but when you think about we all know this in financial services we create the products and services that our people deliver to our customers and clients. So every time we can drive efficiency and effectiveness of how we build our products and services has a direct impact on that and so that's another avenue as well and so for me ROI there's the must do we don't spend a lot of time debating it these are the foundational things you must do but then as you start scaling up in these other areas whether it's in consumer where you can start driving personalized solutions at scale or are advisors and how you think about in wealth management or how you think about things from a markets perspective in the future. That's when you actually start looking at how can you measure how did I serve that customer? How did it drive revenue? And by the way, we have to change in many cases the metrics in terms of how we measure value and success in the future. But that's some of the ways that we're thinking about the ROI question.

97:38 >> Fantastic. Thank you. And with our last question and thank you so much. This has been remarkably insightful. If we reconvene a year from now, same dynamic dream team on stage here. And we think about the question a year from now, what's one thing you want to be able to say you did differently? Not two or three things, one thing you did differently to take this kind of AI transformation to the next level. And you know, in 30 seconds or less for each of you, Peter, let's start with you. We're here a year from now. What's one thing you did differently?

98:14 >> I I Well, first we should ask you what you're developing in the next year and then I can answer more accurately because I'm sure whatever I say will be wrong, but assuming you're not going to do anything. I think it's getting it out to the enterprise. What Lori said was great is a hard question. But getting stifled by, you know, return on investment when you're going to have to compete in a world that's going to be very proficient with large language models, better decision- making. I hope that I'm here to tell you that we've rolled it out to the enterprise.

98:41 is we've had great learnings and our company is much better run. We have much better results as you know as a result of taking that action. >> Fantastic Marco. >> I mean it's very simple. Right now you have a lot of internal metrics to to measure success. I think a year from now we will see success or AI will be seen from our client side. Our clients will clearly see that we're leading on AI. That will transform the way our clients actually even interact with us. And so that's really what I want to happen.

99:12 Ultimately, the client success is really what matters. >> Fantastic. And Lori? >> Yeah. I think ultimately the value like at the end of this, if we're not creating value, if AI is just not embedded in the in the framework, there's probably way too many things to say. We wish now that we're on the other side of it, we could say we wish we'd have done better, faster, etc. I think making that pivot, we had traditional AI embedded deeply, but rewiring faster governance the way we work. I would do that differently now sitting from this chair right now in this moment.

99:46 >> Well, thank you. Thank you all three of you and please join me in a warm thank you for our amazing panel. Oh. Well, hopefully you got a chance to learn something new, chance to see how we're thinking about this space and how excited we are to work with each and every one of you on this journey in the next year and of course reconvene a year from now to make them good on their promise. I want to thank everyone who's dialed in from our live stream for joining us. We appreciate it. And that concludes our session, but it doesn't conclude our time together because we'll have an hour to mix and mingle and grab a drink and connect with one another. So, thank you so much for being a part of this day and thank you for joining us on this journey.