Cloudflare CEO: We're Ready To Block Millions of Websites From AI — With Matthew Prince

0:00 In an internet where AI bots now outnumber humans, what happens to the web and can news information publishers and small businesses survive? That's coming up with Cloudflare CEO Matthew Prince right after this. Welcome to Big Technology Podcast, a show for cool-headed and nuanced conversation of the tech world and beyond. Well, on the web, AI bots now outnumber humans. So, we have more AI traffic than human traffic and that's obviously changing the dynamics and the incentives for news publishers, information publishers, and of course small businesses. So, what is going to happen to the web? Well, today we are lucky to be joined by Matthew Prince. He is the CEO of Cloudflare. He was with us almost exactly a year ago talking about Cloudflare's efforts to prevent the web from disappearing. The web is still here, but maybe not as healthy as it was a year ago and obviously not due to the failure of those efforts, but the force of AI marches on and today we will talk about what that battle is going to look like.

1:00 Matthew Prince, Matthew, it's great to see you again. Welcome back to the show. >> Thanks for having me on. >> All right, let's go with the headline here, which is that recently Cloudflare has found that AI bots outnumber humans browsing the web. I I can't tell whether that's surprising or not. I think part of me would say, "Hey, you know, of of course that's happened and it's surprising that it just happened." the the sort of conventional wisdom has just always been a lot of bot traffic on the web, but just talk a little bit about the escalation in AI traffic that you're seeing, whether that's, you know, crawling for new information or agents taking action and what it says about the rise in in these type of AI applications recently.

1:44 >> Yeah, you know what? I'm I'm certainly surprised. you know, I was asked at Web Summit in November of 2025 when when we thought that automated traffic would pass human traffic online. And we pulled all the data and looked at it and we and our best estimate based on trends was that it was going to be in the second half of 2027. I it's up by Southwest in March of 2026 was asked the same question again. We pulled the same data again and we're like, "Huh, that's interesting. It's moved up. It's now we think going to be in the first half of 2027." And so I was I was floored when our team came to us and said when my team came to me and said it in May of of of 2026, "You won't believe it, but we now see that automated traffic has passed human traffic for the first time in the history of the internet." And and I think that that that really does just show how fast this is progressing and how fast the actually the derivative is is changing where the the slope of the line is getting is getting steeper.

2:42 And if we if we just assume the current trends, what's what's blows my mind is that if you just fast forward 5 years from now, we estimate now that automated traffic online will be a thousand times what human traffic is. And that's not because human traffic is going to drop, it's going to be because automated traffic is just going to continue to rise like crazy over over that period of time. And that's going to put you know, enormous strain on the resources of the internet. It's going to put enormous strain on the infrastructure. And and it's a lot of what you know, the team at Cloudflare is thinking about is how do we make sure that the internet is ready for what is going to be, you know, just a complete and total onslaught of of of automated traffic.

3:25 >> Right, and I'm sure you've heard of dead internet theory, but those who haven't, it's this theory that one day the web is basically going to consist of of bots and automated content and there are the bots will be manipulating that content for algorithmic curation. So what you see in your social feed is effectively a dead internet that looks like it's real. This is obviously a little different than that. >> Yeah, I don't think actually the trends we're seeing are kind of the opposite of that. It's actually interesting for you know, Cloudflare started and launched in 2010 and and around 2015, the web stopped growing. In fact, it plateaued and then it was even declining slightly over over the last you know, 10 10 years.

4:10 And then right around kind of October of 2025 the the web started growing again and it started growing at a rate which which most closely resembled kind of the mid-2000s. And the content that's being created is not like I would not describe it as as automated sort of slop. There's some of that, but there's always been, you know, there's always been some of that. What I think has happened instead is that, you know, people who were not particularly technical, people who didn't have a lot of resources, but they had a interesting idea, they had a story to tell, they had something to share, all of a sudden were able to use, you know, incredible tools like, you know, Lovable or Replet or, you know, what the team over at at Base44, you know, Wix is doing. Like and then have those tools be able to build pretty sophisticated applications. And and that's like for the for the first time almost in Cloudflare's history, we're seeing an explosive growth in things which are getting created and instead of it making the internet worse, I actually think the internet's getting better right now.

5:20 It's just the interface that we're all going to consume it through is going to be very different. It's it's going to be much more mediated through some sort of an AI interface, but I think it's incredible that you know, you can even if you don't have technical skills now put together an application, put together a website and bring that online and again, I think that that's that that that's remarkable and I and it's actually making the the web is getting healthier, not less healthy right now.

5:50 >> Okay, I actually want to press you on this one because one of the studies that was done about this showed that like for instance, while there are many more apps on the Apple App Store for instance, the reviews are going down, right? The seems like the weekly active users hasn't really increased. So, yes, there's been this ability to create a lot of new stuff, but it also seems like a lot of that I mean, I know you said it's not slop, but what something close to it? This What do you think?

6:20 >> I again, I would I see every day is just people who didn't have the resources or the ability able to create things which are valuable. And so yeah, is there going to be stuff that's not valuable? Of course. but but I think that the net has been that we're getting more things out there. I do think that we're going to move more and more away from kind of an App Store model because that's not going to be interface where we where we where we reconnect with it. And and the App Store in some ways was an enormous threat to the web.

6:47 Like it it took people away from the web. It was probably one of the biggest forces between it and Facebook were like the two biggest forces that really made the web less interesting. I think now putting things on back on the web, and then making sure that they're accessible across the different AI systems in a way that makes sense for creators and and and make sure that creators get compensated is going to be something which actually think will will will reinvitalize the web, and it's and it's exactly what we've you know, been been seeing. And and and again, I have both just personal examples from, you know, colleagues, family who have been able to get online and and tell things, but then also professional examples where every day I'm getting letters from people who are like, "We're I'm able to do something that I was just never thought I would be capable of doing before." And using some of these again incredible tools that are out there combined with the infrastructure that someone like Cloudflare provides you have very cost-effectively. I think it's actually making it so that humans assisted with AI are are more creative than they've than they've ever been.

7:55 >> Okay, I'm just going to pause to say when I built the website for our summit which happened in June. And when I'm saying I am should really put quotes around it. It was mostly the Claude code harness hooking into the Cloudflare agent. That went out and set up all these accounts for me and then and then built the site. So >> I mean it's it's extraordinary. Like what what like compare that with what you would have had to do. In the past which is you know hire a whole >> I wouldn't be able to. There's no way I could even start.

8:24 >> The resources would like you alone wouldn't be able to start. So you'd hire someone to do it and and yeah so that that that has enabled you to do more interesting things. And I think that that's that's that's what technology at its best does is it it empowers more people to have access to something. Then they had they had before. We're doing this. You know remote. Even just 10 years ago like what would it have cost to do like a live video setup with you know 4K video you know in two different places. I mean it would have been hundreds of thousands of dollars just to have the studio time or or you know set up cameras in order to do this. And yet we've been able to to do this. I think AI is no different. It's going to enable more people to be able to be creative and do interesting things. And and and if we get the incentives right. Then hopefully it also makes it so that we can have more more entrance into into the marketplace. I think there's like figuring out. What's the business model of the next five years of the internet I mean the next over the next five years figuring out what the business model of the next 30 years of the internet like that's the most interesting and important question in technology today.

9:33 By far. >> Right. >> If we get that right. I think it's it's we're about to be on a golden age of content creation. And and and it could be incredible. If we get it wrong, you know, there's there's a we can we can it's easy to list what the parade of horrible could be. >> Yeah. And so let's let's dig into this, right? Because on there's two sides of it. There's the content creation side and then there is the browsing of that content. And this is sort of picking up a little bit about what we spoke about last year is more and more that content is being crawled or accessed by AI. Right? So you've seen this this massive increase in AI traffic to websites in the past year to the point where it's now overtaken humans. So do you have any visibility into whether that is AI bots accessing it to train new models primarily, agents using those websites as a human would or you know, maybe a query that at test time. So as the bot is trying to find an answer it goes to a site and it and it ends up, you know, bringing that back into the chatbot interface without a person actually going to the website itself. So what are the proportions if you have any sense of that?

10:50 >> I mean it's it's all of the above. And so I think all of those will drive drive things. There's there's there's definitely an uptick in training as more model providers are are coming online. I mean I don't think that you know, like open AI is, you know, crawling a little bit more than they did, you know, a year ago. Anthropic is crawling a little bit more than they did a year ago. But but it's more that we've just got more people building models and I think that's great. Like we should have like the world shouldn't be five you know, frontier model providers. It should be 500,000. And how do we make it so that that's that's that's available. I think at the same time you've got sort of agents that are operating on their own. That's probably the smallest of the categories right now.

11:25 I think that that will grow and become a pretty large category. But right now that's sort of agent to agent kind of commerce and and things is is is is less like that that's that's more of a future thing than a present thing. and then I think you are getting more of the, you know, AI acting almost as at on on behalf of humans going out and accessing things. But I think it's it's almost, you know, it's wrong to say that they're acting exactly like humans do. I mean, the scale is just dramatically different. If I'm shopping for a digital camera unassisted by an agent, I might visit I mean, if I'm really interested, visit five websites, you know, and on every website maybe visit I don't know, 10 pages or something like that. My agent is going to visit a thousand websites and visit every page on all of them. And so like the load that that then puts on this, especially if there's no there's no financial disincentive to just crawl everything, right? Is is pretty substantial. Now, I think you've got to balance that against you know, what's what's the benefit to the user in the fact that agents are infinitely patient, that they you know, that they that they will look through everything. I think that's that's that's really good for end users.

12:34 But there's servers behind, you know, every one of those requests. There's bandwidth that has to, you know, get paid for. There's content that had to get created. And so there's got to be some exchange of value. And if the AI companies are just taking without without returning some sort of value, then that then that then that just doesn't make sense. And so that's that's again what I think we've got to we've got to figure out. >> Right. So, I think that what you know, when you talk about this and your view is that we're eventually going to go to I think you said this a thousand times more bot traffic than human traffic, right?

13:06 In 5 years, which is crazy. Okay, first of all, just wait. >> you a sense of how like how intense this is. The internet European internet came this close to completely collapsing in COVID. Because in COVID over period of 2 weeks, internet traffic doubled. And especially doubled for very for video streaming, which was a which was a very high bandwidth. And so I think but I remember being on a phone call with senior leaders at Netflix, Disney, Akamai, YouTube, you know, all the major streaming platforms.

13:43 and and it was basically like this Mexican standoff where where where where nobody wanted to reduce their the quality of their video stream because you know, they're like "People pay for us because we have, you know, 4K video and we're not going to reduce it." But what was happening was everyone was their kid like in Europe, everyone was putting their kids in front of the TV, you know, streaming first thing in the morning and that was unfortunately lining up with Asia where everyone was getting, you know, was was turn after dinner was turning on, you know, something to watch and it was just causing so much traffic that we were we were running out of capacity online and eventually people had to we had to cut their video streaming resolution down in order to make sure the European internet stayed online. And by the way, if the European internet had fallen over, then Asia would have been next. The US would have been okay for a little bit longer, but at some level like there's a finite capacity to all this stuff. So you've got to be able to support it. And that was at 2x. At 1,000x, it's going to be completely different. And so again, we're spending a bunch of time trying to figure out how do we make sure that the technologies are ready. So like WordPress, most popular platform for hosting content. It's out there it's run well on a programming language called PHP, not the most efficient programming language. And so we're like, "Listen, we want to support the WordPress community. We want to make sure WordPress is successful, but if it costs" I was meeting with one of the largest WordPress hosts a little bit ago and they said, "You know, it is it costs us about $3.17 to host a website on WordPress and then we charge like $12.99 to and that's that's sort of what their business is."

15:20 I was like, "If traffic went up a thousand X, what would happen? And they're like, well, then it would cost us over $3,000 to host that website. And again, the average WordPress user isn't going to be willing to spend over $3,000 to host their WordPress website. So, we've got to change some of these fundamental technologies. And again, I'm thinking the best case, it's still the user interface and the ecosystem and everything that people have come to love about WordPress, but with the underlying technologies made much, much, much more efficient so that you can actually, you know, scale to what's coming. And that's that that that volume of what's coming is just is just again, it's it's almost inconceivable to to to think about.

16:03 >> All right, we have to spend some time talking about this cuz this is very interesting. So, first of all, to anticipate a 1,000 X increase in web traffic coming from bots in 5 years. I mean, yes, like, you know, we we've seen a or you've seen quite an increase recently, but a thousand X's is a crazy number. So, what is going to make up that thousand X? Like, what makes you believe that that's going to be the case?

16:29 >> I mean, we just look at the trends right now. I mean, in in May, we saw that that you know, the automated traffic crossed it and we kind of extrapolate out what that rate of change is and then we assume that again, the the the exponent is going to is going to increase on on on that cuz cuz it's you know, I still way under penetrated with what's there, but we've got to invent new technologies in order to support it.

16:54 So, just to give you some some substance. Let's imagine that just every knowledge worker on Earth had an agent that was working for them 24/7, right? Helping them do their job. And I And again, like, that's that seems reasonable. Like, every knowledge worker is going to have some agent that's that's working for them. Very few do today, really, but but over time, like, that that becomes becomes reasonable. Just with that, and putting GPUs aside, just looking at CPUs, if every one of those agents ran in a traditional container, like, you know, which is which is not even a VM, not even its own server, but just a container, right? you would need more than 40 times the current CPU production in an in a calendar year in order to support that level. And that's with one agent. I would guess that most knowledge workers are going to have many agents that are going to be working on their behalf. So, we just don't Using the current technology, we don't have the ability to support what's coming. And so, again, what we're spending a lot of time at Cloudflare is thinking about like, okay, like, we went from servers to VMs to containers in order to get more efficient, but even containers, you've got to pull in an entire operating system, you've got to pull up all the tool chain, all this stuff. It's pretty heavy weight. So, what's even lighter weight in order to support what's what's coming. And a lot of what we've built with our developer platform, which we call Workers, and and leaning heavily on technologies like Isolates that allow you to, you know, have much, much, much lighter-weight services, we think that those are the sorts of things that the internet's going to have to have if it's going to be able to continue to scale to support the the the level of agent traffic that just is inherent to to to what what I think the future holds.

18:42 >> So, just to put a fine point on it, the increase is going to be agent-driven, right? That 1,000 increase is agent-driven. >> Or humans interacting with their agents, right? So, but just going to be automated. Okay, and again, the way to just think about it is, again, we've if if you've used ChatGPT, if you've used Claude, if you've used Grok, like, and you play with them and you just ask them a question, just count how many websites, like, you ask it a simple question, and it just goes across everything. Like, now imagine that not in the hands of you know probably about 1% of global knowledge workers like in terms of using it in a regular basis. Just just scale that up to 100% of knowledge workers.

19:22 And instead of it being something that you use maybe once or twice a day, it's something you're using for every task and it's really easy just see how a thousand X is not in five years is not only inconceivable but it might actually be a low estimate. >> Yeah, that is crazy. So basically last year we talked about the economics for publishers might not make sense because if you're paying somebody to write you know an article and Anthropic crawls it and drops it into Claude then they sort of get cut out of the value proposition.

19:51 >> That's right. >> But now a year later, you know, as we've seen this rise in agentic traffic, the problem actually is much much broader because now website that you might not have even needed to invest a lot into making the content for if they get faced with this increase of traffic could cost much much more to upkeep and then the economics of you know, even sustaining them become questionable if you can't find a way to do it more efficiently.

20:17 >> So I think the thing that's that's really important is that different content creators, different publishers, different app developers are going to have different preferences that are there. And at Cloudflare what we believe is we want to make it as efficient as possible for the companies that want their content in AI and we want to create the systems and incentives for those companies that don't want their content in AI and especially if they're not compensated for it to be able to control that. But let's talk about the majority. The majority of companies that are out there, the majority of content that's created, they actually want that content in AI. Like Cloudflare, like our knowledge base is a way of us sharing how to use Cloudflare really well. And we want to make sure that every AI system has it and has access to it. And so like for that, what we have to do is figure out how to make it as efficient as possible. How do we make sure that we're not delivering a big heavy HTML page but just a markdown that is is actually required. How do we make sure that we can actually get systems into AI as efficiently as possible with caching and and other things that are out there?

21:15 And our goal has to be, how do we make every time we serve a page 1/1000th of the cost of what it used to be, cuz that's the only way that we're going to stay ahead of the demand, which is coming down for those companies that want to have their content in AI. There's a different class of companies, and they tend to be companies that are, you know, either ad-supported or subscription-supported. And in those cases, AI, that's where the people are feeling like their their content is being stolen from them. And in those cases, again, different from the majority of companies, but in those cases, those publishers want to have the controls to say, "You can only get my content if you compensate me for it, because you're going to hurt me in terms of advertising revenue and everything else." The efficiency still matters. We still want them to be much more efficient, but then we also want to give them the tools to be able to say, "You know, I want to enforce what the rules are, where you can't have my content unless you pay for it." And then we want to make it as easy as possible for agents to actually pay for that content and be able to have a real exchange of value.

22:11 >> Okay, so there's these two buckets. One is the bucket of companies that want their stuff crawled, which is most of them, but then we do get to the news publisher side. And you know, there is a risk here, and we've talked about it on this show a lot, that the incentives for news publishers and information publishers will go to zero if the bots just crawl the information they put on their websites and then present it to them in the chatbot format. I think you've mentioned that people don't really click the footnotes anymore. They just trust what the bot says. And so, let me share again the stats about where things looked last year. So, last year you told us that 10 years ago on Google, if you were to Google would crawl your page twice and send one click for those two crawls.

22:55 Six months ago, it was six crawls for one click. Once Google released AI overviews mid last year. It was became 18 crawls for one click. OpenAI had 1,500 call calls crawls for every click that it sent Anthropic and Cloud were about 60,000 crawls for every click that it sent to publishers. Where do those numbers stand today? >> Yeah, everything has gotten basically assuming you're a publisher from the publisher's perspective, it's gotten twice as bad. They just double every one of those those numbers over the last year. now from end users perspective, I think what you see is actually something quite different, which is seems like the world has gotten better. Like the AI systems are smarter.

23:36 They seem like they're responding better. They're giving you the answers that you need without you having to double-check them. And those two things are actually completely consistent because it's not like the the UI changed. It's not like the behaviors of the of the AI companies changed. What changed is the results got better. And as the results got better, people click through to the sort of original sources much less. And that I think is indicated by the fact that you've had, you know, again a very substantial drop in further drop in that. And that's that's the trend. Like over time even Google has announced that the sort of the era of 10 blue links is going away and that everything is going to look much more like an AI interface. And and I think that that's the change. And if that's the case, then the business model which has supported the web for the last 30 years which has been advertising and subscription-based isn't going to be the business model that supports it for the next 30 years. And we've got to figure out what that is. For sure some of that is going to be deals between the big publishers and the AI companies. We're seeing those increase substantially. But we've got to have something that actually works for everyone. And so you know, one of the things that we're thinking about is how do you have actually micro payments where if you're accessing a piece of information that a site can say, "Listen, you can have this, but it's going to cost you a fraction of a penny." And the minimum that's going to cover the the infrastructure cost of serving it. But in the case of you know, people who are doing real journalism or academic work or other things, maybe it helps cover the cost of actually supporting the creator that's behind it as well. But it's it's crazy like the volumes. Like here here's the interesting technical problem. So Visa, largest payments network in the world, handles at peak during the holidays somewhere around 20,000 financial transactions per second. Cloudflare handles about half a billion internet transactions per second. Those are not financial transactions those different stakes. But we think that somewhere between 1 and 10% of those will day one be monetizable through agents with with a with a micro-payments transaction, which means that if you're going to build something like this, you've got to build a payments infrastructure that can day one support somewhere around 10 and then be able to scale to over 100 million financial transactions per second in order to be able to support what's there. Now, these are going to be really, really, really small transactions, but they're still financial transactions. You've still got to account for them and and do that.

25:53 That's an incredibly interesting and hard technical problem and and we're looking for both, you know, external partners that can solve that, but we're also working on how internally we can build that. And those are the sorts of things that you have to have in place in order to then support what is a new business model for the internet. And that's and again, I I I I can't imagine any problem that's more exciting to be working on right now.

26:14 >> Yeah, so a year ago basically the solution that you proposed was or sort of implemented was a system where if you want to crawl a publisher page, you would have to pay them a micro-payment and if you didn't pay it, then you'd be blocked. so >> that's not exactly I don't think we had the infrastructure to do and we still don't have the infrastructure to do the micro-payments. I think what we what we put in place was a set of tools that allowed those content creators that were typically ad supported or subscription supported to be able to control who had access to their site.

26:47 And what we saw from the largest publishers that are out there, you You the the People Inc., you know, the the Condé Nast, New York Times, you know, the Financial Times, the Atlantic, like those publishers actually went out and used those tools in order to then control the AI companies and then strike deals with them. I think the next step now is both how do we make sure that all of the AI companies have to play by the same rules and the the big boogeyman out there is is Google.

27:18 because Google is basically saying, you know, if you want to be in search, you also have to let us use our AI thing. They're using one crawler to do two things and they're trying to leverage their sort of monopoly that they had yesterday into the monopoly tomorrow. That gives them a huge advantage. Google, to give you some sense of these, more than two times as much of the web as OpenAI, more than four times as much as Anthropic, more than five times as much as Microsoft.

27:41 Like they just have this enormous crawling advantage, which is going to give them an advantage in AI over time. We've got to figure out how to either bring everybody else up or get Google to play by the same rules. And I think we've made some real progress on that over the over the over the last year and and we're headed towards a place where we're going to treat Google the same as everyone else. And then and then I think the second thing, that that we have to do is we actually have to have the infrastructure in place to support the folks who aren't going to be able to go out and negotiate deals on their own. They're They're not Condé Nast. They're not People Inc.

28:10 They're They're, you know, small blogs. They're They're, you know, mom-and-pop shops. And and we got to be able to make it so that even they can essentially have a rate card where they say, "Listen, you want my content? Here's the Here's the way that you pay." And we can make it easy for those transactions to take place. But it's important to remember that's just for the ad-supported subscription sites that are out there. The vast majority of the web, people want their content to be in AI and we've got to make that easy for them as well.

28:35 >> Right. So, then how do you read this Wall Street Journal article that came out, recently? It said Google was a lifeline for publishers, now some are thinking of cutting it off. the article talks about how Reddit has discussed shutting off the technology like Google's technologies access to its content for AI use. It's also got some quotes from people who you've mentioned. I mean come from companies that you've mentioned, right? Neil Vogel from people says turning them off and blocking them entirely is 100% on the table.

29:04 USA Today's co-chief executive Mike Reed it's time to take a stand and say enough as is enough. Is this is the right way to read this article that like now that these publishers have put into place your technology that might allow them to block the AI crawling, they're now doing the the sort of negotiating in public where they're telling Google, "Hey listen, if you don't give us the terms that we want, we are just going to use Cloudflare's tech and block you."

29:30 >> Well, if Google is just another AI company, then they should be treated like any other AI company. And if every other AI company has to pay for content, so should Google. Like Google is not special just because they were again the business model of the last 30 years of the internet. They're not going to be the business model, at least not that way. It's not going to be ads, it's not going to be subscriptions for the next 30 years. And so as that changes, Google should have to change with it. And and I'm encouraged that again all of the publishers are are are coming to that conclusion independently. I'm encouraged that they can then use our technology to do that. And what we've said at at Cloudflare is that by mid-September we're going to treat Google the same way on if you have an you know ads, if you're a business that's ad supported or subscription supported, then we're going to restrict Google from being able to access your content. You can always opt out, like you don't have to like you if you if you're like, "No, I love Google, they've always been good to me, they can have access to my content for free." Go ahead, it's that's that's fine. But the defaults for us starting in mid-September are going to be that we are going to block Google by default.

30:34 And I would imagine that a bunch of the largest publishers in the world are going to follow suit. >> Wow. and so what does that do to like, let's say a publisher's you know, ranking in the search algorithm? >> Well, I think again, I think my hope is that Google is going to come around and say we're going to pay for the content cuz if they don't have the content, then then Google's product is is going to be, you know, significantly worse. I I would say that ranking, like thinking about SEO is like it's just not relevant anymore. 10 blue links are going away. Google has said that the 10 blue links are going to go away. And so, if you care about being in Google, that's fine, but you should also care about being in Open AI and you should care about being in Claude. And and again, the vast majority of content creators, that they want that, right?

31:17 And so, that that's we're going to make that as easy as possible. But if you're ad supported or subscription supported, like then then you're then those businesses are dying. And you've got to figure out a way to negotiate a a different business and it can't be that Google gets special treatment. And so, and so I again, I think that what the industry is saying is that the old world of SEO is gone and we've got to invent a new business model for what the future looks like.

31:43 >> Yeah, this is from a New York Times article writing about some of what we're seeing with Google. it says users are writing queries three times as long as the keyword heavy questions they asked of traditional searches. People are spending one to nine more minutes in AI mode than on traditional Google searches. one October study from Growth Memo, a newsletter focused on search and marketing, found that in about 75% of sessions, users never left AI mode for the web. so basically what I think you're saying is we're heading towards this kind of seismic moment where publishers, you know, they've played nice with Google cuz they had still a lot to lose. if, you know, they sort of block them, that could be a big portion of their revenue.

32:24 But at this point, they've basically seen like they have very little to lose left and yet they're still delivering a lot of value and we could be seeing in September basically this moment where this key source of information to Google goes away. >> Yeah, and it's you know and that's Think about what a seismic threat that is to Google in some ways. Like the whole page rank algorithm is is about building a chain that attributes information. If all of a sudden a huge portion of that chain just drops out of their visibility, it actually breaks the fundamental way that they can do they they can do search and they can do ranking online. And so I think that that th- they are finally recognizing that that is a a systemic threat, that the business is changing. And I'm hopeful that Google who's been, you know, this incredible patron to the to the internet has been, you know, provided more for it the last 30 years than anyone else.

33:18 that they recognize that the internet's changing and that they should be supporting it. And and again, I think actually from Google's perspective, from their business perspective, from the right thing to do as a Googler is how do we make sure that we're the ones who are out there actually supporting content creators and and doing it in in an interesting way. And I and I am surprised that Google has dragged their feet as much as as they have as opposed to leaning in and saying, "We're going to actually help build the future, invent the future, and support content creators in the future."

33:49 And that's going to be again different than what existed before. But again, done right, I think we could be a a a a a place where this is truly a golden age of of content creation online. >> So okay, so I'm getting it. So that Wall Street Journal I was wondering where that what the story was with that journal article that all of a sudden all these publishers were speaking up about this, but they finally have effectively the technology to do it and now they have a deadline for Google. And if you say it in let's say July, you give them two months.

34:16 >> Yeah, and they've seen and they've seen the revenue that Google is is driving go down and down and down. Now that again, that doesn't mean that AI doesn't drive revenue for certain businesses. And Google keeps publishing these things saying, you know, look at how great, you know, AI is for e-commerce or these other things, and that's right. The vast majority of the web wants to be in these AI systems, and we need to make it as easy as possible for them. But, if you're ad-supported or subscription-supported, this is an existential threat to your business, and you've got to make sure that there's there's something there. And what I think is also is just right is it needs to be fair.

34:54 So, can't be that you charge Open AI and Anthropic, but Google gets everything for free. Like, it's got to be that everyone plays by same rules. And I think we're at that tipping point where even Google is starting to realize that they the economics of the internet are changing, the economics of media creation are changing, and that they're going to have to get there. It's actually interesting to think about like the conversations with with like some of the other AI labs where they're like, "Well, none of this matters because eventually we're just going to everyone's going to have universal basic income, and that's how people are going to get paid." And I'm like, "Okay, but that's you're you're living so far in the future. Let's kind of work our way back, and before we get there, maybe we have some steps that we have to go through, you know, along the way."

35:33 And so, again, I am optimistic. I'm way more optimistic for media companies today than I was a year ago. And and I and I really think that 2026 is going to be the year that in the media space, in the content space, that you're going to see a real shift and and that and that content creators are going to get paid. Google, by the way, is one of the best at paying content creators. You look at YouTube. Why do content creators publish their content on YouTube?

36:00 It's because YouTube like YouTube pays content creators better than anyone else. Like, they're they're not they're not against this across their their entire business. They they know how to do this in various places. I think what we've just got to do is as help them understand that like The New York Times and The Atlantic and and People and and, you know, Vogue, like they deserve their content creators that deserve to get paid the same way that some, you know, influencer on YouTube does.

36:27 >> So, you think Google's going to pay up before this deadline? >> I don't know if they'll pay up before the deadline, but I think it's inevitable that they'll pay up. because again, I think that they they they need the content and they understand fundamentally that it journalists and content creators and academics deserve to eat. >> How many of your clients like do you think will actually go ahead with the blocking of these AI crawlers? I'm talking about not not your whole client base, but just the news publishers.

36:56 >> Just the just the folks who are ad supported. So, again, we've we've >> Yes. >> done this at like if you're if you're not ad supported, none of this applies to you. Like you can you can set whatever you want. But if you're ad supported, by default, starting in September, we're going to, you know, switch this over and you can anyone can opt out. But but we expect that the vast vast majority of people that have ad ad or subscription supported businesses will will do this. So, there will be millions of sites that will drop off of Google's radar and at the same time that we will very much tighten up all of the other controls to on on AI crawlers. So, right now, it's basically put sort of a no trespassing sign and they'll stop line and say, "Don't don't come and get this content." And most people are actually pretty good at respecting the the no trespassing sign.

37:37 Starting in mid-September, we're going to actually make it not just a no trespassing sign, but we're actually going to have basically put a bouncer there saying, "Unless you're compensating creators, you don't get this content." >> Millions of site could sites could drop out of Google's ranking. >> Correct. >> That's a big deal. >> Mhm. I mean, it's a a huge percentage of the internet uses Cloudflare. And so, again, I think we're in a position to help fix what is a a fundamental market problem where Google is leveraging their monopoly of yesterday in order to generate a monopoly tomorrow. And I think that if we do if if we don't fix that, like who's going to? And and if we don't fix that, like as you you said, the incentives for content creation just go away and and that makes that makes everyone worse off, including Google.

38:27 >> Okay, so big big stuff coming up ahead. I want to take a break. When we come back, we're going to talk about how this might apply to or or how this new technology shift is going to reshape small businesses, which is not something that we've heard a lot about, but if we go to an agent type of web or super app dominated, you know, way to browse and interact and and do commerce, then potentially small businesses are at risk. So, we're going to hear Matthew's perspective on that when we come back right after this.

38:57 And we're back here on Big Technology podcast with Matthew Prince, the CEO of Cloudflare. so, Matthew, I've heard you speak about this a little bit, but basically your perspective is as we go more agentic, there are some serious risks to small businesses who might just kind of get lost in the super app shuffle. you've done it much more articulately than the way that I just described it. So, can you explain sort of what the stakes are for small businesses again as this trend continues to progress and we go to a web that's, you know, so much more bot than it is human.

39:27 >> Yeah, so, you know, I think I think first of all, like a year ago, what we were talking about what we were worried about was what's going to happen to media. And I think we've actually done a ton of work and and made real progress on how do we figure out a business model that's going to support media going forward. But as we've thought about that, we thought more generally about how's this going to have other impacts across across the economy. And and I really do worry about small businesses.

39:48 Now, that doesn't mean small businesses are doomed, but I but I think we have to be extremely deliberate at understanding what the risks are and then trying to figure out, you know, what the solution is. The world that I'm trying to play for is one where there is not five AI companies, there's 500,000. There's not, you know, a a tiny number of content creators that everyone can be a content creator and put something out. And that that really importantly, that we don't consolidate businesses into a few big mega corporations, but it still becomes a place where I think the internet has done historically incredible good job where new businesses can start up and thrive going going forward. And and I think that's what we have to play for is how do we have as many small businesses possible. So let me let me walk through what I think is the problem. So in order to understand think about you're a you're a small business. Let's say you're starting something to be a new you know B2C a a DTC a direct-to-consumer brand. I don't know you're making I don't know new light bulbs or makeup or who knows what what possibly it is.

40:47 What do you do in order to reach customers? The answer is you go on something like Instagram or a Facebook or even a Google AdWords search and you start to buy the ability to get your product in front of consumers. And consumers, you know, again they're scrolling through their Instagram feed and they see your you know slickly produced video that you maybe you raised some venture capital or you got you know small business loan in order to put together. And if you touch kind of the emotions and speak to consumers, then the then the companies that do that are actually going to rise and and and thrive over over time. And I think a lot of if we're really honest, a lot of the small businesses that we do business with, we do business with because we have some either emotional connection to them or because they're convenient. Like they save us money. Why do I shop at the local bodega even though I know it's more expensive? It's cuz it's convenient. And like I have limited time. And so I'm not going to you know go find whatever the cheapest place is for eggs. I'm just going to go to wherever is is most convenient. And that's what humans do. And if you think about brands, like what a brand is is a shortcut for humans to understand value and quality without having to do a ton of research.

41:59 So I see a Marriott logo on the side of a building, I kind of know what the experience is going to be when I go in. I see a Walmart logo on the side of a building, I know what the experience is going to be when I when I go into that. What I think is interesting is like brands are going to change radically. Like it's not going to be a couple of like swatches of pretty paint that agents are going to care about. As more and more commerce gets done through agents, what it what a brand means is going to be very different because agents have like they have almost infinite patience.

42:30 And they don't have emotion to sort of say, oh, you know, I I have this brand affinity to Marriott Hotel so I always stay at Marriott Hotel. Instead they're willing to do really in-depth research in order to figure out what's right. And at first you might think, well, maybe that's good for the small guy. But I think it's actually going to be really hard for new entrants because new entrants don't have very much information online. I launch a small business trying to sell my new light bulb or makeup brand. And like how do I get my first customer if all customers are going to be agentic in terms of that? I think that's a really interesting and hard problem because the agents have infinite time to research every single possible detail and so they're going to look for where the richest sources of information that are online, which are going to be for the established big companies that are out there. And if we can't be in a place where new entrants can easily enter the market, then what happens over time is that the big companies will acquire each other, will have massive consolidation, and it actually for the end consumer might seem just fine because you can get hyper-specialized I care I I only want brown eggs from certain hens, right? And maybe, you know, Tyson Farms can produce that for me, you know, very efficiently.

43:41 But how does my local farmer find a way to actually reach this cuz they're just not going to have nearly the depth of information that's out there. And so what I think is the thing that we have to be continuously thinking about is in a world where we're increasingly having commerce sort of federated by agents and that that's the interface that people are going to do it. How do we make sure that a new brand without any real information in the market can actually be able to exist and thrive because if that doesn't happen, then agentic commerce becomes this massive tool of consolidation and that's going to favor the big companies over the little companies and over time like I I was I was meeting with the former CEO of PayPal who before that ran small business at Intuit. And he said, "I worry that in the future, there are only five companies.

44:26 One that holds the money, one that holds the real estate, one that makes things, one that ships things, and an AI company. And obviously that's that's not what's going to happen. But some level of consolidation is I think the natural trend of this and and what I've I've been doing to kind of sometimes you can you can you can sort of see the future by watching kind of science fiction and like go back and watch like episodes of The Jetsons, the cartoon show.

44:51 And it's like, you know, we I as a kid I watched it and I thought, "Oh, that's kind of fun and funny." It's an incredibly nihilistic and kind of depressing world. There are only two employers in the world of The Jetsons. It's Spacely Sprockets and Cogswell's Cogs. Like George Jetson's job is to basically push the button but the robots do everything else. And all of the commerce that George Jetson does is basically through his helpful maid robot Rosie. He's like, "Go get me eggs." And she goes off and does it and it comes back and George seems perfectly happy.

45:18 But behind the scenes, like do you think that Rosie is going to shopping at like the local bodega? Of course not. She's going to wherever she can get the most information about what the quality of eggs is and that's going to be some mega corporation that's out there. I worry about that and I think we need to worry about that. And again, I'm not I'm I'm nervous about it, but I think the first way of solving it is actually articulating what the challenge is and then thinking about how can we make sure that new entrants have a shot in a world where more and more commerce is being done by infinitely patient, you know, deeply researched, unemotional agents that are acting on our behalves.

45:53 >> Can I give the counterargument? and that is that brand is actually something that favors the big companies over the small companies because they have the money to spend. And so, like you might be much more much better suited to work with like a small business that you've never heard of, but because you, a busy person, don't have the time to research them, you just say, "Okay, I'm going to target." >> Go I'm going to buy Nike as opposed to, you know, whatever.

46:22 The new I I think that might be right if you can get that. But But the question is how do you And then And that's that's the happy story. And that's what we want to get to. But the question is how do you How do you get the information into the system? How do you get enough customer reviews? How do you get Like And then how do you encapsulate that? So, like what is a brand in the world of agentic commerce? I don't think it's pretty swirls of paint. I think it's actually like a true attestation of quality and experience. So, like how do I take Amazon star ratings, but bundle that with what's your actual like shipping fulfillment time? How many returns do you get? How many chargebacks happen How do I aggregate all that information, package it up into some sort of module, and then make it actually verifiable so that when my agent comes and sees it, they go, "Oh, I can see that this is a quality merchant, and I'm going to be able to do that." I think you have to have that, but then you also have to think before I have any reputation, I'm just trying to prove that my product is better, how am I actually going to be able to succeed in doing that? I don't think it's impossible, but I think that left to its own devices, agentic commerce is going to be a consolidating factor. And so, we have to be extremely deliberate in making sure that we're giving small businesses the tools to be able to do this. And again, I don't I don't CloudSwarm can do this on our own. I I would love to be part of a solution, but I'm spending a lot of time with, you know, Shopify, you know, PayPal, Visa, MasterCard, American Express, you know, Intuit, the companies that actually deal with these small businesses, and saying how can we bring together the tools in order to make sure that they can succeed in this future because again if they're if they're all just sort of following what the old SEO playbook is, that's not going to work for the future. It's got to be something else and I think it's really one of the most important questions is how do we make sure that these small businesses have the tools to be able to again live in in what I think you described as as the optimistic outcome. And again, I'm optimistic, but I think we should also but in order to be optimistic, you also need to worry about what could go wrong and and articulate it and then figure out what are the solutions to that problem.

48:26 >> Okay, so last question for you. What does that look like from a technical standpoint? Is it just like a small business has a portal in like the back end of chat GPT and just like spends, you know, the small business owner spends their weekends like dictating all the things about their business or how what could it look like and what are the solutions you've discussed so far? >> I I like I like when we talked last year, I didn't really have a sense of what like the solution for media is and I just spent a whole bunch of time talking to media, talking to payments companies, talking to AI companies saying, "Hey, this is the problem. How do we solve it?"

48:57 >> I think we made a lot of progress. >> My hope is that when we talk next year, I'll have a better answer to your question, but it's only going to be because you know, people who hear this say say, "Hey, I had an idea." and they reach out, right? Or or you know, me with Toby from Shopify and he goes like, "Hey, we've we've been working on this. Can you help distribute it for us?" Or or you know, American Express says, "Hey, we've been working with all these small businesses. Here's something we found works." I mean, we're trying to just gather as much information. I don't think we're going to solve the problem, but I I hope that we can be a part of the solution to the problem because like we could be in a place, done right, where this is actually helping more small businesses reach customers and it and the better product finding the best possible consumers for that product. And done right, that that's amazing. And then and then by the way, we get to then worry about the next problem, which is that AI is this massive divider between, you know, the rich and the poor. And and if we're not and if we're not in an ad-based business model anymore, which I think is likely. I mean if if you're spending thousands of dollars a year for your agent, you don't want it to be influenced by any ads at all. You want it to be completely honest. So, I think the wealthy parts of the world are going to have just huge advantages over the poorer parts of the world. And so, that's that's that's the next problem and that that we have and and and again, we're starting to think about it. I don't think we can solve it entirely ourselves, but I think we need to talk about these things in order to make sure that the future that that we're that we're building is one that, you know, is is is as utopian and and and not dystopian as as possible.

50:28 >> Yeah, well, we we hope so. So, Matthew, we got a good routine going. We've We've done this 2 years in a row around this time. So, we'll have to check in again next year. We'll see you then. >> and and I'll tell you what the solution for small businesses is. >> Okay, it is a deal. It is a deal. All right, thank you so much for joining, Matthew. Great to have you on. >> Thanks for having me.

50:46 >> All right, thanks everybody for listening and watching and we'll see you next time on Feat of Technology podcast.