Inside SaaStr's 21+ AI Agent Stack: 2.25M Sessions, 614 Meetings, $2M in Revenue. The Agents #006

0:00 We are going to go through as much detail as we can on our agent stack. We've talked a lot about it at this event, but we're going to go into as much detail as we can, show you backends, show you how they really work on the other side. Really, we could do an hour on each and maybe we'll do that over the course of the year on our live webinars. We're going to do as much as we can on this session just to really give you a sense of how they work. Maybe it's our 401 class. Um, I'll let Ameilia kick it off and we'll do a deep dive. We won't do all of our 20 agents, but it looks like we're going to do about seven or eight of the ones that that really make a difference.

0:29 >> Yeah, I tried to force rank maybe the top ones that we use every day. Um I had I actually used Gamma last night when I was tired after the VIP party to make you a little agent uh card stack, if you will. Some are coded, some are not. So, I'll explain the differences. Um and they've been used over a million times and collectively our agents have had multi-millions numbers of interactions. Um, and so I thought it'd be fun to just make a little Everyone's been asking us what our stack is this week. And so I thought I would make them as Pokémon cards. Um, so that you can also collect them all if you want. Um, catch them all. Uh, all right. So, a few of them here that we'll go through today, but just to give you a quick run through of some of our top agents that we use every day. So 10K, you've heard us talking about hopefully you came to the session on Tuesday where I walked people through how to build your own AI VP of marketing. 10K is our AI VP of marketing that we use. If you did miss that session, it's on the live stream. So you can go back. I do a whole hour on how to deploy your own AI VP of marketing. Uh QB who we now call QB is our AI VP of customer success. He's handled all the sponsors you guys see here today with some human human interactions of course, but he's done a lot of the heavy lifting to help the sponsors and our customers that we're loading into the SAS annual this week. There's Annie, who you guys all know because that's the SAS annual website. She actually turned into an agent the more that we did with her. Um she has a parking pass app. She has a few other things she can do and so started as a website turned into an agent. We'll explain how that works next. Amelia AI, which you can also demo over there if you want. That's our inbound agent that runs off qualify from Salesforce. Um she we've had her since like last summer, I think. And she is the one that is driving all of our inbound sales, handling all of our attending questions like you guys have been asking her like Wi-Fi, where's this, where's that, that's Ameilia AI.

2:12 So she's part seller, part customer support, part marketer. Um, she does a lot of things. Then there's agent force. I made it look like a ghost because we use agent force primarily to follow up with leads that our sales team never wanted to follow up with. So I call that one, you can call it agent force. I sometimes I call it king blue cuz he's a ghost. Um, so agent force reviving our dead leads that we never followed up with. The two other ones we use for sales as well. So Ava, that's artisan.

2:37 You guys have all seen Ava on their booth this week. We use her for a mix of both cold and warm outbound. And then Monica, who's newer to our staff, we use it for completely cold outbound um to prospects and companies that look and match our ICP. So it's really Monica is really good at taking look alikes for companies that you already have and that you've already closed and then just automatically filling your pipeline with companies that look exactly like that.

3:03 You're listening to the official SAS AI podcast brought to you by Hey everybody. Starting a business can get expensive fast. Web plate here. email somewhere else, business phone with another provider. Northwest Registered Agent gives you a complete business identity in one place with free tools, resources, and built-in privacy from day one. Get more at northwestregistered agent.com/sasterfree. That's northwestregistered agent.com/sasterfree.

3:33 S a s r e. Starting a tech company is thrilling. It's also really messy. Just ask anyone who's done it or anyone who's tried. Better yet, ask EY. They've seen startups in their best moments. Go to ey.com/teartups to learn more. We do have other agents we've deployed. I don't know how much you want to touch on these, Jason, but I just wanted to show the top. >> Yeah, we can keep going. If you haven't seen it though, you can and we'll update it with all the content from today and this week, but if you go to sasser.ai/agents, that's continually updated with everything here, the best sessions, all the decks, all the slides. It's good today, but by next week it'll have everything from here organized for you.

4:11 So you can come back to that. >> Cool. So let's start with 10K. If you missed any of our previous demos on what 10K does, because he does a lot of the heavy lifting for us. One thing I wanted to prep is because we just showed you our stack, some of the ones that we built ourselves. We built on Replet. So I'll call out in these slides what it's built on, what's powering all of our different go to market agents. But one kind of common theme I noticed as I was building out this deck and just thinking about our stack of AI agents, if you will, because folks were just asking, I was like, "Hey, what's in your stack?

4:40 What are you guys using?" is a lot of these did not start as full-blown agents. It's just something where we wanted to replace something. I wanted to replace our notion dashboard because I got tired of copy pasting things from our different data sources. And so 10K really started as just a dashboard back in January. That was his thing. I just wanted to hook up to Marquetto and Salesforce and really simplify life and it really was just a dashboard for a few weeks. But then 10K evolved, if you will, and now does a whole lot more. He owns the number. He helps us track our daily revenue across all of go to market. He helps with our forecasting, which is super important, helpful for something like this where you have really time-sensitive maybe releases or product things that you guys are doing as equivalents. He knows all of our campaign performance now. So he's tracking all of our marketing campaigns in real time. He pushes out our daily marketing ideas. We we've been talking about this week our top three ideas that we get every day for SAS comes from our AI VP of marketing.

5:38 And it's interesting too. I did a little test comparison this week where on Tuesday I I rebuilt our AI 10K VP of marketing on Replet and then yesterday I did a little bit live with Elena lovable. And actually the ideas it came out with were different. It's like the LM were so different that it came out with four different ideas. >> You tried to rebuild 10K level as much as you could. And what what was different?

5:59 >> So the the biggest like the dashboard still, you know, kind of looked similar like there were elements that were the same, but the underlying brain was different because the the biggest test for me was like when I gave it to Replet, I think the top four ideas it gave was like more email based and then the ones that lovable came out with were more like, okay, you need to do ads, you need to spend on this, you need to do ads. Oh, the idea is that the seeming brain we got to be careful not to anthropomorphize AI, but the idea is that the lovable version were different than replet.

6:27 >> Exactly. Yeah, but they I gave them both the same spec and the same inputs. >> We could run we should run an experiment. We'll follow up. We should get them both running for real >> and have them debate and argue their three ideas a day. They each come up with three ideas a day. It's too many. Let's have them compare and contrast because for reasons that we probably are not smart enough to understand Ted K is very biased on email marketing right and the version on lovable is very bias on advertising >> advertising brand >> brand a little like Elena >> I know >> that's pretty interesting that that they came different conclusions >> I was like it actually matches the marketing style of those internal teams I I don't know maybe it's unintentional that however their internal teams are using it for marketing as like >> oh rep's nerdier and lovable is more marketing >> so it's a little funny >> yeah it is we'll thread some stories in this about how it is surprising how seeming agents with seemingly similar data sets and LM's come to different good but different conclusions I'll share a story from this morning later the one thing I would add to this Emil if you were here on the first day in the morning for the 21 session Emilia gave you a recipe she gave you the skills we'll post it online you can build your own 10K we don't want to duplicate it here but the one thing I would add um I think Emilia hit on it. The simplest way you can get some of this done, like the most basic version of this, and a millia show you'll go deeper, but this headless Salesforce idea that's in the media if you haven't tried it, it's really effing powerful. Okay? And and they just Salesforce just launched something called Headless Salesforce if you're on Salesforce. And it some of that is actually a product that doesn't even exist yet. It's on Mulesoft, which they bought years ago. And a bunch of stuff that most folks are never going to use.

8:06 You're not using you in the room using Mulesoft. Okay? Nobody. Okay? But most of you are using Salesforce. The Salesforce API is really good. So if you do nothing else, spin up a replet or lovable or vzero from versell and tell it and then just get and tell it you want to go into Salesforce and build I want a type of dashboard or analytics I don't get today. Okay. And we'll show you like for example for me I wanted I want real-time visibility into attendance and ticket sales here every hour. That just doesn't work in Salesforce, right? So that actually took two APIs biso and sales. My point is now I can interact with Salesforce data in a way I never could and this is you can get a quick ours started off as a dashboard for tickets and attendance but it could have been a Salesforce dashboard you do not if you really want to do something cool >> just go to go to you can do it in cloud code too but I actually think reputable or vis and just say connect me to Salesforce with their connectors and I wanted to do this and watch some magic happen they already works headless and agentic you can get 10% of what we do in an hour and maybe do something really tool that you would never be able to do with Salesforce on its own because these agents are so powerful.

9:12 >> Yep. So, just a little bit of the back end of 10K. So, I I did a version of this for each of our agents that I showed as our top agents. Um, so 10K was built on Replet. His first commit was in January of this year. So, he's really not even he's only four months old actually. Um, but it's funny. I asked each of the agents how many commits we've done. So, this one has almost a thousand, which is a lot. Um, seven to eight commits a day. So, that's between myself and Jason. And then for fun, I put how many lines of code each one of them was. But uh the more of and then there's there's 10K has the most external APIs wired into it. So this is what we mean when we say headless Salesforce is we're using Salesforce just directly via the API without logging in directly to Salesforce. I still log into Salesforce. Jason doesn't have a Salesforce login and I saw like use the authenticator app when I go move things around. Um that's not in the API yet. But um so that's what we mean by headless if you guys are wondering what what all that means. Bisbo. So that's what we use for ticketing. That's how you guys all bought your tickets. Um and then Marquetto, which was our uh marketing automation platform, um is still hung up to this for now. And then Slack, that's how we get our daily every day when we get our daily marketing ideas. We get an email with it, but we also get a Slack update. Um QB also has Slack. So QB will Slack as a morning customer report essentially of, hey, here are the accounts you need to follow up with. Here are some of the sponsors who maybe didn't finish what they were supposed to be doing. Here are people that are overdue. And then it'll also slack us and say, "Hey, here are some folks in your customer base that we think you need to follow up with because maybe they haven't done anything or they're super behind on things or they're approaching a deadline that they haven't quite met yet." So, it can do a lot of that automated for us. And so, I really like uh 10K and Slat. And then you'll see clerk in there, too. But those ones at the top are really I think one through six are the big ones for 10K.

11:02 And then we talked about this a little bit, but I want to just force rank kind of like the top three things I think 10K does for us. He had a different list of what he thinks he does for us. The top three things well, but yes, number one, he is still a dashboard, but he is a living breathing dashboard. So we can talk to 10K. So to Jason's point, he can just talk to any of our data from Salesforce, Marquetto, Bisabo, and he can ask a question like how many VCs are coming this year? How many CMOs do we have coming to the summit today? What sessions are maybe looking a little bit late? Should we move them around, add a new speaker? And so we can just talk to our data every day. Um I think an interesting point here is I don't know if you guys heard the session with Denise yesterday from Snowflake, but it's also on our YouTube channel. She was talking about how she she has this like bold statement that the dashboard is dead, but really it's because she has dashboards, but she has agents. She just talks to her agents. And for her at scale having her, she's got multiple versions of 10K. for her the biggest thing at scale was that and she doesn't have to argue anymore between sales and marketing what the number is the number just is the number and so for us it's still I think prevalent across for your guys' scale to have something that is yes a dashboard but it's one that we all agree on because it's directly connected to the API and so there's never any nuance or people lying about the numbers or maybe pulling it from the wrong dates or trying to make it look better than it is um if the numbers are just the number because it's just pulling it directly from the API. And so I think that was a good insight that she brought up that I hadn't really thought about previously.

12:33 >> I think I hit that uh with the OMJ session. Um uh the uh so if you were there, I don't want to be too too duplicative. Um I don't have too much to add. Um I I guess it's just a reminder that um Amelia said when she had the lines of code in the commits it it sends six to seven commits a day and anyone that's a developer engineer here might roll their eyes and say you shouldn't be doing six to seven code commits a day. It's like a bad sign. But we're not we are tech from the perspective of replet. We're doing six seven commits from the perspective of us as >> the boss of the agents or the agents of the boss of us. We're just improving the application each day, right? Um >> and so I guess just the lesson from this email when last week >> 10K started to create better emails than humans is just how much if you keep engaging with the agents every day, they will get better and better. And that's the key. Whether it's just to train something like Artisan or Monaco or something or qualified, if you train them, they will get better and better every day or if you have agents that are somewhat more alive like 10K and QBR if you talk to them more every day. Their context windows will get richer and richer and they will perform better and then somehow last week they will create a better human out of the blue than a human can. And when I asked Amjad Sioppplet how he did it, he didn't quite know. When I asked their head head field, he didn't quite know. But that that's the key is the more time today uh the whole open cloud thing is really dangerous that you could not just that it has security issues and breaches and can leak your confidential information.

14:12 I I think it also confused people that an autonomous agent requires no work. It can go off on its own. The number one lesson we've learned is the more time you invest with the agents just like most humans the better they get. And just like most humans, you don't really need to go over the basics with your agents if they're well, if they're if they're performing well. If they're not performing well, you have to go over the basics again and again. But even your best people, the more if you can just spend a little bit of time with them each day or each week, they will get better when you start to mind with them.

14:42 And it's literally the exact same with agents. Yep. >> And that's a little bit different than the narrative on on people that aren't doing it, right? We think it's the opposite. >> Yep. >> Okay. And so QB, we've I realized we did a workshop a while ago on how we built QB, but he is our AI VP of customer success. So we can do a fresh one after this um where we walk through how we built QB kind of what he does now because when we did that webinar or podcast a while back, it was like he was he didn't have all the features and functionality he has now because we again we keep shipping.

15:11 >> Yeah. >> Lines of code into him. But he again he started as 10K was just a dashboard. K was just a project management tool. So we had an anti beta project management tool we were using for customer support basically to manage all of our sponsors um to get information from them and because SAS events are like kind of niche and weird. No project management tool was great at this, right? They were all like out of the box, just you get what you get. And there wasn't a lot of customizations we can do. And so it required a ton of human followup, manual emails, manual calling people, texting people. It was a lot of manual labor outside of the project management tool that we had in years past. And so we started this as a vibe coding project of I just wanted to save us some time, maybe get some it has saved us quite a bit of of of budget by building him versus like having humans follow up with these folks all the time. And so he he really turned into a sponsor, what I call now a self-service agent where he can manage questions, he can intake logos, he can intake websites, he can remember all these things about our different customers. And he really has become not only a self-service agent, but also a personalized asset management. So he's collecting all. It used to be a pain in the butt for us to get everybody's logo so that we can put them on the sponsor wall, put them on all the signs here. Um, it was honestly annoying because because they people email you files, sometimes they're too big. But now you just upload it to our agent. It's much better. But the better part of QB now is even though he started as just a dashboard. Quickly I realized, okay, this agent's pretty good to the point where he can send each of our there's almost like 150 sponsors because we have some non boo sponsors too. And he can email them personalized, right?

16:54 This is something that no human can do. Like no human CSM wants to have a 100 accounts. They all want to have like five. And so that was the magic of Q.B. was that he knows all of our sponsors cold. He knows all their assets. He knows their logos. He knows what they do. He can he uses whatever quad to research about them. He knows more about our sponsors and like probably some of your best CSMs know your top customers.

17:17 >> Can we try to ask QB some questions we never tried before? >> Let's do it. >> Let's see what happens. >> Okay, QB. Here's QBing. >> It's QB. Okay. Let's ask QB. This is your show, right? Let's ask QB. Yeah. What what sponsors are most at risk of not renewing? >> Okay. >> No idea what I was going to say. Never asked. Um but QB has a lot of data. Let's find out. You probably had something else you were going to ask, right?

17:40 >> I think it's a good thing to ask today, right? >> It might fail the test or it might surprise us. Let's find out what QB has to say. >> He's thinking. But while he thinks, I'll just show you guys around KB a little bit. If you've never seen the front, you can go to serssponsors.com. You can play around with Q.B. He does like you'll hit a wall eventually if you're not a sponsor of ours because you'll have to log in um so that you don't get their data or anything weird like that. But you can see like he started as a dashboard. So he's still he's still our project management tool here. He's got all the information about the events. There's certain sponsor loading times that they had here. Um this is where they all picked their their booth that on the floor that you should see here now.

18:21 These are all the things he did originally which again these are all things a human had to do and he created these nice overlays for our maps which our designer used to have to do but now he can work on the stages and there's a lot he can do but now he's so here's his assessment and who's at risk >> who's at risk artisan I that's a fair one he's flagging that they they never really logged into him so he's like >> that's a signal >> they never talked to me they had a customer success they never talked to me, right?

18:52 >> They never talked to Q. But they did talk to me a lot. >> So lack of contact. Okay. Okay. Flag that one. >> Salesforce. Interesting. He says they they were incomplete. Um and they went dark for a little bit. That's a good flag. But again, I talked to the Salesforce team and so does David a lot. Google Cloud, same thing. Like these are folks we It's funny. He flagged all the people that we the humans spend more time on than the agent.

19:18 Uh there's a few others here. Pylon. >> Pylon complained the most to him. >> Okay, I see. >> That's why he said it cuz I asked him before cuz QB reached out to all the SP instead of a human reaching out to him, QB reached out on his own, talked to them, asked what their issues were and how he could help. And Pylon said their logo was wrong and a bunch of other they complained a lot. So he still gave me I can't quite read it from here, but a medium sounds right. It's not not a high risk, right? We fixed the logo before it went live, but their amount of frustration was a signal that a human might not know how to rank properly or hide from or ignore. Right.

19:53 >> For sure. >> Yeah. >> Any other interesting ones that he thought >> Lightfield and Monaco never did their VIP nominations. That's a that's a good that's a fringe thing that we do for folks. We let our sponsors nominate some of their customers to come disaster. He's saying they never did it, but they are top they're some of our top sponsors. And so maybe it's a it wasn't a high ROI for them. >> Okay. Ask you one last question. You can do it. I know a second. Um you don't even need the empower mode, but keep it in there. Right. What sponsors did you talk to that seemed happiest?

20:24 >> Okay. >> It's the flip side, right? Let's just see what he says cuz he talked to them. >> That seemed he'll forgive my typos. >> This is why I think traditional customer success is just dead. They won't talk to 120 people and like all they want is a commission on the renewal. They'll hide issues. They won't surface issues because it doesn't help them that help them hit their number or goal any longer. Uh I think QB is better. Who's that? Is this who's the happiest? Is that what he say?

20:56 >> Yes. So he says there's no explicit this was great signals in the chat yet, but here's who I think have the highest signals of satisfaction. Viden because they wanted to post more on social. Um, Launchpad just returned a lot to the chats. Uh, Gamma, um, seems pretty happy with what they're doing. They never pushed back, never escalated. That is a good signal, actually. Uh, then he lists out a lot of sponsors all at once. Uh, asked once, got the answer, and left pattern. So, that's a good sign, too.

21:22 They got what they needed from QB. And so, he's saying that those folks are probably fine because I got them what they needed, and they went on their merry way. >> Yeah. >> So, pretty good. >> I think this I don't know what you guys think. I think this is pretty good. And again, this this agent and agent's kind of a loose definition, but this agent's less than 90 days old, right? >> Correct. Yeah. >> And again, it started off as a just dashboard to track assets. Uh almost a preAI workflow. There's a lot of asset trackers if you're in the creative industry or do anything. Now we can talk to it and it can identify issues. And I think that analysis was pretty good. We never done it before. That was I just made that up with I thought it was pretty good. I would say >> top 15% of CSMs I've ever worked with already. And you can already see looking at okay that's good but it lacks some of the context of human conversations that we had. We'll probably a million and I will probably ramble. Think about that and we'll talk with cubulator. How could we integrate some of that context?

22:16 >> My email for sure. A simple thing is to hook up to your email. Yeah. >> We could hook up to sorry what's the application where we listen on your phone? >> Oh, whisper flow. >> Whisper flow. any of these things that have APIs, maybe we just hook them up and actually if they have an API, it's really easy. It goes and figures out what the best emails are in whisper flows in in 10 or 15 minutes and maybe we ask again in a month. Maybe we'll do it on on the agents on our podcast and we'll say our complaints because we've never done it before. But I would already give that like a B.

22:43 >> It missed a whole it only has the context it has. It missed the human side, but for the digital stuff, >> it was pretty good in remembering all the chats it had with maybe 500 or,000 people force ranking the issues and identifying them. um better than human hiding from issues. So that's QB. >> That's QB. He's got an interesting API stack that differs from 10K. So they they have a lot of crossover, but noticeably too he's got he's got clerk is number one on the list because that's what we use for signal fine on. And so that was one I I've talked about it previously. It's gotten easier in replet now that it's native. But it used to be a little hard to actually hook that whole thing up. And I'm not again I'm not a technical person. It was pretty hard for me to figure out single sign on. But I got it done in a weekend and now it's even easier. And so that one has remained top of the list because you got to imagine people are inviting other people to interact with QB and they can just add it on their own. They can catch up. They can see what other folks are doing with QB within their org. And so that one makes the top of the list for QB.

23:38 >> Yeah, it's a pretty simple stack to get that much value. It's not even logging into Salesforce, is it? >> No. It's funny when I It's funny. When I went through it, I was like all this time I really thought QB had Salesforce, which we'll hook up next. Um so he can write some of this information back to it. But he doesn't that's that answer you just gave you was without even having any Salesforce data so it'll get even better.

23:58 We already talked about this so I'll move on to the next one. So Annie is somebody you all know because this is our disasterual.com did not again did not start as an agent much like QB and 10K started as just a replacement cuz last year's sastasteranual.com was on Squarespace and we were just limited by how much we could do on sasanual.com cuz we're not developers. We're >> we were just limited to Squarespace. You can't vive. You can't change. All you can do is put images and some videos.

24:26 >> It wasn't enough this year. So, we just I quickly built a V1 on Replet. It's okay. Has a little some purple gradients, but um but we could get it to do whatever we wanted to do. Right. >> Correct. Yeah. And it started as a website. Uh I don't even remember. Oh, I guess it's on the next slide when we did it, but now it does a lot for us. Right. So I asked actually our SAS annual website which we just call Annie because annual um how it would describe itself as an agent cuz I was like okay 10k pays very clearly we call it our AI VP of marketing we call QB our AI VP of customer success I said Annie what would you call yourself >> and it said it's our AI event producer hybrid because it's managing >> human and what's the hybrid >> I think hybrid of producer and also technical yeah event manager it said it was and it was also our technical producer because it has all the website and the agenda stuff.

25:15 >> I It's always interesting whether you when you ask somebody including agent what they want their title to be. Sometimes I remember asking Ameilia years ago what title she want. She said DP marketing. She said okay get that's your title came right out of her mouth when she started. Now she's she officer but Andy came up with quite a title. >> It really >> quite a title. Yeah. >> I was like you're I'm not sure that that spec is on job bite but uh I like it.

25:36 Yeah. >> I was like are you trying to say that you did some of the physical stuff? >> That didn't quite work out. Um, but yeah, it was a little funny, but it does again it runs. I was like, "Okay, but to your credit, you are running the website. You're running the agenda. You are sending a lot of the attending newsletters you guys hopefully all read this morning. That's all coming from Annie." Yep. >> Um, and then obviously I it became a when I added this now famous parking passes app, which was previously, if you don't know the story, for you guys to get a parking pass to come disaster, we had to have a human on our team split up the 5,000page PDF. So what I did is I first I just used claude co-work to split up the PDF, add all the addresses to the bottom of the PDFs and then I hooked it up into Replet in our existing website at the time and said, "Hey, can you help me create just some agentic um workflow so that you can automatically send a parking pass from this long dock of PDFs that's now been separated to people who need it. So when you guys request a parking pass, you just put in if you're an attendee, sponsor, speaker, how many days you need, and then it sends you your parking pass automatically versus you guys having to previously was a form fill. Last year fill out the form, wait for one of us to send you your parking pass, and that took a while.

26:48 It also it's also um in the background something else I think it's doing that I don't give it enough credit for is it because I've hooked it up to Vector it can see our active website visitors and run targeted campaigns to folks based on what they're doing on their website. Qualify does a bit of that too. So there's a little funnel action there. But this is Annie built on Replet first commit back in November. That's when we swapped off of Squarespace. um he she or she Annie has the most um commits out of all of our agents and the highest amount of commits per day. And I think that's because we were just safer.com became a living breathing agent and kind of co-pilot for the event as it calls itself an event producer versus it just literally being a website that was static.

27:31 >> Can we do a Can I ask Andy some questions? >> Sure. >> You know what's interesting on the side? I It's interesting these 46,000 lines of code. It doesn't really matter how many lines of code you have. I think again a lot of engineers and maybe critical of it on X, but I wrote a SAS post literally two weeks ago about 10K to that it was $257 a month and I think 18,000 lines of code. Now we're adding apps inside of it, micro apps and now they're 4650,000 lines of code. Um there's clearly some slop that has gone in. If you've gone from 186,000 lines of code to 45,000 a week, um but it's also part of adding more and more apps in one scenario. um that probably eventually will slightly break, but it is it's just as an aside. It is a lot of that's a lot of going from 18 to 45 in two weeks is uh if it kept going at that logarithmic or exponential rate, we would have a larger code base than like Salesforce or Meta. Maybe I'm exaggerating.

28:24 >> Um Annie, because I Let's see how well it does. Okay. Um write me a follow-up email to everyone at Saster AI 2026 trying to get them to come to 2027. Okay. see how it does. >> Since today is the last day and you are on stage right now. >> What did you say? >> Oh, I just told Annie um write me a followup email for Saturday annual 26 to come to SAS 2027 since today is the last day and you are on stage right now.

29:00 Smiley face. >> It's good. Let's see how good it Okay. It said subject line that's a wrap. It said it wrote it as you. >> Okay. Which we didn't ask it to do. >> I didn't ask it to do that. It is interesting like I I did I ran this experiment the other day with like a final email to our database and I asked Annie to do an email through the database and then I asked 10K to do an email to the database and see which one did better. It's funny. They were actually equal in like their conversion rates. But one of actually 10K wrote it as Jason and Annie wrote it as like one of those fully designed HTML emails, but the content was the Jason one was more like punchier. The Annie one was a little bit funnier. So it's a little funny now that now Annie has swapped to writing it as Jason's.

29:46 That's a rap on faster. Lock in 2027 today at the low price we'll ever offer. Preheader. I'm writing this from the stage. Um 2027 tickets are not live yet, but they will be later today. and they'll never be cheaper than they are right now. I'm writing this from the stage of disaster AI annual about to do the closing so forgive the brevity 10 days 10,000 AI leaders the CEOs of Clavio Gamma Webflow Rubric um all on one campus the deploy summit um six AI workshop tracks the pitch comps and more first meeting introductions than any week of the year. Thank you for being here. None of this works without you.

30:20 Now the ask come back next year. >> But it's pretty good. >> It's pretty good. I it's not a 10 like that crazy repo one where we pulled every attendee but it's it's better than 95% of humans would do and it did it in minutes and you can see the context >> it has the agenda built into it right so it's able to pull the speakers that we had this year it knows the numbers of folks that are coming knows the type of tickets someone that joined your team on their marketing team last month would be unable to write this email in a week they wouldn't know unless unless they could access the data themselves it's pretty good I I think it's that that that passes the test right >> you guys are going to get that email later >> it might do this email and then Annie and then I'll share a story. You go Annie is so Annie's hooked into Bisbo and Resend.

31:04 Did you have a slide? It's not hooked into Salesforce. >> It's not hooked into Salesforce. Um but she is hooked up to Marquetto to send like the attending. Okay. >> For now. >> It's okay. Annie has a different set of context. I'll tell you a quick story. Surprised me. Um I think it's interesting. So this morning we were coming to get here and I said to Millia, "Everything's been I think it's been great." Congratulations, Amelia. Great job. I think it's an amazing job.

31:28 >> Thank you. >> Um but there's been so much going on. I think I thought we forgot to remind folks to come to the founder VC brunch at 11:00 in the VC lounge over there. I said, she's you're right. I forgot. I'm like, can I ask 10K to do it real quick because it's risky. I'm doing it literally in five. I'm sending an email in five minutes to thousand something people. Yeah. And but she gave me the permission because it's not higher. What if the email's a little bit off? Like it's to you guys. It's not the end of the world.

31:54 >> But what's interesting, but she says, "No, ask Annie." Because Annie sends better emails like you saw. So I said, "Okay, I go to Annie." I said, "Annie, please look at everyone coming that's either a VC, a founder, or CEO, and invite them to the brunch." Anniey's like, "I don't know how to do that. I only see 17 people coming that are VCs or CEOs." >> I don't know why, how you have all the data. How can you think there's only 17 founder COBC's?

32:18 And she I should have screenshotted enough. She was very direct that she's unable to do this. Like she was she's like, "Oh, you know what?" She said, "You're going to have to upload me a spreadsheet for me to be able to do this job." They're like, "You have access." So, it's reminded that agents, they get confused. The context is great, but so Annie had the great enough context to write a really good email with all the speakers and type, but it didn't it couldn't remember that it had access. It couldn't put the >> Okay. So then I go to 10k who told me not to use but 10k has access to even more in sales source than others and it's like no problem. It did do a deep analysis. It it went through 10,000 records here in in minutes. It pulled up all the founders VCs. Um came up with a list and then it said hold on I confused lights light field serum with light speed. Those are not 55 VCs. Those are SCRs. I'm removing them from the list.

33:05 Prepared the list. It said can I send you a sample? Send me a sample. It was pretty darn it was about it was almost as good. Okay. >> And um they said, "Can I send it all to 1500 people?" Then it researched and said, "Oh, we've never used this version of the API at rescend to do a mass send." I researched. It'll work fine. >> Um can I send it? I said, "Yeah, send it." >> I it proofed it. Great. It was going to be >> But >> but Annie used a prohibited email to send it. An email we're not allowed to use from the sending. It's in the memory.

33:35 It's in the rules. Broke a rule for multi-year now. If you hired a marketing manager, they might use an email. They will they might use >> they might send email a gung-ho SDR might email people they're not supposed to email. And I I I will add this later. And I said, Annie, how could you? This is in the core memory. How could you prohibit it? She's like, there's no excuse. I did not I forgot to read the memory. Annie said, I forgot to say it's worse. And Annie said, it's worse than that because this is irreversible. When you send an email from the wrong address, I cannot undo it. So she said my job is to call in the architect and the way replet works is replet runs on claude but when you have something complicated it actually brings in open AI 55 it calls it the architect for a proof it brings in a different model to look at something take another look at see and see if it's an idea called an architect he said there was no excuse this is exactly the thing I'm supposed to call the architect to and I didn't I'm terribly sorry there's nothing I can do about it what should I do um you want me to block anybody that emails you started to scramble for ideas So, it's pretty interesting one that Annie couldn't do it even though it was Amelia thought it was better for regular couldn't find it. Um, 10K did it, but because of the and it was partly my fault because of the pressure of time. I wanted to do this in an Uber coming here too fast. I didn't slow it down a little bit. But, um, when I started on this journey, this would have seemed like a huge issue to me. That's unacceptable.

35:01 How could >> First, it wouldn't have gotten done. Yeah, we got but when I started on this journey it would have bothered me because I'm like how can you send to an email that is prohibited but you got to step back a human could have made this mistake too right so this is why you're going to get better reminded me slow it down just a little bit because these agents are so productive and could have sent a thousand not just a thousand emails but a thousand different emails by the time we started this session there's no way I could have viewed and interacted and so they not only do they come to different outcomes you've got to spend more time with them >> Yep >> because they will inadvertently and goal seeking cut corners and it just any there's no I'm sorry there's no excuse.

35:38 It's clearly you're not allowed to use the female address and I use it anyway. Um so a story of the day >> but humans do the same thing. >> Our agent is forgiven. >> Yeah. >> Okay. I'm going to talk about Ameilia AI and also I think my Pokemon card is very cute but all right so Ameilia AI I launched last summer post last before this. Um I wanted to solve some of the inbound issues we were having on then our Squarespace website which was we had a lot of folks and engagement on our s just sanual.com but we weren't able to really fully capture a lot of it. I couldn't really do anything with it to book a meeting.

36:13 It was similar to a parking pass unfortunately like you had to talk tell our team in a contact form which sponsorship you wanted um how much like budget you had and who we should follow up with. And then then I would personally round robin it as as quickly as I could which sometimes was not as quickly as I wanted to be. And then one we had slightly more folks on the sales team and then I would round robin it to one of our AES and then our AES would take a slight delay and then they would follow up. So it' be like a kind of like two or three day process if you inbounded to our website. So then we added a millia which runs off qualified from Salesforce now and she just does this automatically. So we did spend a lot of time with my SDE qualified Cam training her like going through again the nuances of okay because it's for SAS we have two different customer buckets right there's all of you guys so all of our our kind of like self-s serve customers if you will that can just buy a ticket off the website but then there's all the sp potential sponsors and current sponsors that are also hitting the website coming to the website looking for information and so we built all that nuance context um back into qualified when we first launched.

37:18 the qualified I would say I put the most amount of training into it does also scrape um sashan.com in real time safer.com in real time so it's every day it's updating crawling all of our websites so that's why a millia AI is so good like some of it is context I've built into her so anytime I push a release to 10k um QB or Annie I also try and make sure I push it to qualified if it's not already there if it's something on the back end I try and make sure I put it into her memory so she has it too and that's why an AI is pretty good because she's got a she's maybe one of the biggest knowledge bases of all our agents.

37:51 >> I didn't know it's the most trained agent we have. >> It's the most trained agent. So that's why it's pretty good. Um but you can see that not only she has a pretty good repai. Uh she's booked a lot of meetings for us. She's had 2.2 2 slow down for a minute because it's so important. Yeah. >> Um >> the and every everyone should have an agent on your website automating inbound. I'll get to that in a minute.

38:15 There's no excuse. I'm shocked how many folks I've talked to that are running AI startups that just have a contact me form. Okay, I'm going to come back to that next. But by by adding AI, essentially adding AI to a old school calendarly or chili piper type link, we booked 614 good meetings out of that >> just for this annual. >> Yeah, 614 meetings pro and our average ticket size is about 85K. >> So that's pretty pretty good pretty high ROI to book 614. Now obviously they didn't all close or we'd have 60 million worth of sponsors here and so 10 million but that's a pretty efficient agent right pretty good right and almost zero complaints booking it this way right >> the only complaint was like sometimes >> like she so she'll write right now she'll round robin to her reps or myself but primarily David and so it's weighted heavily to book it'll book a lot most with most of the meetings with David because that's how I programmed her round robbing and then some of the meetings with me like if I happen to be free or if If it's kind of categories I'm good at of closing deal, which is like the gamma, the >> It weighs that.

39:20 >> It weighs that. Yeah. Yeah. It weighs like the It weighs our Salesforce data of like, okay, what does Ameilia close the most of type of sponsors? What does David close the most of? Which David has a big mix because he just has a lot of the a lot of the customer base. But mine's pretty more nuance. So, it kind of weights that and then it'll say, "Okay, for a while it was giving me all the agent companies because it was like, oh, David has a lot of like old school companies." So it was giving me all the agenda companies. I was like, "Give Ameilia AI, real Amelia, all the agenda companies." I was like, "No, no, no.

39:49 That's going to be everybody, dude. You got to stop weighing against that." And so it was pretty funny for a while. So we opened it up. Um, but she'll weigh against that and kind of book our meetings accordingly, which is pretty fun. And then it's funny our I think our I think I got a now that Slackbot is hooked up. It was like, "Hey, you should get back in the round cuz I'm off the round robin for the last two months." It was like, "Hey, do you want to put yourself back on the round robin and qualify?" I was like, "Maybe in two more weeks." But it was pretty fun.

40:16 >> Is there I want to ask you about this. Is there any is there anything worth seeing itself? Do you have this is when we're going to look at live or not? >> In qualify. Let me just Sorry, just to go back to Brown. Let me just understand that slide because this is new to me too guys. 614 meetings 2.25 25 million sessions on annual.com and sastaster.com together probably just >> just the website for this event had 225 million sessions >> I think so maybe it's s annual too or sastaster.com too >> I don't think so and four 42 so interactions is probably individual chats with the agent 42,000 chats the agent handle >> that's not that's pretty pretty effective um I want to let's have nearly show it if there's anything to say just but the point I want to make design. To me, this seems like the most captain and obvious thing to do. I'm shocked that 100% of people don't do it. Replace whatever you have, at least on the part of your websites that where you're trying to interact with prospects and leads and have an agent that qualifies them that's extremely well trained. It's so much better for the prospect and the customer to talk. Okay, do should I sponsor Saster? I'm whoever. I'm I'm Monaco. Is there anybody like me doing AI sales agents here? Yes, there's 10 other folks doing like this, right?

41:34 How here's my ICD. Here's what I want. Here are the agents. It should be able to answer all your questions honestly without drama. Give you all the data that you want that is fresh. It's not dated. It's not last year's perspectus or data. And then it can decide with some intelligence who to send that lead to in real time and they can schedule the meeting instantly with the agent. Like it's you saw the metrics. We could we have two and a half three people. We could never hit the number of metrics that you saw the 2 million sessions, 600 something appointments. we would need like three BDRs that would like quit every 3 months. So, I know this is like qualified is not the only vendor.

42:05 There's plenty of other vendors. Just buy somebody that does this. Train the agent and I can't imagine you won't have a lift versus a crappy chatbot. >> Yeah. Two things I'll just briefly show on qualified. So, this is her contact studio just for annual. So, if I just show you guys the top travel here, if you can see it, there's annual, there's London, which it was on, and then it has a slightly different one for annual in person, which is the demo counter over there. Because we just wanted to compact her memory a little bit for SAS annual.

42:29 We were like, she doesn't need to have all of Sasra.com in her brain for SAS annual. So, we we kind of toned her back for the inerson one so that you guys would have better interactions with Ameilia AI when you ask her about more like venue specific things versus things on saser.com. Um, but this is her brain. You can see she literally was updated this morning automatically. So she's pulling in sessions that we updated on the website. So that again if you ask her a question and I made a change on Annie with our staffing website qualified already knows this. So she's going to give you the right answer. If you say hey where did this session move to? What's the agenda for today? She can give you that right answer which is pretty cool. And then another cool thing it does is it does do a lot of email campaigns for us. So you guys probably got some of these email campaigns but you'll see here she's got two basic like I don't know if you call it top of funnel, middleunnel, whatever funnel.

43:13 Um, but she's got two um, uh, triggered campaigns here that the agent will do automatically based on what you do with her either on the website, whether you chat with her or not. U,, she has these two top performing uh, triggered campaigns as I call them. So, one is for sponsors. So, if you visit the sponsor page and you don't quite fill it out, but we know who you are either in Marquetto or Salesforce, the agent will send you a follow-up and say, "Hey, do you want to book a meeting with David?

43:37 here are some similar um some similar companies um and the that are current sponsors of yours. Here's the meeting router to book the meeting. It also will exclude anybody who's a current sponsor so that if they were just browsing the website cuz they're a current sponsor, maybe don't send them, hey, do you want to sponsor because they're already a sponsor. >> So, if you don't finish if you don't finish the flow and ask to talk to a human, but you start it, the agent will follow up. It'll find it'll find similar customers, look alikes. It will make sure you're not already a customer and then it will follow up on its own with the >> meeting. Yeah.

44:08 >> Yeah. No human can do that. >> No. And then similar thing for um tickets. Maybe I reverse them there in my mind. But similar thing for tickets. So if you hit the website and we know who you are and you don't buy a ticket, we know through their like website tracking do anonymization or you're already in our sales force it will email you a VIP code which a lot of you used to come to SAS.

44:30 So, it sold hundreds of thousands of dollars of tickets by either you if if you talked to Ameilia AI on the website, she gave you a code and then if you didn't use it, she followed up with you and if you're on the website and maybe you didn't interact with her, but it's been a couple days and now you're visiting the website again, she would just send you a VIP code to try and get you to come this. So, pretty good.

44:48 >> It it doesn't it's a niche thing. But one interesting thing it does that I think is hard for humans is it actually and again this wouldn't work with enterprise deals but it automates discounting. >> Correct. >> So there are most folks one way or another keen here with a discount. We would like not to do it but it's such a piece of human psychology we can't help it. If you buy a ticket tomorrow you're going to get by far the cheapest price ever. Do everybody a favor. Buy a ticket tomorrow it'll be half price. So that's an inherent discount. But no matter what the price is, we have so much data over the years says it's better to mark it up 20% and give people a 20% discount. We hate it as founders. We hate it. We have the data to prove you have no choice, right? So, but what we can do is rather than let humans forget a code or make up if you ask for it, the agent just gives you the right discount and it can change over time going up to that.

45:39 There's no reason to give people a discount today. The three people that will buy a ticket today, there's no reason, right? And it just be because you can't we all love to not have discounting but to truly guard rail it so it's automated it's a gift because it removes friction from the sales process and it's again I don't think humans are able to do this like almost all especially sales reps when they have no ability to close a deal they'll instantly discount again when they smell when a human smells a deal going away and they already gave you 20% off the next answer is will you buy my product for 25 are you listening you don't have a core integration I need. How about 30% off?

46:15 >> I think I'm going to buy our competitor. How about 34% off? And it doesn't move the needle. It creates a lot of issues. So, you can create guard rails here. I didn't get the throw out till I watched what um Amelia was doing. It's super efficient because it removes all the drama from discounting, right? It's almost like a a a pseudo CPQ system. Um but it does it in real time because you have rules, right?

46:37 >> Yep. Just a couple of things I want to touch because I don't want to get Adam up here so he can do his we're running out of time. Okay, I'll talk a little bit about agent force. I showed it a little bit with Adam Alano who was here yesterday who's the president of SMB. This is actually the this is actually the 1.0 Asian force builder. There's a better one now that we're on Fluency just upgraded me. But I think the goal here of Asian force what I've learned this week actually was I think a lot of folks are like okay it's Salesforce so it's going to be really hard to deploy an agent. like all the other things I had to put into Salesforce were kind of hard like how easy can it be to get my agent force up and running and there's a nuance quick answer I'll give you which is one how we use our agent force which is for ghosted leads is a very bounded use case right now we will use it for other things but right now King Boo's really good at agent force of reviving leads we never followed up with um some of them are still older leads of them are still newer leads where he's doing more of a re-engagement campaign of leads who maybe said no to me and David and he's going back out to them later and be like hey do you want to just come this year see what it's like for next year and again he's his primary goal is to follow up with folks that didn't give us a yes but I really like agent force because to that point now one it's gotten a lot better since we launched it back in October there's a 2.0 No builder. Um, it has the highest open rate. And why is that? Just quickly I'll touch on is because it has all of our Salesforce data and so it's got a lot of now also it has all of our qualified data since qual or since Salesforce block qualified and momentum which we were using as separate agents now it's got all of that in it and so it's got the most context automatically. Everything you guys just saw >> I kept asking a does this one integrated with Salesforce? Does this remember the agent first? The advantage is if you run on Salesforce and it'll be true when HubSpot finally launches agents, but it has every it has it all. So if you can do it in agent force, maybe do it at agent force, right? That you can because it's going if you're on if you're on Salesforce because it just has the maximum context, right?

48:34 >> It really is a path of least resistance and I know the team over there at the booth will help you get set up. >> Yeah. >> Today if you want them to. >> So it it really again it just has a lot of context. You don't have to move things from one place to other. if you were already using things like qualified or romantically kind of overnight and so it was a gift that they bought those agents we were using.

48:53 >> So are we out of time on this one? >> I I just want to hit artisan and monica death. >> Okay. Artisan um how do we use artisan? So Ava who you guys have seen in the booth too. We use her for slightly warm outbound and so a lot of campaigns that are not inqualified not agent force. We listen we have a big go to market votion here. I'm realizing we have a lot of different agents that do a lot of different nuance things. Um, we use Ava for what I call slightly warm outbound.

49:20 So, these are folks where we have past sponsors, past customers, past attendees, if you will. Um, between attendees and sponsors. And I would go into things like, okay, if their email is still valid, cool. We can load it into Ava. She can interact with them. If they're if they've moved on, she can help find like the right contact information, whatever. And she can also build look alikes, which it's pretty good at. and then she can help us. I use this primarily for us on the ticket side. So if you have something that's a lower ASP, you can use something like an artisan to do as I call it lukewarm outbound as a sales agent because she again she's also hooked up to the website so she knows everything we've done. I get really segmented with Ava.

50:01 So I give her super specific campaigns like here are alumni after annual 2024. Here's what was different about 2024 specifically versus now 2026 so that when you email these people, you have that specific context to use in your emails when you're doing their follow-ups. >> Yep. And one insight I had to synthesize all of this. I know we're all just about out of time. Yeah. So, but >> I was having a conversation with an AI CEO last night that their own products kind of at the bleeding edge and um and and also was head of marketing and synthesize everything. They're like, "Oh, this doesn't we haven't gotten this to work." And I went through one simple heristic with her with both of them that the head of marketing and the co >> for your a another way to think about this. You have your a leads. Your a leads are so hot a human's going to fall off it in 60 seconds. Someone comes in and says I want to buy a million dollar sponsorship or half million. Even your laziest rep is out out there at the mall or watching movies is going to respond to that email in 60 seconds. Like I got a million dollar budget I'd like to sign today. Don't put an AI on those accounts. Put it on the B leads. The bee leads are the ones that have some signal, some score, but they're not worth the a human's time in your organization. Every organization of size, humans will just not follow up with the B leads. The C or D leads may or may not have gold in them. It's a topic we'll run out of time on. But put our something like Artisan on the ones already in your database that you have contacts with that have some score, some signal, but a human, it's not worth a human's time. That's 500k of gold for us. Okay? This is not business or even a business, but we need the $500,000 so you you can eat.

51:34 >> Okay. With that $500,000 be bring your own sandwich. Okay. The artisan got us an extra $500,000 so then to have that's the be leads. And so >> I want let me Neil do one more break. But that if you think about it that way and you train it, you realize it's going to work because you have be leads in your database. >> Yeah. I haven't given artists in a lot of content. We can talk about them more later. I can maybe squeeze some I can help answer questions during the AMA later. Okay. Monica, this is our last one. our newest one. So, Monaco was also here. Um, references. I kind of Oh, I did a little copy paste error there. The Monaco is the one that you guys can see here. This one I put on cold outbound.

52:13 So, even though Artisan can do cold outbound, the reason I like Monaco for it is technically we're not their ICP because we have this giant a agent stack. Sam will be the first person to tell you this, but I like it because I basically gave it the best sponsors across all the years. and it said a millilia, I got it. It took all of our ICP data, all of our close one history. We did have to export it from Salesforce, which is a little ones, but and then Monaco just automatically found look alikes and booked a bunch of meetings, closed some pretty big logos, still for in a very short amount of time. So, this is the one that I think idles the least amount of time out of all of our agents because it is self-fulfilling. It'll just keep going out to different ICPS. It'll keep filling your fund. And so I like Monica for that reason because I really don't have to think about it which is quite nice.

52:59 >> Yeah. >> So that's why we added to our stack even though it's not I know we're not through ICP and it's not per but I really like it. >> Yeah. It's a We'll run out of time and we'll do another follow on deep dive. But the Monica one is cool because it fills its own funnel. >> Yeah. >> If you can get it to work if you can train it if they think you're the ICP.

53:15 It is the only tool we right now that constantly fills it with lookalikes. It's just doing a clever use of lookalikes. And as you dig into these things, just like be leads are good for SDRs, some of the art underneath is just getting good at look alikes. If your sponsor is is or is like field like and if your sponsor is Salesforce, why isn't HubSpot here? They should be here. But it's not that hard to figure out that since everybody but HubSpot is here, they should be here, right? And so maybe the team followed up with the wrong person to HubSpot. So Monaco will say, we'll go out and figure out, okay, who is the right person at HubSpot to talk to? And what it did a real instantly good job of then we got a break. that deal may or may not close, but it instantly identified a really good buyer and gets you a meeting. Okay, so this whole lookalike thing, it's actually not that a lot of this stuff if you you're going to realize in a year is not that complicated. Okay, it's just clever and the what you saw today when you looked at the backends of a lot you saw forms that look like other forms for web. It's the there's some clever rappers and guardrails and things, but you can do all of this yourself. You can do it all yourself. So, thanks everybody. Um, all right. Thanks everybody.

54:22 Costs can add up quickly when you start a business. So stop paying for five different services. Northwest Registered agent gives you complete business identity in one place with free tools and resources to actually launch your business for real. Get more at northwestregisteredagent.com/strophe. That's northwestregistered agent.com/sas tref e. Why knows you don't start a company to burn cycles on regulatory hoops or discounted cash flows, but you can't ignore it. That's how risk compounds.

54:52 What you can do is work with EY. They'll help you get it right early so you can stay in builder mode. Go to ey.com/teartups to learn more.