Claude Code for Outbound: How We Run Outbound Experiments

0:00 Here I want to share a sneak peek into what the Experiment Outbound uh dashboard looks like. So, um for our clients, they get access to uh the stash where they can see all of the experimentation that's happening, as well as all of the contexts and the guardrails and the tests and the experiments that we run um and the results, as well. So, let me start with context cuz I think this really kind of sets a nice boundary of um what's possible and what's not. So, if you look here on the on the left side, um just going to zoom in a little bit.

0:41 It has all of the context files that are stored in a repository. So, if you go into something like um company, we're looking at uh this is a little bit meta, but we're looking at um the company context for Experiment Outbound or EO for short. And you can also see how that file changed um its context over time. So, you can see different versions. Uh we're just pulling this straight from uh the repo, right? And then what's actually kind of interesting is when you look at it on the graph view, you can actually see how all of the different um context files are interconnected with one another. And this is how um we we have an algorithm that basically um allows us to uh move from one node to another depending on their relationship, and then be able to pull adjacent um contexts so that we have a well-rounded um summary of all the contexts that would be relevant. So, for example, if we look at a play, let's say that we have a play called growth leader experimenting uh on outbound, right? So, a head of growth who needs experimentation rigor on outbound, whether it's black box it can't measure or produce results uh they can't explain, The job is bringing structure testing to the channel, right? And then it'll have references of persona head of growth makes a lot of sense. What's their job to be done? What are potentially some signals that trigger this play? What are some insights that we can share or our our views, our POVs, right? What are some proofs and what might be their objections and what are their alternatives if they don't go with us, right? So for example, um ROI on certainty, we can uh click into this and actually see what what this says.

2:32 And this gets pulled into um the LLM generation. Uh so whether it's email or LinkedIn messages, we can pull this context in uh to to generate. And how does that happen? So I've shown you what the context uh files and how they're interconnected, what that looks like. So if you actually click into any of these, you can see for example compose full catalog. You can see what kind of um context were pulled in. So AI SDR tools, uh clay agency partners, that's probably the alternatives, right? These are alternatives. Um so this is all the context I get pulled in.

3:11 But uh just going back to that other one cuz I want to show you what that looks like. Um for each prospect, right? If it's um whether it's LinkedIn or email, we'll we'll we'll pull their LinkedIn profile, we'll match their persona. Um we will score their persona and if it's relevant, then we'll go and get the company's LinkedIn profile, we'll try to find their emails. Um we'll get their company website to get uh a richer context on the company.

3:40 We'll classify if they're B2B or B2C, right? For us, we sell into B2B so that's important to us. We'll get the person's LinkedIn posts. We also store all of their information um in our database so that we know when the last time we reached uh out to them was, what type of angle was used, what were the actual messages, what were the enrichments, do we have to rerun them, were they fresh, right? Um And then we look at the company signal detection, we look up news, we look at their hiring, fundraising, partnerships, key executive hires, things like that.

4:17 And ultimately, we write we generate an email and then we're able to enroll it into a campaign, right? Sorry, email or or LinkedIn message. So this is what it looks like. We also have a lab which is sort of your pre-flight test. So before we actually launch a campaign, we will run different permutations of context and enrichments and run these labs to see what the resulting output looks like.

4:48 So let me So you can see um the emails that were generated using some of the context and some of the enrichments so we can iterate based on these results. Really quickly, this just pulls in the context, this pulls in sort of the same workflow graph that that I showed you, right? Runs through it for given prospects that are that are real people and then and then generates these emails and see what it sees what it looks like, right? So we can say generate, you know, three email sequence or generate um uh uh an a LinkedIn email and maybe a connection request if you want to send something that's not blank, right? And so that's what it looks like. And then when once you're done with all these labs and you feel good about the the quality of the output, you launch a campaign and then we're able to pull those in as well. And then we overlay the different um AB ABC testing that we can run. Base So basically we can pull in different context files. We can run different enrichments for a given cohort in the same campaign. So, for example, even if you're targeting legacy SaaS CTOs, we can run different permutations of context and enrichments to maybe do signal-driven or maybe a second-degree connection, right?

6:11 Or and then compare that to the baseline and see, you know, how many leads were enrolled in each. This was random assignment, so the number's not exactly split, but you can see the email sent, the replied and positive. And obviously, if you get to a significant enough volume, then you can drive statistical significance to a certain extent, right? Um, uh, if you if you're able to get, uh, the the sample size to be large enough, right?

6:40 And so, each campaign we tend to run a lot of AB testing. We're always iterating on the context and the enrichments to see what provides an uplift in the performance. So, that's what it looks like. Um, so, hopefully that was helpful in terms of grounding your understanding of what experiment outbound looks like. Thanks for watching.