How I Use AI as a Product Manager

0:00 This is incredible. I've seen so many MPS slide decks and a lot of them look exactly like this. Hey Sachin, really excited today to talk to you about how to be a more productive PM with AI on the job. And And you've developed a framework, a kind of a model, and you're going to basically walk us through that map. Can you give us a little preview of what that map's all about? Yeah, totally. I've been thinking about this idea that AI can really supercharge the role of PMs. But the reality is it can do it across so many different things. Strategy, design, execution. And so, a lot of PMs are finding this kind of overwhelming right now. And so, the thought was, what if I put together a map that helps all PMs start to understand where AI could really help them in their role versus where they should probably wait for AI to advance even further. That's That's the idea of the map. That is so cool. Okay, so if someone's sitting there like, I just want to get better at AI in my day-to-day, we're going to tell them how to start, we're going to tell them what things maybe to avoid, and we're going to actually do some live demos later, too, so they can get really practical.

1:09 That's right. Yeah, so I think, you know, I want to dive into a bit of what are the areas where PMs should be using AI versus not, but I like to make this stuff real. And I'm excited to so show some real demos of this in action. Awesome. Let's get into it. So, in this In this framework you've developed, what are the four dimensions for what a PM does? You know when I think about the PM role, it really breaks down into just these four things.

1:36 PMs own the vision for their product, which is articulating how the world is going to be a better place if they're successful. They also own the strategy of the product. This idea of how your product's going to win in the market that you're in against all the competitors that are out there. They then own the design of the product. And I use the word design pretty broadly. I'm talking everything from customer insights to data insights to actually what the product's going to look like. And finally, execution. All that matters in the end is whether you get something shipped as a PM, and PMs have a lot of responsibility around that. So, those are the four dimensions that I think about. Now, those break down into about 10 specific deliverables that PMs are responsible for. On the vision side, it's actually putting together a product vision and a vision narrative. On the strategy side, it's this idea of putting together a concrete product strategy document.

2:30 Now, when we talk about design, we're talking about gathering customer insights. We're talking about gathering data insights based on all the data we can collect. And ultimately, we're translating that into a road map and a set of product specs to build. And these days, we're also turning that into a set of prototypes. Finally, when we talk about the deliverables of execution, we're talking about this idea of actually all the tasks you do to execute, project management work that we're responsible for PMs, coming up with product decisions, ultimately coming up with things like OKRs. So, that's really the framework I think about about what the PM role is and the deliverables that we're responsible for. Nice. That is That is super helpful. where And where can AI help us?

3:14 You know, this is the really interesting part that I think many people when they think of AI, it's all about time savings. And it certainly does that. It makes us faster, right? It can accomplish tasks much faster than I. For example, like listening to a bunch of customer interviews, gathering those insights, synthesizing them. It can do in minutes what takes me hours. So, AI definitely makes us faster. But actually, where I get excited is that AI can in fact make us smarter.

3:42 You know, later I'm going to show you in a second here an example of how I can give it a product strategy and actually have it critique it. And the output is actually fantastic. It's making me so much smarter at this. And so, I think AI has the benefit to not only save us time, to up but to also up level our work. And finally, I love that it actually gives me superpowers. You know, what's really cool about this is I'm now doing the work of a data analyst because I can have AI write SQL queries for me. I'm doing the work of a market researcher because I can have it conduct an end NPS analysis. And so, I'm getting these superpowers beyond my traditional role thanks to AI as well. So, at a high level, I understand this. If we go one level deeper and get a little bit more practical, where do you see AI's having a good application right now within your framework, and then where is it still leaving some room to be desired for improvement? Yeah, it's a great question and something I think a lot about.

4:41 And the best way I've thought about this is to go through each of the deliverables that PMs are responsible for and ask myself, where can I apply AI really well versus not? So, let me tell you, I've tried AI on every one of these deliverables. But what I'm highlighting here are the ones where you get incredible bang for your buck, really high-quality output, and where I think AI is going to give you the most leverage as a PM. And really, where it's worth PMs diving into right now. So, let me talk about what those are. The first one is product strategy. Now, I want to be careful how I talk about this. AI is not yet great at generating a compelling strategy, but it's fantastic at things like market research, doing all the research on the competitors in the landscape, researching your customer segments. It's incredible at research.

5:30 What it's also really good at is once you've digested that research and come up with a product strategy, it's also incredibly good at critiquing that product strategy. I've gotten such good insights based on that critique. And so, those are two areas where I use it for product strategy, research and critiquing, but I don't have it come up with the strategy itself. That is still on me, the human's job. Now, let's talk about design. There's a lot of value in AI for design as well. Customer insights is my favorite one. I am now using AI for basically analyzing all my survey data, basically conducting actual customer interviews, not only coming up with the script, but now we have AI-moderated interviews where it can actually go run the interview. And in fact, it can then synthesize the the transcripts from those interviews and find patterns across them. AI is incredibly good when it comes to helping us gather and synthesize customer insights. Same with data analysis. I use it now to write all SQL queries and to answer all my data curiosities. I remember before when I was at LinkedIn, I might have 10 data questions. I decide my to ask my data team three of them.

6:40 Because it's a lot of work for them to go and do all the data analysis, prepare reports, and whatnot. Now with AI, I get all 10 of my questions answered. And that's where we're seeing a lot of leverage on the data insight side. Also, prototyping has become huge for AI. Instead of working with my designer to come up with a static mock-up, I can now come up with full-fidelity interactive functional prototypes using AI tools. And so, that's a huge unlock because now I can visualize my own ideas, I can put those in front of customers, gather feedback, and I can do all of that before engineering even gets involved. Now, there's a ton of value as well when it comes to execution tasks.

7:21 Things like I now never have to worry about writing a meeting summary. AI does it for me. It even comes up with meeting agendas for me. And so, a lot of the sort of hygiene stuff I have to do around meetings, AI has completely taken care of. And so, that gives you sort of the landscape of where AI has is having incredible value. Now, I do want to be clear that there's parts of my job I still don't leverage AI for.

7:44 Vision is probably the key one. It is not great at coming up with a big, bold, opinionated vision of what's going to happen in the future. That's still on me. And that's okay, because I'm getting so much leverage on these other parts, I'm not going to try to force-fit AI where it's not yet useful. And so, that's an example where I I find AI lacking, and these are the other areas where I'm stead focusing all my time on using AI. From our last conversation, the way that I really like that you said it is that AI is going to be really good at giving you consensus-driven stuff, but it's not going to give you anything different. So, that's why it's lacking on that product strategy piece.

8:23 That's exactly right. It's one of these things when we think about both vision and strategy, AI is good at giving you the common pattern, the consensus-driven ideas. But as we all know, when it comes to strategy, it's the non-consensus opinionated bets that ultimately win in the market. it's Elon Musk deciding to build a performance Tesla vehicle when at the time people were just using these cars for saving money on gas. Very non-consensus at the time, but ultimately becoming the reason the company was successful. Same with vision. It can't really come up with this big, bold thing that's never existed before cuz again, it's going to that consensus, the patterns it's seen from the past. And so, those are the areas where we continue want to leverage our own human taste, our own human judgment, particularly on creating a strategy and creating a vision.

9:17 Now, like I said, AI is still incredibly good at all these other parts. And so, that's where we're going to spend most of our thinking time on how do we use AI? And I'd love to bring this to life with with some demos. Awesome. Let's get into it. Yeah, so let me give you set up the scenario here. I was recently hacking away at some like financial planning tools just for myself. And so, I was thinking, oh man, this is kind of a cool product idea. Maybe I should think about commercializing it. And so, I wrote up a little product strategy document. And you know, this was like a just an hour of my time trying to put together my thoughts around, well, if I built this product strategy financial planning tool, what would it look like?

9:58 And here's what I wrote down. I have sections for the problem I'm trying to solve, the target audience, the value propositions, the business model, growth strategy, how I think we're going to win against competitors. So, you know, it's a it's a pretty detailed product strategy that I was able to put together. Now, I want to show you how I then take this and give it to AI to critique. So, what I did here was create a project inside of Claude Code, sorry, inside of Claude.

10:28 and in that project, I set up some very specific system instructions. So, let me walk you through that. I say the goal of this project is to provide a product strategy critique to each provided product strategy. And this is the key part. Your task is to play devil's advocate and point out flaws or limitations in the product in the provided product strategy. Don't be nice. Point out in detail why the product strategy may not work, what questions remain unaddressed, and where the strategy falls short.

10:59 and so, this is key. Sometimes we complain about AI being too sycophantic, being too nice, being too people-pleasing. Well, turns out, you can just tell it not to be, and it actually does a good job of it. So, that's the first part of my prompt. Now, the second part is I'm actually training it to be good at product strategy. And so, what I did was I then, inside of the project knowledge, I took all of my course content on product strategy, literally copy and pasted here. And so, in my course, I have detailed guidelines on things like monetization strategy. And so, I can show you, for example, this is literally from my course copy-pasted. In this lesson, we're going to learn about a deep dive on on monetization strategy.

11:42 And so, I popped it all in here. And so, this is like, you know, hours and hours of content on product strategy. And then I tell it, use this to come up with a critique. And so, at that point, it's not only going to critique my product strategy, but leverage all my best practices on product strategy to do so. So, let's take a look at this. I've now had it basically pasted in that same product strategy I just showed you, and said, "Please critique it."

12:09 So, now it came up with a full critique of this product strategy. So, it's saying, "Hey, good news. At least you talked about all the six dimensions of product strategy that are important, things like target audience, problem to solve, and things like that. But here's where the fun stops. It's now telling me, "Look, there's some major problems in your product strategy. It turns out, your your target audience is too broad and unfocused. You define three distinct segments, dual-income professionals, FIRE enthusiasts, and financial advisors, as if they're all equally valuable starting points. This violates the core principle of focus. The strategy reads like you can't decide who to serve, so you're hedging your bets against multiple audiences."

12:51 That is spot-on. You know, like I came up with three, and you're right, the next thing I should do is go do the research and pick which one's the best one. and it's saying, "Hey, you have no genuine bull's-eye approach. You're not picking the narrowest viable audience." And this is something I teach in my course, and so that's why it's able to leverage those best practices. Now, if you just told Claude or ChatGPT, "Critique this product strategy," you're going to get something very generic, but because I've trained it on what great product strategy looks like, I'm getting incredible level of critiques for this. let me take another example here, and say let's say, you know, competitive advantage critique. This is the weakest section of the entire strategy.

13:38 And so, what I love about it, it's even being opinionated. What's good, what's not good. And And I'm saying here, it's saying, "Depth of model is not a moat. Any financial planner or competitor can build a sophisticated model. Complexity is easy, simplicity that works is hard." And you know, it's actually kind of right. part of what I was writing was, "I'm going to have the most complex model. It's going to be more sophisticated." And it's fine if it's sophisticated on the back end, but the user doesn't care. And so, it's calling that out. And you know, I think it's kind of funny when you get to the sort of the summary of it here. like I said, it it's trained to not be nice. It even says, "This strategy reads like an MBA case study answer, not a battle-tested plan from someone who deeply understands the problem space."

14:25 So, obviously, amazing best practice here, the way that you have trained this model to know what good strategy looks and talking more about like the the the brute-force way of just if you just ask Claude to critique your product strategy, it's really not going to give you much at all. Is there a kind of middle-of-the-road solution for people that want to have something high-value, but not too too time-intensive? Totally. And this turns out to be a best practice for all my prompts. What you want to do is give it an opinionated perspective on what great work looks like. So, let's take the product strategy example. Maybe you have your favorite product strategy book you've read, whether it's from Michael Porter or from any of these other like luminaries on product strategy, you can just basically tell Claude Code, "Hey, can you take Michael Porter's work or his best book and summarize it?" It will then go find an online summary and give you a detailed summary. And then you can say, "Hey, can you use that summary to critique my product strategy?"

15:24 And because you're actually giving it a specific set of opinions and best practices around some leader you respect in that field, it actually gives you great answers. So, for example, I do this when I'm generating a customer interview script. If you ask AI to say, "Hey, here's here's where I want to interview, can you create a script?" Very generic response. If I instead say, "It turns out there's this fantastic book called The Mom Test, which is all about running great customer interviews, give me a summary of the book." And then it will give you the detailed best practices of the book. And then I'll say, "Generate an interview script based on those best practices." And now it's as if the author of Mom Test, this guy Rob Fitzpatrick, basically wrote a customer interview guide for me. It's a fantastic approach you can use around all your product work. This is the superpower thing that you're talking about earlier. That is so cool. next up, we're deep diving into this this customer insights section. I think you're going to walk us through mainly like basically, what it actually looks like for AI to use customer in- insights. Like, let's say I have a bunch of NPS survey on a spreadsheet. How do I take that and make that into something that's actually insightful?

16:39 That's right. Let me show you that. So, you know, at my startup No Choy, we collect every week a bunch of NPS responses. So, in product, we ask users how likely are you to recommend this product or service on a scale of 0 to 10, and then we ask them, "Why did you give us that score?" And so, what we're looking at here is just a dump of the the the raw survey responses. We have date, email address, NPS score, and then the verbatims. I've also annotated this with some segmentation data. So, for each of these users, I'm seeing how many clients they have integrated, which is, you know, if they're using the Mac app, the web app, and the mobile app, that would be three clients. I'm also seeing whether they signed up with an email that was personal or corporate. And finally, I'm seeing whether they're on on a free or paid plan. So, that's just additional segmentation information I might use when I'm analyzing this.

17:30 Now, I then could use any of these tools. Claude, I happen to like Claude Code, but I had it then put together a full NPS analysis report. It generated this HTML report for me from just giving it these survey responses. So, I took the CSV V file, gave it to Claude Code, and then it was able to create this report. And so, when we look at this report, it's showing me the NPS score overall. It's showing me number of promoters, passives, and detractors. And then it's giving me this beautiful graph of NPS score on a monthly basis. So, I can see exactly how it's trending. I can mouse over, and it's interactive, so I can see the specific score. Looks like most recently, we were at NPS 50.

18:16 I then get a full breakdown of the NPS score on a monthly basis. It then goes and does segmentation analysis for me. So, how does NPS scare score compare amongst people who signed up with a personal email address versus people who signed up with a corporate email address. And so, that's useful. It even went and calculated statistical significance to tell me whether there's a significant difference here or not. Turns out there isn't, cuz these scores were so close.

18:44 It even did that same for subscription type. it's doing that same analysis with number of clients, so we can see as you integrate more clients, you get happier with the product. and so, that's super valuable. Now, I did three segments. I could have done 10 segments, and it would just take another minute or two. and that's the beauty of the amount of power we can throw into these tools. This is so cool, Sachin, and you you you've brought up this point that's being discussed more nowadays, which is PMs are the bottleneck now. So, what do you think about like, how does AI-assisted customer research change the pace of discovery?

19:23 Yeah, it's made it significantly faster if we know how to use these tools. This is a great example where I would have previously done this kind of analysis in a week, or my market research team would have done this kind of analysis in a week, cuz there's just a lot of data to crunch to get this. And so, now what's happening is now it takes no time to do the analysis, and that's been great for two reasons. First, I'm saving time so I can go do other work, but more importantly, at LinkedIn, we used to run NPS's quarterly because it was took a week to go process the results. Now, with no choice, we run it continuously.

19:59 Every week, I'm getting this report, so I'm getting new insights every week. And so, that's an example of how I can increase the volume of insights I'm actually leveraging. another great example of this is with AI-moderated customer interviews. I can go run 10 interviews myself on a one-on-one basis, but I can run 100 if AI is helping me out conduct those interviews. And that's getting me way more feedback. Now, of course, I still want to be listening to individual interviews, but again, I can do 10, but now I'm scaling that effort and getting way more feedback than I would have ever gotten before. And so, that ends up being the key part here on how we end up scaling our feedback in this way.

20:42 This is so so cool. the next one that we want to dive in with you is the data analysis piece. Like, what what is what is AI-assisted data analysis look like in practice for a PM who isn't a SQL expert? Like, just realistically, what what's possible now that wasn't possible before? This is one of my favorite use cases, and I built a scale in Claude Cole code called answer data curiosity. And the whole idea is that I have so many data questions as a PM. They come up every single day. And what I can now do is just give it a natural language question. And so, here's the question I gave it. I said, "How many note views have there been each month for the last 3 years?" And I just give it that in English. It then goes inspects my database, understands the database schema, and writes the appropriate SQL query. And so, you can see here as an example, the query came up with. It shows you this, but if you want to auto edit or verify it, you can to build some conviction and trust in it.

21:46 but otherwise, you can just trust the data. And now you get a visualization of results. And so, not only did it run that SQL against my my SQL database, it then created a pretty chart that again is interactive, so I can go see these results. it also gave me a table that I can export if I wanted to take this to Excel or whatnot. and this is really powerful because now I used to have 10 data questions, pick three of them to go ask my data team, and then frankly, just let the other seven fall on the floor. I no longer have to do that. I can just get those answers incredibly quickly with the power of AI. Now, I've shown it here in an ad hoc basis. This is like, you know, something I might ask a question to, but you can equally do this as a weekly dashboard. So, here's an example of we have a bunch of customization features in Notejoy, and I like to see are people even using this stuff? And so, I can click into like desktop themes, and I can see which theme our default theme is the most used, the dark theme is the next most used. and I'm getting these rich graphs, data tables for each of the customization options.

22:55 And again, I can just schedule this report to run every week, and it required no work. You know, I set up this the dashboard the first time around, but at this point, it's just running and giving me this data automatically, saving me a ton of time that I used to waste on data analysis. I now at spend spend it on just the insights. Is there another like step down of more detail for a customer insight kind of value add here?

23:22 Yeah, you know, these days, one of the things I love doing is asking AI to then generate a summary executive-level presentation for me for that analysis. And what's so cool here is I'm using a tool called Gamma, which is a AI-powered presentation software. And keep in mind, this entire presentation I'm about to show you is 100% generated by AI. So, I'm getting the executive summary, the NPS score of 41, the total responses, promoters, and detractors. And I'm getting beautiful art. I didn't pick any of these visualizations.

23:58 Gamma chose it automatically. and AI chose it. I'm then getting a breakdown of promoters, passive, and detractors, and a beautiful visualization of it. Keep going here, I can see my NPS trend that it's now graphed for me on this beautiful graph. it then goes into segment analyses, like number of NPS score based on personal email users, does the same for thing for subscription type. And what's so cool here is it's then summarizing those customer themes.

24:29 and what's powerful here is it basically got the same data as that HTML report, but then AI decided, "How do we make this executive level? How do we make it beautiful? How do we layout each of these slides?" I gave it no guidance. Like, it's not like I'm giving it a detailed template here. This is the power where we're getting to. Now, you think about this, I used to again ask my marketing team to go run these analyses, go create the Excel present the PowerPoint presentation, and now I'm able to do this all myself really quickly, really giving me these superpowers as a PM. Yeah, this one to me is the most one of the more like relevant superpowers for me. Like, I I I think this is incredible. I've seen so many NPS slide decks, and a lot of them look exactly like this. So, it's just really really impressive. and maybe even like for a lot of different roles, too. Like, data analysts, people presenting to executives outside of the PM world, an amazing amazing value add here.

25:29 Sachin, where does this break down? Like, what are the failure modes a PM needs to watch for when they're leaning on AI to interpret data? Yeah, you know, setting this up initially can result in some SQL queries that are inaccurate. And it's not that it's getting it wrong, it's that none of us have perfect database tables, right? We have these issues where we didn't name them properly. We didn't name them descriptively. We have this old version of a table, and now we use a new version of a table. Just the realities of production. And so, AI doesn't necessarily know that. And so, you might get a inaccurately written query, and when you look at it, you're like, "Yeah, I get why AI got it wrong." But of course, we want it to get it right. And so, there's some techniques here to improve this. One of my favorite ones, frankly, is just have your data team document natural language questions and the SQL pairs, and take the last 10, 20, 30 of those that they've done, and dump that into your project knowledge, and then AI is incredibly good at using examples when it's trying to decide, "How should we actually format this?"

26:35 And so, that makes your queries so much more powerful and more accurate. >> That's another good concrete takeaway is just like earlier for the middle-of-the-road solution of using the book. If you give it an exemplar or a model to work towards, it's going to have a much less general opinion and a much more kind of elevated opinion. That's so cool. That's so that's so true. I talk about this a lot in my course, this idea of shaping the output.

27:01 That like, we want to show it great work. We want to show it great examples. We want to show what greatness looks like, and that helps us shape the output from the generic meh that AI often gets to something that you feel really proud of as the work coming out of AI. Such a valuable walk-through, Sachin. Thank you so much. I want to wrap with one question here. If let's say there's a PM who's watching this, and they want to walk away and do something differently right now, and they just want to start small. what do you recommend they they they start with?

27:31 I think take any one of these scenarios and open up your whatever tool you have, ChatGPT, Claude. You can do this in any of those tools, and try it on your actual product work, right? If you're working on a product strategy, run a critique. See what you get. See if you can improve the output with the techniques we talked about by showing it greatness. You know, maybe you have a data question. hook it up to your MySQL database using a MCP server, and see if you can actually ask your question in plain text English and start to get some data. You know, learning at the high level of what AI is possible to do is definitely step one, but the next step is to get in those tools. Start actually trying to do this for your own product work because I I firmly believe that it's going to unlock a lot of productivity in your own work. Make sure you start to bring AI to life as a PM. And good luck to everybody out there in your interviews.

28:28 >>