transcribe

How to Build Effective Product Loops in Claude Code | Tyler Folkman | Chief AI Officer, JobNimbus

Aakash Gupta · 1h 8m · transcribed 17d ago
More from Aakash Gupta Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Section Insights

# 0:00

The Future of Product Management and Engineering

What does the shift from traditional roles to product building mean for PMs and engineers?

The conversation highlights the evolving nature of roles in tech, suggesting that in the future, titles like engineer or designer may become less significant as the focus shifts towards product building. The speaker emphasizes the importance of upskilling for PMs, designers, and engineers to adapt to this change.

  • The distinction between roles in tech may diminish over time.
  • Upskilling into product building is essential for PMs and engineers.
  • AI is making product management more people-centric.
# 13:45

Implementing AI in Product Development

How can AI be effectively integrated into product development processes?

The speaker describes a process where AI-generated outputs are reviewed and refined by humans, emphasizing the importance of maintaining a feedback loop. Instead of reverting to previous versions, the focus is on tweaking and improving current outputs.

  • AI can enhance product development by creating a feedback loop.
  • Tweaking current outputs is often more effective than reverting to past versions.
  • Maintaining a human review process is crucial for quality assurance.
# 27:30

Creating Efficient Documentation with AI

What are the best practices for using AI in documentation?

The speaker suggests that documentation should be concise and visually appealing, ideally limited to one to three pages. They argue that excessive detail can lead to inefficiency and that AI should be used to streamline the documentation process without overloading the reader.

  • Documentation should be concise and visually engaging.
  • AI can help streamline the documentation process.
  • Avoid overloading readers with excessive detail.
# 41:15

Prototyping with AI

How does AI assist in the prototyping process?

The speaker explains how AI can generate prototypes based on simple prompts, allowing for real-time adjustments and variations. This approach reflects their internal processes at Job Nimbus, where multiple prototype variants are created to cater to different needs.

  • AI can generate prototypes dynamically based on prompts.
  • Creating multiple variants of prototypes is beneficial.
  • Real-time adjustments enhance the prototyping process.
# 55:01

The Role of PMs in Agile Environments

What is the significance of PMs adapting to agile methodologies?

The speaker discusses the importance of PMs being able to make strategic decisions quickly and effectively, emphasizing that the ability to solve problems end-to-end increases their leverage within the company. This agility reduces bottlenecks and accelerates the speed to value.

  • PMs need to adapt to agile methodologies to enhance efficiency.
  • End-to-end problem-solving increases leverage and speed to value.
  • Reducing internal bottlenecks is crucial for agile success.

Transcript

0:00 We talk a lot about vibe coding, but we don't talk as much about vibe PMing. >> Everybody keeps saying loops are the new prompts. What does that actually mean? >> I honestly believe it's a very powerful skill for engineers to develop. In 5 years, I'm not sure we'll refer to ourselves by functions. There's nothing magic about being an engineer or a designer and that being in our title. Tyler Folkman, the chief AI officer and head of product at Job Nimbus, which just raised Utah's largest series B round ever at $33 million.

0:30 >> I know we talk a lot about AI versus people and replacing them. But I am a huge believer that with AI, it's more people ccentric than it's ever been. >> That's kind of like career reputation suicide, I feel like, for a PM at this point, right? Like do not ship anything. So should every PM designer and engineer be aspiring to upskill themselves into a product builder? >> In my opinion, yes. Unless Before we get into today's show, please take a second to check that you're subscribed on YouTube and following on Apple and Spotify podcasts. If you want access to all of my favorite AI tools, I've gotten them to give you an entire year of their paid plans. Check out bundle.acosg.com for an entire year of bolt.new, air table, speechify, descript, magic patterns, linear, dovetail, arise, and mobin. And now into today's show.

1:30 Everybody keeps saying loops are the new prompts. What does that actually mean for product managers? How do you actually build a loop that's effective? What are the steps to improving that loop? Today we've brought in somebody who has the data scientist and engineering background. He is the chief AI officer. He leads up product and engineering at one of Utah's most exciting startups, Job Nimbus. Tyler is going to walk you through not only how to set this up, but how to improve these loops, how to think about these loops.

2:02 Nobody is teaching you how to do loops specifically for product work. If you are an engineer, if you are a designer, if you are a founder, you're going to learn so much about how you can potentially vibe PM. If you are a product manager, you're going to learn how to make loops and use loops. So, I hope you enjoy this episode as much as I did. Tyler, thanks for being here. >> Yeah, thanks for having me. It's great to be here.

2:22 >> So, let's get started. How do PMs and product people use loops? >> Yeah, it's something we talk a lot about. I feel like AI more than most things has a lot of hype and not a lot of action. So, I'd love to just show you kind of what we do and try to be really safe that I don't show anything I'm not supposed to. So, what I've got up here showing is the actual system that I use every day to do loop engineering, context engineering, whatever you want to call it. And we'll talk a little bit about the differences, but the system I'm in for people that want to follow along is called Herder, which lets you essentially orchestrate agents and was built with a ability to essentially run itself. So, I've built this thing called orchestrate, which I can then tell it to spin up another agent to research contractor needs in 2028. And it actually knows in Herder, which you'll see it do in a second here, how to spin up an agent, how to open a new pane on the side here, which I can move between. You see, I'm working on some different leadership stuff, hiring, and these green dots actually tell me what's done. like this one's ready for me to look at. This yellow is it's moving and then this one I've already looked at. So to get to your question, this is really the pane of glass that I use to manage multiple loops, which is kind of starting with getting an AI helping you.

3:47 >> Why would I use her instead of VS Code or Cursor or Claude Code in the terminal? >> Yeah, it's a great question. I think and I was actually pretty skeptical maybe a year ago about terminal based coding agents but they've gotten so powerful and the systems around them like herder have also been made in such a way that's powerful that it feels super native to work in the terminal where it can access what it needs to access. I get that this isn't as maybe sexy as VS Code, but for me I can actually just have like a terminal open that's an editor if I'm doing some coding. I can also in that terminal spin up an agent in Herder.

4:25 I can even like split PES and do all sorts of work. So, it's easier for me to have control over the agent loop cycle as opposed to having to rely on VS code to release things at the pace that the industry is moving. And that's one of the things that's really fun about the community tools because most of them are using AI to help build the tool. They move very fast and tend to evolve very quickly as AI improves.

4:48 >> And then we're using this term agent. I feel like agent is like used on so many things. What makes it an agent here versus just another cloud code instance? >> Super solid question. For me, the difference is an agent is doing something on its own and making decisions versus you consistently prompting it. So like GPT chat GPT came out and most people are interacting with it at a level of question answer, question answer. I've heard people refer to this as almost like smarter Google search. Agentic is more you're giving it a task with a way to verify itself and asking it to execute on that by itself as much as possible. So you're taking yourself out of the loop more than you are with just like a prompt. That way you can actually scale yourself where I think prompting kind of got hard is you're kind of constantly in the loop and you're not really feeling the gains from what you get from Agentic.

5:41 >> Amazing. So what do we do next with this research contractor agent? >> Yeah. So it's off. It's doing its thing. It's actually going to report back when it finishes. I believe it made it in, let's see, right here, customer research. So, if I click this, it's actually doing research and drafting some thoughts for me, which isn't really the point yet of this. And to kind of show what I was thinking outside of her with loops, one of the things I was super interested in is like, can I use loops or agents to help me prepare for this podcast, right? Like, that's the most meta thing I could possibly do. So I pointed at like a bunch of stuff we do at the company and asked it to build various demos that we could kind of run through to show different things. And so what I'm doing actually is kicking off a predefined skill or agent that I've built that knows the things I want to talk about. It will then kind of walk us through some different examples. So one of the things I asked it to do which makes it a little bit more of an agent loop. And you'll see that some of these things I'm not actually passing. Is it's doing a pre-flight check before I record am I showing anything sensitive?

6:44 It's saying yeah, you might be showing some sensitive stuff. I think we're we're okay. Is the repo in a good state? I'm not actually in a sandbox, which I feel okay about actually. It's got some fallbacks that are a little bit overkill. And then some manual things I'm supposed to do like did we connect Riverside and all that other stuff. So I did this on purpose to kind of show a few things. one AI when you ask it to can be a great partner to help you do better. Like I wouldn't have thought about half these things and it's kind of forcing me to check them. And I did run this before and I kind of decided I didn't care about like a sandbox. We're okay for this podcast. But every time now if I ever wanted to use this in the future, that loop is starting off with conditions that I've told it are required for success. And now it's asking me, is this a rehearsal or is this actually a live run? And and we're live. So, let's say it's live. Do we want to start at segment one or start where I finish my rehearsal? Let's start from the beginning. And you can see, in my opinion, how this goes beyond kind of prompting and building an actual system to help me prepare or do work in a way that I feel good about and isn't just like AI slop.

7:52 >> And you called it a loop specifically. What makes it a loop? >> Yeah, which we'll see in the end here. But one of the things it'll do assuming it all runs as planned is towards the end the way you close that loop is you feed back information into the AI to make this whole system better. So, and I actually have a little graph we can run through in a minute. It'll pull it up. But looping kind of assumes that you learn and it builds like this flywheel. So any good loop has this end that says, "Hey AI, let's review what I did. Let's look at the log of all this back and forth and how do we take that and improve it.

8:29 So that's really where the loop comes from. If you hear people talk about looping, it's this idea of self-improving agents as it pros to a static skill that the skill could just sit here and be the same forever. Or we could take this entire experience and feed it back into the skill and have the AI improve it. And I guess what I get nervous about there is is it going to save all the intermediate versions? Am I going to be able to easily revert back if there was some version of it that I liked before?

8:57 >> Yeah, for me, Git is the main way I handle that. So very normal for engineers, a little less normal for product people, though I think that's changing. Git lets you version control these things. So actually one of the questions that popped up was saying, "Hey, your git is unclean. What do you want to do?" Oh, here it goes. It pulled it up. But I think that that is the right way to think about it which is you can go back in time like git. Maybe someone will make something more agentic and there are companies working on this than git. But with git I can kind of commit and move back in time. And those beeps you're hearing is actually it telling me I'm done with something. So if you hear those through the podcast that's how I manage it. And one of the things I'd love to talk about is the importance of having your system so that you can roll back and also adapt to new models because what we're seeing is these loops that you might built or these skills or agents or whatever you want to call them, different models perform differently and you might go to a better model like Opus 5 or Fable 5 and find that your skill gets worse. And that's kind of a paradox to work through because sometimes you might have forced things that don't need to be forced anymore. Yes, I felt that way about a lot of my course skills. Here's a quick word from our sponsors. I used to think I had a retention problem. Turns out I had a messaging problem. I was sending the same onboarding emails to every new user, whether they activated on day one or never logged in again. I had no idea who was slipping or why.

10:18 Customer.io changed that. Every message I send is now based on what users actually do in the product. Someone hits a key activation moment, they get nudged to the next one. Someone goes quiet, they get a different path entirely. Their AI agent makes it fast. I describe the campaign I want and it builds the full journey form. Triggers, timing, copy, even branching logic. And when I want to know how something is performing, I just ask the agent directly and it tells me what to do next. They also have an MCP server which means AI tools like Claude can see directly what's happening in your customer.io workspace. Your segments, your customer data, your attribution, all of it. So instead of explaining your business context every time you need help, Claude already knows it. Notion used customer.io IO to personalize their onboarding and hit nearly 50% open rate, improved conversion by 6 to 7% with localized campaigns, and pushed open rates up another 20% through AB testing. The idea is simple.

11:12 Customer.io helps you deliver more impact from every message you send. If you're a PMR founder and your onboarding is still oneizefits-all, try Customer.io at customer.io. If you've worked at any company bigger than 30 people, you know this one. The CEO sets strategy. By the time it reaches the people actually doing the work, it goes through three or four layers of translation. Half of it gets lost and nobody finds out until the quarter is over. That's the problem Ariso is built for. It's an AI operating partner for every manager and team.

11:39 Connects to where work actually happens. The meetings, the messages, the docs, and it turns all that fragmented activity into a clear picture of execution. Managers get real coaching grounded in their team's actual work, not generic advice. Teams stay aligned with strategy as it changes, not as it was last quarter. And leaders see where execution is drifting in weeks, not in the postmortem. One shared memory for the whole org. Everyone finally working from the same picture. If you lead a team, check out ariso.ai/ashosh.

12:08 That's a r iO.ai/ a kh. So what are we looking at here? >> Yeah. So when I was preparing for this, I started asking myself some of these questions. What is a loop? How can we visualize it? and working with AI. Obviously, this is what I kind of think of is like part one is you kind of fetch your own stuff as much as possible. So, you can see in the thing I gave before research X. I didn't actually go do the research for it and give it to it. I said go figure this out. It does some amount of work. Then there's usually a gate if you can have it, which is how do you validate that that work was correct?

12:40 This is almost always the most important part. As much as possible, being able to make this deterministic is critical. there is this idea of like LLM is judge where it can decide and sometimes that's necessary but we're seeing like with this idea of mirror coding that if you have a lot of really solid tests around a piece of code AI can actually replicate that code pretty agentically and the learning from that isn't that like let's replicate all this code that's already been written it's AI is really good with clear boundaries and clear definitions of done and success that it can work through so that's kind of like the part three and then if you look at how it shapes it kind of writes out that thing and then allows it to learn and iterate and improve upon itself, hence making it a loop. If you miss the learning piece, then you're mostly just running what I would call like a skill, which isn't a bad thing. A skill goes to a loop when you feed back the learning so that it continues to improve.

13:34 >> And so is there a step in here like before it creates the next version I guess between create the artifact and fetch its own inputs that it is pushing to GitHub or something? >> Yeah, exactly. So generally what I would do is if it passes the gate then you would push a PR to GitHub for a human to review which if you're just yourself building your own skill that would be you. You'd kind of look at the decision-m it made how it wants to update things and then merge that into the the canonical main branch such that you are always using that going forward.

14:05 But to your point before have a way back if something needs to go back because you don't like it anymore. >> And are you often needing to revert back? How often does that happen? You know, I tend to with AI not revert back as much anymore because I can often what I think of as revert forward a little easier, which is to say if I don't like something, I almost tweak it forward as opposed to go back because often it's less of it's unanimously better in the past. It might be more of there's this thing that I used to like that I'm not quite getting enough of that I want to tweak the new version with, so I just fix forward.

14:39 >> Okay, awesome. So, conceptually, I get it. How does this work in practice? Yeah, if we kind of go next here, which the AI will kind of walk us through some of the things we do, what we do at job Nimbus to kind of convert this into practice is we try to think of how we develop products and I think of products pretty holistically. That could be a product for our customer or if we're doing something internally like using AI to develop our products or write code.

15:05 How do we think of it as a loop? And if we kind of go back to this this loop of fetching its input, doing work, the gate, writing an artifact, this all works pretty efficiently if it's code, but what do you do when you need to get like the gate is a human or a customer that doesn't work in your company? and right now it's kind of working on showing this, but how do you really get that into the loop so that it's fast?

15:29 Because I think a lot of what companies are finding out is you might have a billion ideas, right? can generate ideas faster than ever, prototype them, even build them in some cases, but is that something that's valuable and viable for the business? You know, if you go to kind of Marty Kagan's four risks, that's still really hard to figure out. And a lot of times it's because you're missing a clear gate. If we go back to this picture, there's not a clear defined gate that you can quickly iterate on because it's your customer. You can't just constantly lock them in a room and say, "Hey, do you like this? Do you not like this?" And even if you could, you have a bunch of customers. So how do you manage that is a challenge and one of the ways we're trying to adapt to that is actually one being way more customer centric than we've ever been before like using the savings that AI gives us on other things and spending it with the customer and two trying to mind anything they've already given us with the the most AI or efficiency and tech we can.

16:23 And so like we have a bunch of recordings and calls and interviews and that might have been a whole team that had to work through that. And now with you know cloud code you can point it out all those transcripts pull stuff out and really more efficiently build that gate to at least filter out some of the bad ideas. And that's sort of what my AI is working on right now. It's actually using our product skill which the first step is to build a bunch of prototypes.

16:45 So it's actually going through and building just some random prototypes around payment follow-up. Not really anything we're particularly building right now, but as an example, it's going to build me three. And I just kind of wanted to showcase if you think about that loop is doing something at the first step. While I'm not even doing anything other than talking to you, my AI is doing actually a lot of work to build ideas, the hard part of the loop is going to be validating that it passed the gate.

17:10 >> Wow. So, where did this prototype come from? I missed that part. How did >> Yeah, the AI just was it's still working on the it's going to build three of them, variant C and B. And just when I said keep going, it started working on it. So this was built while we were talking. >> And what did we do to kick this off? Like why did it know to start a prototype? Can we look inside that product skill?

17:29 >> Yeah, I go and that's not in here cuz I kind of demoed it and I can go find it. But if and in fact this is a good example of her. So if I wanted to spin up my own kind of pane here, I just say new and then I can go into our product area. wait, not that one. Really the best one to show would be like forge. And it might be a little bit hard to walk through the exact skills, but if we look at all the things in here, you will see this.claude.

17:58 And this claude is actually got a bunch of skills and commands that let us do a lot of this stuff. So we essentially built our own prototyping system. And by we, I mean the UX team here is awesome at kind of being at the cutting edge. And so we've got a bunch of skills that let us do really cool things like create an iOS prototype, an Android one, port components, pull requests, run content. And there's more outside of this. This is just kind of an example, but really what these skills do is take a lot of our beliefs in how to build product and design and convert them into something a little bit more deterministic. It's not code, but we do try to kind of build those into canonical shapes. And then we can say to the AI, use this skill to generate 10 prototypes to solve this problem for our customers. And then we can start iterating on the feedback loop. Got it.

18:49 And I think this is an important point. You're having your domain owners create skills for their specific area. And so the UX team is creating the prototyping skills and I guess the product team creates the product skills. >> Yeah, exactly. And you can see it kicked out two more. So, like we try to get it to spin out variations of an idea. And these ones are pretty basic for speed, but you can see like payment followup. Then there's like a crew schedule one, there's a different crew board idea, here's another one about what you might need to know today. Because the idea with AI is how do we actually test a lot of ideas very quickly because AI is really good at that without building.

19:30 >> Yeah. >> As opposed to building the thing we think is right and then just doing the normal product cycle. like you're kind of if you're mostly using AI to build in production that's valuable but man can you use AI to build prototypes way faster like instantly anyone can build a prototype and go talk to a customer about it. >> So now that I have these prototypes what do I do next? >> Yeah. So for us the next step would be and unfortunately we don't have this skillified quite yet in a great way but we've injected all of our customer research calls transcripts into our data warehouse. So you'll actually have an opportunity to synthetically kind of have a customer inspect the prototype through AI.

20:10 >> So it will kind of try to think like our customer. We get, you know, for anyone that might be a customer that that's not the highest bar, but it is a good bar to catch some maybe lowhanging failures that we missed, right? And the reason we like that is it lets us very quickly move through this loop, right? You need a gate. And so we can quickly go from maybe a hundred ideas down to the five best ones. And then we start getting with our customers, setting up calls, talking to them, getting into the office with them or on the field so that we go from bunch of ideas to some ideas to a few ideas to hopefully the one that we then finally go build and then do AB testing with in production. But the idea is you lower the amount of effort until you get more and more validity of your idea from the customer. And so who would be running these customer outreach? PMs, engineers, designers, anybody can do that or how does that work exactly? Because sometimes like the UX research function in particular has like a very strong point of view about who can run those and how they should be run in order to get accurate insights.

21:13 One of the things we're working on, and this is newer muscle for us because traditionally it has been more of a UX product run thing like most companies really trying to maintain the standards and do things right and that's really important, but we're trying to build the muscle of everybody doing this weekly if possible. In fact, we try to aim for like twice a week. We're not there. We're still working towards that. But if you think about with AI where the new bottlenecks come, this is going to be one of them. Like getting in front of customers and talking to them. And if you try to gate that too much, it's not going to scale with the speed of AI. So, we're trying to figure out how to really invest in our people such that anyone can start to talk to the customers and know some of the like obvious traps of like leading the witness or biasing them too much. You know, the only thing worse than no data is really bad data. So, we don't want that. But we really believe it's possible with AI skills and training and really leaning into the people side that we can get anyone to talk to customers and really validate ideas. And that is a new muscle. Like some people have never done this. Might be a little uncomfortable, but we think that's that's okay.

22:20 >> Makes sense. So what has it done next? >> And you can see as we go through this, if you don't use claw as much, it's doing what they call hooks. So it's trying to make sure it does everything that I've asked it to do essentially. so now it's showing you kind of a representation of the onboarding that we do here where you know we talked about how we really on board with AI and want people to have a good experience and what we found is a lot of having a good experience is having like a clear road map to success. So like stage one is like setting up all of your stuff.

22:51 stage two is like understanding the customer and market for where your your team is. The different strategy and principles we've got that all codified. the product process, the loops that exist or the skills and actually shipping something. And so these are just some of the highle buckets, but we actually can basically build a simple doc that explains the person, their role, what team they're on, who their mentors would be, who they need to connect to. We feed that to AI. In fact, AI really interviews us for that because we've codified it. And then it generates a whole onboarding flow for them that they can work through. We use linear for our project management. So it creates them a linear board. They come in on day one and we still talk to them like don't worry there's a lot of human elements here. But for when they're kind of on their own and working through what to do. They don't actually have to remember from the conversations. They have clear hey you should meet with these people.

23:44 This is your mentor. This is where you start reading about the product strategy and our process. And we found that the onboarding is way more efficient and also just way more enjoyable because you're not spending so much time just trying to get up to speed. So, how do you build an onboarding like this? Basically, it's like a HTML walk me through getting onboarded on my job. >> What we found is AI is really good at systematizing existing process. So, if your onboarding is already kind of wild and all over the place, the first step is to write down what you want the process to be. Then you can start thinking about where AI can help. So, for us, like the key pieces were like setting up your laptop.

24:25 Okay, cool. We can totally codify that into the onboarding. Setting up meetings with people. Actually, the AI now will just use my calendar and set up the meetings for you. So, that's just automated. I just tell it who you need to meet with in the first week and it finds time and sets it up. Another one was customer research in the product, like really understanding the product. So, we built a product tutor. We put it into our repo. So, when you come in, we say, "Hey, go here in the terminal, run this skill, and it will actually walk you through the product." And then if you have questions, you can get answers right there. And so it's going to be different for everybody depending on where you find value. But if you can write down your system, then it kind of usually becomes pretty clear. Okay, this AI could help with this. It can't like talking to customers. We don't want that just happening in docs. We want you out in the field talking to customers. So we'll have that being set up, but not AI. thinking through some of the like pros and cons is something we want humans doing. But there's a lot we found especially in onboarding that's just process. It's just getting onboarded.

25:28 It's just kind of being up to speed on everything that's already been done. And my worry with long documents like these >> is that AI will create this fancy thing and ultimately you just spend more time editing it >> to get it to be accurate than you would have if you had just done like a simpler version yourself. So how do you build this in sort of like the right compounding way? Yeah, one of the things we're still working through honestly is managing that because yeah, everyone got this like crazy writer that can write anything for them in in as much detail.

26:03 And a lot of companies historically valued that in some way, even if it wasn't explicit. Like if you showed up to a meeting with like a big document, you'd it kind of implied you did a lot of research, right? So as humans, we seem to have associated value with length. and I've got I can't even tell you the number of documents shot over to me that are like 10 to 20 pages of AI and I'm just like yeah that's not happening that's going straight to my AI which kind of defeats the point right so what do you do what we're thinking about is something should be AI written for AI in my opinion like part of the onboarding that we've done is not for you to read it's to give context to the AI to essentially walk you through the things and answer your questions so if you want to know like why did we make this strategic decision and we give all that context to the AI, that's a lot. and if we give it as much as we can, we would never expect a human to read all of that. But then it provides context for the AI to to answer. So that's essentially docs for AI. What we found is now, and this is more of a newer thing, we're leaning towards more visual, more succinct docs for humans.

27:10 And AI could still help you, but you got to spend a lot of time as a human really cutting it down. Maybe adding more visual components that people wouldn't read, but actually understand, like a graph of data or even a picture that would have been really hard because most of us aren't designers, can really help. So now, if someone's sending me something that they expect me to read as a human and not have it AI ingest for me, I kind of think of one to three pages as the max. Prefer it comes across as like an HTML and somewhat visual. And then if they really want like a ton of details, I think it's okay to think about AI as kind of an interpreter for that. And then there's a rare case where you actually want the humans to go really deep into like a 10 20page document. I think that still exists. But if the person used AI and not a lot of their own thinking to generate that document, I don't think it's reasonable that the other people will use their own energy to consume the document. Yeah, that's got to be the most frustrating thing is that sometimes it feels like people are spending roughly as long as it took me to read this as to write it and nobody wants that.

28:14 >> Nobody does. I a lot of times I think you're kind of almost kicking the effort over to another person, right? Like you just keep bouncing the level of effort around where it's like, "Oh, I need you to do this thing. Okay, I'll have AI do it and then I'll send it back to you." And you're like, "Okay, I mean I could have done that." So like what was the value add? Exactly. So, how do we create like a nonslop loop? Maybe you can help us like program a loop from scratch.

28:40 >> And non-slop is kind of interesting because you hear the word slop thrown around a lot. And I would argue that some AI slop is actually better than human slop that I've seen before AI. And so, one thing I like to remind people is before AI, we did not live in a perfect utopia of only good documents and good code running around the universe. A lot of stuff was bad. people don't like to remember, but we used to copy and paste code from Stag overflow, right?

29:06 Like it it was not that different than AI. just less efficient. So for me, when I think about writing a good skill, and we could even go through this. go back here. I'll just make a temp directory. So, like if you're going to make a skill, honestly, one of the first things you can do is, and I'll just type here, but I actually think if in the best case, you're going in here, and we'll just do skill.m MD, which is very basic.

29:36 It's not including any code. I would write the first pass by hand. There's actually a lot, not a lot, but I've read some papers that suggest that skills authored by humans are often better than the ones authored by AI. And I think the reason for that is you know more and can express more of like what you want this thing to do. And if you offload that to AI, you might tell yourself it's it's kind of like, you know, ebikes are kind of popular. I live people like, "Yeah, I got an ebike, but don't worry. Like I still want to get exercise and we'll pedal." Once you get on the ebike, you're not pedaling. Like that's just the truth for most people. So with AI, once you kind of get on the AI loop, it can be really hard to be like, "Now I'm going to inject my thinking." because the loop's moving so fast, right? It's like you're on the ebike and it's moving and it's fun and you're getting the dopamine. It can be a little bit hard to get off that treadmill. So, how do you start? I'd start like a human and just be like, "Help me make good decisions.

30:34 I specifically struggle and I'm just going to make stuff up with decision fatigue and would love someone to use more what's like a good word for this?" contrarian thinking, but also help push me to make a decision. And you can tell I suck at typing now since AI when enough thinking's been done.

31:04 I also like to know latest research and have a partner that questions, pushes me, uses the socratic method and calls my BS also be succinct. I always add this because man, if you use Opus 5, the thing likes to talk. The average length of responses between like and opus 4.6 they're like in one realm and then opus 5 is just like so wordy.

31:41 >> It's so wordy. I did it definitely for me regressed. It sounds more like AI than any AI I've used. It's insane. >> Quick thought experiment for you. Is there anything in this video you should be trying on your own? If there is, try it. Take a screenshot, post it on LinkedIn or X and tag me. I'd love to see what you're learning. Now, a quick word from our sponsors before we get into the back half of the pod. I used to live in report purgatory. Every team had a different number. Every weekly review started with someone reconciling spreadsheets. We stopped hiring more analysts and gave the reconciliation to an AI employee instead. Victor is an AI employee that lives on Slack and Microsoft Teams. It connects to 3,000 plus tools your team already uses, ships real deliverables, and every action goes through your team for approval first.

32:27 It's the closest thing I've seen to a small team running like a much larger one. Let me share three things that Victor does that changed how my team operates. First, Ask Victor has replaced our Monday metric scramble. Someone types flag any customer whose usage dropped 40% week over week and draft an outreach loom for the account owner to approve. 90 seconds later, the next step is ready. Second, scheduled tasks run the work that nobody wants to remember.

32:55 Every morning, Victor checks overnight support tickets, drafts replies for the on call to approve, and escalates anything mentioning churn. Nobody had to ask for it. Finally, spaces ship internal tools in minutes. Ask for renewals dashboard. Victor builds it with a database and post the link to your channel. The team can stop opening four tabs to get the same view. So, stop chatting with AI and start working with it. Get started at victor.com. There's $100 in free credits with no card required at vikor.com/kashgupta3.

33:27 You can find that link in the description. Quick question. Does your AI coding tool have a design system or a theme? Because there's a massive difference. A theme is a few hex codes, a font, and a logo. You paste it in and your AI built app sort of looks like your brand. Sort of. The buttons are wrong, the spacing is off, and if your engineer tries to use the code, they're rewriting every component from scratch. A design system is your actual component library, your buttons, your modals, your token definitions, the code your engineers already ship with. Bolt new design system agent ingests your real system. Upload your npm packages, your CSS, your documentation. The agent builds a complete design system from your actual code. Every project on bolt new uses your real components. So when engineering pulls the code, they see their own primary button, their own design tokens, their own architecture, nothing to rewrite. Every other AI tool gives you a costume. Bolt at new gives you the actual uniform. Check it out at bolt.new/ aos. Imagine learning AI product management, AI product strategy, AI product leadership, advanced PM with Claude Code, all from Frontier Leaders at OpenAI, Anthropic, and Google for less than $10 a day.

34:39 Actually, you don't have to imagine it. That's exactly what product faculty's AI builder fellowship gets you. One yearly membership, six live cohorts, not recorded videos, or another content library you never finish. real live sessions with Frontier AI leaders, hands-on AI build labs, executive insight sessions, capstone projects, unlimited retakes, and continuous access to new programs as the AI landscape evolves. You'll also get every new certification they add while you're a fellow at no extra cost. The goal is simple. help you master Frontier AI skills to become an AI native product builder and operator, not just someone who talks about AI. Purchased separately, these programs are worth more than $19,245.

35:30 The lowest price certification alone starts at $2,700, but founding members can join the entire fellowship for less than $3,600 per year. That's less than $10 a day. Soon, the price increases to $5,000 a year. So, if you're serious about becoming the person your company turns to for AI, this is one of the highest ROI decisions you can make this year. Join the AI Builder Fellowship at productf faculty.com. And so, we can go a lot further on this and there's like a lot more depth to skills, but if I just do that and I, you know, write quit and and then the way claude looks for skills is you can make a claude and we can move that skill toclaude and we'll just make it decisions. MD. Pretty sure it'll find that. so we've got there. You don't see because it's a hidden folder. And then if we go open claude. Okay. Help me decide whether to invest in robots for roofing. Just making stuff up using our decide skill. And so if this worked correctly, you can see that it pulled the skill aside, right? So, we're starting the loop off with a skill, which is essentially like something to help us start interacting with an AI in a way that we've kind of defined the boundaries. And so, you can see it's starting with phase 0, which I didn't even say phase zero. It's just kind of inferring that I need to know what invest means here. So, it's using some of that socratic things I said. It's asking me questions. So, it's asking me clarifying questions. I'm, you know, in a perfect world, we'd actually just be talking to it. So, if you use voice, that's really efficient. But we could talk through and say it is reversible and the blast radius well maybe not reversible now I think about what I asked it is not easily reversible man don't don't take spelling from me the blast radius is large and cost of weight small like let's just throw that in there and this is really you know I picked this because it's not something we're thinking about right now so it's completely just out of nowhere And it's kind of like a big question. So it's saying one way door, high blast radius, urgency. And you'll start to see that it's using, you know, tech type words because it's trained on all this corpus of the world. So you know, two-way doors, it's thinking about it's think about the blacks radius, the urgency.

37:47 That's combination is the textbook case for not deciding yet. And I tried to purposely give it something to try to push it in that way so this doesn't go on forever. So it's like saying, okay, you probably don't actually to decide now who owns this call. And I can say the CEO. And then after this what you would do to close the loop in the most simple way possible and you can build all this into like systems and go to level you know 10. But you can say based on this conversation how should we improve our skill to be better and more effective next time? So this is kind of where the loop's coming in more manually where we're saying hey we just had a conversation. You've seen the conversation. it. I don't know what it's going to recommend because this is a very short conversation to kind of show the point, but it'll come back with something. We will riff with it on that and say, "Okay, yeah, yeah, yeah, let's do this.

38:41 Let's not do that." And if we go back and look at the skill, we'll surely be a lot better and a lot more involved in the ways that we failed before. And that's really where the loop happens. And this can be hard to remember. And so what I've seen a lot of people do is build in hooks which essentially is like before AI closes out the session or when it's closing out the session it will force a kind of improvement loop and that's something you can build in claw that's called clawed hooks. So yeah it's giving me some changes and now it's saying do you want me to apply those changes and this is where I would say again this is the time to get off the AI treadmill use your brain think about what you actually want to put into it because AI will always have suggestions on how to improve things. never not going to say that. And if you always accept what it says, it will become too much. The skill gets too too opinionated, too much context, too many tokens and it kind of dilutes the whole value.

39:35 >> Yeah. It tries to create like mega skills that have like 10 use cases into one, >> which is completely not what if you look at like the best practices even from like anthropic themselves is like not the way to do it. The AI just really seems to like that. >> Wants to solve all the problems with a given skill. So how do we turn this into a loop? Like I got the the basically we had the human gate and then we gave it feedback and then it improved itself.

39:58 But do we want to set it off to go autonomously on its own now and start making decisions around roofing robots or what what happens from here? >> Yeah. Yeah. And then you can set this up as a loop and ask me what you need including a learning hook before closing out sessions that learns from the log. Yeah, totally. The one thing Claude actually supports is looping, which if you do sloop, you'll see it run a prompt or slash command on a recurring interval. So I personally think loops are more than just like a cron job, but one of the things of a loop is you want it to run on a schedule. So you can actually tell Claude, hey, do this for me or help me know how to do it. And that's one of the best advice I think that I give people is if you don't know how to do something, ask AI. That's not like always the end point, but it can often get you further along and help you kind of at least know the questions to ask. You said, "Hey, can we make this a loop?" I said, "Yes." And I told AI to do it, and it I know it knows how to do this. So, it'll go set it up. It can set up the hook. It can set up the loop. It makes the skill better. And then in a few minutes, we've kind of gone from a one-off kind of prompt skill to a system that loops around one specific thing here, which is like making decisions around robots on roofs. But you could imagine doing that for a lot of things.

41:14 I know people that use it for like preparing for the day. every day goes off, does its thing, takes any feedback, updates the loop, and goes back. So, you come in with like a fresh set of like here's what I need to get done today. >> Okay, these prototypes are pretty cool. How did it decide on these particular prototypes? >> Yeah, so if you look at this demo skill that I've made, and this is where I'm cheating a little bit, the skill has step by steps that it's running through.

41:39 So when I told it to go next, you'll actually see that it's going through here. It's reading the skill. It's then got some prompts which is saying to create varants for some specific prompts. And these are really basic prompts like if we go back to the actual things that it generated. What I said is create a crew schedule prototype for a contractor in a product like Job Nimbus. So really basic prompt and you see it working through that. So the the demos or the prototypes were not already built. It's building them live. So every time I run this, it's a little different. But the skill itself had the boundaries of this type of prototype, how many to make. And you can see it's variant A, minimal, variant B, fullfeatured, and C creative. And that actually does reflect what we do in our internal system where we try to have at least three variants and of different types. One that's a little bit more full, one that's more basic, and one that's a little bit more creative. So it's reflecting in more of a demo state.

42:37 But this is exactly how we do it on our side. We just spend a little bit more time kind of refining the context into the prototype where here I just gave it a few sentences on what to build. >> Yep. I have a morning planning loop. What other loops and hooks should PMs be building? Like what's the ideal set of them? >> One that I've really valued for myself is customer outreach loop. So that every day or week or whatever the cadence is that makes sense for you is actually identifying customers from your data whether that's like from Pendo or Amplitude or a database that you should be reaching out to. Maybe they've had change in product usage, maybe they're using a product that you're working on and you can even just have it email them if your company is okay with that and you feel like you've set up the right guard rails. It's a really good way to keep close to the customer without having one of the biggest challenges, which was just the time to go find who to reach out to and send them a message.

43:31 One of the best loops I think you can do as a PM just to stay close to the customer. Another one I see that comes up a lot is just like a project management loop. So pull tickets from linear, Jira, whatever you're using, see where they're at, when they were last updated, what risks there are, who's, you know, not got work in progress or needs things and kind of synthesize that down so you're never having to go do all that manually.

43:53 >> So I got two loops. Those both sound brilliant. What other ones do I need? And also, what hooks do I need? hooks that I find useful is one I did already talk a little bit about which is like a post like you can set up a hook when Claude closes a session to do specific things and that's just really valuable like it's storing all the logs on your machine so it can inspect the logs the things that have gone through and it's essentially like building out your own kind of eval harness so one of the things you can do is like build a hey this thing finished run it against where it ran into issues like where do we have back and forth what could we improve? And it can actually just message you with that out when it closes and says, "Hey, this is like some issues." If you want to go even further, you can automate that process. Again, there's pros and cons because like we said, Claude might overmate and try to jam every possible thing into your improvements. Other hooks are like startup hooks that are useful. So like when it starts up, you might want to know things like what repo you're in, what where it's at on the git, like is it which branch, is it the main branch? And then as you're in the work, you can actually set up tool hooks. So, one that I think a lot of companies have, including us, is like if you try to run like rm-rf, like remove everything, it won't do that. Like, it'll catch on that hook as opposed to telling Claude not to delete everything in a prompt. The hook is actually deterministic and it'll trigger on specific things like a bash command and stop that. Or sharing credentials, right? Like you're not allowed to share credentials. So we have a hook that if you're trying to do anything that kind of looks like that, it'll stop you or stop Claude more importantly because maybe it thinks it needs to do that. So that's just kind of some of the ways it adds some determinism into the looping of Claude because Claude itself is a loop, right? It's essentially asking you for something, taking it, iterating, and then coming back to you. And so at each kind of phase of that loop, they let you inject code through hooks. So it will always happen. Whereas in a prompt or a skill you say never share a password.

45:51 Claude can just be like I didn't read that today. I mean I know we've all experienced this where Claude's like yeah you're totally right. You said that and I forgot. doesn't do a lot of good if like all your data got blown out of the water. >> So you create a bunch of safety related hooks. It sounds like the key loops. Just to review I don't know if I remember them all off the top of my head but there is one around like work in progress and things happening in linear.

46:15 There's one around customer outreach. There's one around synthesizing support and what feedback you're getting. What other ones should I be thinking about after that? >> Yeah, some of the big ones I'm trying to think on the product team that people use. This one's less of a loop and more of I'd say a straight skill because it's not necessarily looping as much, but we've tried to start building skills around thinking so that it's less about doing and more about pushing the thinking of the people on the team. So like if you're on doing more of the product work, it's like pushing your thinking on like did you talk to customers? Did you look at the benchmarks in the industry for these things? Where does the data come from?

46:55 Did you think about this risk or feasibility? Because we're almost trying to force you off this AI treadmill at certain points as opposed to just say, "Hey Claude, here's, you know, the CEO told me to go build this feature. or can you go do all the research and tell me what to build and why to build and blah blah blah blah blah. Like that's not actually like a terrible place to maybe start. But I find that because it's so addicting that you actually just finish there. Like you get all this stuff back and you're like I can't possibly go and evaluate this.

47:27 So I might as well just send it up the chain and like see what happens. That's not doing product work. That's more kind of outsourcing your thinking. >> That's kind of like career reputation suicide. I feel like for a PM at this point, right? Like do not ship anything that just AI just like developed and start to pass that up the chain. You're just offloading the work to people above you who are even busier than you, >> right? And and it's not like to be clear like I don't see that happening on our team. I think we've had some not like go all the chain, but there's been things that maybe gotten higher than you'd wanted to see. And I think it's not because people don't want to do quality work. They're feeling like they have this tool that helps them move faster and so they want to keep moving faster because now everyone's using it. People also aren't unaware. If you went back like a year and you were doing this, people maybe were like, "Holy crap, this guy's insane or this girl's insane." Now people are like, "That's just AI." Like, I know this is not like thoughtful.

48:21 So, think about it. Use AI to help you do research, to push you to think, have it even give you its ideas. But if you end up in a meeting and your answer is Claude said that or told me that or did this thing outside of just like normal data gathering like it's like a code machine or a searching machine for you. That's bad. And I I hear that from time to time. People are like, "Yeah, Claude said it would take this long to build this feature." I'm like, "Whoa, Claude does not know how long it will take."

48:49 And that was more like six months ago that pe that I would hear that when Claude was newer on the scene, especially for PMs. I'm like, "No, Claude does not." You ask Cola how long it'll take six months ago and it's like a week and you say go build it and it'd be like built. >> It was crazy. It would overestimate everything. Okay. So, I think I got a good sense of what loops are, what hooks are, when I need to use them. Talk to me a little bit about from product to engineering. What engineering loops should be out there? What engineering hooks should be out there? And then PMs like interfacing with engineers. When does the PM work? PM loops end and the engineering loops begin.

49:27 >> Some of the most important loops for engineers right now are quality loops. The thing that people don't talk enough about in my opinion is if you ship let's say twice as fast with AI and your quality rate maintains the same rate. Let's say you have a 1% bug defect ratio or something, your customer experiences twice as many bugs. If your quality doesn't improve, the customer doesn't experience your defect rate. They experience the number of defects that you push out there. And so we're seeing this. I mean, you look at big companies, their like uptime is the worst it's been in a long time. And my opinion is it's not that their quality got worse actually, it's that they're just shipping more code and so more things go wrong at the same rate. And we've seen that. And so we've really started and tried to invest in quality improvements through loops for engineering like our standards checks automated endtoend type testing. There's a billion things you can do there. But if you don't have AI working on the quality side, you're going to move faster and your customers will feel like you're gotten worse at building things even if you didn't because they will experience the velocity of more bugs. So that's where I would start. If you're looping loop on quality for engineers >> and should PMs be pushing PRs where does that go till should PMs be working on engineering tasks at all?

50:47 >> I think it's definitely appropriate for PMs to work on engineering tasks where it makes sense. Like let me give an example. I don't know if I'd put a PM working on like our back-end billing system and making upgrades to it that could impact people's money flow and and all that. There's a lot of like important decisions to be made there that are more architectural. I think it's fairly reasonable to push some front-end changes where we have good, you know, decoupling from the back end, you know, good APIs, where we have good CI/CD, good quality testing. Like the more you trust your system to do a lot of that checking, the more I think it's okay for anybody, UX, PM, edge to push.

51:25 And so, like I kind of mentioned before, you are at the mercy of your systems before AI. Great systems do really well with AI. If you're a company that had really bad systems and relied a lot on humans and slowness to kind of protect you, you don't want PMs coding in there because you barely want probably your engineers coding in there. But great systems, I think there's no reason PMS can't jump in and help. And the one thing I would be careful of is make sure it's agreed upon in the team because it can create some shadow work for engineers because you want code reviewed. that's the same for any engineer. And I've seen PMs be like, "Yeah, I just pushed this. I was working on the side. Can someone take a look?"

52:04 And now you've generated maybe hours of work for someone that wasn't planned on. And you might get frustrated because you're like, "Hey, I'm trying to ship stuff." And they're frustrated because it wasn't like agreed upon that that would be part of their workload. And as an engineering community, this is one of the things we're struggling with is how do we manage the code onslaught? because you could have AI generate basically infinite code. So how do you do that? Systems is one thing people are thinking about but you know the dirty truth is most companies engineering systems weren't perfect before. So they're not perfect now. And I tend to push quality like I said on the loops because I almost think that the quality advantage of AI can beat the velocity advantage because it lets you ship safer.

52:51 >> Okay. So that's one part of it which is PMs doing engineering work. It sounds like PMs shouldn't be trying to become engineers but if they're doing some front-end changes where again they have put in probably more time on it than the reviewer would need to then it might make sense. >> What about the other side which is engineers doing vibe PMing? The failure points of vibe PMing are interesting because failure points of vibe engineering can be potentially more obvious. Like you shipped a bug, you created a problem. So you think like what's the failure mode of vibe PMing?

53:26 It's it's essentially shipping something that people don't want, right? Or spending time going down a road that's not viable for the business or not feasible or or not valuable. if you think about the four risk factors that I often come back to, feasibility engineers already know that we're good. Usability is generally handled more by design. So we can think about that in a minute. So then you got like vibe PMing is more thinking about is it viable for the business which can they make money like is it a high margin or margin you need and is it something people actually want and would pay for the value side.

53:59 So how do you vibe PM? In my opinion, engineers are more empowered than ever to have AI help them mine data, right? To go out into the industry to see what competitors are doing, to look at calls, to even go find customers that they could talk to, right? You can very easily do all that. The risk is that they do it poorly, right? And you go down a path that doesn't make sense. One of the ways you can kind of mitigate that is speed, which if you have a bunch of ideas, but you're very quick to to throw them away, then it's kind of okay, like cool. So, I think if you're an engineer thinking about doing more product stuff, one of the advantages you have and where I would suggest starting is you can generally build pretty fast, especially with AI. Get a space where you feel comfortable that that's a two-way door kind of to use the AI decision and go try to do some of the piming thing. Talk to the business. How do we make money? Talk to the customer.

54:55 What do they need? Don't overthink it. Get something out there behind a feature flag and get it in front of a customer and talk to them. That is very low risk. Same. It's like the version of like shipping a button color change or something for a PM in code. Like I think there is a vibe PMing. Where I don't want engineering's vibe PMing is like, hey, we need to make a strategic decision on how to invest money that could impact three years from now where this company is. like that's not viing that's like really let's get deep into this thing and use the skill set and so in the same way I think of engineering there's this gradient of hey we really need best-in-class trained product thinking but not all problems are that way just as not all problems are like really hard engineering problems and so I'm thinking about where I want engineers to lean in it's like hey we wanted to make this change we heard customers complain about this thing it's on the road map for us it's not a huge thing but we need to kind of figure some of the things out.

55:55 Can you do that? And the reason that's so powerful is if you as a person can go end to end from hey here's a problem to hey here's a solution that's making the business money or the customer happy again your leverage has gone up exponentially in the company and the speed to value went up exponentially. all those human bottlenecks internally go away, all those different decision points and like getting people together. that was a d-risking thing, right?

56:22 We did the triad to derisk things because things took a while to build. If you can build things instantly, you don't need as much d-risking. You still need some because your customers aren't okay with constant change and giving you feedback is their main job. But the bar I do think has changed because the time to build has gone down which is why waterfall existed right in manufacturing. It's like the time to create something physical takes a long time. So you spend a lot of time de-risking it up front. And we placed a lot of emphasis on this episode too on de-risisking in terms of creating divergent prototypes and putting those in front of synthetic users, putting the best ones in front of users. How important is that for what features do you need to do that and which do you skip that?

57:10 >> never skip it in my opinion. It's the kind of correlary to engineering is we used to talk about like what your test coverage is and now in my opinion it should just be 100%. It's basically free like to tell the AI on the unit test side that's not all tests but on unit test to tell AI to write 100% coverage is basically free why not do it before we didn't because it costs time and energy and there are good reasons if you're doing product work why would you not take 10 minutes and have AI spin up a few different ideas for you to consider push your thinking a bit do some synthetic analysis don't just like take it what it's what it says but like see if it's got some points you're like oh yeah I didn't think about that Our customers have said that and that's a thing I would have missed is just insane to me. I mean we know from research from I think out of Microsoft that a third of the product things we release won't deliver value like no a third will. So like twothirds of the thing either flat or don't do anything for the business.

58:06 So why would you not take some time to basically get more app bats for free? And that and that's where like the synthetic stuff can be useful. That's also where like having really good relationships with your customers so so useful. Just get some more reps. because AI makes it free. It wasn't free before. You had to have someone design this stuff. You know, if you're newer to your career, maybe you haven't experienced that, but it definitely used to take a lot more time.

58:32 >> So, we touched on the PM and the engineering loop side of it. The other portion of the triad, of course, is the design. What loops are they building? How did these all come together to vibe product develop? >> Biggest loops you'll see on UX is prototyping. Like they were doing a lot of that anyways through like Figma. I you know Figma was like a darling in tech and I think that AI has really hurt them because like claw designs come out.

58:55 A lot of people I know were kind of vibe coding internal tools like design tools to work with their systems. And so I see design really leaning in heavily on like how do we iterate faster in a way that matches how we think about design and our components. And so you can totally build that into loops. And I think of like what is vibe design? The one we haven't talked about. It's really like a PM or an engineer spinning up something that's designed in a way that wouldn't make your design team immediately like throw up. And the goal again is like it's there's a gradient. Some things need a lot of design and really be thoughtful. Some things don't. But we used to always go through design for that because they had the skills, the tools. Now with AI, if you have a good design system, like it'll use our component library, use our system. So if you're wanting to make some changes to the front end, like I think you should totally be able to do that. And having your design system know those boundaries is helpful. So when people use it, you're like, "Yeah, that's something that we feel comfortable non-design experts kind of riffing on." But you never learn this if you don't try. And so when I think about the triad, it's really more coming together and what people might be calling like a builder.

60:06 And I think that coming together is going to take longer than we think. It's not like some people want to act like today everyone's a builder. Like it takes time. It takes time to learn new skills, to build systems, to do all that stuff. But there are people today that have found a way to merge design, product, and engineering into one person themselves with AI that are creating tremendous value. Whether it's for a company they're in or their own business, I bet you probably, you know, running your own basically product catalog of podcasts and newsletters. You probably feel like you've merged a lot of skills that you didn't have before.

60:44 That I think is an exciting future. And we got to get there. We got to train people. We got to build the tools. We got to invest there. It just doesn't come for free. But I really think designers are in a very powerful place where they more than anyone often understand the customer, which is the hardest thing to replicate with AI. AI can code. AI can do the research, but they can't really replicate the customer. So if you have that knowledge and you're willing to think a bit more about the business and how they make money and a bit more about how to build things, I think you can totally go end to end as a designer and even if you don't do that, your prototypes will become so much better and you can get them moving so much faster because of all these AI tools. so yeah, I'm I'm a huge fan of this idea of like design, product, engineering kind of coming together and people will still have expertise for sure and that's important, but it's just fun to work across the whole stack on certain problems that make that a reality.

61:44 >> And have you hired any of these full stack product builders or are you still hiring specialists? >> Yeah, we hire a lot of them actually now. So we we we hosted a hackathon. I think it's the largest hackathon in Utah when we did it. This probably still is because it wasn't that long ago and we got hundreds of people to show up and the person one of the people that won it is now an SVP at our company and he's a builder like he built a product with a small team end to end in like two days like we asked them to build an actual product for for our user base and it was crazy. And then another person that I think got like third, we hired two onto a team. And what we're actually finding is not that skills don't matter, like engineering skills matter, design skills matter. All these things that people have learned matter, but if you have the ability to take those skills and then inject AI in the right way and kind of push a little bit maybe outside of your traditional comfort zone, it just opens up a world of opportunity for you to create so much value that wasn't really possible before. So, I'm super pro this builder mentality. I don't Some people take that as like, oh, he doesn't believe skills matter. I think they matter more than ever, but I think because they matter and with AI and systems, you can actually upskill yourself more than ever.

63:00 >> So, should every PM, designer, and engineer be aspiring to upskill themselves into a product builder? >> In my opinion, yes, unless you find yourself in a very specialized role. For example, like on engineering, if you're like a security engineer or like you're working at database level research at Google on like how to build the next, you know, high scale, high throughput backend or database or whatever it might be, those type of specialized roles really need special people with special skills that are going to always be important.

63:31 A lot of SAS companies don't have those. They tend to generalize better and if you can become a product builder, they will love you because they want to solve problems that people will pay for and love using. And often that's not a incredibly complex engineering problem. It's not often incredibly complex design problem or a incredibly complex product problem. It's the merging of all of those things in the right way that's the hard part. It's not like most like if you looked at most companies codebase no one would be like wow this is like the Picasso of code and most things are not designed in a way that we're like wow put this on my you know on my wall because it's so beautiful. So it's kind of like that super nexus of all those things and that took three people before and maybe it takes three people forever I don't know but one of my bets is it doesn't for all things going forward and I I mean I'm not I can't predict the future so I'm not putting money on it but that's what I would say. So what's amazing is how quickly you ramped up on all of this even at the seauite level when you probably have to spend 90% of your time in meetings and with stakeholders and those kind of things.

64:43 So for the other leaders out there how do you ramp up how do you personally find the right time to do this build this skill set because if you're going to start asking this out of your team you obviously have to lead by example like you are doing. Yeah, I one of the things that I benefit from is I've always enjoyed tinkering. So, I spend a lot of my free time outside of work doing stuff. So, that helps. Like, I won't lie. Spending extra time learning is always a good thing. The thing that I think we've done that's actually been really good and our CEO really pioneered this was we don't we're in person Tuesday through Thursday. And what we've done on Monday and Friday, which is remote, is we really discourage meetings. And it's actually the first place I've worked at that's done this somewhat well. So usually on Monday and Fridays I don't have any meetings which wow, you know, my team's like almost 100 people. So that's like not a bad place to be if you're leading a team of like any amount of size where usually you're just moving through meetings. So let me tell you, if you get in a room for eight hours with a lot of time and AI and a lot of things you want to learn and explore, you can do a lot of damage.

65:47 it's really cutting that time into your schedule. So most people don't have that luxury or that culture. What I would recommend is treat it as important to your role and to the adoption of these tools. Saying it is shown. I mean a lot of companies are doing research on this like McKenzie. Saying things like use AI doesn't do nearly as much if anything versus this is how I'm using AI. And that is if that's what you need to accomplish in your role, the best thing you can do is use it and show the team you're using it. And then you start to learn too where the failure modes are because often highle leaders can talk about things that they read on like X or probably your, you know, newsletter.

66:26 They're like, "Hey, they're talking to this guy Tyler and he said, "Everyone's building. Why aren't we building?" Right? And there's just so much nuance even what I'm saying about how we did this, the pain, what we're still learning. You really got to experience it yourself to be able to speak to it. >> I feel like every leader now needs to just carve out some icy tinkering time. >> And most leaders love it. It's like so fun. Yeah, it's way more fun than being in meetings all day. This has been amazing. If people want to learn more, if they want to find you online, where do they go?

66:53 >> So, biggest place, Substack. I have a newsletter that I kind of write my tinkering about. It's called the AI Architect can find my name, Tyler Fulkman or the AI Architect. I am on LinkedIn. LinkedIn, I feel like, has suffered from AI more than most places. So, I do post there. I'll be honest, I don't read it as much. So, if you engage with me there, it takes me a little bit longer. But, yeah, reach out. I love chatting with people. Like one of the things I'm really passionate about is community and bringing together people just having a good time. Like I really love AI, local AI, tinkering, building.

67:20 So if that's your thing, definitely reach out. And if you're in Utah, like we're always looking for great builders. So Job Nimbus is is a super fun place to work here. >> Sounds like a place where you can actually improve your AI skill set instead of being stuck in meetings, which is another bonus, too. Subscribe to the AI Architect. You will see the link in the show notes or down below in the description. Tyler, thank you for being so generous with your learnings and time today. Yeah, thanks so much for having me. It was super fun.

67:44 >> See you guys in the next episode. I hope you learned as much from today's episode as I did. If you can do one thing that's totally free that would help the show, it would be to check that you're following on Apple and Spotify podcasts. Check that you've left ratings and reviews on those platforms. Check that you're subscribed on YouTube. Leave a like and a comment on this video. And then share it with your friends. We're trying to make better and better podcasts. After 2 years, we think we've gotten something pretty good going. So, let us know what we can do to make it even better, who else we should interview, and we will put on the best shows we possibly can. Finally, don't forget my offer for the bundle. You get an entire year of my paid newsletter, plus my favorite AI tools. Bolt, new, air table, speechify, descript, magic patterns, linear, dovetail, arise, and mobin. That's $27,000 worth of value for just $150. So check that out at bundle.ac.com com if it interests you and I can't wait to share our next episode soon.

Summary

The discussion centers on the evolving role of product managers (PMs), engineers, and designers in the context of AI and product development, emphasizing the importance of "vibe PMing" and the concept of "loops" in product management. Tyler Folkman, the chief AI officer at Job Nimbus, argues that as AI tools become more integrated into workflows, professionals across roles should aim to become product builders, leveraging AI to enhance their processes and outputs.

- Loops are seen as a powerful tool for product managers to create self-improving systems that enhance productivity and decision-making.
- The distinction between prompting and agentic behavior in AI is crucial; agents can operate autonomously, reducing the need for constant human input.
- PMs, designers, and engineers are encouraged to upskill into product builders, merging their expertise to create value effectively.
- The importance of customer-centric approaches and rapid prototyping is highlighted, with AI facilitating faster iterations and feedback loops.
- Quality assurance loops are essential for engineers to maintain product integrity while increasing shipping velocity.
- The conversation stresses that while AI can automate many tasks, human oversight and critical thinking remain vital to avoid pitfalls in product development.
- A collaborative environment where PMs, designers, and engineers work together is encouraged, fostering a culture of building and innovation.
- Tyler emphasizes the need for leaders to engage in hands-on learning and experimentation with AI to effectively guide their teams.

Questions Answered

What does the shift from traditional roles to product building mean for PMs and engineers?

The conversation highlights the evolving nature of roles in tech, suggesting that in the future, titles like engineer or designer may become less significant as the focus shifts towards product building. The speaker emphasizes the importance of upskilling for PMs, designers, and engineers to adapt to this change.

How can AI be effectively integrated into product development processes?

The speaker describes a process where AI-generated outputs are reviewed and refined by humans, emphasizing the importance of maintaining a feedback loop. Instead of reverting to previous versions, the focus is on tweaking and improving current outputs.

What are the best practices for using AI in documentation?

The speaker suggests that documentation should be concise and visually appealing, ideally limited to one to three pages. They argue that excessive detail can lead to inefficiency and that AI should be used to streamline the documentation process without overloading the reader.

How does AI assist in the prototyping process?

The speaker explains how AI can generate prototypes based on simple prompts, allowing for real-time adjustments and variations. This approach reflects their internal processes at Job Nimbus, where multiple prototype variants are created to cater to different needs.

What is the significance of PMs adapting to agile methodologies?

The speaker discusses the importance of PMs being able to make strategic decisions quickly and effectively, emphasizing that the ability to solve problems end-to-end increases their leverage within the company. This agility reduces bottlenecks and accelerates the speed to value.

© transcribe · For agents Built with care and craft by Gokul Rajaram