The 7 Biases of Product Teams by Shreyas Doshi of Stripe for Elevate2020 by Plato

0:00 [Music] so hello everyone my name is shreyas doshi in this session i'm going to share some

0:30 lessons and frameworks i've learned from my work at yahoo google twitter and most recently at stripe and what you're about to see in here has been four years in the making it all started when i asked myself this question in the year 2016. why are so many products unsuccessful even when really smart people and teams are building those products this presentation is my answer to that

1:01 question and it has to do with the biases we have when building products but before we get into answers and the biases themselves let's get oriented we'll start with a quick perspective on why we should study these biases in the first place we'll then take an in-depth look at the seven biases what they are and how to spot them you'll also hear a few product management stories along the way inspired by real events and lastly we'll

1:35 cover how to deal with these biases in our work so that we can improve our chances of future product success part one why should we care about and study our own biases to understand that let's begin with a look at the inputs outputs outcomes framework i really like this simple framework for the work we do as product managers and product people now as a quick recap inputs are things like user feedback

2:05 metrics industry landscape technical effort outputs are the actual artifacts we create to help our customers achieve their goals such as the product itself its documentation and any other interactions that our customers have with our product and outcomes are the results and the progress that our customers and our company achieve because of these outputs so as product people our challenge then

2:39 is simple so definitely not easy it is how do we collect the right inputs convert them to the right outputs so we can get to the right outcomes and that is where decisions come in mind you we make countless decisions while building enhancing and marketing our products the quality of those decisions is ultimately what determines the impact our product makes that's why i like to say that you do not rise to the level of your plan

3:11 it's easy to make a great looking plan you fall to the level of your decision making similarly okrs are useful but your actual outputs and outcomes are a function of the decisions you make during your execution annual planning is useful but the decisions you make while creating your product strategy will determine your long-term success more than your annual plan we're tempted to do so many things at

3:41 once but it's our prioritization decisions that determine how much we actually get done so if you show me 10 teams with similar ability and ask me which one will produce more wins over the long term i will tell you that it's the team that makes the most effective decisions so how do we get to more effective decisions everyone understands what the person on the left is saying here about making effective decisions

4:12 there are many variants but it's roughly be clear on what you want to achieve collect the right inputs understand them and then thoroughly evaluate the options before you to pick one of them that's all well and good but what the person to our right is saying is equally important and that is to avoid making bad choices there's no better person than charlie munger to drive that point home all i want to know is where i'm going to

4:39 die so i'll never go there for a sufficiently smart and motivated group of individuals decision making is less about making brilliant choices and much more about avoiding bad ones and that is why we need to study the biases that stymie us as product people so what are these biases in the next few minutes we'll walk through the seven biases i've seen time and time again both in the companies have worked at and those have advised over the years

5:11 but first let's start with a product management story once upon a time a team has a new product idea the company already has a pretty large install base of customers using its software for customer support the company is very customer driven and so the team's first step is to talk to customers to evaluate their new product hypothesis andy is our pm here he talks to a number of customers

5:41 and asks them whether the new product would be useful and whether it would solve their problem of high customer support costs almost all customers say yes some even say that they want to participate in the beta as soon as it's out andy and his team are quite excited by this feedback they report their findings to the executive team and get the green light to build the product great the team works hard executes almost flawlessly on version one of the product

6:12 and launches it in just six months now andy goes back to the customers who had expressed interest in the product early on he tells them that the product is ready for deployment and then he waits and he waits some more and then some more unfortunately even six months after its initial launch hardly any of those customers have deployed the product remember this for later it's important

6:42 it's been a while since version one of our product launched so it's now time for andy's product review with the executive team things are not great adoption wise or revenue wise but andy is an optimistic person at the product review he directs attention towards the positives here's what we've learned our mvp isn't sufficient we need to make it easier to implement and adopt our new product we need features x y and z

7:14 the team gets the mandate to build out those features from chris the cpo who's been quite bullish on this new product since the very beginning fast forward a couple more quarters and adoption is still not where it needs to be time for another product review andy still very customer centric in his approach says we know from talking to customers that we have the right product we just need to improve our go to market strategy and

7:44 execution chris the c cpo is still quite committed to the product since they made a major investment here they start discussing ideas on how to improve their go-to-market approach with joe the vp of revenue and marketing bundling cross-selling email campaigns reorganizing the sales team no option is off the table they make the changes on the go to market front another year goes by

8:16 adoption is still flat so finally at the next product review almost three years after work began a decision is made it's time to sunset this product now at this point learnings are captured and shared widely in the org we haven't failed we have learned of course edison's light bulb quote is repeated several times i haven't failed i've just found 10 000 ways that it won't work

8:49 so what really happened here let's take a closer look now clearly there can be many possible causes for product failure but in this case the main observation is a simple one this product should not have been built in the first place let's look at the main reason why when andy talked to customers about the specific problem of customer support costs they naturally focused on that specific problem and with that focus on a specific

9:20 problem they excluded other more important problems that they were facing that is why these customers could never find the time or resources to actually adopt and deploy the product when it was ready here's what andy should have done instead after talking to each customer about the problem of support costs he should have asked them to stack rank that problem versus all the other problems they were trying to solve for their business and for their organization that is where the real truth would have

9:51 emerged this customer problem stack rank acts as a great test of the true priority of the customer problem and the product solution you're discussing if you're more curious about the customer problem stack rank we can discuss it during the q a section anyway back to our story there are at least three biases that got in the way of andy chris joe and team here beginning with the focusing illusion first described by daniel kahneman

10:22 nothing in life is as important as you think it is while you're thinking about it and the interesting thing is that while kahneman was probably talking about everyday life the focusing illusion applies to product management as well it is one of the biases i see quite frequently and it's common because it's largely invisible companies that like to think of themselves as customer driven are especially susceptible to it here are some ways this bias can

10:54 manifest in team discussions i want you to note that no single one of these is by itself a definitive sign of this bias but i suggest keeping your eyes and ears open when these things are said repeatedly by your team the second bias that andy and especially chris the cpo fell prey to is the ikea effect for products it comes from the observation that something we built together with our own hands is more valuable to us than if

11:25 we had to purchase exactly the same item pre-assembled it's the same with the tech products we built and that is why chris was not ready to pull the plug on the product early on even though all the signs were already there with the ikea effect for products we base our current product decisions on past product execution because we are too attached to the work we've already put in and don't want it to have been for nothing you'll encounter this effect

11:54 quite often with products that are straddling that unfortunate zone of not being a complete failure and not being very successful either what's interesting about the ikea effect is that it makes us less attentive to our customers actual needs instead of being rational we try to rationalize now time for another story this time a very short one over here the engineering

12:24 manager is keen to get their eng team started with software development for a new project the problem though is that the product manager isn't yet ready with the requirements but she's also feeling incredibly guilty that engineers are blocked on her somewhere in her training early on she was taught that engineers being blocked on writing code is the cardinal sin of product management so what does our hard-working product manager do she spends the weekend completing the

12:56 prd she knows that she should have done more customer research and more competitive research before calling the requirements ready but she promises herself that she'll fix this in version two of the product and we all know how these stories of will do better in the next version end they rarely have a happy ending what you just saw is the most common bias i see especially on high energy teams and ambitious startups

13:27 shane parish said we waste years because we cannot waste hours and that isn't too far from what happens in these situations because we can't stand the idea of taking a couple more weeks to get the product spec or experience right up front we end up building something sub-optimal and now that we have that thing the ikea effect kicks in making it much harder to fix our own bad choices later on the bias for building fallacy is most

13:56 common in organizations that value speed nothing wrong with that per se but if you go speedily in the wrong direction you will end up in the wrong place that's why teams should value velocity much more than speed velocity being a combination of speed and of direction i thought this is a perfect illustration for our next bias it's inspired by a scene from the tv show i love lucy what are you doing behind the sofa i've lost my car keys

14:27 where did you last see them in the kitchen then why are you looking for them in the living room well because the light is better over here now while this may sound like it's made just for tv i will tell you that something similar happens in product development all the time i call this the execution orientation fallacy where we're so focused on execution that we craft flawed product strategies and make suboptimal product decisions because we base these decisions on what

14:58 we know we can easily execute today we avoid things that seem hard to execute even if they will make a major positive impact this is especially common in product cultures where shipping something anything is lauded and rewarded when i talk to product leaders and startup ceos the execution orientation fallacy comes up all the time they want their teams to think bigger to not be phased by current limitations

15:30 they want their product managers to exhibit higher agency and ambition now on to our next bias this one's named after abraham maslow who needs no introduction one relatively little known aspect of his legacy is called maslow's hammer if the only tool you have is a hammer you will treat everything as if it is a nail maslow's hammer manifests in product development when we use a process

16:01 of framework or a skill that we are most comfortable with or one that our company endorses say via core values or its mission or operating principles even when its utility is limited or perhaps even counterproductive in a specific product context consistency process being data driven are often used as hammers at many companies now i don't want you to take away from this that

16:32 data is bad or process is bad my main point is that instead of thinking about these things in a very binary fashion either use data to justify all product decisions or no product decisions the context is what's supremely important here our decisions must first and foremost be informed by the current context not by a tool a process or a best practice that may have worked for us in a different context

17:03 two more biases to go we often encounter situations where another company's product is a fiasco and when we look at such situations we might ask ourselves what was that product team thinking how did they miss something so big so harmful and so obvious and the answer is that such teams are likely affected by this next bias russian roulette for products in it we don't account enough for scenarios that could under certain conditions lead to a

17:35 catastrophic outcome for our product or a pr nightmare for our company russian roulette for products is especially common in companies where moving fast and breaking things is viewed as a virtue and companies where following orders is expected and rewarded pre-mortems are a good antidote for russian roulette for products and if you're interested in learning more about pre-mortems i'm happy to cover them in the q a section and lastly the authority approval bias

18:07 where we devise products and plans based on what's going to be easier to get us approval or confirm the pre-existing beliefs of an executive we aim for a successful product review meeting which comes at the expense of a successful product this is the easiest to understand and explain because most of us have vividly seen this at some point in our career whether it's aligning a project with the ceo's pet project just to get funding

18:38 or team members not bringing up important facts concerns and ideas with the fear of triggering an executive at the company so there you have it you now know what i've observed across many companies of all sizes over the past 15 years and what took me the last four years of introspection and research to really understand the last thing i'll say before moving to the final section of my talk is that people will at times point out

19:07 to me a situation where they saw one or more of these biases and yet the product was extremely successful and they asked me what i make of it to which my response is surely successful products can emerge from severely suboptimal environments but that's just survivorship bias at play if you start with a sample size of hundreds of products surely many more than a handful will be successful despite all odds but remember as individuals each of

19:38 us has finite time we have one life one career one team and it's much wiser not to play that lottery for the one two three or five products that we may be responsible for instead we should aim to maximize odds of success from day one it's the same reason why we prefer being in cars that have seat belts and airbags it's the smart thing to do in this last part we'll look at the three main ways that we can

20:07 combat these biases as product managers and as leaders within a product organization and with as most things in life we need to start with self-awareness we need to recognize that these biases are real and that each of us is susceptible to them if we sense ourselves falling prey to one of them that doesn't mean we are stupid that recognition is in fact the key that unlocks greater wisdom second name it to tame it make your

20:39 teams more aware of these biases by name add these names to your team's shared vocabulary so you can create mutual awareness and accountability while ensuring psychological safety for everyone it's much easier and safer for people to be able to say that we might be falling prey to the ikea effect here than to say that we're not thinking straight about our priorities that is the power of this shared vocabulary it helps us see reality and talk about it objectively with each

21:11 other and as they say the team that sees reality best wins and we must remember that lasting org-wide change cannot come through surface level treatment that's why i love this quote from taylor swift indeed band-aids don't fix bullet holes that's why the best long-term solution is culture change and the ultimate onus for avoiding these traps is on pm engineering design leaders

21:43 if our product decisions will make or break our company doesn't it make sense to have core values and operating principles that counteract these product biases so that we can make better product decisions so friends i'll leave you with perhaps the most important part of my talk today a view of how each of these biases you saw can be counteracted with an operating principle you can establish and depending on the scope of your role

22:13 you can do this at the level of a single team a single org or the entire company to counteract the focusing illusion we need deep customer empathy we need to really understand their biggest problems and work towards solving those big problems to counteract the ikea effect we need a culture of rigorous thinking or reasoning from first principles to fight the bias for building fallacy we need to recognize that sheer speed doesn't mean much unless it's in the

22:45 right direction that's why velocity is what really matters the antidote for the execution orientation fallacy is to cultivate high ambition and high agency to find a way to get what you want without waiting for conditions to be perfect or otherwise blaming the circumstances to counteract maslow's hammer we need to recognize that context comes first the same tool can either help us or harm us and the way we decide is with an

23:16 appreciation of our current context to avoid getting killed in a game of russian roulette is to understand that preventing problems is better than having to do heroics to save the day afterwards and lastly leaders should counteract the authority approval bias by fostering constructive dissonance which is about going after the right answer and not to just to try to prove ourselves right note that this need not be one size fits

23:47 all you likely don't need each of these operating principles that's where context is important so i urge you to identify which biases your teams are most susceptible to and adopt the corresponding operating principles accordingly and with that i hope this session is helpful when you go back to your work tomorrow when you're making major prioritization decisions for 2021 annual planning and when you're devising a new product strategy next year

24:18 but perhaps even more importantly i hope that our shared journey today will help you build happier and more capable product teams and for that i wish you all the best thank you very much for your time all right we're back live thank you so much that was really great talk thanks for recording that uh we actually have a bunch of questions so we're gonna go into the q and a section right now uh one of our top four questions was can

24:47 we have the slides and of course everyone gets the recording um the the other one is by vivek and he asks how do you make decisions in a fast-paced environment where you may not have enough time to collect data or do research yeah and i think this is very common in companies with uh silicon valley culture so they don't have to be in silicon valley but you know they embody a lot of the silicon valley ethos

25:15 uh and um you know i get this uh i i get this question and i see the situation regularly where teams uh are really biased towards action their bias towards speed right and i think that is a fabulous thing i wouldn't work in an environment that did not have that drive and that energy to make an impact quickly but i think what we forget is what speed actually gets us right and so that's why i like

25:43 to say that what we ought to be aiming for is velocity we ought to be actually moving with energy uh but in the right direction uh oftentimes a fast-paced environment gets equated to writing code quickly or getting to writing code quickly so what i tend to do instead is yes let's move fast but let's do it in a way that all of us as an entire team that includes design product management engineering research we're all moving fast towards first

26:14 understanding the customer problem extremely well and then once we have a good understanding of that problem we will be able to build a product that meets their needs much better than if we rush to just start writing code and ship things and so i think like yes move fast but move fast in the right direction and what i've found is that you know while it might not seem very satisfying in week two of the project where you're still not writing code uh

26:43 what ends up happening is you end up delivering results much faster than you know shipping code finding out it doesn't work shipping more code and sort of uh you know just meandering your way hopefully and in many cases you never reach success but hopefully meandering your way to success much more a believer in intentional success uh which i believe is can and should be done in a fast-paced environment nice okay move fast but in the right

27:10 direction that should go up on the wall for everyone listening um max has a really great question uh often these biases can be enforced by stakeholders rather than the pm how do you have any advice on managing stakeholders to avoid these craps um well i would say the first thing is perhaps send them a link to this presentation if you can but also more seriously i think uh you know what the the kinds of challenges i've described in this

27:37 talk uh it is going to take us time as an industry and say you know if you're in a company for a company to get to a recognition of these things it's not going to be overnight and i think the this is why i call out product leaders as well over here which is actually the middle management of a company is extremely vital here because they are the ones who are going to sort of spot these biases if they are

28:05 coming from company leadership and have the confidence and the ability to push back against these biases right oftentimes it's very hard for an individual contributor product manager or an in or an engineering manager uh or an individual engineer designer etc uh to push back on the ceo of the company uh and so i think like you know it's really important that people at all levels of the company truly understand these biases so that's my first point about this

28:33 um and i know it's not an easy answer but i really think that that's really how this sort of change will come about is it it needs to it needs to be brought about by the culture itself the product culture at the company the second thing i'll say is uh that i have also seen instances where i talk to you know ceos company leaders often through my advising or through my work and what i often find is that they

29:01 are the ones who are lamenting the existence of these biases right so so in many cases like you might already be at a company where your company leadership your ceo is actually quite supportive of you know not speed but actually velocity but somehow that message gets lost before it reaches the the individual team that's working on the product and so again that's why i think middle management is really important because they're supposed to translate what upper management wants

29:32 to everybody else to the rest of us all right well we have less than a minute to go so i don't think we can take any more questions but everyone's going to be getting the recording next week you can forward it to other stakeholders in your organization um and if you have more questions uh how can people find you online yeah you can always find me on twitter i tend to be a little active on twitter and so feel

29:54 free to send over your questions or comments over there uh and uh i'm also happy to uh you know sort of get dms and whatnot my dms are open on twitter as well all right thank you so much for joining us have a good day

30:25 [Music] you