[NOT A PUBLIC VIDEO] Why Products Fail: The 7 Biases & Fallacies of Product Teams

0:00 hello everyone my name is shreyas doshi in this session i'm going to share some lessons and frameworks i've learned from my work at yahoo google twitter and most recently at stripe and what you're about to see in here has been four years in the making it all started when i asked myself this question in the year 2016. why are so many products unsuccessful even when really smart people and teams are building those products this presentation

0:34 is my answer to that question and it has to do with the biases we have when building products but before we get into answers and the biases themselves let's get oriented we'll start with a quick perspective on why we should study these biases in the first place we'll then take an in-depth look at the seven biases what they are and how to spot them you'll also hear a few product management stories along the way

1:06 inspired by real events and lastly we'll cover how to deal with these biases in our work so that we can improve our chances of future product success part one why should we care about and study our own biases to understand that let's begin with a look at the inputs outputs outcomes framework i really like this simple framework for the work we do as product managers and product people now as a quick recap inputs are things

1:38 like user feedback metrics industry landscape technical effort outputs are the actual artifacts we create to help our customers achieve their goals such as the product itself its documentation and any other interactions that our customers have with our product and outcomes are the results and the progress that our customers and our company achieve because of these

2:08 outputs so as product people our challenge then is simple so definitely not easy it is how do we collect the right inputs convert them to the right outputs so we can get to the right outcomes and that is where decisions come in mind you we make countless decisions while building enhancing and marketing our products the quality of those decisions is ultimately what determines the impact

2:39 our product makes that's why i like to say that you do not rise to the level of your plan it's easy to make a great looking plan you fall to the level of your decision making similarly okrs are useful but your actual outputs and outcomes are a function of the decisions you make during your execution annual planning is useful but the decisions you make while creating your product strategy will determine your long-term

3:09 success more than your annual plan we're tempted to do so many things at once but it's our prioritization decisions that determine how much we actually get done so if you show me 10 teams with similar ability and ask me which one will produce more wins over the long term i will tell you that it's the team that makes the most effective decisions so how do we get to more effective decisions

3:40 everyone understands what the person on the left is saying here about making effective decisions there are many variants but it's roughly be clear on what you want to achieve collect the right inputs understand them and then thoroughly evaluate the options before you to pick one of them that's all well and good but what the person to our right is saying is equally important and that is to avoid making bad choices there's no better person than charlie

4:09 munger to drive that point home all i want to know is where i'm going to die so i'll never go there for a sufficiently smart and motivated group of individuals decision making is less about making brilliant choices and much more about avoiding bad ones and that is why we need to study the biases that stymie us as product people so what are these biases in the next few minutes we'll walk through the seven biases i've seen time

4:40 and time again both in the companies i've worked at and those have advised over the years but first let's start with a product management story once upon a time a team has a new product idea the company already has a pretty large install base of customers using its software for customer support the company is very customer driven and so the team's first step is to talk to customers to evaluate their new product hypothesis

5:11 andy is our pm here he talks to a number of customers and asks them whether the new product would be useful and whether it would solve their problem of high customer support costs almost all customers say yes some even say that they want to participate in the beta as soon as it's out andy and his team are quite excited by this feedback they report their findings to the executive team and get the green light to build the

5:38 product great the team works hard executes almost flawlessly on version one of the product and launches it in just six months now andy goes back to the customers who had expressed interest in the product early on he tells them that the product is ready for deployment and then he waits and he waits some more and then some more unfortunately even six months after its initial launch

6:09 hardly any of those customers have deployed the product remember this for later it's important it's been a while since version one of our product launched so it's now time for andy's product review with the executive team things are not great adoption wise or revenue wise but andy is an optimistic person at the product review he directs attention towards the positives here's what we've learned our mvp isn't sufficient we need to make it

6:42 easier to implement and adopt our new product we need features x y and z the team gets the mandate to build out those features from chris the cpo who's been quite bullish on this new product since the very beginning fast forward a couple more quarters and adoption is still not where it needs to be time for another product review andy still very customer centric in his approach says we know from talking to customers

7:14 that we have the right product we just need to improve our go to market strategy and execution chris the cpo is still quite committed to the product since they made a major investment here they start discussing ideas on how to improve their go-to-market approach with joe the vp of revenue and marketing bundling cross-selling email campaigns reorganizing the sales team no option is off the table

7:45 they make the changes on the go to market front another year goes by adoption is still flat so finally at the next product review almost three years after work began a decision is made it's time to sunset this product now at this point learnings are captured and shared widely in the org we haven't failed we have learned of course edison's light bulb quote is

8:15 repeated several times i haven't failed i've just found 10 000 ways that it won't work so what really happened here let's take a closer look now clearly there can be many possible causes for product failure but in this case the main observation is a simple one this product should not have been built in the first place let's look at the main reason why when andy talked to customers about the specific problem of customer

8:47 support costs they naturally focused on that specific problem and with that focus on a specific problem they excluded other more important problems that they were facing that is why these customers could never find the time or resources to actually adopt and deploy the product when it was ready here's what andy should have done instead after talking to each customer about the problem of support costs he should have asked them to stack rank

9:17 that problem versus all the other problems they were trying to solve for their business and for their organization that is where the real truth would have emerged this customer problem stack rank acts as a great test of the true priority of the customer problem and the product solution you're discussing if you're more curious about the customer problem stack rank we can discuss it during the q a section anyway back to our story there are at least three biases

9:46 that got in the way of andy chris joe and team here beginning with the focusing illusion first described by daniel kahneman nothing in life is as important as you think it is while you're thinking about it and the interesting thing is that while kahneman was probably talking about everyday life the focusing illusion applies to product management as well it is one of the biases i see quite frequently and it's common because it's largely

10:17 invisible companies that like to think of themselves as customer driven are especially susceptible to it here are some ways this bias can manifest in team discussions i want you to note that no single one of these is by itself a definitive sign of this bias but i suggest keeping your eyes and ears open when these things are said repeatedly by your team the second bias that andy and especially

10:48 chris the cpo fell prey to is the ikea effect for products it comes from the observation that something we built together with our own hands is more valuable to us than if we had to purchase exactly the same item pre-assembled it's the same with the tech products we built and that is why chris was not ready to pull the plug on the product early on even though all the signs were already there with the ikea effect for products we

11:17 base our current product decisions on past product execution because we are too attached to the work we've already put in and don't want it to have been for nothing you'll encounter this effect quite often with products that are straddling that unfortunate zone of not being a complete failure and not being very successful either what's interesting about the ikea effect is that it makes us less attentive to our customers actual needs instead of being rational we try to

11:49 rationalize now time for another story this time a very short one over here the engineering manager is keen to get their eng team started with software development for a new project the problem though is that the product manager isn't yet ready with the requirements but she's also feeling incredibly guilty that engineers are blocked on her somewhere in her training early on she was taught that engineers being blocked on writing code

12:21 is the cardinal sin of product management so what does our hard working product manager do she spends the weekend completing the prd she knows that she should have done more customer research and more competitive research before calling the requirements ready but she promises herself that she'll fix this in version two of the product and we all know how these stories of will do better in the next version end

12:52 they rarely have a happy ending what you just saw is the most common bias i see especially on high energy teams and ambitious startups shane parish said we waste years because we cannot waste hours and that isn't too far from what happens in these situations because we can't stand the idea of taking a couple more weeks to get the product spec or experience right up front we end up building something sub-optimal and now that we have that thing the ikea

13:22 effect kicks in making it much harder to fix our own bad choices later on the bias for building fallacy is most common in organizations that value speed nothing wrong with that per se but if you go speedily in the wrong direction you will end up in the wrong place that's why teams should value velocity much more than speed velocity being a combination of speed and of direction i thought this is a perfect illustration

13:52 for our next bias it's inspired by a scene from the tv show i love lucy what are you doing behind the sofa i've lost my car keys where did you last see them in the kitchen then why are you looking for them in the living room well because the light is better over here now while this may sound like it's made just for tv i will tell you that something similar happens in product development all the time i call this the execution

14:22 orientation fallacy where we're so focused on execution that we craft flawed product strategies and make sub-optimal product decisions because we base these decisions on what we know we can easily execute today we avoid things that seem hard to execute even if they will make a major positive impact this is especially common in product cultures where shipping something anything is lauded and rewarded

14:52 when i talk to product leaders and startup ceos the execution orientation fallacy comes up all the time they want their teams to think bigger to not be phased by current limitations they want their product managers to exhibit higher agency and ambition now on to our next bias this one's named after abraham maslow who needs no introduction one relatively little known aspect of his legacy is called maslow's hammer if the only tool you

15:24 have is a hammer you will treat everything as if it is a nail maslow's hammer manifests in product development when we use a process of framework or a skill that we are most comfortable with or one that our company endorses say via core values or it's mission or operating principles even when its utility is limited or perhaps even counterproductive in a specific product context

15:56 consistency process being data driven are often used as hammers at many companies now i don't want you to take away from this that data is bad or process is bad my main point is that instead of thinking about these things in a very binary fashion either use data to justify all product decisions or no product decisions the context is what's supremely important here our decisions must first and foremost be

16:26 informed by the current context not by a tool a process or a best practice that may have worked for us in a different context two more biases to go we often encounter situations where another company's product is a fiasco and when we look at such situations we might ask ourselves what was that product team thinking how did they miss something so big so harmful and so obvious

16:57 and the answer is that such teams are likely affected by this next bias russian roulette for products in it we don't account enough for scenarios that could under certain conditions lead to a catastrophic outcome for our product or a pr nightmare for our company russian roulette for products is especially common in companies where moving fast and breaking things is viewed as a virtue and companies where following orders is expected and rewarded

17:28 pre-mortems are a good antidote for russian roulette for products and if you're interested in learning more about pre-mortems i'm happy to cover them in the q a section and lastly the authority approval bias where we devise products and plans based on what's going to be easier to get us approval or confirm the pre-existing beliefs of an executive we aim for a successful product review meeting which comes at the expense of a successful product

18:01 this is the easiest to understand and explain because most of us have vividly seen this at some point in our career whether it's aligning a project with the ceo's pet project just to get funding or team members not bringing up important facts concerns and ideas with the fear of triggering an executive at the company so there you have it you now know what i've observed across many companies of all sizes over the past 15 years and what took me the last four years of

18:33 introspection and research to really understand the last thing i'll say before moving to the final section of my talk is that people will at times point out to me a situation where they saw one or more of these biases and yet the product was extremely successful and they asked me what i make of it to which my response is surely successful products can emerge from severely sub-optimal environments but that's just survivorship buyers at play if you start with a sample size of

19:04 hundreds of products surely many more than a handful will be successful despite all odds but remember as individuals each of us has finite time we have one life one career one team and it's much wiser not to play that lottery for the one two three or five products that we may be responsible for instead we should aim to maximize odds of success from day one it's the same reason why we prefer being in cars that have seat belts and airbags it's

19:35 the smart thing to do in this last part we'll look at the three main ways that we can combat these biases as product managers and as leaders within a product organization and with as most things in life we need to start with self-awareness we need to recognize that these biases are real and that each of us is susceptible to them if we sense ourselves falling prey to one of them that doesn't mean we are stupid

20:06 that recognition is in fact the key that unlocks greater wisdom second name it to tame it make your teams more aware of these biases by name add these names to your team's shared vocabulary so you can create mutual awareness and accountability while ensuring psychological safety for everyone it's much easier and safer for people to be able to say that we might be falling prey to the ikea effect here than to say that we're not thinking

20:37 straight about our priorities that is the power of this shared vocabulary it helps us see reality and talk about it objectively with each other and as they say the team that sees reality best wins and we must remember that lasting org-wide change cannot come through surface level treatment that's why i love this quote from taylor swift indeed band-aids don't fix bullet holes that's why the best long-term solution

21:08 is culture change and the ultimate onus for avoiding these traps is on pm engineering design leaders if our product decisions will make or break our company doesn't it make sense to have core values and operating principles that counteract these product biases so that we can make better product decisions so friends i'll leave you with perhaps the most important part of my talk today a view of how each of these biases

21:41 you saw can be counteracted with an operating principle you can establish and depending on the scope of your role you can do this at the level of a single team a single org or the entire company to counteract the focusing illusion we need deep customer empathy we need to really understand their biggest problems and work towards solving those big problems to counteract the ikea effect we need a culture of rigorous thinking or reasoning from first principles

22:13 to fight the bias for building fallacy we need to recognize that sheer speed doesn't mean much unless it's in the right direction that's why velocity is what really matters the antidote for the execution orientation fallacy is to cultivate high ambition and high agency to find a way to get what you want without waiting for conditions to be perfect or otherwise blaming the circumstances to counteract maslow's hammer we need to recognize that context comes first

22:45 the same tool can either help us or harm us and the way we decide is with an appreciation of our current context to avoid getting killed in a game of russian roulette is to understand that preventing problems is better than having to do heroics to save the day afterwards and lastly leaders should counteract the authority approval bias by fostering constructive dissonance which is about going after the right answer and not to just to try to prove ourselves

23:17 right note that this need not be one size fits all you likely don't need each of these operating principles that's where context is important so i urge you to identify which biases your teams are most susceptible to and adopt the corresponding operating principles accordingly and with that i hope this session is helpful when you go back to your work tomorrow when you're making major prioritization decisions for 2021 annual planning and

23:50 when you're devising a new product strategy next year but perhaps even more importantly i hope that our shared journey today will help you build happier and more capable product teams and for that i wish you all the best thank you very much for your time