Do You Answer The 5 Most Important Questions In Product Management?
Ask the Tough Questions First, And Then Start to Build
Nobody Knows The Answer
Let’s be honest. We don’t know that much. There are a lot of smart people with a lot of smart ideas, most of which fail. This is one of the hard truths we must accept. Most ideas will fail. This is especially true for product management. It operates in an environment of extreme uncertainty. There are so many unknowns and dynamics in play that it’s impossible to expect a product’s success. So, should we always work on leap-of-faith assumptions? No. We must test our ideas and prepare ourselves to be proven wrong. And that is not easy. It stings when we are wrong. But, we have to get over it. Product managers can’t have that pride, they can’t assume to be always right. Product managers have to accept that ideas are just ideas. And they must prove they can keep their promises.
Five Critical Assumptions
We base almost everything we do on assumptions. Some assumptions have a very good foundation. They are either based on experience or even statistical data. But they are still assumptions. We often assume that the user will see the value in our latest feature or that it’s obvious how to set up this new app. Sometimes we are right and sometimes we are wrong. But, there is always room for improvement. There are five critical assumptions we must test as product managers:
Value risk
Usability risk
Feasibility risk
Business viability risk
Ethical risk
The success of our product efforts depends on the certainty of our assumptions. We must work hard to identify our assumptions and get actual data on them. This is what Product Discovery is all about. We need more than a gut feeling or consensus. We need evidence! How much evidence? This is the product manager’s call. There is a way of overdoing it. But, the bigger risk is not doing enough. This is certainly true for me. There are a lot of reasons not to do discovery: “Not enough time”, “Not enough resources.”, or “Not necessary, it’s obviously right what we’re doing.”. None of those reasons outweigh the importance of product discovery. Product discovery will lead to less waste, less support, higher customer satisfaction, and many more. In short, it will make your product better. And I still have trouble following through with that advice myself. So, for you and also for myself I summarized the critical product risks below.
Value Risk
Will the customer choose to use this? This is arguably the most important question in all product management. If people don’t need what you are offering them, you have failed. You have wasted time and resources. It doesn’t matter if you are developing a new product from the ground up or if you are releasing new features to an existing product. You have to spend time validating the perceived value for the user. If they don’t see any value, they will not buy your product, they will not use that feature.
Asking For Approval Is Not Enough
People are nice. If you propose an idea and you are passionate about it, they will be supportive: “Yeah, that sounds like a great idea.” or “I can’t wait to use that product.”. That is very nice, but also deceptive. They don’t know that they are deceiving you, but in most cases they are. So, it is not enough to ask for approval of an idea. Often, existing users come up with ideas. They request a feature they are missing that would improve the product. You might think: “Great. The value is there, otherwise, our user wouldn’t ask for it!" But this is also deceptive. People often don’t know what they want. There is a great mantra: Listen to your customer, but don’t listen to your customer. What does that mean? Feature requests give you a great insight into your customer’s world. They hint at a problem your customer has. Don’t fall into the trap and stay in the solution space. Dig deeper and figure out the problem the customer is trying to solve. Why is there a need for this feature? This is also true if your biggest customer “demands” a feature.
Users show real value not by words, but by actions. So, if you want to test the value of a new product or a new feature you must look at what people do, not what they say. There are many ways to test for value. You can be creative, but you have to be aware of the space your product is operating in. You should never jeopardize your revenue, brand, customers, and employees. Following are a few examples of value testing techniques:
The Fake Door Demand Test
A very low-effort test. You put a button or menu point for the new feature exactly where you believe it should be. When users click on it, they are redirected to a page announcing this feature. The necessary analytics should be in place. You need to analyze how many users found this “new feature” interesting enough to click on it. Additionally, you can add a simple form on that page where you invite the users to take part as early testers of this new feature.
This test can also apply to new products. In that case, you would design a landing page for the product that looks like the real deal. But when users want to sign up for a trial period they are redirected to a landing page. Here you explain that you are currently evaluating the need for this type of product and ask if they are willing to talk to you.
Customer Interviews
Customer interviews are the single most important way to confirm your value proposition. But, you have to do it right. Customer interviews are not meant to be a quantitative test of value. You don’t have to ask hundreds of people the same questions to have a meaningful sample size. Each customer interview is a new puzzle piece to get a more complete picture of the problem. And again, it is not enough if the customer is only talking. You need to see them interact with your idea. So, there must be a usability test that precedes the value test. The usability test can be performed with various prototypes. Wireframes, interactive prototypes, or high-fidelity prototypes with fake data are all possible. It all depends on the scope of your value testing. Yet, low-fidelity prototypes are not meaningful enough to get the most out of the value testing that follows the usability test. There are multiple ways the user can demonstrate value to you (remember, only saying it is not enough!).
Ask For Money
Nothing shows perceived value more than the amount of money someone is willing to pay. It has value if they are willing to pay for it on the spot. Some products would be too expensive to put on your credit card. In that case, ask them to sign a non-binding letter of intent to buy.
Ask For Their Reputation
We value our reputation. We care about what people think about us. If the user is willing to share this product with his friends and family he sees real value in it. Providing the email addresses of colleagues is also a good sign of perceived value.
Ask For Their Time
Everyone is busy. Only very few people have too much time. So, if the user agrees to spend significant time helping you with your product, it’s a good sign of value.
Remember, this is for a qualitative test. You don’t have to show the same prototype to all interviewees. The moment you learn something, apply this learning and start a new iteration. The faster you learn, the earlier you can provide real value.
Usability Risk
Can the user figure out how to use this? The greatest value is worth nothing if it can’t be realized. If the user can’t find the new feature or is unable to use the new product, all your efforts will have failed. First, you need to understand the workflow of your users. Imagine how they would use the new feature or product. Figure out the user’s main activities and don’t get lost in minor workflows.
Most of the time, the product manager, the product designer, and (if possible) an engineer should be present for the usability test. Some engineers won’t be forthcoming when it comes to being available in those situations. But, try to rotate through all the engineers in your product team. Product discovery is a team effort and is most efficient when all product team members take part. Follow these general guidelines to get the most out of your usability testing.
Make the Subject Feel Comfortable
The subject has to feel comfortable. If they don’t act normal you won’t get the insights you are seeking. So, the choice of location is important. If you can make the time, it is always valuable to meet the subject on their home turf. There are so many observations you can make. How big is their screen? What kinds of distractions are present? How fast is the internet connection? It is also the most convenient way for the subject to take part in your test. Depending on the product, public locations are also possible. A nice café is a good choice. It’s a relaxed atmosphere and offers enough space to do some testing. A lab environment built for test studies is not a good choice. It makes the subject feel observed. A second important point is that the subject is not being tested. She cannot fail, only the prototype can. This must be clear and is a good starting phrase when the product test is about to begin.
It’s Not a Product Demonstration
You know the purpose of the usability test is to learn about your current design decisions. Yet, we often fall back into the mode of presenting our product. This is not a product demonstration! We don’t want to highlight why the subject should use our product. This is not the purpose. To keep ourselves out of the presentation mode and in the observation mode we only have to remember one thing: Keep quiet! The subject has the space to figure things out and we can make important observations. Does the subject go through the workflow without any issues? Are there some parts of his journey that feel clunky? Or is she even giving up because she can’t figure out what to do next?
Know When to Speak and What to Say
You should keep quiet, but there are some moments when you can speak. But, don’t give any help. If you see that the subject struggles to find the button or the menu item, don’t tell her where to find it. This robs you of important insights into your design. It’s like in a courtroom, don’t lead the witness. What you can and should do is act like a parrot. Typically, the subject will be rather quiet. But, whenever she says something, repeat it like a parrot. For example, she goes: “Does it save my settings when I click next?”, and you answer: “So you wonder if your settings will be saved?”. This has many advantages. The subject stays in the doing state and doesn’t transition into an asking state. You don’t want to answer questions about the workflow. You want to learn if the subject can figure it out. It also helps you to stay out of demonstration mode and keep observing. And last, it gives you (or your colleagues) time to document the observations.
There are moments when you can speak to the subject. When you are not sure what the subject is trying to do, ask them. The answer can contain valuable information.
Feasibility Risk
Are we able to build it? This is often the first question that comes up and the first question we try to answer. And unfortunately, it is often the only product risk that is evaluated. The questions about feasibility come naturally, especially to engineers. Do we have the skills to build it? Does our current architecture support this feature? Will it scale? Are we aware of all dependencies? Most of our product efforts don’t lead to many open questions about feasibility. Our engineers already built something similar and know what they need to consider. But with innovative technologies, a lot of uncertainty comes into play. This could be novel machine learning algorithms or bigger infrastructure topics like using Kubernetes.
When it comes to feasibility, product managers must be aware of a few things. Engineers can’t always give a direct answer to all technical questions. Often, they do know the answer because the question doesn’t stretch to technologies they don’t know. But, sometimes engineers need to investigate. Don’t push for an estimate on the same day you ask the question. Let them check, read the documentation, and think things through. Otherwise, they will give you an answer (a very conservative one) to get rid of you.
Business Viability Risk
Does this solution work for our business? Value risk, usability risk, and feasibility risk are all important. The business viability risk is of another nature. It doesn’t reflect the product’s general quality. It takes the specifics of the organization into account. This is the product manager’s responsibility. The product manager is the key communicator to all other departments that are affected by the product. No stakeholder should be surprised by your most recent product developments. Even worse, no stakeholder should block the shipping of your new features or new products. Nothing is more demotivating for the product team than to find out that contractual obligations negate all their efforts. Typical stakeholders are marketing, legal, sales, business development, customer service, finance, and of course the executive leadership.
Testing the business viability risk is a matter of keeping everyone in the loop. This is easier if you know all the relevant constraints of your stakeholders.
Ethical Risk
Should we build it? This is the final risk that you should check. Ethics is a difficult playing field. Don't confuse it with politics. I’m a big supporter of this consideration. Modern technology distributes a lot of power and responsibility to those who develop new products. Our products can have a profound impact on the happiness, health, and security of people, to only name a few. This responsibility should not be neglected but embraced. It is motivating for the product team to see their values represented and heard. Bringing these issues up can shape an organization’s culture. This is a good way to have a workforce full of missionaries, and not mercenaries.
What Are Your Assumptions?
Maybe, product discovery is the core of how your product team is working. That’s great. For all others, try to figure out what your assumptions are. Write them down. Think about how you can test them. And then go on and do it. Believe me, it is time well spent.