As part of the product manager hat, you need to explore and find out what product should your company build. This falls under the first of the 2 broadest umbrella of Product: Product Discovery and Product Delivery.

Once you have found a problem and got a solution in mind, you need to test for validation. You do not want to build something that you think people want but actually would not pay for it.

In this article, we will talk about:

1. What types of risk do you need to consider for every product idea?
2. How do you test these risks?
3. How can communicate your data and key insights for a product commitment?

Source: reforge

1. What types of risk are there?

Every type of prototype has a purpose of test. It can be either value, usability, feasibility, viability. Depending on what you wish to test for, derive the level of fidelity you need to prototype. Also, for complex products, you can use prototype as specs to show engineers what you are going for

Types of risk you need to check for

1. Value: Will the user or customer choose to use or buy this?
2. Usability: Can the user figure out how to use this?
3. Feasibility: Can we build this?
4. Business Viability: Is this solution viable for our business?

Start your idea validation down the order of value, then feasibility before you go to business viability. You want to make sure you have data to go back to business stakeholders on whether this idea will work with how much confidence.

Always align on the problem that you think your users have today. That is your biggest assumption in which all your solutions will revolve around.

In your usability test, one thing before you jump into the tasks: see if they can tell from the landing page of your prototype what it is that you do, and especially what might be valuable or appealing to them - this is a first-time visitor context, which you can only get from first time visitors to your website.

Keep your users in use-mode, and not critique-everything mode. In usability tests, what matters is wheterh user can easily do the tasks they need to do, not their opinion on how things should be positioned. Do not ask: “what is the 3 things you will change in this page”, watch more on what they do than what they say. Be comfortable with extreme silence during the user test.

Why do you want to test for value?

One of the biggest possible waste of time and effrot, and the reasons for many startup failures, is when a team designs, and build a product. But when they finally release the product, they find that people won’t buy it.

How do you test for value?

• Qualitatively: focuse on the response or reaction of the customer. Do the customers love this? Will they pay for it?

Tells us why something is happening and what to do to correct the situation.
This is about rapid learning and big insights, not to prove anything. When you talk to users, you don’t get your answer from any one user, you talk to every user like another piece to the puzzle. You need to have enough pieces to understand where you’ve gone wrong.

Expected number: 2-3 qualitative value test every single week.
How do you do it?

1. Interview first

short user interview, make sure (i)the user has problems we think she has. (ii) How she solves these problems today, and (iii) what it would take for her to switch (customer interview techniques)

You have to do usability test before you can do value test.
the user has to first understand what your product is and how it works. After the usability test, the user should know what your product is all about and how it’s meant to be used. Then can we have a useful conversation about value

The magic happens when you have at least an engineer to be in the usability test sessions. When you are focusing on value test, you need a high fidelity user prototype.

Users can pay for your product with a few ways:

• Money
• Reputation (are they willing to promote your product to their colleagues or contacts?)
• Time (are they willing to spend significant time with you to work on it.
• Access (are users willing to provide you their access credentials for a switch there and then)

If you have two drastically different responses to the same product, question what could be different. Different type of customers? Different type of problems? Different type of users with different skillsets or domain knowledge?

• Quantitatively: test efficacy, how well this solution solves the underlying problem. In ad tech for example, you can test the click rates, generate generated.

Tells us what is happening.
Used to prove any hypothesis of value by collecting evidence.
You want to collect sufficient evidence to be statistically significant results for b2c features with high traffic.

Gold standard for quantitative testing is A/B testing. The user doesnt know which version of the product she is seeing, and this yields data that is very predictive. Note this is not the same as optimization A/B testing where we experiement with different calls to action, different colors on a button. Opt testing is normally surface-level, low-risk changes which is often tested in a split test (50:50)

In discovery testing, we should 99% the original product but only 1% the live-data prototype with tighter monitoring.

5 Key users of analytics in strong product team

1. Understand User and Customer behaviour

Can be used to identify features that are not being used
confirm features are being used as expected
Gain better understanding of the difference between what people say and they actually do

2. Measure Product Progress

3. Prove whether product ideas work

4. Inform Product decisions

Spirit of data beats opinions.

5. Inspire Product Work

Types of analytics

• User behaviour analytics (click paths, engagement)
• Business analytics (active users, conversion rate, lifetime value, retention)
• Financial Analytics (time to close)
• Performance (load time, uptime)
• Operational cost (storage, hosting)
• Go-to-market costs (question cost, cost of sales, [customer discovery] programs)
• Sentiment (NPS, customer satisfaction, surveys)

How do you test for demand?

• If users do not even want to sign up for trials, its a fatal problem.

Fake door demand test

You put the button or menu exactly where you believe it should be. But when the user clicks, instead of taking the user to the new feature, it takes the user to a special page that explains that you are studying the possibility of adding this new feature, and am seeking for customers to talk to about this.

How to make this work? - the users cannot have any visible indication that this is a test until they clicked the button.

How do you test for feasibility?

Questions you should get your engineers to answer:

1. Do we know how to build this?
2. Do we have the skills on the team to build this?
3. Do we have enough time to build this?
4. Do we need any architectural changes to build this?
5. Do we have on hand all the components we need to build this?
6. Do we understand the dependencies involved in building this?
7. Will the performance be acceptable?
8. Will it scale to the levels we need?
9. Do we have the infrastructure needed to test and run this?
10. Can we afford the cost to provision this?

Putting an engineer on the spot without time to investigate and consider, you are likely to get conservative answers, partly designed to make you go away.

Putting the engineers close enough to hear the customer’s pain directly, receiving sentiment response from the customers, the engineers will probably be considering the issues for some time. Don’t ask engineers can you do this? Ask them, what’s the best way to do this and how long will it take?

How do you test for business viability

Use Pilot Team technique to bring out technological changes to your organization. The core objective of pilot team is for comparison. How well do the pilot teams accomplish their objectives vs the others or compared to how they did in the past?

Ideally have people who are:

• open to new ways
• people who are co-located to reduce the complexity of timezones or logistical issues
• and ensure the teams are largely in control of how they work and not dependent on other teams that still works in the old way

General stakeholder management Tips

Who are your stakeholders? Anyone who have the power to veto your product from getting to the users.
They could be your management team, Business Partners, Finance, Compliance team, Legal team, BD (ensuring whatever you propose do4s not violate any existing contracts or relationships)

Build trust by understanding their constraints and concerns. Understand our customers, the analytics, technology, industry, and business. The best way to convey understanding is to share openly what you understand of their constraints or concerns.

Do not show stakeholders the solution after you have built it. Show them the solution at discovery stage. Align your understanding of their concerns at discovery stage. If there are opposing opinions between stakeholders and PMs, don’t have to give in because of seniority. Take it as a sign to gather more evidence and consistently move the conversations from opinions to data.

PPTs are terrible for testing business viability. High fidelity user prototypes are ideal, and do not trust a sign off on anything other than a high-fidelity prototype.

Do not discuss products in a group setting. At least it will only yeild mediocre results. Meet privately with each stateholders and show them the high-fidelity prototype, give them the chance to raise any concerns.

To forge a strong product culture in your company, it will then take more than a few people to bring it down. Talk about a product driven culture from onboarding. Let your new hires know they are joining a different type of company that takes pride in how they work and in using best practices.

Talk about this culture explicitly in the interview process. Emphasize to them that the new company is interested in them because of their mind and their ralents and of course they wouldnt want to bring with them the bad practices of their former company.

How do you communicate product learning with your team?

What is a good product team?

• Good team have missionaries like passion pursuing product vision.
• Good team set their inspiration and product ideas from vision and objectives. from observing customer struggles from analyzing the data cusomters generate from using their product. From seeking to implement new tech to solve real problems. Bad teams gather requirements from sales and customers.
• Good team understasnd who each of their key stakeholders are, understands constraints of biz units, committed to invest in solutions that work for users, customers, and within constraints of biz. Bad teams gather requirements from stakeholders - no thinking on their parts.
• Good teams are skilled in many techniques to rapidly try out product ideas to determine which ones are truly worth building. Bad teams hold meetings to prioritize roadmaps.
• Good teams love to have brainstorming discussions with smart thought leaders from across the company. bad teams get offened when someone outside their team dares to suggest they do something.
• Good teams have product, design, and engineering sit side by side, embrace the give and take between functionality, and the UX, and the enabling technology. Bad teams sit in their respective silos and ask that others make requests for their services in the form of documents and scheduling meetings - no innovations can happen in such formats can they?
• Good teams are constantly trying out new ideas to innovate, but doing so in ways that protect the revenue and protect the brand. bad teams are still waiting for permission to run a test - if you want to test, test it rapidly.
• Good teams insist they ahve the skillsets on their team, such as a strong product design, necessary to create winning products. bad teams don’t even know what product designers are.
• Good teams ensure that their engineers have time to try out the prototypes in discovery every day so they can contribute their thoughts on how to make the product better. Bad teams show the prototypes to the engineer during sprint planning so they can estimate. There is no thinking together. Thinking of designs together are important to create missionaries.
• Good teams engage directly with end users and customers every week, to better understand their customers, and to see the customer’s response to their latest ideas. Bad teams think they are the customer.
• Good teams know that many of their favourite ideas won’t end up working for customers, and even the ones that could will need several iterations to get to the point where they provide the desired outcome. bad teams just build what’s on the roadmap, and are satisfied meeting dates and ensuring quality. Not saying those are not important but rather not more important than actually seeking to create a winning product.
• Good team understand the need for speed and how rapid iteration is the key to innovation, and they understand this speed comes from the right techniques and not forced labor, bad teams complain they are slow because their colleagues are not working hard enough.
• Good teams make high integrity commitments after they have evaluated the request and ensured they have the viable solution that will work for customer and biz. Bad teams complain about being a sales-driven company.
• Good teams instrument their work so they can immediately understand how their product is being used an dmake adjustments based on data. bad teams consider analytics and reporting a nice to have. How then do you measure the usefulness of your product, the traction, pick up rate of your product? How do you know you have actually built something users want and are using heavily?
• Good teams integrate and release continuously, knowing that a constant stream of smaller releases provide a much more stable solution for their customers, bad teams test manually at the end of a painful integration phase and then release everything at once. Once a week is good enough. Keep to it with the quality control.
• Good teams obsess over their reference customers. bad teams obsess over their competitors. Know your competitiors moves, but know thy customers.
• Good teams celebrate when they achieve a significant impact to the biz results. Bad teams celebrate when they finally release something - Good teams keep releasing until they have gotten a product that has value to the business.

Top reasons for…

Top reasons for loss of innovation

1. Loss of customer centric culture. Having that direct and frequent contact with them.
2. Compelling Product Vision.
3. Focused product strategy
4. Strong Product Managers
5. stable product team
6. Engineers in discovery
7. corporate courage
8. empowered product teams
9. Product mindset - serve the company’s customers in a way that meet the needs of biz
10. Time to innovate

Top reasons for loss of velocity

1. Technical debt - consider the evolution of your architecture
2. Lack of strong product managers
3. Lack of delivery management
4. infrequent release cycles - good teams release multiple times a day.
5. Lack of product vision and strategy
6. Lack of co-located, durable product teams.
7. Not including engineers early enough during product discovery - helps to think of cheaper solutions to the same problem
8. Not utilizing product design in discovery and instead having them try to do their work at the same time the engineers are trying to build.
9. Changing priorities
10. A consensus culture

How to establish a strong product culture

1. Culture of urgency => speed is highly prioritized
2. Culture of high-integrity commitments
3. Culture of empowerment => teams feel they have the resources, tool and permission to do whatever necessary to meet their commitments
4. Culture of accountability => feel a deep responsibility to meet their commitments
5. Culture of collaboration
6. Culture of results => is the focus on output or is the focus on results?
7. Culture of recognition => teams often take their cues from what is rewarded and what is accepted.

Is an innovation culture in any way inherently at odds with an execution culture?
Does a strong execution culture lead to a stressful (or worse) work environment?
What types of people, including leaders are attracted to and needed for each type of culture?

• Places with strong execution culture are pretty tough places to work at.

TLDR;

Product consist of 2 main umbrella: Product Discovery & Product Delivery.

As an engineer, you find yourself delivering on clearly written specs based on the business team’s wants or demands. But in the space of product, we know proceed one step prior to the specs of wanted products. We ask ourselves, why do we want to build this product? What problem are we trying to solve with this new product?

We learn to move with purpose, and to do that, you need a collective effort moving towards the same direction; a product driven direction.

There is really just so much to learn.

Broadly, You have to find the right people

1. Principles of Strong Product teams

2. The product manager’s role and responsibilities

• Deep knowledge of the Customer
• Deep knowledge of the Date
• Deep knowledge of your Business
• Deep knowledge of Your Market and Industry
• Smart, Creative, and Persistent

3. The Head of Technology’s Role in a Product Driven company

Broadly, you have to build the right Product

1. Product Vision

• Principles of product visions

2. Product Strategy

• Principles of Product Stragey

Broadly, you need the right Process

1. Product Discovery

• Principles of product discovery
• Discovery Techniques

2. Prototyping Techniques

Broadly, you need the right Culture

And we managed to cover some of the principles of culture here in this article.

Reference:

• Reforge