svpg blog

Product Marketing Contribution

Sat, 28 Apr 2012 22:37:00 -0400

Jane is supporting the launch of Product X, a new release her company is really excited about. She is on the marketing team. Armed with her launch checklist, she schedules a meeting with John, the product manager. At the meeting, John answers all of her questions, draws a market segmentation on the white board, and talks about the key features and why they are important. Jane takes lots of notes and asks John to review what she sends him.

The first thing John gets is a press release. The features are mixed-up. There is no positioning – just announcing it’s “now available!” – and the quote sounds like it came from a Web 2.0 robot. John inserts corrections. There are so many it’s like a rewrite.

Next, John gets the copy from the website. His corrections from the press release aren’t in it. The feature descriptions are wrong in an entirely new way because “the copywriter” took a pass. John is starting to worry that Jane and the rest of the marketing team don’t get his product.

John asks the head of marketing, Bob, “Are we all on the same page about the product launch?” Bob assures John, “We’re in great shape.” Bob says this because when he reviews the launch checklist, everything is on schedule. John assumes Bob meant the marketing of his product is part of some grand strategy connecting it with what the market needs to hear.

Assured, John accepts Jane’s next meeting request to “review product positioning.” “At last,” he thinks, “I’ll see it all come together.” Jane shows up to the meeting and asks, “Now tell me how do you want to position Product X and what features do you want us to talk about?”

Does this confusion or frustration feel familiar? You are not alone and probably suffering from the need for Product Marketing.

The Problem

What makes Product Marketing distinct from corporate marketing is it’s specific focus on using products—most technology companies’ greatest asset—to drive market strategy and growth. When a company is a start-up with one product, “marketing” and “product marketing” are often one in the same. The need for it as a distinct discipline becomes much clearer when a company branches into multiple product lines or is highly technical or complex.

The Job

My partner, Marty, wrote elegantly about the necessity to split product management and product marketing into two separate roles http://www.svpg.com/product-management-vs-product-marketing/, and this assumes that model.

I have yet to find a company that practiced the two roles as systematically and well as Microsoft, where I began my career. They recognized that the two jobs generally required different skills and people to be done well. Product management (which goes by the program manager title) defined the products to be built. Product marketing (which goes by the confusing title product manager) made sure the world knew about the products and why they should care.

It’s the “why they should care” part that requires far more product and communication skill than most realize. To hone a compelling strategy and messaging requires deep customer understanding, an analytical mind, broad understanding of the industry and business, and—most important—a really good understanding of the “why” of a product.

It’s insufficient to just know a feature set or the business concerns of a C-level executive. If you get the in-depth “why” for a customer as well as the “why” for a product, you can dynamically position your product based on the specific goal or activity at hand. It’s this skill—the strategic application of product to achieve marketing goals—and the ability to communicate about it in a clear way that’s done insufficiently in many companies and is the domain of the product marketer.

Product Marketing Responsibilities

Good product marketers have command of:

Competitive analysis (what is the competition saying and how is it shaping perceptions in the marketplace?)
Market and customer research (what matters in and beyond our category, what’s the engagement model of the people influencing product conversion or the perception of our product?
Product positioning (what is it, why is it different, and why should the world care?)
Product communications/PR/social media (what can we say that will make others do the talking about us?)
Marketing communications and campaigns – advertising, email, interactive, search-engine, events etc. (when to use each and to what end)
Sales support, salestools (this includes demos and the product knowledge required to build a compelling one)
Vertical/affiliate/evangelist programs
Measurement of marketing programs (ROI and mapping to business goals)

In consumer companies, product marketers focus on enabling activities driving acquisition, activation, retention, and referral.

In enterprise companies, product marketers focus on shaping the product's perception in the business and competitive domain and enabling sales channels.

Product marketers’ work also bridges product and corporate marketing teams, ensuring new product enhancements shape company messaging and strategy.

What to Do?

In all the companies I work with, when we look at product planning through a product marketing lens, the what and when of features inevitably changes. Having product marketers participate in product planning discussions ensures every product effort has the maximum market leverage at it’s back. Remember if the world doesn’t know about a product, believe it is important, or cares, it doesn’t matter how useful or useable a product is.

Even in organizations with existing product marketers (typical of most enterprise companies), the edges of the relationship between product mangers and their product marketing counterparts varies—due to skill differences, product maturity phases or often simply the lack of shared understanding of expectations. Don’t let old habits get in the way of improving roles. Make sure product managers and their product marketing counterparts talk often so both feel invested in and understand market strategy.

Adapt until things work well, as each company and group has to find a groove that works for their teams. The ideal relationship is one in which the product manager feels like her product marketer is an essential part of the team. That’s also how a product marketer knows he is doing his job well.

However you get there, make sure the world knows why your product matters and invest in great product marketing.

Defining Good

Sun, 15 Apr 2012 22:37:00 -0400

I consistently get asked questions like the following: "Just look at Facebook/Amazon/Google (usually one of those three). Don't you think they have a terrible product? How could they possibly be so successful?"

No matter how many times I hear the question, I always have to take a breath before I answer.

I think there are three relatively minor confusions underlying this question, and two very large confusions. But they all get to the heart of what makes great products and great product people. So in that spirit, I thought I'd share my answer here.

First the minor issues:

1. Product people often fall into the trap of confusing themselves with the actual target customers. Just because you or I don't like something doesn't actually mean much, if we're not the target audience.

2. Very often these backseat designers are pointing to secondary tasks that frankly don't really matter. Good product managers and designers know that there are a few critical tasks in every product that need to be exceptionally good, and if this comes at the expense of some secondary tasks, that can be a very good trade to make.

3. Anyone that has been working in this industry for a while, and has actually been running A/B tests, knows that we're very often surprised. What we think would happen with a given design or feature doesn't always work out like that. Unless you have access to the detailed analytics, it's very possible your intuition is wrong. Mine is wrong so often I have come to expect to be wrong. But I've found that helps me much more than it hurts me because I've learned to be very open to the data and to learn quickly.

Now the big issues:

1. How do you actually define good? If you define it by what you personally like, then maybe these sites are not very good. But I define good using the product scorecard. The prioritized set of business KPI's that the business is looking to the product team to find ways to positively influence.

If you define goodness in a social network as the ability to rapidly grow and deeply engage users, its hard to argue that Facebook has been anything less than off-the-charts successful.

If you define goodness in an e-commerce site as KPI's such as conversion rate, average shopping cart value, customer acquisition cost, and repeat use, then it's hard to argue that Amazon has been anything less than stellar.

If you define goodness in a search engine as the number of search results (organic or paid) that the user needs to evaluate before he finds what he is looking for, then it's hard to argue that Google hasn't done an outstanding job.

2. Even when teams focus on the key KPI's for their product, they still often can't see the forest through the trees. What I mean by that is that too many people focus on what's right in front of them, which is typically usability issues. I'm a huge advocate for removing friction cause by poor design, however, I also know that it's mostly about value. If you focus on providing real and sustained value, that will overcome a lot of sins. If you don't provide that value, I really don't care how usable the product is.

The companies I highlighted above provide a ton of value to their users and customers. Sustained value.

All of them make mistakes. All of them work to improve their product every day. But mostly they focus on providing real value to their customers and driving the key KPI's of their business.

That's how I define good, and I'm hoping that's how you define it as well.

Live-Data Prototypes vs. Production

Tue, 03 Apr 2012 18:22:00 -0400

I have written earlier about the differences between user prototypes (simulations intended to test the user experience), and live-data prototypes (actual code intended to send live traffic to in order to test real behavior). See http://www.svpg.com/product-discovery-with-live-data-prototypes/

We use a live-data prototype (usually in the context of an A/B test) to prove something works, and we perform user testing to understand why it doesn't work, and what we can do to correct.

I'm happy to see so many teams now doing live-data prototypes. However, I often find that teams confuse live-data prototypes with production software. The purpose of this note is to clarify the difference.

First, let's be clear for those that do not necessarily know what is meant by the term "production software."

Generally this means that this is software we can run our business on. But specifically this means:

All of the critical use cases have been implemented and tested
The necessary level of test automation is in place
The software has been architected and engineered so that it can work at the necessary scale and performance
The associated SEO work that may be necessary based on the changes
If the product is a global product, the software has been internationalized and localized
Activities like code reviews have been done to ensure that the software is something the team can maintain and stand behind
The sales, customer support and marketing organizations have been briefed on the differences and trained if necessary

Many non-technical people often wonder why it can take real time to go from something that's just working to something that can sustain the business, and in large part, these are the reasons. Especially for consumer internet companies operating at scale, these can be very non-trivial activities.

While I would argue that the items in the list above are critically important, I would also argue that most of these items are not necessary for the purposes of a live-data prototype, and in fact work against our pursuit of minimum viable product.

Remember that the general principle of product discovery is to find the fastest, cheapest way to validate our hypothesis. We use a live-data prototype to prove that our idea works. Anything beyond that purpose will very possibly prove to be waste.

Confusion about the difference between a live-data prototype and production can lead to a couple significant problems:

The first issue is that if the team is not very clear that they are building a live-data prototype rather than production, the team often over-engineers their live-data prototypes. This is bad because it hurts our ability to iterate quickly and can lead to significant waste. So we must be very clear with our product team and with the items on the backlog, whether this is normal production software quality work, or whether this is a live-data prototype.

The second issue is that because the live-data prototype is live in the sense that there is something on the site that some people can see and use, others in the company can easily become confused as to its status. Is this something we should be selling? Is it something we should be advertising? Did we even agree to do this? Companies, especially those with direct sales forces, can end up selling something prematurely. Nobody wants to go back to a customer and explain that it was all a big mistake. We must be very clear with the company when something is simply a live-data test, and when it is something we can sell and support.

So clearly there is a big difference between a live-data prototype and production software.

While I don't want you to think of live-data prototype as anywhere near as expensive to build as production software, I also don't want you to view it as trivial.

It must work enough to handle the live traffic. It must have enough analytics in place that we can understand how it is being used to measure it's success. It must run well enough that we're not causing problems for our customers. As just a rough guide, I find a live-data prototype to be somewhere between 20% and 50% of the work of building production software.

There is one special case I run into occasionally in certain types of companies such as e-commerce. Sometimes the performance of an idea is essential to accurately assessing whether or not the idea actually works. In other words, it might be an otherwise very effective idea, but due to slow performance, the benefits are overshadowed. In this case, some amount of the performance engineering may need to be done during the live-data prototype work in order to ensure a fair test of the idea.

One final note. Remember that if your live-data prototype proves to be a good idea, the team will still need to "productize" the live-data prototype, which means to turn it into production quality software. So you'll need to plan on that work. The good news is that you'll have the evidence and confidence that this hard work will not be wasted.

The Product Manager Contribution

Sun, 18 Mar 2012 20:19:00 -0400

Recently I was asked by a very smart CTO: "I understand the need for a great user experience designer, but if we have a strong designer, and that person is paired with a strong technology lead, do we really still need a product manager?"

For many teams, the contribution of the product manager is not at all clear. Either because they simply don't see the contribution because it's not happening, or because they don't understand the significance of what is being done.

In this article I'd like to spell out very clearly what the product manager needs to contribute if they are to carry their weight on a product team.

But first I need to provide a couple caveats:

First, It is absolutely possible that the technology lead or the user experience designer is sometimes capable of covering the role of the product manager as well, and there are some real advantages to this, but most people, once they understand what's really required, usually back off of trying to cover two roles themselves.

Second, in an early stage startup, the product manager role is most appropriately played by the CEO or one of the founders (see http://www.svpg.com/the-ceo-as-head-of-product).

That said, most teams know and acknowledge that the product manager is the product owner in the Agile sense, and as such the main deliverable they are responsible for is the backlog (what the team will build). However, to some the backlog may just appear as a simple list of user stories. It can look like just a few minutes work, where the hard part is in designing and building out the stories.

Sadly, in many teams, they are right. The backlog is generated by someone that is ill-equipped to succeed. So how is it that the product manager can make the many choices necessary to come up with a strong, prioritized backlog?

Let's look at what goes on when the product manager is skilled and doing the job the team needs and deserves.

The product manager must contribute five essential ingredients:

1. Deep Knowledge of the Customer

The foundation of product is deep knowledge of the actual users and customers - qualitative and quantitative knowledge. The product manager needs to be the acknowledged expert on the customer. This is what informs the many decisions that must be made. Without this deep customer knowledge, the product manager is just guessing. This means both quantitative learning (to understand what they are actually doing), and qualitative learning (to understand why our users and customers behave the way they do). See http://www.svpg.com/product-manager-vs-product-owner/.

2. Deep Knowledge of the Business

The second critical contribution is a deep understanding of your business. This means understanding the needs of your various stakeholders (how your business works); your business model (how you make money); the dynamics of the ecosystem you operate in; and certainly the economics of your product. The product manager does not usually have P&L responsibility, but they absolutely are responsible for understanding the financials. See http://www.svpg.com/make-a-friend-in-finance/.

3. Deep Knowledge of the Industry

The third critical contribution is deep knowledge of the industry you are competing in. This not only includes competitors, but also key trends in terms of technology, customer behaviors and expectations. Your industry is constantly moving, and we must create products for where the market will be tomorrow, not where it used to be yesterday.

4. Reference Customers

The above three contributions are what allows the product manager to make informed decisions. But it's not enough. The product manager needs to take personal responsibility for delivering the initial set of reference customers. This means knowing them personally, and getting them to the point where they don't just like the product concept in theory, but they love it and use it in practice. See http://www.svpg.com/charter-customer-programs/.

5. Motivation

Finally, because product teams are comprised of people, and because the motivation level of those people can easily make or break the product (especially the quality of the product, and the velocity of the team), the product manager must contribute a genuine passion and enthusiasm for the product. This means sharing customer's pain and sharing knowledge of the customers, the business and the industry so that the team is both able to help, and inspired to help. See http://www.svpg.com/product-evangelism/.

Certainly product managers need other skills and knowledge to do their job well (see http://www.svpg.com/developing-strong-product-owners), but these five contributions are what you absolutely need to bring to the team.

If you are a designer or a technology lead, and you've been asked to cover the product owner role as well, then this is what you need to sign up for.

One final note. In certain companies, there is so much in terms of business and industry knowledge, the product manager may be supplemented with what are called "domain experts" or "subject matter experts." Examples of this would be companies building tax software, or creating things like medical devices. In these cases, you can't expect the product managers to have the necessary level of depth in addition to everything else. But these cases are fairly rare. The normal case is that the product manager needs to have the necessary domain expertise.

MVP vs. Minimal Product

Wed, 07 Mar 2012 13:06:00 -0500

To continue on the series of articles describing the critically important concept of Minimum Viable Product (MVP), in this article I wanted to contrast this concept with what I call "Minimal Product."

So often I meet teams that tell me that they have identified their MVP, and I ask them how they have validated this, and I'm told that they've tested their prototype on customers and the results are excellent. But then I ask them how they tested MVP, and they proceed to describe a series of tasks that they measured whether or not users could successfully use the product.

Essentially, they have determined that their proposed product works. Or at least, it could work if the customers either choose to use it, or somehow had to use it (e.g. their employer made the purchase decision).

Unfortunately, there are countless products that would work if people choose to use them, but the problem is that people don't choose to use them. Maybe because they don't like the pricing. Maybe because they are happy with what they use today. Maybe because it is very hard for them to switch even if they wanted to. Or any number of other reasons why they don't buy the product.

This is why I consider the most important question not "could you use it?" but rather "would you use it?" (see http://svpg.com/the-most-important-thing/).

I use the term "Minimal Product" to refer to this smallest possible product that actually works. Unfortunately, that's often a long way from Minimum Viable Product which not only works, but is something that people actually choose to buy and use (see http://svpg.com/minimum-viable-product/).

So often teams confuse the two, and then they're surprised when the product doesn't sell. Or they'll blame marketing and sales.

Moreover, it is this "viable" part that's usually the hardest part of the equation, and is the highest risk, so it's where you need to focus most of your time in discovery.

Social Contract for Product Managers

Fri, 17 Feb 2012 13:13:00 -0500

If I were starting my career in product today, I would do anything I could to get into a very innovative program at Stanford called the Stanford Design School (aka "d.school"). I absolutely love the curriculum and the faculty. But this article is not really about this program.

One of the most creative minds I've ever had the good fortune to work with is Michael Dearing, and he co-teaches a graduate course called "Launchpad."

A friend of mine told me about the "social contract" that the instructors require every student to explicitly agree to as part of the application to get accepted to this special course. Here is the heart of that contract:

    •    We will not complain about the workload.
    •    We will launch our product / service in the market.
    •    We will think with our hands through rapid prototyping.
    •    We will use both sides of our brains (creative and analytical).
    •    We will approach every activity with an open mind and a beginner's mind.
    •    We will help design and iterate LaunchPad this quarter and in the future.

When I read it I thought that this was a remarkably good characterization of the commitment and mindset required to be a very strong product leader. If we were to rephrase the last item to "We will strive to continuously improve how we create and launch products" then we've got a pretty strong social contract for strong product leaders.

I've written earlier about the skills of a strong product leader. I think this social contract complements this list of skills well by emphasizing the commitment and mindset necessary for creating anything of real value.

I'm hoping to see many more multi-disciplinary programs like the d.school start up around the world, and I'm especially looking forward to the products the graduates will create.

Product Management Then and Now

Mon, 23 Jan 2012 09:09:00 -0500

Occasionally in my work with technology product teams around the world, I run into product managers that are still practicing the role as it used to be defined back in the PC era of technology. These organizations are inevitably frustrated, as the role was not terribly effective and often not respected.

There are many possible reasons why these organizations have never moved forward. Perhaps the leaders are simply perpetuating what they learned many years ago. Perhaps the organization received "training" from one of the many non-technology firms that try to apply their models of the past to Internet-era companies. Perhaps the old role has been institutionalized in a formal corporate product development process.

In any case, after I explain the new role to the team, I find that it sometimes helps to highlight the key differences.

I think this probably works better in person, but I want to try this in written form. Let me say up front that this is a little bit exaggerated (but not much) to shine a light on the key behaviors.

Organization:

Old: Marketing
New: Product (Product Management plus User Experience Design), a Peer to Technology and Marketing

Education:

Old: MBA
New: Computer Science or User Experience Design

Spends Days:

Old: Writing Requirements Documents
New: Product Discovery / Pursuing Minimum Viable Product

Learns About Customer Behavior:

Old: With Focus Groups
New: With User Testing and A/B Testing

Makes Case For Project Funding Based On:

Old: A Business Case
New: Customer and Product Discovery

Reads:

Old: The Wall Street Journal
New: TechCrunch and GigaOM

Deep Knowledge In:

Old: How To Use Excel
New: His Customers

Loves:

Old: To Be The Boss
New: To Apply Technology To Solve Problems

Sits With:

Old: "The Business"
New: His Product Team (Designers and Developers)

When Things Don't Go Well:

Old: He Blames The Developers
New: He Blames Himself

Strives To Please:

Old: His Stakeholders
New: His Customers (because he's learned that's the only way to really please the stakeholders)

Makes Decisions Based On:

Old: Opinions
New: Data

Communicates With Stakeholders:

Old: With PowerPoint
New: With Prototypes

Attitude:

Old: Believes His Ideas Are Great
New: Knows At Least Half of Ideas Won't Work

Worries About:

Old: His Competitors
New: Taking Care Of His Customers

Secret Weapon:

Old: Killer Features
New: User Experience

Strives To Create:

Old: Profit
New: Value (because it's the best path to sustained profits)

This last point may not be so obvious to people and it is the subject of an upcoming article.

Measuring Innovation

Sun, 11 Dec 2011 20:07:00 -0500

Measuring innovation is a popular topic lately. Many product teams use Product Scorecards to keep their focus on outcomes rather than output. Eric Ries introduced the term “Innovation Accounting” for this purpose as well.

However, as much as I like and advocate for these techniques of measuring true improvement to your products rather than just adding features, if this is all you look at, over time you run the very real risk of falling into The Innovator’s Dilemma.

I am perhaps overly attracted to the concept of building enduring companies. I attribute this to having spent the first ten years of my career as a developer for Hewlett Packard, which back then was a company that prided itself on continuous innovation. But they had a tougher measure of innovation than many companies today. We were taught to measure the percentage of revenue that was coming from products and services introduced in the past few years.

My mentor during those years advocated that at least 50% of your revenue should be coming from products introduced in the past 3 years. He argued that even the best products have a natural cycle, and with good continuous improvement on those products you can stretch that cycle out, but still, every product has its day, and eventually competition and shifts in consumer behavior will take its toll, and sales and use will diminish.

Many large companies today use the “grow revenues through acquisition” strategy to building new sources of revenue. Certainly that’s one route, and occasionally the acquisition is one that is truly synergistic and could legitimately be viewed as a form of innovation.

But when people ask me for my favorite examples of truly strong product organizations, I cite the product teams that have proven their ability to not only improve their existing products, but even more importantly, to repeatedly deliver entirely new major streams of revenue for their companies. Examples include Apple, Amazon, Netflix and Zynga (in the case of the latter two examples, please don’t confuse recent questionable business decisions by the leadership with any fault of the product teams).

When I meet a company that is still getting nearly all of its revenue off of products introduced more than 3-5 years ago, I feel a real sense of urgency to help them get serious about innovation.

If you’re in one of these companies that has gone many years without new sources of revenue, and are harvesting the innovations of the founders, and you’re wondering if it’s possible for your company to learn these skills, one of my favorite examples today is the team at Barnes and Noble. They are now consistently producing highly rated new products and services, even while competing against two of the best product companies of our age. Their product team is giving their company a fighting chance to avoid the fate of the rest of their industry.

I hope more teams will track this additional measure of innovation. Interestingly, when I find companies that are very aware of this measure, they are much more open to the possibility of a pivot. Pivots are often the best source of these new major streams of revenue. Yet when you only are looking at innovation within your specific product, pivots are all too often viewed as a distraction rather than an opportunity.

Product Manager vs. Product Owner

Tue, 06 Dec 2011 11:11:00 -0500

All too often I run into companies that have resigned themselves to having two different people covering the product role.

Usually the way they split it is they have one person responsible for interacting with customers and stakeholders (which they often call the product manager), and another to interact with the development team and manage the backlog (which they usually call the product owner).

The reasoning is typically because they don’t have someone with either the skills or the time required to commit to covering both.

There are many “product managers” that are not technical enough to effectively engage with the developers, yet management hopes to utilize them. And there are many “product owners” that show no inclination or ability to get out of the building and interact with customers, yet management knows this is critical.

As appealing as this strategy may sound, I want to use this article to try to explain why this approach typically yields very weak product and little innovation.

I have written earlier that this approach has two common negative consequences. This first is that there is no clear owner (neither person takes responsibility for the product), and the second is a common lack of respect or understanding between the two (the “product manager” doesn’t appreciate the technical complexities, and the “product owner” doesn’t appreciate the customer’s pain).

However, this approach has an even more fundamental issue as well:

In order to make the many hundreds of large and small decisions a product owner makes every week, he needs to have deep understanding of the customers. Deep customer knowledge is what informs the decisions. It is actually the main thing a capable product owner brings to the party and it is what distinguishes him from the others on the team.

Similarly, when interacting with customers and identifying problems and opportunities, it is the knowledge of the technology and what is possible that informs the discussions and the potential solutions. This is what distinguishes a product person from other roles such as marketing, user research or sales, and why it’s essential that the product person has the direct customer interaction.

It is precisely this combination of deep customer understanding with the ability to apply technology to solve customer problems that enables a strong product person.

I know this doesn’t make it any easier to find people that are willing and able to do both, but I do hope that more company leaders come to understand how essential it is to find product people that can cover both aspects of the role.

MVP vs. Product Vision

Mon, 31 Oct 2011 04:40:00 -0400

Earlier I expanded on the notion of Minimum Viable Product (MVP) and I promised a series of articles that explores aspects of MVP that often cause product teams confusion. In this article, I’d like to discuss the relationship between the MVP and the Product Vision.

As a reminder, the Product Vision describes the types of services you intend to provide, and the types of customers you intend to serve, typically over a 2-5 year timeframe.

When I meet a startup, or begin working with a team that has an ambitious new project, we typically start with the Product Vision. I’m a big believer that if you don’t know where you’re heading, then you don’t have much chance of getting there.

However, it’s important to acknowledge that every Product Vision is predicated on a set of beliefs about what customers will find valuable, and that we hope can one day sustain a business or business unit. It’s important to get these beliefs out on the table and set about validating them.

One of the most common mistakes I find is that teams embark on product discovery and MVP but without a clear focus on the customers they are trying to serve.

So we typically start by identifying a core set of Earlyvangelist prospective customers. Remember, our intention with an MVP is not to try to please everyone, but rather to find a set of potential customers that believe in the vision, and are willing to work together with you to discover a solution (also remember we don’t expect them to give us the solution – we just need to be able to test out whether our solutions work for them).

Our hope and intention is to try to come up with a product that can make our product vision a reality.

That said, it’s possible that we’ll discover along the way that our hypotheses about value just aren’t reflected in our customers, or we might discover we need to pivot to either different customers, or different solutions, or different problems to solve.

But mainly we’re hoping to iterate our way to a solution that these Earlyvangelist customers find enough value in that they’re willing to buy, they can figure out how to use, and you can deliver with the time, talent, technology and money you have available.

I don’t mean to gloss over the product discovery techniques of this rapid iteration using various MVP Tests - I’ve written about this many times, and more will come in the MVP context – but if we view the MVP as the smallest possible product we could discover that’s sufficient to sell to our Earlyvangelist customers, then there’s still going to be a long way from this MVP to the product described in the Product Vision.

Just because our Earlyvangelist customers will buy something doesn’t mean that everyone in the target market will buy. If you remember the technology adoption curve, the Earlyvangelists are just the small but highly motivated group that’s really feeling the pain, and desperately need a solution. To move from Earlyvangelists to more mainstream customers (early majority), we’ll need to continue to develop the product (typically by expanding the scope of customers we’re trying to serve, and doing product discovery to identify solutions to their needs).

This is why it often takes weeks or months to discover the MVP to serve the Earlyvangelists, but then it can take years to expand the offering to the point that it meets the much broader market needs, and fulfills the product vision.

So, while the Product Vision inspires the MVP, the MVP precedes the Product Vision and is usually the first real test of the Product Vision.

Probably one of the most visible examples of the relationship between the MVP and the Product Vision could be seen with the original iPhone. While there were many prototypes (MVP Tests), the original iPhone device (what Apple believed was the MVP) had many limitations (even missing copy-paste) but for the Earlyvangelist customers, they could see the vision, they found real value, and they embraced the device. But of course that was just the beginning of the product line and not the end. Each release since then has expanded the target market to meet the needs of a broader range of customers, and come another step closer to realizing the product vision.

Hopefully this all helps to put MVP Test, MVP and Product Vision into perspective. There is lots more to talk about but I’m hoping this can serve as a foundation. Please continue to keep your questions coming.

Lean Thinking

Thu, 13 Oct 2011 15:13:00 -0400

A while ago I posted an article on people that I think have something really valuable to say to product leaders. One of those people I discussed was Eric Ries, author of the blog http://www.startuplessonslearned.com. I also promised that I’d share a glossary to map the nomenclature and concepts he uses with the terminology I use in my writing and my work with product teams.

I held off a little on this because he has been working on a book which has just recently been published, called “The Lean Startup” and in this article I’d like to discuss the key concepts in the book and provide that mapping.

First, I have to say that I consider this one of the best new books on product in a very long time. Since I read it I’ve been encouraging everyone I work with to buy the book. Eric has a strong mind and is also a gifted writer, and the result is a powerful combination.

I think the book is relevant whether you’re new to product or experienced. I have been working on technology products for about as long as anyone, and I added several techniques to my arsenal, and I’m pretty confident you will find real value in the book as well.

I also believe the majority of the material in the book is applicable to all technology product teams, and not just early stage startups.

One worry I had going into this book was that he might try to take the concepts of Lean Manufacturing and force fit them into a technology product innovation context, but he really doesn’t do this at all. He takes some of the relevant principles from Lean Manufacturing and applies them where it makes sense. His key premise is that Lean Manufacturing is about removing waste, and since one of the most egregious forms of waste is to build a product nobody wants, maybe we can apply some of the techniques to reduce the waste in creating products.

Using my nomenclature, I consider this a book primarily about product discovery. I define product discovery as the process of coming up with the minimum viable product, and that’s at the heart of the book. But more generally he’s trying to share techniques for building a sustainable business: “The goal of every startup experiment is to discover how to build a sustainable business around the company’s vision.”

Glossary of Concepts:

Many of the terms we each use are essentially the same: validated customer learning, customer/user testing, pivots and split testing are examples, but there are some places where he uses different nomenclature, and I highlight these here (term that Eric uses, term that I use):

Build-Measure-Learn Feedback Loop = Product Discovery
Minimum Viable Product (MVP) = MVP Test
Innovation Accounting = Product Scorecards
Cross-Functional Teams (small but secure, empowered and motivated) = Dedicated Product Teams
Customer Archetype = Persona
Value Capture = Monetization Strategy
Small Batch = Incremental/Continuous Development and Deployment
Large Batch = Big Bang/Waterfall
Early Adopters = Earlyvangelists
Portfolio Thinking = Portfolio Grooming
Metrics = KPI’s (Key Performance Indicators)

There was one key term that the book uses differently than I do, and that’s the term “prototype.” Eric associates the term prototype with the engineering perspective, where a prototype is mainly about assessing technical feasibility. He tries to draw a distinction between an MVP Test and a feasibility or usability prototype here: “Unlike a prototype or concept test, an MVP (Test) is designed not just to answer product design or technical questions; its goal is to test fundamental business hypotheses.”

In my use, there are different forms of prototypes for testing different things – a feasibility prototype would address technical risks, a usability prototype would address interaction design issues, but the primary purpose of prototyping in product discovery (user prototype or live-data prototype) is intended to assess the value of the product.

His big point is that the most important thing is to assess whether or not the product can support the business – especially value and growth. And he’s trying to point out that just because something’s usable or feasible doesn’t mean people will want to buy it or that you can build a business around it. In my writing, when I use the term “prototype” that’s precisely what I’m focused on as well.

Great Discussions

The book ties to cover quite a broad range of topics, some lightly and others more rigorously, but several of the discussions were especially noteworthy:

- Minimum Viable Product

Most of the book is about this concept of rapidly testing and iterating our way to coming up with something that will sustain a business. The book shares quite a few good techniques for validating the market/demand, validating the solution, and validating the growth engine.

- Pivots

One question I get from so many teams is when to pivot and when to persevere (iterate), and the book talks about this explicitly as well as implicitly with good real-world examples throughout. Also a very useful taxonomy of types of pivots. Overall I’m hoping this book goes a long way to helping teams not only recognize a pivot opportunity, but embrace them.

- Growth

For many teams I meet, establishing strong value in the product is only half the battle, and they often struggle trying to get their growth engine working to the point that they can acquire new customers in a way that you can build a viable business around. The book has a good discussion of the main ways technology products grow.

- Innovation Accounting / Product Scorecards

Probably my favorite section of the book covers what Eric calls “Innovation Accounting.” There are several very relevant examples from companies you’ll recognize and the book does a good job of highlighting some of the big benefits as well as the pitfalls of business and product KPI’s. This should be required reading for anyone responsible for coming up with the scorecards for their teams.

- Importance of Qualitative and Quantitative Learning

A hot button for me is when I meet people that only believe in one or the other of qualitative learning or quantitative learning. But Eric truly seems to understand the importance of each: “Qualitative learning is a necessary companion to quantitative testing,” and “poor quantitative results force us to declare failure and create the motivation, context, and space for more qualitative research.”

- Sustained Leadership

I’d like to end this article with probably my favorite quote from the book, which I think does a good job summarizing why these techniques are so essential to modern product teams: “Sooner or later, a successful startup will face competition from fast-followers. A head start is rarely large enough to matter, and time spent in stealth mode-away from customers-is unlikely to provide a head start. The only way to win is to learn faster than anyone else.” Amen.

The Greatest

Wed, 05 Oct 2011 21:45:00 -0400

It may have been Muhammad Ali in the boxing world, but in the product world, it’s hard to argue that Steve Jobs wasn’t the greatest ever.

He tackled immensely difficult problems, and generated products that came to define their categories and literally change the world. Just one such product can define a career, but a stream of them defines an icon, and leads to the most valuable company in the world.

I can’t imagine a world without Apple, and I am grateful that I don’t have to.

I know that the talented employees at Apple have been preparing for this eventuality, and have been practicing by asking themselves “what would Steve say?” for a while now. While I do not believe he can be replaced, I do think his lessons can be learned, and his passion can live on.

My sympathies to all of the employees of Apple, past and present, that knew and worked with him.

Minimum Viable Product

Wed, 24 Aug 2011 20:51:00 -0400

One of the most important concepts in all of software is the notion of minimum viable product (often referred to as “MVP”). But if you’ve been around software products for a while, you know that term is used in many different ways, and while the term intuitively resonates with people, there’s often a lot of confusion about what this really means in practice.

You can find it defined as the smallest possible experiment to test a specific hypothesis, all the way up to the the tangible realization of a product vision.

I’m not arguing here that any of the definitions out there are right or wrong, but I am arguing that the confusion is not helping product teams, and this is simply too important of a concept to have so much ambiguity.

I have long defined minimum viable product as the smallest possible product that has three critical characteristics: people choose to use it or buy it; people can figure out how to use it; and we can deliver it when we need it with the resources available – also known as valuable, usable and feasible.

I love the concept popularized by Eric Ries of the smallest possible experiment to test a specific hypothesis, but I refer to that that as an “MVP Test” so that people don’t confuse an experiment with a product.

But beyond the definition, it’s equally important to recognize that it’s not enough just to think you have defined MVP (which is what most product owners do – it’s their opinion), you need to have evidence that you have this – we call this validated customer learning.

Further, I define product discovery as the process of how we identify the minimum viable product and get this validated customer learning.

While my definitions above hopefully sound straightforward, I still get many questions from product teams about this critical concept:

How do we rapidly converge on an MVP?
How do we know we have actually achieved MVP? What level of “proof” do we need? How many customers need to agree to buy it?
Is the MVP intended to be something that we sell, or is it just an experiment?
If it’s something we intend to sell, what if a customer won’t buy it? Does this mean it’s not MVP?
Who is responsible for discovering the MVP?
Who makes the decisions regarding MVP?
How long should we keep trying to achieve MVP before we give up?
What should we do when we identify a pivot opportunity?
Do we need to wait until we have identified MVP before we start building production software?
Is MVP the same as “minimum product?”
Is MVP the same as “product vision?”
Is MVP the same as MMF (Minimum Marketable Feature)?
What happens after we have achieved MVP?
How do I go from MVP to backlog items?
Is MVP a moving target? Can I lose it once I’ve found it?
Is there only one MVP for a given market or could there be many?
Do we handle MVP the same way for products for consumers as we do products for businesses?

These are all important questions and I can understand why people can get confused, so I thought I would try to address these with a series of articles. I’ll do my best to make this abstraction real, so you can see the value and hopefully put this fundamental concept to work for you.

Please feel free to send me additional questions and I’ll try to incorporate them in as well.

Competition

Tue, 02 Aug 2011 15:55:00 -0400

One of the constants in our business is competition. Very occasionally you find a company that has established a monopoly position, but for the most part, if the market you’re serving is a real market with real customers with real needs, you either have competitors already, or you will very soon.

Yet so many companies struggle to determine how to respond when a new competitor emerges. They are worried that this competitor will somehow make their product offering obsolete, or undercut their pricing, or one way or another steal away their customers.

As a result, many companies resort to a cat and mouse game with their competitors. They stress over each other, they chase each other’s features, and they try to market against each other’s weaknesses. All of these responses to me only serve to distract them from what they really need to be doing.

What these companies don’t really understand is that customers don’t leave us for our competition; they leave us because we fail to meet their ongoing needs.

Please consider this. Think about the companies you know that that thrive in the face of many strong competitors. And think about the companies you know that have lost their customers.

I try to get the companies that I work with to focus their energies on providing real value to their customers. Unless they do this, their customers are probably just waiting for a good opportunity to leave. On the other hand, when your customers know that you continuously strive to deliver great solutions and great service, it generates true loyalty. The kind of loyalty that you’ll need when you make mistakes, or when new competitors emerge.

It is amazing to me how much energy goes into chasing competitors, yet how little energy or resources goes towards addressing the very real concerns of customers.

It is true that markets evolve and our customer’s needs and expectations can change over time. Especially with major technology shifts, customers that were once happy with your product may no longer be satisfied. A common example is when customers now expect mobile access.

While many product leaders and company execs stress out too much about their competitors, I also find many that completely ignore competitors. Usually because they assume their competitors can’t possibly be as smart as they are. These people also I think miss an opportunity.

I’m a big believer in having product leaders understand all of the players in their market. I think you can learn valuable lessons from all of your competitors.

My suggestion is that the product leader should study each competitor and identify the three key things that they think the competitor does that provides real value for their customers, and the three major areas where they think the customer’s needs are not met. The strengths and weaknesses may be in the functionality, or it may be in their policies, or their customer service, or pricing, or anything else. It’s important to take the holistic view when evaluating competitors.

The key is to constantly drive value for your customers. It’s really the only way I know to competitor-proof your business. So stop chasing your competitors and start focusing on providing real value for your customers.

Project-based Funding

Sun, 24 Jul 2011 09:07:00 -0400

If your company is one that still allocates product development funds based on approval of projects, then you still have the old “project-based funding model.” This is mostly a situation in either large companies, or those that have an IT-style legacy, but the mindset often exists even in small companies too.

Unfortunately it’s a very bad model.

I have written about this before but not as explicitely as I am here.

There are three fatal problems with the project-based funding model:

You very likely have no real clue when you propose a project for funding if you should really even be pursuing the project. Even though you might pretend otherwise with a cleverly crafted “business case” the truth is that you probably have no real evidence if your customers are going to like this, and you probably also have no real idea how much it will cost to build (because at the project proposal stage, you don’t even know what “it” is). So you don’t know the revenue and you don’t know the required investment, so of course the ROI estimate is less than meaningful.
Creating strong products is not a series of projects that come in sporadic bursts of a few months each over several years. Strong products result from getting your concept in front of customers and rapidly and continuously learning and improving. Further, to make this progress, you need not just continuity of investment, but also continuity of the team.
Very often in good product work we find that our initial ideas are not quite right but if we change direction somewhat, which we call a “pivot,” we can often uncover major new sources of revenue. These are the very sources of revenue that companies depend on for another several years of rapid growth. However, with project-based funding, the consequence is that this sort of pivot is effectively discouraged.

The project-based model is not only bad from the product and customer point of view. The model is very bad from a financial point of view.

The financial business cases that the decisions are based on are unacceptably flawed. The inefficiencies and lack of velocity caused by the lack of continuity means we get less from our money. The accountability so desired by finance is missing because you don’t know if the problem was that the team did a poor job executing, or if the problem was that the leaders picked the wrong project to pursue. And most importantly, it doesn’t generate the business results we need or expect.

The main alternative to the project-based funding model is the dedicated product team model. In this model, rather than funding specific projects, you instead are funding product teams which work on distinct products.

Rather than have the teams focus on delivering specific features or projects, they are accountable for delivering against prioritized business KPI’s. The team decides which features or projects or other work is necessary to deliver the necessary business results. Some of the ideas will pan out and others will get quickly killed. The product team is empowered to decide what is most appropriate to deliver, yet they are also held accountable to the results.

Product teams can be funded and staffed at any time, but they usually last for at least a year or two. You can also decide to increase or decrease the allocation of staff to that team as the needs of the business change.

One of the reasons that the project-based staffing model is often found in old IT-style legacy companies is because it fit well with the outsourcing model of staffing projects and the associated "Statements of Work" style of working. But just to be very clear, this model doesn't work for product companies for the reasons above and more.

In any case, the dedicated product team model will not only help your money go further, but most importantly it generates better results in terms of speed to market, degree of innovation, and business results.

This is just one of the important dimensions of product portfolio planning. If you think your company has issues in this area, you'll want to read Principles of Product Portfolio Planning, Dedicated Product Teams and The Product Scorecard.

Product Evangelism

Sun, 17 Jul 2011 18:18:00 -0400

In my last article I wrote about the importance of product passion, and I said that one of the reasons this passion is necessary is for product evangelism.

Product evangelism is, as Guy Kawasaki put it years ago, “selling the dream.” It’s helping people to imagine the future, and inspiring them to help create that future.

If you’re a startup founder or CEO, this is a very big part of your job, and you’ll have a hard time assembling a strong team if you don’t get good at this.

If you’re a product manager at a large company, unless you’re good at evangelism there’s a very strong chance that your product will get killed before it sees the light of day, and even if it manages to ship, it will likely go the way of thousands of other large company efforts and wither on the vine.

I’ve always believed that the product owner needs to be the product evangelist for the team. If the product owner is responsible for the backlog, and the backlog is what the team is working on, the product owner needs to ensure the team understands the reasons behind the backlog items.

This not only helps the developers do their work, but more importantly, it motivates the team to actually want to do this work.

There are several techniques to help communicate the value of what you’re proposing to your team, colleagues, stakeholders, executives and investors. Here are my top 10 pieces of advice for product leaders in terms of selling the dream:

Build a high-fidelity prototype. For many people, it is too hard to see the forest through the trees. When all you have is a bunch of user stories and backlog items, it can be very hard to see the big picture and how things hang together (or even if they hang together). A prototype let’s them clearly see the forest and the trees.
Share the pain. Show the team the customer pain you are addressing. This is why I love to bring developers and stakeholders along to user testing. For many people, they have to see the pain themselves to get it.
Share the vision. Create a product vision showing where you hope to be in 2-3 years. Not a list of features and not a spec, but rather, what types of services do you intend to provide, to what types of users? A set of product principles complements this well to share more of the nature of the product you’re working to create.
Share learnings generously. After every user test or customer visit, share your learnings – not just the things that went well but share the problems too. Give your team the information they need to help come up with the solution.
Share credit generously. Make sure the team views it as their product, not just your product. On the other hand, when things don’t go well, step forward and take responsibility for the miss, and show the team you’re learning from the mistakes as well. They’ll respect you for it.
Learn how to give a great demo. Especially for stakeholders, we’re not trying to teach them how to operate the product, and we’re not trying to do a user test on them, we’re trying to show them the value. A demo is not training, and it’s not a test. Yes, it’s a form of sales. Get good at it.
Do your homework. Your team and your stakeholders will all be much more likely to follow you if they believe you know what you’re talking about. Be the undisputed expert on your users and customers. Be the undisputed expert on your market – your competitors and the relevant trends.
Be genuinely excited. If you’re not excited about your product, you should probably fix that either by changing what you work on, or changing your role.
Learn to show some enthusiasm. Assuming you’re genuinely excited, it’s amazing to me how many product leaders are so bad and/or so uncomfortable at showing enthusiasm. This matters. Absolutely be sincere, but go ahead and let people see you’re genuinely excited. Enthusiasm really is contagious.
Spend time with the team. If you’re not spending face time with every designer, developer and QA person on your team, then they can’t see the enthusiasm in your eyes. If your team is not co-located, you’ll need to make a special effort to travel there and do this at least every couple months. Spending a few minutes with every last person on the team pays off big in their level of motivation and as a result, the velocity of the team. It’s worth your time.

Note that if your company is mid to large in size, then it’s normal to have product marketing that plays the role of evangelist with your customers and your sales force. You still may be called on to help out on the big deals and big partnerships, but you’ll need to focus your evangelism on your team because the best thing you can do for your customers is to provide them a great product.

Product Passion

Mon, 27 Jun 2011 01:07:00 -0400

One topic I’ve never written explicitly about is the need for product passion. I’ve referenced it at the top of the list of traits for good product leaders, but it’s easy to take this for granted especially since the people I surround myself with professionally are generally very passionate about products.

However, lately our industry has seen a resurgence of what I’ll call “product frenzy.” I hesitate to use the term “bubble” because I’m not sure if it is or not, but certainly there is a form of frenzy going on with people starting product companies at an incredible pace – more than I remember in the late 1990’s.

In the late 1990’s we had many people coming to Silicon Valley to try to get rich quick. They didn’t care what they were working on as long as they thought could flip it quickly for big profits.

While I think there is some of that going on again now, I am seeing a different sort of problem, that is fueled largely, but not exclusively, by the new mobile market.

Instead of trying to build a company around a vision, many people are essentially equating building a mobile app with building a company. Build a great app, people will love it, someone will buy us.

Essentially the table stakes for starting a company have dropped. It can cost very little to build a mobile app. Of course, a few of these apps are impressive while most are not, but that’s not really my point in this article. Here I wanted to talk about the difference between those that are pursuing a vision where the app is but one step, and those are just chasing their latest app idea.

Let’s consider three recent startups:

- Read It Later indeed has a very successful mobile app, but founder Nate Weiner is in hot pursuit of a much bigger prize; to be the leader in the space of content shifting – being able to read your favorite content on whatever device you want, whenever you want to view it, online or offline.

- Flipboard has created one of the best original apps for the iPad, but founder Mike McCue views this as but a step along the way of reinventing how we interact with and consume media.

- Lytro may not yet have a mobile app, but founder Ren Ng’s new generation of digital camera technology is not just trying to create a cool new consumer device, but rather to redefine an entire industry.

These are but three examples of founders of product companies that are pursuing their passions. Not with the view of turning a quick app that makes some easy money, but rather they set out on this journey well aware that it’s going to take many years of hard work to build out their vision of the future.

Let me contrast this with the six different “startup founders” I met with in just the past few weeks that were all just trying to come up with an app that people liked enough to actually install on their phone. In each case I asked them what they were really trying to achieve with their app and mostly I got confused looks in response.

It’s possible that our industry will evolve to be more like the feature movie industry. Lots of people create apps, a few make it big, and the audience just moves from one blockbuster to another. The casual online games industry is working largely like that now. But even there the best companies are working to create larger ecosystems and not just point games.

If you believe as I do that creating great product companies is a marathon rather than a sprint (albeit a marathon where we’re running 5-minute miles) then to keep the team and your customers motivated for the duration, you need to be pursuing a dream worth getting excited about and staying excited about.

I’ve been using startups as the example here but the same holds true for larger companies. If you’re trying to create major new sources of revenue for your company, you need to have a vision that is compelling to people not just for a few months but for several years. You need to show your team, your execs and your stakeholders that you have a longer-term vision, and you must demonstrate the passion required to get the rest of your company excited.

Getting them excited and keeping them excited is a big part of the product leader’s job, and is known as product evangelism. In an upcoming article I’ll talk about the techniques we have for product evangelism, but they all start with a sincere passion for products that solve real problems.

The Two Core Competencies

Wed, 01 Jun 2011 15:26:00 -0400

Good product teams must be good at product discovery, which means they must get good at learning quickly. They need to be able to zero in on the appropriate target customer, identify the key problems to solve for those customers, and typically the most difficult part of all, apply technology and user experience design to come up with good solutions that will solve those problems.

Our general principle is that we want to learn – test our theories – as fast and cheap as possible. We have two general approaches to doing so: one qualitative and the other quantitative.

User prototyping and user testing:

User prototypes are very quickly created simulations of the product you’re proposing to build.
The main reason we create them is to be able to very quickly put them in front of real target users and customers, generally face-to-face, and test the response.
The big benefits of user prototypes are that they’re quick and easy to create, generally hours or days; they don’t require developer resources to create; and we can get significant qualitative insights into how the product would be used, whether it will address the customer’s problem or need, and whether people would actually choose to use it or not.
A user prototype is not about statistically significant results; it’s about big insights and rapid learning.

For more on user testing, see http://www.svpg.com/the-most-important-thing/.

Live-data prototyping and split testing:

A live-data prototype is real code that’s typically just deployed to a subset of users in some form of an A/B test.
The big advantage of a live-data prototype is that we can gather statistically significant results and prove something actually works or doesn’t work.
Another advantage of a live-data prototype is that we can also test them face to face in a user test just as we do with a user prototype.
The big disadvantage of live-data prototypes is that since it’s real code, it needs to be written by developers and it typically takes days or weeks to create rather than hours.

For more on live-data prototyping and split testing, see http://www.svpg.com/product-discovery-with-live-data-prototypes/.

Because of our general principle that we want to learn as fast and cheap as possible, this generally means a user prototype, however, for many things we simply can’t learn whether something works without live-data, and in those cases we need live-data prototypes.

When trying to decide which technique is most appropriate for the situation, one general rule of thumb is that we can best prove something works with live-data prototypes and split testing, but we can best understand why something doesn’t work, and most importantly, what it would take to make it work, with face-to-face user testing.

But the bottom line is that as a product organization, we need to get good at both forms of learning.

The Most Important Thing

Tue, 10 May 2011 18:31:00 -0400

There are several skills and activities that are important when coming up with great products. In my last article, I argued for the absolute necessity of having good data about how our products are actually being used.

But in the this article I want to argue that the single most important thing product owners can do to add value to their team, their product and their company is a user test.

A user test is when we put the product (or product idea in the form of a prototype – either user prototype or live-data prototype) in front of our target customer and gauge their response.

There are two parts to every user test, and they need to be carried out in this order:

1. Determine if the user can figure out how to use the product.

2. Determine if the user would actually buy, or choose to use, the product, and if not, what it would take to do so.

The first part is a usability test. It is pretty straightforward to do – not hard to do yet the learnings come quick. But consider this the warm-up.

After the usability test, the user now understands what the product is trying to do and how it is intended to help, so you are all primed and ready for the moment of greatest learning, which is the second part: the value test.

Getting users to actually choose to use or buy a product is our ultimate test, and the purpose of the value test is to determine if we’ve met that test. Most of the time we haven’t. But now you’re in a position to learn what you could do to change that.

What makes a product owner useful to his company and team is attending every one of these user tests. To me it is absolutely inconceivable and inexcusable for the product owner to not be at every one of these tests. If at all humanly possible, the lead designer and lead engineer should be at every one of these tests as well.

The product owner typically is there to observe during the usability test (if you have a user researcher they typically drive this portion, and if not, the designer typically drives), but then during the value test the product owner is there to drive the discussion. Don’t leave that user until you have figured out the answer to the question: “what would it take to get this person to actually buy this product?”

The final point I’ll make is that you want to be very open to the pivot. In fact, the fastest and most consistent way I know of to discover potential big-win pivots is through these face-to-face user tests. You may realize you have the wrong target customer or user; you may realize that you’re solving the wrong problem for that user; you may realize you have the wrong monetization strategy; or you may realize you have the wrong approach in your solution.

Mostly we do these user tests using a user prototype, but you can also use your current live product, a live-data prototype, or even a competitor’s live product.

I warn teams that the worst day on the project is usually the first day you actually do user testing. But it gets better quickly. Start by doing at least three user tests a week. Don’t make any really dramatic decisions until you’ve tested on at least a dozen users.

If you haven’t yet done this type of user testing, you are very likely in for some big surprises. But the sooner you jump in, the sooner you’ll learn what you need to do in order to capably steer your product.

Flying Blind

Mon, 02 May 2011 23:08:00 -0400

I know this topic is going to sound far-fetched to many of you, but I am finding too many product teams out there that either aren’t instrumenting their product or site to collect analytics, or they do it at such a minor level that they really don’t know what users are doing on their site or how their product is being used.

My own teams and most teams I work with have been doing this for so long now that it’s hard to imagine not having this information. It’s hard for me to even remember what it was like to have no real idea how the product was used, or what features were really helping the customer versus which ones we thought had to be there just to help close a sale.

Certainly this is easiest to do with cloud-based products and services, and most of us use Web analytics tools like Google Analytics or Omniture SiteCatalyst, but sometimes we use home-grown tools for this.

But good product teams have been doing this for years not just with cloud-based sites but also with installed mobile or desktop applications, on-premise software, hardware and devices that “call home” periodically and send the usage data back to the teams.

A few companies are very conservative and they ask permission before sending the data. But mostly this just happens silently.

We should all be anonymizing the data so there’s nothing personally identifiable in there, but occasionally we see in the news that a company gets in a little trouble for sending raw data in the rush to market.

Sometimes in the press they think we’re tracking this data for nefarious purposes, but we’re simply trying to make the products better – more valuable and more usable – and this has long been one of our most important tools for doing so.

The way this process works overall is that we first ask ourselves what we need to know about how our products are used, then we instrument the product to collect this information (the particular techniques depend on the tool you’re using and what you want to collect), then we generate various forms of online reports to view and interpret this data.

For everything we add, we ensure we have the necessary instrumentation in place to know immediately if it is working as we expect, and if there are significant unintended consequences. Frankly, without that instrumentation I wouldn’t bother to roll out the feature. How would you know if it was working?

For most product owners and designers, the first thing we do in the morning is to check our analytics to see what has happened as of last night. We’re usually running some form of test almost all the time so we’re very interested in what’s happened.

There are of course some extreme environments where everything lives behind very strict firewalls but even then the products can generate periodic usage reports to be reviewed and approved by systems administrators before being forwarded (via electronic or printed reports if necessary) back to the teams.

I’m very big on radically simplifying products, but without knowing what is being used, and how it’s being used, it’s a very painful process to do this when you don’t know what’s actually going on. We don’t have the data to back up our theories or decisions so management (rightfully) balks.

My view is that you just need to start from the position that you simply must have this data, and then work backwards from there on the best way to get it.