Data gives you power. You can argue for months about the value of a particular feature. If you bring in measurements that show how people use the feature, you can win the argument in a few seconds.
The power is within your grasp. If you run an online service, your machines are collecting a stream of big data, sometimes very big. They are logging speed and capacity usage, errors, transactions, completed purchases, dropoff rates, bounce rates. The DevOps motto is "measure everything."
We found that most product managers throw most of this data away.
According to Microsoft product CTO Adam Pisoni, "What you realize when you begin to be very data-driven is a very large percentage of our ideas are bad." Measurements help you find the good ideas.
Users pay attention to fewer than half of the features and feature changes that developers add to software. If you can figure out which features they are ignoring, and also ignore those features, you can double your development capacity at no extra cost. Actually, you double your effective capacity for everything related to product releases - product management, development, and marketing.
Watch what users do, and NOT what they say. You can't rely on user surveys to figure out which features user want, because users tend to ask for more features than they actually use. They also ask for bad features. They have limited imaginations. As a product developer, it's your job to transform those bad feature requests into good features, using your special skills, in-depth understanding of the innovation landscape, creativity, and measurements.
The extra features that users ask for can damage your product because of a strange psychological effect that I call "averaging down." When you add an unfinished feature to a high quality application, users will notice it and "average down" their opinion of the quality of the application. Even if they never use the feature, they will be less likely to recommend the application.
The unveil process gives you many opportunities to test the usability and usefulness of new features. You can use feature flags to show changes to your own team, or to specific users. You can use feature flags to do A/B testing, where you show changes to a subset of your users (group B) and see if they show any difference in feedback or usage patterns compared with group A.
Users are especially untruthful when you ask them about the price they are willing to pay. They will always ask you to underprice your product, which is the same thing that I would do if you asked me. They complain loudly at the price points that will make you the most money. If they weren't ready to buy, they would not bother to complain. You should test your pricing with new package offers, and make pricing decisions based on the amount of money you collect from actual purchases. For inspiration, go to Starbucks and look at the way they are trying to get $4 for a new kind of coffee when their competitors are charging $1.50. I notice that these offers come and go, and sometimes they stick.
Where is this data? If you run an online service, you already have a lot of data. You might be collecting it through services like NewRelic, which tracks server performance, errors, and usage so that you can keep your servers running. They call this "application performance monitoring." NewRelic has started re-using this data for product management, which they call "Software Analytics - Agile business insights from your app." That's what you can do with all of your data.
If your app has a mobile front end, or a partner API, you can log the API calls and report on them. You can also get packaged services that add local usage logging inside your mobile app. This data an important tool for improving engagement and usability, and the top competitors will all be doing it.
If you do online marketing, you probably have tools that follow the progress of users through Web sites and email, as they buy and don't buy. You can use the same tools to find out what they use and don't use.
Add your own logging. You should be streaming important events into some sort of database table where you can do manual or automated reporting. You can turn on the native logging for your software platform. You can enhance the logs to include key information about the user or the subscriber. It's worth the effort.
If your product is installed on the user's computer, you have a difficult challenge. Your attempts at rapid development and data-driven product management will be frustrated by the fact that you don't know what the users are doing, or even if they installed the newest version. In some cases, you may wait a year or more to get feedback from a big enough installed base. Don't let this continue! Add usage data logging to your product. At strategic times, ask for permission to send this data home. Ask to send data home by default, and let users turn the option off. If the system (or the user) detects an error, ask for permission to send data home.
Recently I installed a touchpad that came with a driver. It watched me for a few weeks. Only then did it pop up a message asking to send usage data home and asking me for a product review. It was smart enough to ask AFTER it saw that I was happily and proficiently using the device - a very strategic time.