Productside Webinar
Pricing and Monetization for Products Out in Market
Post Launch Success Series: Part 2
Date:
Time EST:
Your product is live, but the price may not match the value customers get from it. You see how people use the product every day, and that’s exactly what good pricing decisions need.
In this webinar, Productside CEO Rina Alexin with Roger Snyder walks through a real case study with Roger Snyder. She’ll show you how to use AI to study usage, compare competitor pricing, design better tiers and test changes before you launch them.
What You’ll Learn:
- How to influence pricing, even if you don’t own it
- How to use AI to find out what customers value most
- How to design pricing tiers and judge which ones work
- How to test a price change before you roll it out
Welcome and Introduction to Pricing & Monetization
Roger Snyder & Rina Alexin | 00:00:00 – 00:01:15
Welcome, everyone, to Productside’s latest live online webinar, part of our Post Launch Success series. We are in Part 2: Pricing and Monetization for Products out in the Market. We’ll give folks a moment to join, but as you’re joining, why don’t you go into chat and share where you’re from, so we can get a sense of where everybody is sitting today. I will kick off. You know, usually I know, but today you do actually know, for another 48 hours until the next journey begins.
All right, let’s see. Peter Lee has joined. Let’s probably wait for some other folks to join. But this is going to be a fun one.
Absolutely. I’m looking forward to learning more about how we can use AI to help us with what is typically a tough nut to crack.
So, Peter, if you want to go into chat, share where you’re from or where you’re sitting at the moment. Nice. Hopefully it’s still lovely fall weather and not snowing yet.
All right, I think we can get started.
Speaker Introductions and Background
Roger Snyder | 00:01:15 – 00:01:30
Very good. So, welcome, everyone. Rina Alexin, our CEO from Productside, is leading today. I am helping moderate: Roger Snyder, from Productside. Rina, tell us a little about yourself.
Rina Alexin | 00:01:30 – 00:02:10
Yeah, sure. So, I’m calling in from sunny Austin, Texas. I’m the CEO here at Productside. I’ve been running the company for just over eight years, and my role really is, I think, just about everything. I don’t know how many hats I wear. But this is actually one of my favorite things to do: talk about product, talk to product people about things that I think are quite challenging, and pricing has certainly been at the top of that list. So, excited for today.
Roger Snyder | 00:02:10 – 00:02:30
I’ve been with Productside for over 10 years now. Before that, I helped lead product management teams in Silicon Valley, and pricing was always a challenge in multiple dimensions. So we’re going to talk about several of those challenges today. This is a great webinar.
About Productside and Session Guidelines
Roger Snyder | 00:02:30 – 00:04:25
Before we get into the meat of things, let’s talk about a couple of background things. If you’re not familiar with us, we’ve been in the business for over 25 years, and we’ve always been focused on the discipline of product management and product marketing. We help product owners out as well. We want to help people build amazing products, products that customers want and use and repeat and come back and use more. We focus a lot on helping enterprises in their product management practice, their organization, so we look to help them transform over multiple years in our relationship. But we also help individual product managers, and we’re always about helping them improve their careers and their knowledge and expertise. We’re invested experts, and we love to help out.
We want this to be interactive:
- Please pop questions into Q&A and messages into chat. If we can, we’ll take the questions in line, but we’ll also have time at the end of the webinar to answer some questions and go a little bit deeper, depending on how much time we have available
- We have a great community on LinkedIn, tens of thousands of product managers just like you who are having the same challenges as you are. It’s a great community to join, learn from each other, share successes, and ask questions. Use that QR code and become a part of this large community and network with your peers
All right, over to you, Rina.
Agenda Overview and Audience Poll
Rina Alexin | 00:04:25 – 00:06:50
Thanks, Roger. So, welcome, everyone. Today we’re going to be talking about pricing and monetization for products out in market. Our agenda is going to be maybe a little bit of an introduction about the pricing problem, and I hope the bulk of this webinar will be running through a workshop. Kim, I always like to share the Mural link a couple of times, so people can actually get into the Mural, so let’s share that now. We’re going to be running this from a Mural board, which you should have access to. Make sure to also have your favorite LLM, free or paid or otherwise, available as well, so you can follow along with the exercise. Then we’re going to leave time at the end for Q&A and answer some questions that we also received from people who registered.
Now, I do want to say this up front: today is about digital products. We have a lot of content in past webinars about how to price physical goods and what to do about regional and international pricing. There’s even a podcast on pricing changes, so feel free to check out our other content. What we’re going to talk about today really focuses on a SaaS model, a more typical model. We’re not really talking about usage-based pricing tiers or anything. So just know that it’s more about how we leverage AI within the confines of a digital or SaaS product.
This is going to be the only poll, because we’re going to have a lot of interaction with our workshop. If you can quickly answer this, and I know that today it looks like not too many of you have joined, so your voice will be loud but important: who owns pricing at your company?
Roger Snyder | 00:06:50 – 00:07:07
Pick the best answer, if you will. We understand that oftentimes it’s more than one person that owns this, but we just want to get a sense. So, we got finance, and the typical two answers.
Why Product Managers Should Have a Seat at the Pricing Table
Why Product Managers Belong at the Pricing Table
Product managers can influence pricing without owning it by bringing evidence instead of opinions. Analyze how customers actually use the product, compare competitor pricing, design tiers around where value concentrates, and validate any change with a small, low-risk experiment before launch. That gives finance and executives data they can act on.
- Analyze usage: see which features paid and upgrading accounts use more than accounts that stay free
- Compare competitors: check how rival pricing pages package the same features, so you do not look weak
- Design tiers: gate or cap features where value shows up, without losing free users who would convert
- Validate before launch: test with a new-signup cohort, a deal desk test, a fake door, or a free trial, with guardrails and a decision rule set in advance
Rina Alexin | 00:07:07 – 00:08:05
Okay, we’re going to end the poll. Thanks for playing. So, yes, we got finance and the CEO or exec team, and that is exactly what we’re here to talk about. The problem is that product managers really do belong in pricing conversations, but most product managers don’t own pricing. It gets handed to them by finance, by the CEO, by executives, by a lot of other people who demand that product managers have success with their products, where they might not necessarily have as much direct ownership over the pricing structure and the packaging structure. And we at Productside believe that, of course, product managers should have a seat at the table. Even if you do not directly own it, I think product managers bring so much value to the pricing and packaging conversations that they should be involved. And for you listening live or later, I hope you agree with me.
Three Reasons Product Managers Add Value to Pricing
Rina Alexin | 00:08:05 – 00:09:30
So if you’re struggling to get a seat at the table, here are three reasons why your company should really reconsider:
- Context. Product managers obviously have so much context. They have so much understanding of what is valuable to customers. They understand it from a market standpoint. They understand the positioning and the competitive landscape
- Feature-level usage. They also have a lot of information on feature-level usage. We don’t always talk about features as the end-all be-all, but as product managers, most of the time you really know which features customers rely on and which were more of a flash in the pan: good for a little bit, but not really used much
- A testing mindset. Last, and sometimes almost the most important. Product managers are comfortable in this very uncomfortable environment of running experiments, of having hypotheses, of bringing structure to chaos by gaining more and more information through structuring very good experiments. I don’t want to say that all of finance can’t do that, but typically finance lives in an A-plus-B-equals-C type of world, and so the ambiguous nature of pricing, I think, really belongs with the creative thinkers who are comfortable here. And it’s hard
Roger Snyder | 00:09:30 – 00:10:00
Yeah, product managers bring that bridge between understanding the business from the finance perspective and understanding the value, as you said, that customers want to gain from using our product. What jobs are they trying to accomplish? That’s good, and feature-level usage, but ultimately, what outcomes are they pursuing? What are they really trying to do? A product manager should be the person with the best understanding of that, to then make that connection between the value they receive and the monetization that you, as a company, should be benefiting from by offering this product.
How PMs Can Earn Their Seat at the Table
Rina Alexin | 00:10:00 – 00:11:10
So how do you get the seat at the table? That’s the ultimate goal, if you don’t currently own pricing or influence it. Three ways:
- Bring evidence, not opinions. This is probably the most important and hardest, and it’s the easiest one to say. Yes, you have all this context and all this information. Make it comprehensive for the people who might benefit from it. That is going to be the best way for you to get a seat at the table
- Show a hypothesis based on the context and the data. What would you change? It’s an informed opinion. It’s not fact yet, and that’s kind of the point
- Propose a small, low-risk experiment. If you would change something, how would you do it, and what kind of test would you run? Since you are the one with the context and the knowledge and the suggestion, proposing a small and low-risk experiment is definitely a way to get yourself included in these conversations
Now, Roger, in your experience, is this easy to do? Have you seen it done?
Roger Snyder | 00:11:10 – 00:13:37
It is so much easier to do now than it was five years ago. The power that AI brings, and that we’re going to show in the exercises, is enabling product managers to be way more experimental than they were in the past. However, pricing is a particular beast, especially when finance or the CEO owns it, and everybody wants to be careful. “Whoa, whoa, whoa, don’t touch that. Don’t mess with our customers.” So you write that hypothesis in a clear statement, like you said, one page. It’s really important to be able to express it clearly and concisely and demonstrate that, in fact, the experiment won’t harm current situations, in a way that allows for that experimentation. But it’s still a team sport. I say this when I’m training all the time: product management is a team sport. So you offer to run the test. Great, you’re going to lead the test, but you’re going to need help. You’re going to need to find an audience that you can run this test with, and you specify it, and these days you can often automate it. We’re going to use the right tools to choose the right group of folks that we can run this test against, but you still are going to need a little bit of help from the folks that are actually in operations or in engineering, just to make sure that you’ve done this in a proper and controlled way. So it’s a team sport. You’re leading it, but again, it’s so much easier to define a test and be able to hand that to the right folks to get it running quickly than ever before.
I just want to re-emphasize that you may think, “Oh, there’s no way I could do this, because I need engineering, I need operations, and I need finance sign-off,” and so forth. You can get this done. You can demonstrate it, even with the tools we’re going to give you today. Demonstrate with sample data: “Look, this is what we would do. This is how we would do it.” Bring that degree of comfort to the table with the people that need to help, and then you can get this out. It’s so much more powerful than it was, like I said, five years ago. I’m super excited. I’m jealous, frankly, because this was a lot harder back when I was in the chair.
Rina Alexin | 00:13:37 – 00:13:48
Yeah. We’re going to talk about three ways that you can have these kinds of conversations and leverage AI to do it.
Core Foundations: Features, Discovery, and Market Positioning
The Insight Pricing Teams Are Missing: Real Customer Usage
Rina Alexin | 00:13:48 – 00:16:55
That’s exactly the purpose of what we’re going to do today. So, three things that good pricing work requires, and I hope you’ll see the parallels here:
- You need to understand features. If you don’t today, I hope you do. AI can really help here by analyzing usage patterns and upgrade patterns, and we’re going to talk a lot about that
- You have to know the market. That means more than just your users. You need to know who else is offering, who the competition is, what they are offering, and how your positioning compares. AI can really help here, and we’re going to show you some sample ways of doing it. AI can truly collect and compare your competitors’ pricing pages, for example
- You need to know how to run experiments. What a good experiment entails, and how to capture value without losing users when it comes to pricing and packaging. AI can really help draft some hypotheses for you and stress test your thinking. But we’re also going to talk about what it can’t do
So, let’s go one by one. On features, remember this: usage itself is not willingness to pay. In fact, if a feature is the most used, then it’s probably a core requirement of your product. If I think about right now, we are using Zoom, right? Obviously connecting over video is going to be the most used feature, because it’s core to the product. So you can’t necessarily just gate it. You have to be smart about the usage data and the story it tells.
When you look at features, I’m going to encourage you to think about it in a couple of ways, and I think they tie in pretty well. I’m going to try to use some examples. First, you’re trying to identify features where larger, growing, or paying accounts use that feature more. Take a look at those who have upgraded and compare their usage of specific features versus those who stay free forever. An example of that: let’s say you have some kind of project management software, and there’s an invoicing feature as part of it. You’ve noticed that people who have upgraded are the ones who take advantage of the invoicing feature. Let’s say they start by sending one or two invoices per month, maybe they’re a small contractor, and then as they grow they start using five, six. That’s a good feature to think through, because it allows you to gate at a certain value while still allowing the free tier to have access to that feature.
Roger Snyder | 00:16:55 – 00:17:15
So, gating: when you say gating, let’s define what we mean here. We’re talking about either limiting the amount that you can use that particular feature, or, in cases of other kinds of premium features, you don’t have access to that feature in a lower tier until you pass through the gate. Right?
Rina Alexin | 00:17:15 – 00:17:35
Correct. So either no access or limited access. And in this case, I do recommend, for certain features, having limited access, so you can have a tell. It’s almost like an activation metric. If you gated invoicing features completely, then you might actually lose those free customers who would convert over time, because they haven’t gotten enough value out of your product.
Roger Snyder | 00:17:35
They need some invoicing capability, for sure.
Rina Alexin | 00:17:40 – 00:18:55
Exactly. The next one is slightly different but in a similar vein: upgraders use it more. Free accounts that upgraded adopted it more than those that stayed free. Using the same invoicing example, in the first one you want to see accounts that go from two invoices a month to five and six, and that gives you some information. The other one is whether they use it at all. Those that stay free forever just don’t use that invoicing feature, and therefore it’s a tell that it’s not necessarily a good feature to gate.
Next is that the market won’t punish you. That is why, as product managers, we really need to be aware of how our competitors’ products stack up against ours. Not that we want to copy them, but I’m going to continue with the invoicing example. Let’s say that you have a competitor project management software, and they give unlimited invoices in their free tier. Suddenly, even if it is a good feature to gate, if you do so, you might actually look extremely weak against the competition. So you can’t make those decisions without the context of what’s going on in your market.
Roger Snyder | 00:18:55 – 00:19:20
So, it isn’t about being “me too” in terms of a competitive landscape, but it is about expectation setting. If the market expectation has been set by multiple competitors that unlimited invoicing is always in the free tier, then you’re going to be at a disadvantage if you don’t have that same offering. So that’s what you have to be careful about. It’s a good point.
Rina Alexin | 00:19:20 – 00:19:35
Exactly. Now, you could still use that information as an activation metric and set your sales team loose on all of the free accounts that are using that feature a lot, but it might not be an appropriate gating decision.
Overview of Pricing Discovery Surveys
Rina Alexin | 00:19:35 – 00:20:25
Okay. So, we’re also going to talk about how you do pricing discovery. Cognizant of time here, please feel free to take a photo. The webinar is going to be available, so you can come back to this. There are a number of different surveys that you can do to get more information about what matters to your market. The four on this list are:
They vary in terms of complexity. Some of these tools really require very robust surveying tools. But the good news is that this does not require you to make a pricing or packaging change. It’s just learning which direction you might want to consider going. Roger, have you run these analyses in the past?
Roger Snyder | 00:20:25
I’ve done some conjoint and certainly some Kano in the past.
Rina Alexin | 00:20:30
And how easy was it for you?
Roger Snyder | 00:20:35 – 00:21:30
Oh, well, Kano I could do on my own. But conjoint, you want to bring in a market researcher, an expert. Over the years I’ve gained some expertise in conjoint, but even now I would still want to bring someone in who knows the ins and outs of how to properly conduct conjoint and Van Westendorp. I definitely need help. So this is someplace where the product manager needs to know enough to be dangerous, to know enough about those last two columns: what the tools are good for, when you should use them, and then bring in an expert to help you actually conduct this survey research. This is to lay down core understanding, and then Rina is going to talk more about what you do to build on top of this. But yes, get some help from an expert market researcher on those kinds of techniques, for sure.
Rina Alexin | 00:21:30 – 00:21:56
Now, the good news is that for those of you who work at smaller companies and don’t have an expert market researcher, AI can really help here as well. It can be a market researcher with you. Of course, these do get quite complex, so we’re not saying that AI can do it 100% well, but it’ll get you somewhere, get you down the road.
Competitive Positioning and Alternative Data Sources
Rina Alexin | 00:21:56 – 00:25:12
The next one is market positioning. Here is an example prompt. At this point, Roger, I feel like people have this going on a schedule, like a scheduled task, an agent running and doing constant competitive research. That’s really the right way to do it. But if you haven’t even gotten there, start with this prompt. Have it actually do some research and pull competitive pricing pages. There’s a warning here: AI can misread pricing pages, too, and these can change often. So this is not a one-and-done. It’s important to track these pages over time, because you’ll learn a lot more about the moves that your competitor is making, and try to almost reverse engineer some of the thought process behind it. If features start moving between tiers, if there are new limits, new tiers, or new paid add-ons, all of these things can contribute to a better pricing or packaging decision for your own product. As we mentioned earlier, context really matters. Your customer doesn’t just go on your pricing page. They are looking at a variety, and even your existing customers, especially when renewal comes up, if you’re on an annual plan, you’d better know that they are going to take a look. So it’s really important to stay up to date.
Pricing pages are just public information, so you should be assuming your competition is looking at yours. That is why you need some additional data points, some also public but maybe less thought through:
- Review sites: understanding what their users are resenting. Usually, most of the time, you’re going to get your top happy and top unhappy users. You’re not going to get as much of the middle of the road: “I’m frustrated with this product, but not enough to post on a review site.” So it’s not going to give you the full picture
- Release notes, if they’re available
- Help docs. If there’s additional documentation about the product, you might find a little more information on usage limits, plan rules, things like that
- Job postings. If you’re in a field with a lot of large competitors, I understand they’re always probably hiring for something, so it might not be a super strong signal, but it’s still another signal you can take a look at
- Your own sales team. Of course, the best for last. They get a lot more information from their customer discussions. And I think every sales team at this point is recorded, so getting access to some of the sales demos and transcripts, where you can hear objections and learn what some customers might really care about that your product might be missing if your competition has it, is another important data source, and another way AI can really help you
Pricing Experiments and Responsible AI Usage
How to Validate Pricing Changes Before You Commit
Rina Alexin | 00:25:12 – 00:27:55
All right. Last is experiments, and why do we do them? Well, we just talked about surveys. Surveys tell you what customers say. Experiments tell you what they do. I like to view these in the context of risk: if you have a high-risk decision, you want to lower that risk as much as possible. So you need to be smart about what experiment you’re running. That’s where surveys can really help give you some insight into why you’re even doing it. And number two is establishing the right kind of experiment to run that reduces risk to your company. There are a number of different pricing tests that work in B2B:
- New signup cohort. You apply the change to new accounts only. Essentially, you change your pricing page and new accounts see that. I do have a warning, which I mentioned earlier: your existing customer base probably looks at your pricing page occasionally, so know that you can run into issues if you change it publicly and don’t communicate it, and make your existing customer base feel second-rate
- Deal desk test. This is a little more private, and it has to do with your sales team sharing changes directly with accounts. It’s also a great way to involve your sales team, get some champions for a pricing or packaging change, and get buy-in from that team. Many companies pick a very specific subset of the sales team as a pilot team. Even with this, you really need to know the “what’s in it for me” for a sales team. That’s usually the toughest one when you’re making pricing or packaging changes. Of course, your goal should be their goal, to increase their commission by increasing revenue, but it’s always a tricky conversation, because inserting change into a process has risk, and therefore risks their commission
- Fake door. This is almost like a landing page test: you say you have it, and you measure clicks and interest before you even build something. That can be tested publicly
- Free trial. Trying it, seeing usage, seeing adoption, and seeing conversion from that. That one is probably an easier one as well
Roger Snyder | 00:27:55 – 00:28:20
I’ll just make one comment about the new signup cohort. That’s a good protection for finance or sales when they’re fearing, “Hey, this could have a horrible impact on us.” Well, we’re going to test it in a narrow area. You can do things where you limit who actually sees that, even for new signups, by limiting it to one geography, for example: we’re only going to test this in one area for a while. That can be valuable as well.
Leveraging AI as an Amplifier in Product Decisions
Rina Alexin | 00:28:20 – 00:29:45
Thank you, Roger. So, we’re going to be using AI right now, and we always like to bring this up: AI is an amplifier. For the people who are nervous about getting replaced with AI, do not replace yourself with AI. First and foremost, you need to be responsible for doing the thinking. AI can help you spot patterns. It can help summarize things. It can help you stress test your thinking and argue against your proposal. Those are all great things that AI does well, not perfectly, but well. Only you can make the final calls. You will be owning the recommendation, so you need to be well aware of the trade-offs, how to design the correct experiment, all of that. We’re going to be running through a really fast way to get some of those good experiments. As part of the design for the workshop, we don’t have enough time for you to really review and stress test everything that AI is giving you. So this is not the ideal way to use AI, but in this safe environment, we’re going to go with it. When you’re doing the actual work, it’s not the first thing that comes out.
Best Habits for Prompting and Iterating with AI
Rina Alexin | 00:29:45 – 00:30:53
Here are just a couple of habits for how to do this well:
- Iterate. The last one, iteration, is going to be really important. In order to iterate, you need to start somewhere. You need to start with a draft, and then you need to see what’s wrong with the draft. When outputs from AI are wrong, and when we were creating the case study here it definitely took a few tries, remember that you need to fix the prompt, not just work with the answer
- Say who you are. That one also came up. In this particular case, an earlier iteration started giving me ideas like a sales team would, such as changing commission, and that’s not going to work. That’s not really in my purview. So make sure you say who you are: a member of the product team
- Make it show its work. That’s my favorite as well. Give me sources, give me reasoning. That will really help
Interactive Workshop: Plan Loop Case Study
Designing Smarter Pricing Tiers: A Hands-On Case Study
Rina Alexin | 00:30:53 – 00:32:55
All right. So, we’re going to go into the workshop now. We have, I think, just enough time to get it all done.
Here it is, the Mural. So, join the exercise with the Mural, and I’m going to quickly go over some context. We are going to be working for Plan Loop today. Plan Loop is a fictional project management tool. Actually, I don’t think I included anything about invoices here, so my earlier comments have nothing to do with Plan Loop. Plan Loop has a Free, Pro, and Business plan, and your goal is to raise the quarterly free-to-Pro conversion rate, currently sitting at 13.5%, to 16%, and to do that without lowering your signups or raising churn. Everything that you need is going to be on the Mural board. I’m going to walk you through that quickly, but we’re going to run three activities, each less than 10 minutes, roughly:
- First, analyze sample data for packaging conversations that could lift conversions: identify, based on usage data, where there might be some good ideas for us
- Then, choose the right move to make, including some competitive context: we’re going to give the LLM some data around competitors’ pricing pages
- Then, have AI help us design a few experiments and tests
This is completely fictional. A note: pricing and monetization has a number of different levers that we can pull. Pricing changes are one of them, but certainly not the only one. That’s why we’re talking a lot about feature usage and packaging decisions, because I think those are also really important when it comes to monetization.
How AI Helps Product Managers Analyze Usage and Uncover Value
Rina Alexin | 00:32:55 – 00:33:35
Okay, so we’re going to grow free-to-Pro conversions. Here is Activity 1: find conversion opportunities. I see everyone’s joined, wonderful. Download the Plan Loop account data. You can do this by clicking on the image on the right and clicking the download icon in the upper left. There are a number of different ways. Download it to your computer and upload it into the AI chat. After you’ve done that, don’t click Enter. You need to come in here and copy the text. I’m going to do that right now.
Roger Snyder | 00:33:35 – 00:33:45
I did click Enter the first time I tried, and of course it spat out a whole bunch of analysis, but it didn’t do anything [unclear].
Rina Alexin & Roger Snyder | 00:33:45 – 00:34:45
But you’re right. Don’t hit Enter, because you want both the prompt and the context all in one.
I have the prompt here, and it does include some goals that we’re trying to aim for. Is my screen really small now?
It is kind of small, yeah.
I’m sorry, I’m on a bigger screen here, but it’s okay. You can read it on your own screen.
Yes, of course.
It’s telling it what opportunities to look for and to keep a table. I also have my Plan Loop account data, and I’m dropping it in here and sending that message and having it run. So, Roger, are you doing this with me?
I’ve got my previous run. I’m hoping that Peter was in, and Manoj is hopefully in there as well.
Run this prompt now.
Rina Alexin | 00:34:45 – 00:35:30
It does take a little bit of time, obviously, since we’re including feature data. If you want to take a look at the kind of data that’s in here, there are definitions of the data and also all of the accounts. I kept it to 1,000 accounts for the purposes of this exercise. I would hope you’d have more information, but that is also part of the reason why, since there are only 1,000 accounts for us to look at, the conversion we’re trying to identify is only 18 more signups per quarter, which is not a huge number, but it’s related to the exercise.
Roger Snyder | 00:35:30 – 00:36:15
One comment I’ll make while we’re waiting for some results: where do you get this data? This should be something where your ProdOps folks or your operations team should be able to provide this information for you. You have all of the behaviors of your customers monitored, and grabbing this data and putting it in the right format from a spreadsheet should be easy to do. So partner, again, product management teams, partner with your operations team and grab this data, because it’s going to be real usage that your customers are exhibiting, and that’s what you want to be able to measure.
Rina Alexin & Roger Snyder | 00:36:15 – 00:36:30
When I first did it, Roger, I don’t know, it didn’t take quite this long. Mine, well…
What is it saying? A minute and 30? Yeah, mine took about two minutes, I think, last time.
Roger Snyder | 00:36:30 – 00:37:15
The other thing, too, is that if you are missing some of this data, you can actually have AI help you fill in gaps. Synthetic data is something that you can employ, but you’ve got to be careful with it. Make sure that it doesn’t throw bias into this. But if you’ve got, say, usage data that only shows users at a certain range of usage, and you want some more at a higher range of usage, you can actually have the AI take that data and expand it. When it has a good enough base set to work from, that can often be very effective. But now I see results. So let’s talk about that.
Rina Alexin | 00:37:15 – 00:39:30
Yeah, we’ll talk about it. At this point, we like to keep things interactive, so I don’t know, Manoj or Peter, if you want to come off mute, we can allow you to talk if you want to raise your hand to share. But essentially, AI has gone through it and has identified feature usage percentages, the share of accounts using each feature at all, and the differences between the free-stayed and upgraded categories, which is what we’re trying to understand. And it’s offering some packaging changes:
- Move Timeline view to Pro, or limit it on Free, with an upgrade prompt when free users open it. Conversion is 55% among accounts using Timeline versus 11% among those that don’t. Usage is very different when you look at upgraders versus free, like what we were saying earlier: are upgraders using it more?
- Move guest access to Pro, or cap it. Conversion is 52% among accounts using guest access. Among non-users, usage is 57% of upgraders versus 18%. Again, a pretty big difference between those who’ve upgraded and those who stayed free
- Add an upgrade prompt when adding the ninth or tenth user. Taking a look at 11 to 30 seats, free accounts are spread evenly from 1 to 10. If we reduce the users, or just give an upgrade prompt, that’s all it’s suggesting: “Hey, you guys are using this account, it’s time to upgrade,” because you probably are going to need more features from all of your usage
- Raise the prompt at the three-integration count. Free accounts are the cap. Integrations don’t matter on their own. Accounts using [unclear] convert with integrations versus 2%. Okay, interesting. So I guess this one is not as…
Roger Snyder, Rina Alexin & Attendees | 00:39:30 – 00:40:00
Yeah. On mine it was the lowest one too. It was medium. Did you have different ones? I’m curious. Or did you get similar?
No, I had the upgrade prompt. I had the Timeline-to-Pro one and, actually, the integration cap. I didn’t have the other one. I only had the top three.
I did not have the adding-user one. That’s right.
Are you using Claude as well, or using a different…
I was using Claude. Yeah.
Rina Alexin | 00:40:00 – 00:40:58
Ah, okay. So, when I was creating this, I used both Claude and ChatGPT. They actually do hone in sometimes on slightly different things. So that’s also something to consider. I know if your company currently only allows you to use one, remember, I’m going to say this on behalf of hopefully operations officers everywhere: do not use LLMs that your company has not paid for, especially when using sensitive company data. If you have access to multiple LLMs, this is a great one to see what kinds of answers you get. It’s also noting what’s risky and what the data can’t tell us: obviously, cause versus correlation. In this prompt, you’ll see that I ask it to make those clear.
Comparing Your Pricing Against Competitive Offerings
Rina Alexin | 00:40:58 – 00:42:10
So, we’re going to move to the next part, which is giving a little more context to the decision: adding competitor pricing pages and then asking it, based on these opportunities, to make a recommendation. Oh, I don’t know why I’m doing that. What did I do? Cancel. Copy text. So, I am going to go back and copy the text, and I’m going to add to it the competitor pricing pages. Similarly, please download these pricing pages, and I’ll show you what they look like here. This is yours, Plan Loop, and your competitors:
- Plan Loop: Free $0, Pro $10, Business $20
- Task Haven: Free $0, Team $9, Business $18, so slightly less
- StackWork: Standard, Plus, and a custom Enterprise tier. That’s enterprise for you: they love not knowing their pricing, still
As you can see, there’s different information about what’s included in these. Oh, I should have learned last time and run my… while I was doing that.
Roger Snyder & Rina Alexin | 00:42:10 – 00:43:25
This one should go a little bit faster.
Yeah, it does.
I also didn’t do it correctly, so let’s see if this works. It is smart, since I’m in the same [unclear].
Copy my top three ranked.
It might just ask you. That’s what it did for me. Should I use these three?
Please use them. All right, this is a good test of that prompt, because I make it ask if you forget.
Yes, you do. I did it that way when I was writing it, for sure. That’s a good message to all of you when you’re writing prompts: make sure you don’t rely on “you have to fill in this piece.” Add a sentence that says, “If you don’t get the piece you need, just ask.”
Rina Alexin | 00:43:25 – 00:44:15
Just ask. Okay. So, I like this, because I think if we don’t look at competitors, if we just look at our own feature data, we would be led down the wrong path. When I first did this, I thought, “Hm, this one seems obvious to do,” but then when you add the competitive context, you realize that some of these might be a little tricky to do. Right, let’s see. Guest access opportunity: limit, cap free guests at two, unlimited on Pro. Why this move? We’re the only free tier with unlimited guests. Task Haven caps free at two, and StackWork allows five per paid user. Competitive risk: low. A cap of two matches Task Haven exactly. So, we’re going to look like Task Haven.
Roger Snyder & Rina Alexin | 00:44:15 – 00:44:30
Now, the issue is that if we do this, we no longer have an offering that’s better than them.
Right?
So, that is potentially going to be an issue.
That’s the question. Yeah.
Rina Alexin | 00:44:30 – 00:45:25
If I were to see this, I might want to… well, that’s interesting. The Timeline view is the other one. Limited-time Pro trial triggered once a free team is actively using Timeline. Gating or limiting Timeline opens a gap. The Timeline view was the one that looked like, “Oh, this looks like an interesting feature to gate,” but our competition offers it for free all the time. So now you’re very weak compared to them. And then the seat limit prompt: lowering the cap widens our biggest gap. Task Haven is free up to 15 users, so we are already below Task Haven on that. So the only…
Roger Snyder & Rina Alexin | 00:45:25
Go ahead.
I said, “Dang it, Task Haven, you’re setting poor expectations in the market.”
Rina Alexin | 00:45:26 – 00:46:00
There’s another one here. We just talked a lot about packaging changes, so I made sure that we also cover some pricing changes if it makes sense. Pro right now is priced at $10. The Task Haven Team plan is at $9, and StackWork Standard is at $12. StackWork Standard obviously has… we have some better feature sets compared to it. However, is there room to even upgrade based on price? That’s an important conversation.
Roger Snyder | 00:46:00 – 00:46:10
There is, if we go up from $10 to $11 or $12. They’re kind of setting the ceiling on that.
Rina Alexin | 00:46:10 – 00:47:55
The only thing to note is that, again, we have a small sample set, so this pricing change is not going to produce credible results, because it’s a small group of users to begin with. It’s only saying, “Okay, is this worth it?” We would be $2 above Task Haven Team if we raise it to $11. Is it worth it? That’s the decision point you need to make after running this: is the juice worth the squeeze? Is the risk worth us adding $20K, potentially? The same with this: if you cap guests at two on the free tier, going from, I think, five users to two, now you’re at parity with Task Haven. That’s what the AI is suggesting. But what can you do? You can select a different user cap. You can instead suggest that we do limit it, but we limit guest users to three instead of two. That still gives you some differentiation against your competition while offering a packaging change that might benefit the company. The AI is not giving me that, and that’s where the human in the loop is very, very important. AI is not going to be able to make those kinds of strategic decisions, so you have to take a look at the data.
Roger Snyder | 00:47:55
Human judgment. Yay. We still have jobs.
Designing Experiments and Measuring Impact
Prompting AI for Pricing Experiment Designs
Rina Alexin | 00:47:58 – 00:48:35
Okay. So, let’s say that I want to make some changes. The next question is, what is the experiment that I can run? This is optional: if you want, you can also download this page if you want to give your LLM specific experiments to think about. They should also be in here. I think we’re doing okay on time, so let me actually share this.
Roger Snyder & Rina Alexin | 00:48:35 – 00:48:45
Go ahead and start the prompt, though, before you share.
Ah, yes. I don’t learn from my mistakes, right?
Yeah, we all… it takes a couple of reps, that’s for sure.
Rina Alexin | 00:48:45 – 00:49:05
Okay. Let me just start the prompt. Top-ranked move. Okay. And then I’m actually going to give it: gate guest users at two, three. I just copied and pasted it in here. And while that runs, thank you, Roger.
Key Frameworks for Structuring Pricing Hypotheses
Rina Alexin & Roger Snyder | 00:49:05 – 00:51:40
I’m just going to do a little bit about running experiments. This is essentially how to think about experiments when you’re designing them. You want a hypothesis, and the hypothesis needs to include what you’re planning to change and why, and what your guess is. Nobody has a crystal ball. If you do, please let me know; I would love to have it. But you need to have a guess. And here’s an important part. When you work with AI and collaborate and look at data, oftentimes you’ll find that, let’s say leadership is telling you that you need to move your conversion goal from 13.5% to 20%. You structure, okay, what can we actually gain from experiment one, two, three? And you realize there’s no way for me to do that.
Right, right. So…
Exactly. That is actually super, super helpful, because then you can bring it and say, “Okay, if we do this, I’ll potentially get a 1% lift, and that’s a risky 1% already, we don’t know. And if I do this, I might get another half a percent, but that’s basically all I could do this quarter. So, your 20% goal, let’s not do that.”
Right. But now you’re bringing data to that conversation, and that is so much more powerful.
Exactly. When you’re considering the experiment, you need to design it:
- The timing of it
- The audience for it
- The metric that you’re trying to move. You need to measure it and stick with it. If it’s not moving, it’s not moving. That’s also information
- Your guardrails: what you must not do, what you must not see
And do not be afraid of an experiment not going your way. That’s okay. Avoid confirmation bias: don’t ignore it when the results are telling you this is not a good idea.
That is important. If the decision rule is that this must shift conversion from 13.5% to 20%, question that rule before you set it, because you should honor your decision rule after it’s set.
That’s true.
Evaluating Test Results Against Conversion Goals
Rina Alexin & Roger Snyder | 00:51:40 – 00:51:55
All right, let’s go. I know I stopped sharing instead of doing it. One sec.
See, this is why, Roger, you’re the master and I’m usually the moderator.
Well, you’re doing great. Honestly, you had a lot of screens to juggle today.
Rina Alexin | 00:51:55 – 00:55:47
I’m glad I’m here. This is, like I said, a lot of fun. All right, so both checks pass. Cap free gate: I made a suggestion that we cap free at three guests, instead of five, instead of two. That was mine. But keep guests unlimited on Pro, and apply it to new signups only, those that exceed three. So there are some assumptions, and then here are the experiments, and it’s giving the design, two to eight weeks to run it, so we have to make that decision:
- Pricing page fake door. New signups don’t fall when free shows a guest limit. You apply that change to your pricing page. The fake door is split by domain in this case: some people see Free listed as up to three guests, and the others see the five-guest page. It’s basically an A/B test, so you can see which one might affect conversion
- In-product prompt at the fourth guest. This is another experiment you could run: whether teams hitting the cap upgrade when asked. Let’s say you want to establish three, but you are allowing them to actually get to five. When the free account invites a fourth guest, it shows “Upgrade to Pro for unlimited,” and you see how many conversions you get there. I don’t think that one is a fake door. Maybe I just read that somewhere else
- New signup cohort with a cap enforced. You establish a three-guest cap and see how your market reacts to it
Now, the challenge is that this won’t reach the goal. Look at that, like we just talked about the ceiling. With the assumptions that the AI has already given me, it’s saying that this move will only… about 17 new accounts use guests. If half exceed three guests, that’s nine accounts, and let’s say about four accounts a quarter convert for the cap. We wanted to get 18 upgrades, and we got four from the data. Again, this is something you need to iterate on, to see how many changes I need to make in order to get to this 16% goal. The last thing I’d like to point out is that it says don’t just assume this data is 100%. You could have gotten this lift anyway, and you need to be cognizant of that. So, this was just a taste of what you can be doing on the job.
I’ve now explained it a couple of different ways. You need to know if your pricing experiments are even worth it. That’s, in the end, your decision. AI cannot make it for you, but it can help stress test your thinking and identify, okay, you need to do more than just this. And bring whoever makes pricing decisions at your company, finance, the CEO, and so on, the math, and show them why what they are asking for is achievable or not.
Wrap-Up, Q&A, and Upcoming Resources
Key Takeaways and Weekly Action Items
Rina Alexin | 00:55:47 – 00:56:35
Right. So, I’m going to recap very quickly, and I don’t know if we’re going to have too much time for Q&A. Again, the way to get a seat at the pricing table is that you can earn the seat with evidence. You have access to usage data, more so than many of the people who currently own those decisions. You should be looking at it day in, day out. So you have context. You also have structures for how to even consider what is a good gating decision, based on your understanding of the competitive landscape and your understanding of your users and what they value. Very important. Last, you’re the ones with that information, and you can create experiments that will help your company learn more and test those things that will move the needle for it.
Roger Snyder & Rina Alexin | 00:56:35 – 00:57:10
What you can do this week:
- Pull your usage data for your free and paid accounts
- Run competitive research. If you haven’t done a pricing check on your competitors, or if you don’t have an agent, build one. This is the week to do it
- Bring one gate hypothesis or one pricing change to whoever owns pricing. Pricing changes are still on the table here. Have that conversation
This opens the door to having that conversation. I think it’s fantastic.
Correct.
Productside Announcements and Training Courses
Roger Snyder & Rina Alexin | 00:57:10 – 00:58:45
Manoj, any comments or questions, please put them in chat. We have a couple more slides to go over. We have time, so we’ll answer a question, but let’s quickly run over a couple of opportunities:
- Go-to-Market Pack: Use that QR code to download this pack, or click the link that we’ve just put into the chat. It’s a great pack to make sure that you’re working well with your team to go to market and to help you become more effective
- In-person training in Austin: early December. Austin is always a great place to visit. I’d love to visit Austin versus Alaska. Go check it out. Tom Evans will be there, and Rina will even drop in and say hi
I will.
Fantastic opportunity. And this is just Part 2 of this three-part series on post-launch success:
- Part 3: postmortems. Tune in on the 20th. You launched, but did it work? An excellent question
- Part 1: how to kill zombie products. If you missed it, go back and watch the recording. That’s one of the number one things I hear from people: “I don’t know how to end a life. It’s so hard.” We can help
Upcoming courses:
- OPM (Optimal Product Management), in person. As we just pointed out, I will be live online at the beginning of December, so please join me. And then live online in March, and Kenny will be there in January
- AI Product Management: a fantastic course that Tom, Rina, and Dean will be giving over the next few months
Q&A and Final Remarks
Roger Snyder & Rina Alexin | 00:58:45 – 00:59:55
Roger Snyder:
Awesome. I think we have time for… All right. Peter, I think, asked a question.
Question:
How does this approach change for a product that isn’t in the market already?
Roger Snyder:
This whole series is about post-launch success, but in general, a lot of the same questions can be done. You can try to conduct surveys. It’s going back to: you can’t measure real data, you don’t have it. But you can try to get some sentiment analysis going, and you can watch how your competitors are doing over time, as Rina was talking about, to see how things are going with competitive products over time.
Rina Alexin:
Roger, you took the words out of my mouth. Everything that we talked about today is definitely still applicable if you don’t have usage data.
Roger Snyder:
The surveys, obviously, are the best place to start, because that’s a way to get information fastest.
Rina Alexin:
Yeah. And unless it’s a completely, what is it, blue ocean product, a completely new product, there is some data around what the requirements are and what people expect from a product.
Roger Snyder & Rina Alexin | 00:59:55 – 01:00:27
All right, great. Thank you for the question, and thanks so much, everyone, for joining. This was a lot of fun to create, but of course this is just a sample. The hard work is what you do day in, day out, in trying to solve these kinds of challenges for your organizations, and thank you so much for the hard work that you do.
Everybody needs it.
Absolutely. All right. Thanks, Rina. Great conversation.
Bye, everyone.
See you all next time.
Webinar Panelists
Rina Alexin