Webinar: A practical guide to lean product development

Nick Coppolo
Jun 29, 2018

Spire’s Chief Product Officer, Nick Coppolo, walks through the practices, processes, and pitfalls common in early-stage ventures employing lean product development methodologies. Watch the full presentation here.

Feet on the street: A practical guide to lean product development

0:00:00
I’ve been with Spire for eight years. We have a particular focus on new product development. Our clients range from well-established large organizations, fortune five hundred’s to early-stage ventures such as yourselves. And within new product development in particular, we have an additional focus on design thinking and user-centered design. All that boils down to engineering and development. The product itself ultimately reacts to the needs of the customers and users as expressed through design. Thank you for joining today.-

-I’m going to rip through this. Whether you hold questions to the end or ask me in-line, what’s most important to me is that I can make this applicable to you guys. So, no holds barred on the questions. If you want to stop on a particular slide, I would rather it be meaningful to you guys than get through the whole deck, so feel free to interrupt. And like I said, even from a consulting stand point, if I can offer an objective opinion or help you solve a problem or think about something differently, that’s exponentially more valuable, obviously, to you and indirectly to me.-

0:01:19
-So, let’s get started. Can everyone see the screen okay? If you can’t, yell. If you don’t say anything, I’ll assume you can. I’m not going to spend any time of the history of Lean. There are an infinite amount of resources and references. If you’re interested in its birth, certainly its modern birth, post-war Japan, Taiichi Ohno any of the work by Edwards Deming; like I said, an infinite amount of resources written by people eminently more qualified than me to talk about them. But I do think it’s notable that we start out with Steven Blank. Obviously, Lean came into the tech lexicon with the Eric Ries book of 2010, The Lean Startup; which I think is a really important book, but I will say that it takes a particularly populist vent. I think it’s an easy read and I think when most people put it down they say, “Oh, that makes a lot of sense.” That they immediately ask themselves, “Well now what do I do?” So, I aim, a bit at least, to demystify some of the practices and tips today and also emphasize Steven Blank, who in my mind is really the forefather of Lean product development, certainly from a tech standpoint. His seminal book, “4 Steps to the Epiphany”, really is a model, and outline for what he calls customer discovery, customer development method, that certainly Spire draws a lot from. And you’ll see our own particular flavor of it through the slides here.-

0:02:47
-But, in order to begin, we have to agree on what a startup is. A startup is an organization built to search for a scalable, repeatable business model. I’d add one addition to this: unless you’re aiming for an acquisition exit, I would also say scalable, repeatable, and ideally profitable. And once a startup finds that scalable, repeatable and profitable business model, they cease to be a startup and they’re now an established organization whose job is not to search for but to repeat. And, certainly in our own work with the large organizations there’s this interesting paradigm where you teach them a little more to act more like startups to reconnect with the innovative spirit that got them to where they are. And conversely, with startups we’re trying to get them to act a little more like well-established organizations; some of the discipline and rigor, to be a little more repeatable.-

0:03:40
-Why Lean? You know Lean certainly is a buzzword. In our particular industry vertical is rife with buzzwords, but I also think, in it’s proliferation, it’s lost some of its meaning. So, for me it’s important, you guys know certainly already, many, many, many, many products and businesses have been started not employing Lean product development. Alternately, many, many have come to find success in perpetuity using some or all of Lean product development methodologies. Why Lean, for me, really boils down to two things: it reduces the risk of building products that no one cares about, which is pretty important; and then, secondarily, it really tends to maximize capital efficiency, so that we’re spending our cash on features that will show a return on investment. That capital efficiency goes all the way into retaining equity, ‘cause you do more with less. Often times, you know we see it every day, you earn 80% of your revenue with 20% of the features out of the original vision. The goal of Lean, really, is to identify those 20% of features as fast as possible.-

0:05:02
-Couple of principles: again, derived largely from lean thinking, if you guys are familiar with that at the highest level, modified as I said for our particular interests. The classic – Eliminate waste: I won’t go into this. Waste is present throughout, large organizations and small organizations alike. I think that startups can be particularly prone to waste because there’s a palpable energy that runs through small organizations, and they can be prone to a bit of work hard not smart. Waste in its most simple form in terms of being talked about and it’s building features no one cares about. Lifting the fog of war: we talked about this in the build measure learn feedback loops were, I should say, I’m not at all trying to draw a correlation between warfare and product development, but the fog of war meaning slowly making constant probes with the customer to reduce the unknowns and turn assumptions into truths. Customers over vision, this is typically hard, certainly for first time founders. Is that we chase customers and their problems and ultimately people who will pay for our company to become scalable, comfortable and sustainable over that original vision whether we were eating our own dog food as it were or we initially approached or assessed a problem in a certain way. In lean we really want to chase the customer problem and very rapidly abandon the original vision. And in line with that, really, problems first, solutions later.-

00:06:48
-So lean is really about the top, the thinking, the planning, the fuzzy end, the fuzzy beginning of our product development exercise. We want to reduce the amount of waste, certainly during the execution phase. Building and shipping software has a permanence to it. It also has associated costs, but thinking, planning, testing, and concepting is relatively cheap in comparison. The bottom example, you know, thinking and planning, saying yep this is the vision, go execute on it and then, with the premise that the product survives contact with the user, it can be a road to waste and certainly to either pivot or shut down.-

00:07:35
-But I do think it’s important, we’re really using the cycles, and I talk about this later, but we’re really using these methodologies, and today you’re going to see the super set of them, at every phase. So there is low hanging fruit and we don’t want to over-analyze or have an analysis paralysis on things like what color the blue should be if you’re familiar with the google story certainly features that are not necessarily vetted that have a high cost, literal cost that have a tangible risk to them even just in product roadmapping, you know, if we want to capture particularly a new segment or a particular new demographic maybe it’s a laggard certainly how should we enhance the product to capture that market. And then even down at the feature level, we really run these cycles at not just early-stage product market it’s throughout really the product development life cycle.-

00:08:41
-And then lastly, just as preface, some misconceptions, but just the fact that lean is only useful when establishing product/market fit. Lean is useful throughout, as I said, product, the life of the product, of what to build when, for who and why. And really, lean answers those questions. MVPs are shitty, that’s a common misconception. That’s certainly not true. MVPs, need to be, if you haven’t established for them, need to be on brand. And also the definition of an MVP, there’s a misconception around that as a whole which I’ll get into, but MVPs if they are working software then they need to-
-Oh did someone go on mute? Check if you’re on mute. Thank you. –
-You know, every expression as an MVP, I’ll just use this horridly ineffective adjective, needs to be tight. It can be small, it can be testing only one premise, it can be working software with a truncated feature set, but it needs to work, and it needs to, as I said, be tight, and I think that’s a common misconception. Also something that for established businesses, like I said earlier, with brand, it may draw them away from it. As we’ll get through this wrap as well, lean is “Shooting first, aiming later” or “See what sticks”. I think that’s actually one of the misconceptions, the takeaways from the Ries book. Folks that are in the early days of implementing lean product development methodology. Lean is actually super diligent, and in fact you’re constantly guarding against, as I said earlier, analysis paralysis. It’s certainly not “shooting first, aiming later” there’s a very methodical diligence phase upfront. That being said, there are times to “shoot first, and aim later” absolutely. And then lastly, lean asks customers to design your products. What was it, the Henry Ford quote, which I’ll butcher, “If I had asked customers what they wanted, they would have said faster horses.” A lot of misconceptions around what we’re asking customers in early and even late contact with customers after the product is out in the wild. Really lean is about interpreting and understanding customers problems. And customers will generally sway and want to lean on a solution and that can be valuable, but ultimately as product developers, it’s our job to interpret fiscal problems, to correlate problems across a wide variety of segment, and then as product designers, then to solve those problems, ideally with solutions that your customers could never even imagine. But first and foremost, it’s really, understanding deeply customers and having it constantly validated through tight feedback loops that you’re on the right track, in terms of solving their problem, again, ideally in a way they could never imagine.-

00:11:50
-Concessions. This is important: admit we don’t know. So if we’re going to dive into lean product development, there is just this, the concession: we don’t know everything. Vision has led to failure as much as it has led to success, if not more. Assumptions can kill. There are sometimes, like I said, when you want to go on a hunch certainly low costs, time costs and low risk. But certainly, bigger features, bigger initiatives, once the product is out in the wild or big launches, we want to turn assumptions, as I said earlier, into truths. This is important: it goes back actually, there’s a theme here, this goes back to the vision. The only features that matter are those the customer pays for. There are are some features that we all just have to have, obviously sign in, sign out, create account, stuff like that for software products, but beyond those, the only thing we should put any effort into, again, under the theme of reducing waste, every activity should be aimed at generating value for the customer. Just as that is truth, every feature we build should be one that the customer, whether directly or indirectly, wants to pay for. And then the classic: no plan survives contact with the customer. There’s an ongoing theme in lean of retaining your flexibility or the lean principle of delaying decision making to the last responsible moment. You don’t want to get locked into a roadmap. For an early-stage venture, if your roadmap is going, gosh, six months, that’s scary. If you already have a product out there, certainly those time-outs can increase, but retaining your flexibility under the premise that no plan survives contact with the customer, the obvious anti-pattern to that is that we want to establish contact with the customer as soon as possible.-

00:13:42
-Do you have any questions or are we good? I’ll assume we’re good. Everybody is familiar with this. This is the classic build-measure-learn. I’m not sure everybody knows where to start, what these circles entail. And certainly, for us, we’ve found that there’s kind of a missing piece as it were to this, which is, as I said earlier, that diligence phase which is what we’re going to dive into. And when you break this down into parts, at least, our flavor of lean over the last eight years, really looks like this [Define the customer, Define the customer problem, Validate, Pivot or persevere, Define the MVP, Build, Deliver, Listen). And we’re going to dive into each of these steps and, again, I’m going to offer some of tips, some tactics derived from our experience. And I do want to note that sometimes it is right to define the MVP first. You always are combating, I’ll just say it again, analysis paralysis or diligence paralysis. Sometimes it is ok to define the MVP. I’ll go into win that’s a good path versus one that’s not.-

00:15:13
-So defining the customers is really the seminal practice for any lean exercise, and it really should happen throughout, as I said earlier, the product development life cycle. It answers these questions, and I’m a big fan of documenting these things and in traditional user experience design or even ethnographic research, you do persona work. It doesn’t have to be formal, it doesn’t have to be a big deal. But we certainly should be answering the question: Whose problems will we solve? Often times just by your particular category, or if you have some IP it is implicit if not explicit, but throughout the organization, understanding whose problems we’re solving and ultimately whose problems we’re going to go chase until we shouldn’t chase them anymore is important, but who will pay us for the solution obviously is very important. I put those two little side tracks [slide] there, it’s important to know customers versus users. Certainly MVP software, the user is not the customer and then when you go into single/double/multi-sided markets, your users, again, whether it’s b to b or what I call b to e software- business to employee software, the users never pay a penny. Sometimes the end customer, the classic case is big data, where you’re actually selling data derived from the user essentially for free. So it’s important, obviously, not only whose problem we’re going to solve, but who’s going to pay us for the solution, again, falling in line with classic persona: Where do we find them? If you’re filling up the lean canvas, a lot of this stuff, and I actually like the lean canvas a lot. Actually, it’s really good for super early stage, most organizations abandon it pretty quickly, but again, you can find basically this pattern mapped out on a lean canvas as well. Couple tips in defining the customers is: Focus your “customers,” I’m very wary, very very very wary of clients that come in in the early days of a new venture and have just a ton of customers. To me it means they kind of have to be all things to all people, or they really haven’t started the diligence process in identifying those first two questions: Whose problems are we going to solve and who’s gonna pay us? So we prefer super specific, even from a demographic standpoint, all the way down to titles when you’re talking about the buying customer versus the user. And all of that is, enables us to have fewer targets and better aim.-

00:18:00
-Hey Nick?-
-Yep?-
-It’s Aziza, just one real quick question: What are single/double/multi-sided markets?-
-Yeah, ok, I’ll go back here. Good question. Single sided, most classically expressed in b to c. So you build a product and you sell it to a customer who is generally also the user. A double sided market is very common these days. A classic example is AirBnB, eBay, so you’re often a broker between two parties. You have supply and demand. So you have two customers, I’m going to chase the AirBnB model, you obviously have a supply side, these are homeowners, people who have a place that they want to rent and you have the demand side which are travellers, and you sit in the middle of them as the broker. Multi-sided markets, facebook is a good example, again, many customers. You may have a fast product where there is let’s say end-user licence to actually use the software. You may also have an affiliate program there or something like a marketplace if you’re a platform, so you have another customer in the folks that are building modules and services for the platform and then you may sell some of that customer or user data out to a big data company, something like that. Does that answer?-
-Yeah, I believe so-
-Ok-
-Thank you-

00:19:51
-Yeah, yeah. All of that is, again, going back to, these days anymore, certainly for, I’m using an umbrella term when I’m saying this, but for application apps, usually your user is not your customer, even if it’s just a business. If it’s b to b, that would be a single market, you know, Atlassian, or, you know, if you guys use Jira, that’s an example of a single even though the users are not the customers. Double-sided markets are equally common. Multi-sided market gets really complex, and out of the gates, most early-stage ventures are just happy just to have one customer, and then they see there’s derivative revenue streams which invent, again, a multi-sided market with multiple customers. Hope that helps.-

00:20:46
-Reach consensus, so actually, that tip is ubiquitous across all of these steps. Having consensus internally is huge. Now, that being said, if you’re a small shop or you have a single founder who can impose consensus, totally fine. As long as everybody understands this is the customer we’re chasing, and until we decide not to chase them, everybody needs to get on board because it’s really going to drive every activity going forward. And this is huge this fourth bullet–third bullet: Investors aren’t customers. I’ll–slightly tangential but–I’ll rip through it pretty fast.-
-Hello? Did someone just unmute?-
-Yes. We’re still here—
-You guys there? Strange……I’ll keep going. So remember, if, certainly, you’re venture-backed or long or ideally become venture-backed just remember, probably repeating the obvious, or stating the obvious, you are one of many in a venture-backed portfolio and typically those larger funds, in particular, are looking for unicorns. I remember a few years ago, the word platform was ubiquitous, everybody was saying the word platform. But ultimately an investor, like I said, is not your customer, and moving on the advise of an investor, I appreciate the nuance and how tricky that is, could ultimately alienate the customer and cause failure. Be very wary of that. And then lastly, this fourth bullet is: Document. Another ubiquitous statement. You know, I know it seems pedantic, unnecessary, but it’s a communication device and as ideally you guys grow and you have success, if you can reference a core set of documents that almost becomes you know your, not only your customer’s profile, but it almost becomes a core-value document that people can reference really quickly. Also can track the history of the chase as it were.-

00:23:16
-Customer problem. Again, not that profound. You guys can find this anywhere. But I’m still amazed at how few organizations are doing this at a tactical level, so having defined that customer, and also guys, this is not a laborious process. This exercise should be measured in hours, not even days or weeks. What the customers pay points, again, this is the formation of the customer hypothesis. Where do they eat, sleep? What do they drink? What’s everything they hate in regards to our particular category or problem space? What’s the cost of the problem? What do we believe this problem costs our customers? That could be a literal cost. That could be helping a customer achieve a bottom line increase in terms of operational efficiency. It could be a figurative cost or an indirect cost. Certainly for b to c applications the cost is often not, not literally a dollar amount for b to b applications oftentimes it is. What is the problem costing our customers? That will help you later at this potential pricing stage at least, being able to hypothesize around pricing. One of the existing solutions, super important, most of you guys do this in early days, part of your own diligence, and then where do those, and again, in context to the customer where are the existing solutions to succeed and fail? Again, defining the customer is predecessor to this. And having defined that customer, and it doesn’t have to be singular, I said we wanted to be focused, but try to limit it to, call it, two or three. For each of those customers, what are those pay points in your particular problem space that you aim to solve?-

00:25:18
-Tips on the customer problem: you really can apply this and should, it goes back to the whole software product development lifecycle. Applying it to problems great and small. Obviously, there is the reason for being which is usually expressed in the vision of the company. But also later after the product is out in the wild, which is awesome, there are smaller problems. And that could be derived from not seeing some of the behavior you’re looking for whether it’s a classic bounce or your conversion rate for lower, a small problem count be a UX problem, it could be a messaging problem if not just the big, as I said, the big hairy problem. Yeah, we want to apply this practice across the board. If we’re not seeing the type of behavior we’re looking for, we can formulate the customer problem, hypothesis and this exercise doesn’t have to be formalized. It can be something as small as, “I really think the call to action is unclear” Right? “for this customer.” And then, again, we want to focus on problems. Avoid solutions. It’s hard because, obviously, startups get founded oftentimes upon the vision of the solution. We want to shift that paradigm, at least for now, and really focus on a deep understanding of, not only the customer but that customer’s problem. Adequately evaluate all existing solutions. That goes without saying. Don’t skim over that. I’m always still at this point amazed at pretty deeply entrenched early-stage ventures that late in the game, by all accounts, discover an existing solution that’s a direct competitor or an indirect competitor. So, take your time. Evaluate every, obviously the direct ones are easy, but there may be indirect solutions that exist out there that in a matter of two or one quarter could actually be playing directly on the playground. And then, finally, consider user problems in addition to customer problems which goes back to my earlier statement, again, a good many of b to b to b to c , but really it doesn’t matter. Just know who your users are. Oftentimes you have to satisfy your users. I’ll pick on Jira again. If it was, you know, agile or product management software that project managers developed, but then designers found abhorrent to use, then your customer, who’s generally the business who’s purchasing this license, is not going to buy it.-

00:27:59
-Validate. So this is the big one. It’s really kind of the cornerstone of lean. Validation happens, just like all of these practices, throughout the life of the product. And they happen in early days, literally with facetime with customers, that’s, you know what Steven Blank calls getting out of the office. And they also happen in test phase or what we’ll call the analysis of behavior phase. In this sequence, they’re in context to having defined the customer and customer problem, but, again, when you’re shooting first, and you have defined an MVP and it’s the right call to release that MVP, then often your validation is coming in the form of analytics of behavior analysis. First and foremost is finding your customers. You did a bit of that when you defined the customers. The channels, as I said, where he or she eats and sleeps and plays and what that person reads. Finding customers is always tricky for folks. But I can tell you, unless you have a really narrow, narrow, narrow demography, in terms of what qualifies as customers, craigslist is phenomenal. You can also validate and find customers through landing pages or FBN campaigns. If you have existing customers, obviously, they’re a great source of information. You have to be careful there though because they’ve already to a certain extent drunk the Koolaid. Exit interviews are invaluable. If you guys have practices when customers close down your account just pinging them, you’re probably going to have a less than five percent success rate, but some of them will talk to you and that’s going to be just rich fodder. And then leads. You guys have built teams. And hopefully, if you don’t now, you will. That can be contentious, but because obviously a sales guy never wants to spoil his lead. But with permission, or even just, you know, if it’s institutionalized or ingrained in the culture, for you to reach out on what the differentiators were that made that individual become a lead.-

00:30:19
-Check it. Test, ask and observe. So, early on, this is in the form of interviews and observations really born from traditional ethnographic research. Just watching folks deal with the problem in context, physical context of the environment, validating or disproving that that problem actually exists. In later stages, again, you’re using data. You’re testing actual prototypes. And sometimes for the smaller problems, it’s just basic usability testing: If you wanted to perform action ‘x’ what would you do? What would you expect to happen? And then the last little bullet that I’ll pull up here is that there is a direct correlation between the quality of the feedback, and the quality of the test product. The best feedback you’re going to get is also the most expensive. It’s working software, the actual application. You’re going to get the best feedback, but it’s obvious, the feedback may be negative and you’re going to shut down shop. And you will have spent all your cash. Or you may not have enough runway left to fix what the feedback is telling you to fix. So you have to be careful with that, but there’s a host of ways, both early, like I said, face to face, boots on the ground or, as I said, feet on the street, using data, to, kind of a combination of testing a working prototype, whether that’s just something like an envisioned mockup, paperware, or a lofi actual software. And this is important. Be open to happy accidents. You’re trying to read between the lines, and this goes back to we don’t want our customers designing our product. We want our customers to react. People are invariably poor at discussing and understanding concepts and extraction. They’re really good at reacting to things tangible. Obviously, talking and interviewing is super cheap. Building something tangible has varying costs. So, again, a decision to be made around there. But even with something tangible, we see it day in and day out, where they’ll very clearly describe a problem and it’s emphatic to them, but then knowing that there are existing solutions, they can’t describe how they’ve used any of them. So reading between the lines, both for the good and the for bad, meaning that this problemt is not as pressing as they’re alluding to, is important, and then also, when we get to pivoting, we’ll talk about the happy accidents and lets generate, “Hey,” you know, “this customer doesn’t really have this problem, but he does have this, and we’ve heard it twelve times”. Maybe we chase that. The happy accident. That’s not my phrase. That’s an Edison phrase.-

00:33:10
-And then, finally, proving or disproving the hypothesis. It’s the whole goal. Right? So if you’re using data you can see an increase in conversions. If you’re talking to folks, you can see your target customer and the holy grail is a customer that not only identifies with the problem, but has cobbled together some crappy solution and is actively seeking a professional solution and is asking where the contract is or how much it costs, with varying degrees. But all of that is ultimately proving or disproving the hypothesis.-

00:33:48
-We’ll rip through this. Use data when possible. Obviously, the only way you can really use data is if you have something out there that you can derive that data from. Early days, scripts and surveys, or even early days after the product’s out in the wild, but you’re talking about a brand new epic or a whole chunk of features that might be pretty expensive to build, start the cycle over and get boots on the ground. Again, stay as concrete as possible. Typically in your scripts and surveys, you don’t really need to worry about IP unless you’re talking to someone that could beat you. That’s important, but generally your customers aren’t the ones that are going to beat you. Measure everything. There is no amount of data that is too much. Again, that’s when you’re spending a mixpanel or optimizely, don’t be bashful. Even in your license, it’s the best money you can spend. A lot of that measuring will actually be happy accidents. You’re so focused on if this thing works or if this language is right that, without having measured everything, you’re not open to seeing some behavior that’s either desirable or not desirable somewhere else in the application. Quantify urgent and important. This is important. Could be redundant. A customer that has an urgent and important problem that you can solve, is, like I said earlier, the holy grail. Passive or latent problems, problems that the customer can’t identify with, are not a deal breaker, but they are more complex to solve. And from a pricing standpoint, it’s really hard to charge for something that someone would identify as like a nice to have when it is the entire application itself. Observe the context. This is really, in the early days of existing solutions, but also how folks are using your application once it’s out in the wild. It is typically, and I can say it’s an eighty-twenty. Eighty percent of the time, they use the application drastically different than we thought they would when we were in a conference room designing a product. Being able to see that as soon as possible is huge, and it can lead to validating, proving or disproving some assumptions and hypotheses.-

00:36:11
-And then the last two bullets, be aware of false positives and false negatives. It happens certainly when you have things out there in the wild, specifically around user-experience design. You could formulate hypotheses around conversions that could be really expensive, like redesigning a cart. You have an ecom play, when in reality, there was not a clear call to action three pages back. Be really wary of that, and you want to consider all options when formulating these hypotheses to go prove or disprove. And as I said earlier, the higher fidelity product, the higher fidelity feedback. Go ahead.
-Hey Nick, yeah. There is a question that came through: So related to the quality feedback, how do you gauge the data then? If you only have simple MVP, and you got bad feedback, do you pivot or do you reduce the quality of the feedback? How do you avoid looking for the answer you desire?-
-That’s a great question. That almost goes all the way to that third slide. It’s a variation of admit we don’t know, it’s another one called “be honest”. So, in terms of data, obviously qualitative vs quantitative, if you have data, the real measure of quality of data, from a behavioral standpoint, is volume. So, for quantitative data. Not sure if that answers the question. For qualitative, it’s a little harder. But it sounds like this individual has quantitative data all on a super lofi MVP. What I would suggest is take that data and then still, again, feet on the street, be careful who you talk to. Ideally they’re looking at it through the eyes of a child. And get some almost after-the-fact impressions of where they’re getting hung up on the software. What you really want to delineate is between user-experience problems and just the pure product perception of value problem. But let me check myself. Let me read it one more time if you don’t mind Aziza.-
-Sure. So related to the quality feedback, how do you gauge the data then? If you only have simple MVP, and you got bad feedback, do you pivot or do you reduce the quality of the feedback? How do you avoid looking for the answer you desire?-
-Hmm, yeah so I don’t know. Maybe this person, this organization doesn’t have it live. Whoever asked the question, are you using behavior analysis through GA or mixpanel or are you just talking about, “Hey, I’m showing this to folks, and they don’t seem to like it.”? Sounds like you could chime in on that.-
-Yeah.-
-Yeah. Go ahead.-
-Yeah. Can you hear me?-
-Yeah. Absolutely. Thank you.-
-Ok. So the product is a physical product. So it’s a combination of physical and technology, so it’s a product in a location, so I bring it to the teachers. Some teachers said, “Well, I don’t think I need that”, but the other teachers say, “Yeah, I need it.” So, it’s mixed feedback right now. So, ok, which one do I listen to? Does it mean that this is not useful or that the teacher itself cannot get the value for example? Customers cannot see the value. So, there are multiple data. The data itself. It’s more qualitative, more qualitative. I’m not using, like mixpanel or anything like that.-
-Yeah. Ok. Good feedback. That helps. So, I’ll ask another question. Did you validate the problem? For instance, are you combining kind of validating that the problem even exists in the context of presenting a would-be solution?-
-Can you repeat again, so what is the question?-
-Yeah, I’ll ask it in statement form. If you validated the problem, you have really, so there is a temporal, almost sequential, there’s a value in these exercises, in following these exercises sequentially. If you did validate the problem, it’s never a hundred percent, right? If you talk to any number of teachers, let’s talk, let’s use forty, ok? If you talk to forty teachers and thirty-two of them, in some varying degrees, identified with the problem, for some it was urgent, for some it was less urgent, but all of them said, “Yeah, that problem really exists. I would love a solution to that.” or “That problem exists and yeah, I’d be interested in that.” If you put that step first and then, in the validation phase, and you’ve defined your MVP, which we haven’t got to, but no problem, and you get the MVP and you put it in their hands, and then you’re getting the mixed feedback, then I would argue that the problem lies in the solution. If you’re trying to combine their acknowledgement or their validation of the with the solution itself, it gets very, very muddy. So, what I would suggest is to remove the solution as a whole. I know that’s hard, because you’ve obviously invested in it, but remove the solution as a whole if you haven’t already talked about the problem. If you have–and then go back and talk about the problem and make sure the problem actually exists. If you have talked about the problem, and now you’re getting this mixed feedback, then I would say, yes, the solution as expressed in your MVP isn’t adequate.-

00:42:19
-Ok-
-Doing both, I want to be really clear, doing both, and I’ll say that so, again, following this in somewhat a sequence, certainly, validating the problem first before you design a solution, can be hugely beneficial. If you’re trying to do both at the same time, it’s slightly better than a crap shoot. And once you can remove the solution and focus only on the problem, and then having done that with good data, then focus only on a solution, then you’re not worried about false positives and false negatives.-
-Yeah. I typically try to ask the questions about the pay points, before I tell them what I’m doing.-
-Ok, and—
-And—
-Go ahead.-
-That is how I’m trying to seperate that. Knowing that people can be biased once I tell them the product.-
-Yeah. So if you’re getting good feedback, before, you know, opening up the suitcase, as it were, and putting the product in their hands, and getting really good feedback on the problem, and at every company it’s different, sometimes you talk to five people and that’s enough. Sometimes some really need to talk to, you know, a hundred plus. If you’re getting good feedback, and the customer, as you’ve identified it, and defined it is accurate, so you’re talking to the right people, and then you place the product in the person’s hand, the problem is with the product. But that’s not a deal breaker. I don’t even know that that would be a pivot. I would say it’s a redesign. Or it’s obviously listening to why there is a problem with the product, and what previously was, whether emphatic or latent, an identification of the problem the desire to see that product, why this product now fails. And that’s going to serve as fodder for whether to redesign or a reimagining entirely of the solution.-
-Thank you. I’ll process this. Yeah go ahead. I don’t want to take the whole time.-

00:44:17
-Ok. So, in line with that, obviously, this is pivot or persevere. I like using pivot or persevere in, both for it’s original organizational category, business sense, but also in it’s little micro-sense. This in particular apropos the previous question – what is the customer saying? Generally comes into when we’re focused on the problem – the right customer, wrong problem. So when we go talk to customers and they don’t actually identify with the problem at all or there’s another problem they identify which the conversation leads to. In line with the previous question, this can also be right customer, wrong solution. Right? So, we know the problem exists so that would be you know the bottom right here – right customer, right problem. And then you hit a wall with the actual solution. These right customer, wrong problem, wrong customer, right problem, and obviously the holy grail at least in the problem space, right customer wrong problem. This really can be applied to solution down the line as well. So, right customer, wrong solution, wrong customer, right solution, right customer, right problem. So when you have this right customer wrong problem you have a decision on the quote unquote pivot to go chase the customer. Meaning – “hey, I’m sticking with customer A” – as we’ve defined he or she. Or you know what? I’m gonna keep this problem, I’m gonna chase the problem, and I’m go find a customer that has it. And again, just to – cause I’m happy that previous question happened, just to open that up. Can also say hey, I’m going to keep the customer even though the solution isn’t resonating. I’ll keep re-designing the solution until it does. Or, I’m going to keep the solution, and go find a customer that it resonates for. All of those I would say are bigger pivots. But micro-pivots also exist. I’ll use just, like, copy call to action, reason to believe, workflow, once the application is out in the wild or the product is out in the wild, you can really open up the right customer wrong paradigm to all of those. And then, lastly, the holy grail right customer right problem, right customer right solution, we obviously wouldn’t pivot, we’d persevere and move on to define the MVP.
– How are we doing on time? We got ten minutes.

00:47:01
-I’ve said this actually with the previous question, number one is be honest. Be honest with yourself. I think that’s probably the hardest thing in its entire process. As the previous question asked – how do you avoid looking for the answers you want. The only real way to do this is to be honest with yourself. Even by asking the question – implicitly the question itself – is the capacity to be honest. And taking your time. You know again, certainly these are early days, very early days. This is the cheapest phase of your product development. Ok? The minute you start committing, and building actual things, is the minute your burn rate increases. So we really want to take our time. But ultimately when we want to pull the trigger we want to be decisive. And the last, I haven’t talked much about this, not every problem, however emphatic, urgent, and important is worth solving. Ok? And that can just be from a fundamental business standpoint, capital expenditure to get something to market is so high, and the value, or perception of value on the price of that, you would not break even for twenty years, there’s a simpler solution that exists and so on and so forth. So really, don’t chase problems that aren’t worth solving. Finding what’s worth solving is obviously a decision that needs to be made for each individual organization.

00:48:47
-So there’s that general loop. We move into defining the MVP now, but I do want to pause for any questions, cause I think we’re about, oh, 8-10 minutes out. And I did not make good on my, well originally my goal was to rip through it in 40 minutes and have 20 minutes for questions. Aziza are there any questions? If not I’ll keep rolling.-
– I don’t see any in the chat box.-
-Cool. Ok.-
-Thank you.-
-So, let’s talk about defining the MVP. MVP comes in two flavors: A. The minimum features required to solve the customer’s problem. Again, this theme, I’ll, I hope I set out from the beginning, to ingrain it, it’s really a focus on the problem. Which is, I would say antithetical, but counter to how most organizations, how startups start, which is a vision of, whether it’s changing the world or just solving a problem, we really want to abandon that, or at least set it aside. It will start with the MVP, having vetted the problem, we know that it exists with a known customer, we really decide: Hey, what is the minimum feature set, or feature individual required to solve that problem? The flavor, too, B, is equally important, especially early days, which is, and this really, Ries talks a lot about this if you’re familiar with “The Lean Startup” which is The cheapest test to accurately prove or disprove a hypothesis. So, all of those assumptions, even just the test itself whether that’s an interview, a landing-page campaign, a landing page to know where buying search terms, indeed, even a contract itself. I mean the holy grail of lean is selling products that don’t exist to customers that do.-
-There’s a grocery delivery service story, I think Ries talks about in his book, he just went up and down his building in New York City, and asked if anyone would like a grocery service. He formulated a contract, they signed it, and then that early revenue generated the growth of the product and the service as a whole. But the test there was a combination of talking to folks and actually having the contract, you know. All data is great, but there’s nothing like having a signed contract. On the MVP, typically ‘A’ and, again, obviously, a focus on software companies, but even in the previous example, an analog product, an industrial design product. ‘A’ is typically the product or an early version of the product itself, whereas ‘B’, in software terms, could be any variation of fake, vapor, hacks, wizards, landing pages, surveys, etc. There’s a lot of magic. I’ve a particular, oh, penchant for simple and creative, cheap MVP’s.-

00:51:53
-I’m going to skip over this. Oftentimes, the question is: do you actually build something? Especially in light of–maybe I won’t skip over it–in light of the MVP, if you have something accurate, you’re going to get really good feedback. If it’s not right, you also lose customer number one, through, so, it’s tricky there. Quick matrix: If the value of the feature, from the perspective of the customer, based on your research, or based on a maybe a landing-page campaign or something like that. If it’s known that the customer wants this thing, and it’s cheap to build, always build it. Absolutely build it. Again, build the MVP, meaning flavor ‘A’: minimum features. If it’s unknown, and it’s cheap, based on the premise that you’re going to get really good feedback from having something out there, and it’s cheap time cost, so it’s fast and cheap to build, build it. Conversely, if it’s on the other end of the spectrum, if it’s unknown, you don’t know if people value it, or to what degree they value it, and it’s expensive, don’t build it. Design ‘B’ tests, you know: fake, vapor, hacks, wizards, landing pages, surveys. And then the really hard one is, in the to-build-or-not-to-build question, is when it’s well known, the value of the feature is well known, but it’s also really expensive, you know, that’s just, that’s a tricky one, especially for you’re first launch. It’s especially tricky and defining, if you decide to build that, what to build.-

00:53:44
-I’m sure all of you guys have seen this. This is not mine. This is Henrik Kniberg’s. Everybody references this, but I have a particular problem with it, which is why I actually included it, not to use it as a positive example. Implied in this is that you have to actually grow a product to make people happy, but there are a lot of things that folks can do on a skateboard that they can’t do in a car.-
-Litmus test: When we’re defining what features to include in an MVP, you really boil it down to one or all of these three questions: If we don’t have <feature x>, are we still solving the problem? If the answer is yes, you don’t need <feature x>. If it doesn’t do <feature y>, do we still have a product? If the answer is yes, don’t build <feature y>. And this is important: If we don’t have <feature z>, will we have a business? And this runs around your strategy in particular. If we don’t, if we give it away for free, and then charge people later, meaning do we invest in payment processing, a payment gateway or whatever you guys are using, do we invest in that and the MVP now, even though we’re just trying to prove the value of the product, knowing that later, if we have users that aren’t really engaged that later we’re going to have to charge for them, and in fact we actually lose them. Important question.-
-The first two in particular, ‘If we don’t have <feature x>, are we still solving the problem?’ and ‘If it doesn’t do <feature y>, do we still have a product?’, anything that there’s a yes or even a maybe to those, kill, kill it off the back log. Don’t even consider it.-

00:55:48
-Tips for defining the MVP: Focus on what you’re trying to learn, design tests. The test itself could be the software early release or it could be more obvious tests, some that maybe I also mentioned earlier. And this is important and hard, especially when butting up against a vision, is create a culture of “not yet, maybe later”. That’s important around MVP’s. It gets even more hard, the larger you get.-

00:56:21
-Ok. Build and Deliver, I can rip through this in our remaining time. Spire’s a lean kanban shop. You can, if you’re a software company, and have obviously a host of development methodologies, whether it’s scrum, scrumban, kanban, lean, whatever it is, I would recommend that it’s agile and it’s real. It’s actually practicing agile. Waterfall can work if your little waterfalls are small. But for the most part, you want a methodology that is really enabling flexibility. We don’t maintain backlogs, again, trying to reduce commitment. In building a net flexibility, we really want to put many many tiny features, each of them representing shippable value, but in the grand scheme of software development, all fairly small. They move in one piece. And then super key to any lean process, whether you’re scrum or scrumban, whatever flavor of agile, is that you have continuous deployment. You want to, if you just want to change the call to action, right, just change the call of action or this button style or whatever it is, it’s super innocuous stuff, you want to be able to rip through that in a cycle of like an hour and have in product, like live.-

00:57:45
-And then, again, for early stage, careful integrating with services/platforms/solutions. There’s a tacky joke around marriage and divorce which I will tell, but you will be very careful, especially when it concerns your IP. It’s quick and easy to get out of the gate for software companies by leveraging third party service, low and behold if you have success, now whether it’s your IP, or your customer base or core features to something in the third party that you don’t have control over, be very careful about that. Again, in terms of hacking an MVP or not, don’t ever hack your data. If you’re going to hack things, hack the UI. Make sure it’s usable, but you never want to be in a situation, if you’re where for instance in a relationshional database and relationships don’t exist or you don’t have warranty constraints, you know whatever it is, make sure that your data, which is ultimately, indirectly where all your IP lies, you don’t hack that, even if you’re just trying to release something to learn. Don’t obsess of UX. I know that seems crazy, especially coming from a UX shop, but I think that should actually give it credibility. It needs to be functional and use common patterns, but I can tell you for the vast majority of you, bootstrap library is going to be totally adequate. You know, and using best practices on you know, master-slave relationships, that’s totally going to work. Products in the early stage, very very very very rarely fail on the merits of their design. People don’t value the solution, or I should say when people value the solution, even even really cruddy UX, they’ll see through it. And then last, I know this can be contentious for certainly early-stage companies is using well-vetted technology. You don’t want to be inventing tech while also trying to figure out if your product is going to sell.-

00:59:42
-Listening. Always build a listening period after every release. This can be as long as six weeks, but it can be as short as a few days depending on that first stage of volume, the time, the triggers, what you’re trying to measure and how much you need to reach a good consensus that what you’re seeing is real. And then, avoid multiple concurrent tests. Don’t be testing a bunch of stuff. It leads to a lot of false positives, false negatives. And these are all tools we use. Most recently, I’d say from the last fifteen months, we’ve really only been using mixpanel, but we’ve inherited from some organizations some pretty good optimizely instances, Adobe Target very rarely, I think we did it for three clients or so the last eight years, probably should say. Google Analytics, you know, it’s free, gets you out the gate with a good chunk of data, but I would say if many of you can invest, even a free version of mixpanel will get you pretty far. You can keep those data points when you go to the papers.-

1:00:47
-Now we have walked through the process, and I think that gets me here. And we should be right on time if there are any questions.-
-Nick, yeah, a question came up earlier about whether a customer pays for an MVP.-
-Yeah, yeah, so let me define early-stage releases. For us, I always recommend MVP’s which are friendlies, and small and invited and no, not paid for. K, so that’s important, and if I said MVP earlier when I was talking about price modeling, I was wrong. Alpha, same thing. I call it friends and family, it really shouldn’t be friends and family, but invited audience, slightly larger than the MVP, and no, not paid for. Beta? So, beta, that’s two releases out in the wild already. Beta is tricky. You obviously, you have to invest in payment if you’re going to have people pay for it. And a lot of beta versions are still pretty crappy, and it’s also hard to say, hey, I want, crappy in a good way, hey guys, I really want you to be a beta tester, oh and you have to pay for it. So, pretty rarely do we even charge for beta. Where we really start charging is Z1. Ok? And typically, if you guys can afford it, we’ll grandfather in and create accounts for the whole-stream approach that tested the MVP output in beta. Now, all that being said, that could still be a hundred people. You know, if you can stomach it financially, or the powers-that-be will allow for it, grandfather those in, give them free accounts, and then say, “Hello World” with Z1 with full payment processing.-

1:02:50
-So, thank you, Nick. I had another question come up in our chat box. It says, what did you mean by “don’t hack your data”? Don’t manipulate it to your benefit? So, keep it raw or original?-
-Yeah, good question, no. So I originally, I removed it last night, so probably that wasn’t clear. I’m glad you called me on it. I had affection on to hamper not to hack. So, how much investment do we put in, specifically from a tech standpoint on an MVP? So, when I say hack, certainly, like, using an off-the-shelf UI framework like bootstrap, ok, that’s not really a hack, but it’s not an investment. Do I, oh, do I run the whole thing through google sheets and just read in data into clunky tables, so those are all like hacks, ways to get out into the wild, out of the gate as fast as possible. So the two rules around that for me is: even in an early-stage MVP, being very diligent, that doesn’t mean don’t do it, doesn’t mean do it. But putting some diligence around, hey do we get married to a third party service. And then particularly around hacking the data, if you’re actually, let’s just use a relational db, if you’re actually going to be building the platform on (1:04:16) __________, even the very first version, do some modeling and actually build those tables. Ideally, that will live in perpetuity with the correct relationships, even for the truncated feature set of the MVP. That’s all I meant. I don’t mean hack in a type of security or the traditional term hack. I mean, oh yeah, I just through this together, blah blah blah blah blah, and then low and behold, you get great feedback and you want to scale, but relationships don’t exist or the appropriate attributes or columns don’t exist. So really take your time on the back end, even if it’s something small in terms of an MVP release. Hope that answers the question.-

1:04:59
-Thank you. Another question came through: How much does what we’ve discussed change if you’re developing a hardware product?-
-Hmm. It’s a great question. So, I can’t speak to it because my expertise is in software, but I can tell you that even Blank was doing this with hardware in the late nineties. So you can imagine, I mean the barrier to entry, the barrier to getting things, even early prototypes, you gotta play this game. So, it is possible. It’s not my area of expertise, I can’t help you there, other than we’ve had some colleagues/friends industrial design, consult agencies, you know, obviously with 3D printers, that may not be applicable to you guys, getting working prototypes. Ideo does it, if that’s an applicable corollary. But like, you know, bosses, actually non-consumer packaged-good hardware, I think your best source of, ideally reference of inspiration is actually going to be what Steven Blank was doing back in the late nineties, I would say mid-nineties. Again, according to safe techniques. And I would urge you to take a look at “4 Steps to the Epiphany”, he described it in great detail. Just not, I don’t know enough about it to help you there.-

1:06:24
-Alright, are there any other questions? If you have a question, you can take yourself off mute and ask yourself. Alright, well thank you Nick. Thanks everyone for joining. If you’re interested in getting a recording, please reach out to me. My email’s Aziza@innosphere.org. You should have it in your inbox from a message I sent earlier. You already have the pdf of this presentation and Nick’s contact information if you have any further questions.-
-Yeah, feel free, feel free, seriously, feel free to reach out. Love to answer any questions or anything, again, no holds barred if you need to reach out, feel free to.-
-Great! Thank you.-
-Alright. Thank you all.-
-Bye.-

 

For more from Nick, contact Spire.