To Be Continuous: Using Continuous Delivery To Find Product/Market Fit
In this episode, Edith and Paul discuss discuss how you should use the principles of Continuous Delivery to find product/market fit for your startup, tool, or service. Along the way, they highlight common reasons startups fail to find product/market fit, from undervaluing sales and marketing teams, to simply believing that if you build it, they will come. This is episode #23 in the To Be Continuous podcast series all about continuous delivery and software development.
This episode of To Be Continuous, brought to you by Heavybit. To learn more about Heavybit, visit heavybit.com. While you’re there, check out their library, home to great educational talks from other developer company founders and industry leaders.
Edith Harbaugh: So Paul and I were down in New Orleans to be part of a DevOps panel at Collision Conference, and as part of it we both mentored startups. So Paul, do you want to talk about some of your experiences down there?
Paul Biggar: I guess the first experience is that if you’re going to do four hours of mentoring startups in a row, you’re gonna get a little bit tired.
Edith: Yeah, we were supposed to go to dinner and you totally bailed on this lovely paddle wheeler in the Mississippi.
Paul: Oh yeah, I was dead to the world. And so all the companies that I end up talking to. I did this at Web Summit, as well, and we both did this at Web Summit. And every company that comes along kind of in the early stages is like, how do we raise money? And the answer is, to me, always: have you got product/market fit?
Edith: I actually disagree. I think if you’re early in a seed stage, or even an angel, product/market fit might be asking far too much.
Paul: Yes, but I guess… Maybe it’s not product/market fit, but it’s like, are you on the road to product/market fit? Do you know that product/market fit is the thing that you are searching for?
Edith: Yes, so I went through Alchemist Accelerator and now I mentor a lot of the startups and later classes, and at the early stage is this three-legged stool: team, like, do you have a good team?
Paul: Yep, I think I know where you’re going with this.
Edith: Dream, like, is this something that is a big market that is interesting, that is a vision? And then early signs of traction. And they all three go together. If you’re doing a dream that you know you’re not the right team for, that falls flat. And if you have an awesome team you don’t need so much traction.
Paul: So, yes and no. And in fact, I see a lot of people getting into trouble with the team stuff. They’re like a couple of ex-Googlers who find it very easy to raise money, and then they have a dream that’s amazing and then no real idea of how you do that. Or very often, the idea, as is very common in software engineers, they’re like we’re gonna engineer this perfectly and so the company will succeed.
Edith: Yeah, I mean, I’m not saying so much for long-term success, I’m saying for raising money, this is what investors look for. I know some ex-Twitter people who raised a lot of money because they’re ex-Twitter.
Paul: Well, you wrote that article about, oh, what was that company called, with the guy who had a previous exit and he raised 25 million? Famo.us, Famo.us! Yeah, okay. So the idea there was, as I understood, we’re gonna do something new and we’re gonna build the whole thing and then there’s no iteration and there’s no product/market fit, and I think they’ve pivoted and laid off the team and that sort of thing. Which I regard as the sort of punishment that you get for not chasing product/market fit at the very start.
Edith: So I think we actually agree.
Paul: Oh, yeah. We’re definitely gonna agree on this.
I think you need to chase product/market fit, but I think in the early days to have product/market fit is asking far too much of an infant startup.
Paul: So the key is validation. So product/market fit is like the grand validation and the road until that point is validate, iterate, and repeat.
Iterate early, iterate often.
Paul: Yes. So Jesse Robbins, who’s CEO of Orion Labs, is an advisor to Circle, and the way that he put it, and I think we were talking around like Series A, is the seed stage is to get product/market fit. The next thing is to prove out your traction, and that typically lines up with the A, and then the B and later is to scale that.
Edith: And by the way, Paul. Congratulations, as CircleCI just raised a pretty whopper Series B.
Paul: Oh, thank you, thank you. You can date the actual recording of this podcast by the fact that you just said that, so our attempt at having some slightly evergreen content…
Edith: Maybe I knew before you announced it.
Paul: Oh, that would be impressive. So yeah, we just raised our 18 million Series B, which firmly places us in the now-we-are-scaling camp, having achieved our traction, and before that having achieved our product/market fit.
Edith: Paul always talks about how he uses LaunchDarkly, but LaunchDarkly is a CircleCI customer, so we’re pretty happy about it, too.
Paul: So a lot of people skip product/market fit.
Edith: How can you skip product/market fit?
Paul: Well, you can, for example, launch at South by Southwest. Or you can have a really big day on Product Hunt. And you can get tons and tons of traction and not actually get product/market fit. And so a couple of companies have done that. Secret is a good example. They got a whole lot of hype and stuff, but never product/market fit.
Edith: I disagree on that one.
Paul: You think they had product/market fit?
Edith: I think they had product/market fit, I just–
Paul: No, they had signs of potential product/market fit.
Edith: I think maybe we’re arguing a definition, because Secret, I think all apps like that kind of ride the cycle of popularity and then the trough of disillusionment.
Paul: So if you have product/market fit, generally, so long as the market is big enough, I think you have no problem scaling. Or I think you have no problem growing. So you can skip product/market fit if your growth is based on an external factor that isn’t product/market fit. So it’s not, the product is perfect, it’s, Ashton Kutcher tweets about it. Or Secret, you know, there’s a lot of juicy gossip on this thing.
Or there’s another company that I don’t want to mention because it went a bit of a disaster, but they raised a lot of money after TechCrunch Disrupt or something along those lines, got tons and tons of users and survived about three years on it before realizing that they never had product/market fit and laid off 3/4 of their company.
Edith: Who was it?
Paul: I don’t want to say who it was.
Edith: What industry?
Paul: It was in the tech industry.
Edith: What sort of product do they have?
Paul: Every fact that I’ve said so far is actually less or slightly different to the truth here. I’ve used this example before.
Edith: I’ve got 16 questions left.
Paul: Right, well, you’ll probably find 16 different companies that fit the criteria. My point is, though, that you can get traction without product/market fit. And companies that do that, and Famo.us was kind of like that, they skipped their product/market fit and went straight to the 25 million raise. What was that, Airwave and the Chatroulette replacements? Same diff.
Edith: One of the founders was a friend of mine. I think they raised money very quickly ’cause of the team + dream thing, you know, was co-foundered by Sean Parker who founded Napster, and was early at Facebook and everybody’s like, “oh gosh, we need to get in now.”
Paul: So I was doing some testing for that company, and at the time they had about 20 people and they were testing it for scale and that sort of thing for the big launch that was coming.
Paul: No product/market fit. The big launch was going to be the thing that won or lost the company, and there was no evidence that people loved the product.
Edith: Yeah, I think the saddest thing I read was there was a postmortem, I think it was in the Wall Street Journal, and they’re like, at its peak they had like four concurrent users.
Paul: Oh, wow.
Edith: Just something god awful like that.
Paul: I thought they had like 30,000 or something.
Edith: Maybe I’m exaggerating.
Edith: But I think it goes back to something that Steve Blank wrote, which is “Don’t launch.”
Paul: As in, don’t have a launch.
Edith: Yeah, because when you “launch”, you lock yourself into, this is our market, this is our positioning.
Paul: Oh, that’s interesting. So I thought that this was implicit, that you don’t launch because you soft launch and grow from validation and iteration.
Edith: And it goes back, because if you put all your arrows out at the same time and staking out your ground as, this is what we are, then it’s very hard later to move away from that sometimes. So LaunchDarkly never really did a formal launch.
Paul: Right, Circle didn’t either. We kept going for, when are we going to launch, and eventually we were like, why don’t we just open it up?
Edith: We did a blog piece about how we’d hit a billion feature flags a day.
Paul: Okay, that’s your launch.
Edith: I guess it was, but it wasn’t intentionally a launch, it was more like something to be proud of, and we put out a piece on it.
Paul: We kind of launched when we raised our seed round, and we had been out in the open and things for months at that point. But we were sitting in the war room, bracing ourselves for the flood of customers that were coming in from the launch
Edith: It didn’t come.
Paul: Oh my God, none came.
Edith: Yeah, that’s the thing. I mean, I’ve been part of so many launches. People are always fearing the worst. They’re like, oh God. What if our servers crash? Everybody fears the story like Sam Stokes, who was a guest before, when they had this surge of traffic, then they were overwhelmed. What happens 10,000 times for every one of those?
Paul: So we had, I’d say double our signups that day. And we had respectable signups, now they’re doubled, it’s like, oh great, it’s a good day. But it wasn’t… The war room was unnecessary.
Edith: So what was your launch that you thought you needed a war room?
Paul: We were in TechCrunch and ReadWriteWeb, and you know, the usual places that we announce fundraising and two of those stories did reasonably well on Hacker News, and we had a blog post which was doing okay on Hacker News as well. And there’s some Twitter stuff going, so we thought, oh, there’s gonna be stuff.
And I mean, doubling our signup rate is no… Well, it was a small feat, but it wasn’t an incredibly small feat. Just, it didn’t make that big of a difference.
Edith: I think to unpack that, something interesting is, some purchases, and I put CircleCI in that bin as would I put my own company, they have a longer buying cycle.
Paul: So, yes, but that’s why we were measuring signups.
Edith: But I mean even signups, like what we found, and probably the same at CircleCI, is like it’ll be percolating at a company for a couple months–
Paul: Oh, yeah, yeah, yeah. And this is why we all use retargeting.
Edith: Well, yeah, and I get retargeted all the time for CircleCI.
Paul: Yeah, good, good, good.
Edith: So I’ll talk to our customers who are signing up and like, how did you hear about us? Oh, we saw this Hacker News piece, we saw this LinkedIn thing you did, we saw this article, like it’s not this…
Paul: Then eventually we talked to you at a conference and you installed it for us on our laptop right then.
Edith: Yeah, so it’s never like…
I think there’s this myth that you launch, and I think it does happen for consumer products, perhaps, where people are like, oh, I need to buy that today!
Paul: Yeah, sometimes.
Edith: I think for a lot of products, you percolate in people’s consciousness for a long time as a buying decision.
Paul: Yep. There’s an instructive article by Kevin Systrom, the Instagram founder, about why Instagram grew so quick. And it’s a story about the start of Instagram that was conspicuously absent of why Instagram grew so quick. I’ve read it a couple times. Basically the conclusion I came to is that he didn’t know.
Edith: Yeah, I think with apps it is much easier to grow very quickly, and when I say apps, let me–
Paul: The mobile apps?
Edith: Consumer apps.
Paul: Yeah. So I think the people who do that don’t actually know why. I think there’s a bit of, you know, rub the lavender leaf on the cancer and it goes away sort of thing. Well, sometimes it does.
Edith: I think you can tune for more than you think you can.
Paul: I buy that. But I would say that generally, iterating and validating is your key to getting there.
Edith: Yeah, and I think I’ll go further out on this. It really depends on what your product is, how much you can be an overnight success. Like even Slack. So Cici Stallsmith came, gave a talk at Heavybit. She’s like, “The first 70 iterations that were built on Slack, Slack-built.”
Everybody now is like, oh, we’re seeing the end of the adoption curve where it’s the steep up-slope at the end and we’re not seeing the groundswell for years before that.
Paul: When we have these mentorees and that sort of thing, and we’re talking to them about, you know, you want to raise money, you need the team, you need the idea, you need the traction, it’s almost a trope for people to talk about how people keep building products and hoping that people come.
I would say that for 75% of the people I talk to, that’s still the goal. We’re building the product and then people will see how wonderful it is.
Edith: Yeah, I mean, Tomasz Tunguz, he wrote a blog about this. He said you should be spending half your time on go-to-market.
Paul: Right, that’s a really good piece of advice.
Edith: Yeah, and I think there’s this delusion that engineers have, and I’ll put myself as an engineer, if we build something great, people will figure out it’s great. It’s like, no. People won’t even get past your install.
Paul: Yeah. There’s a wonderful story, and I don’t remember where I read it, but it was the engineer who saved up tons of money to start his own company, and when the day came, he quit his job and he went out and he built this wonderful product that was beautifully engineered, and then he waited.
And nobody came, and eventually after a bunch of time he realized that he didn’t have what it took, and he shut down the thing and went to go get a job and every day was kind of the same as it was before, but just a little bit sadder.
Edith: I need to have a moment, Paul.
Paul: I know, right? I read this much before I started startups, and I was like, that’s the depressing thing that you want to avoid.
Edith: Well, that’s why I became a product manager. I was tired of building stuff that nobody used.
Paul: Mm-hmm, right.
Edith: That’s then why I became a marketing person. It’s just like we built this cool thing. If nobody knows about it, it doesn’t exist.
Paul: Yep, yep. So next you’re going into sales? Complete the transition to the dark side?
Edith: It’s funny, I am a sales person. Every CEO is salesperson number one. The number one person you’re selling is yourself, that this is worth doing, then you’re selling your co-founder, that they should quit their stable jobs and come join you on this mission. Then you’re selling to customers, and then you’re selling to investors, then you’re selling to your employees.
Paul: And for each one of these, there’s a traction component. Or there’s a product/market fit component. And any time that you try to focus on making that happen, without there being successive levels of product/market fit.
So for example there’s this company that hired a bunch of people that I thought were really impressive. So they were roughly in a competitive space with us, and they hired three people who I thought were just awesome. And I was like, holy shit. Look at the people that they were able to hire. And three months later they’d all quit.
Paul: And the backstory was that the founder was a complete asshole.
Paul: This is the product/market fit for the recruiting pipeline. You need to have a culture that makes people want to stay.
Paul: You can get as many people in the front door as you’d like, you can do all the marketing you like, but if there isn’t product/market fit, and, you know, this is employee–
Edith: Employee churn.
Paul: Employee culture fit.
Paul: Yeah, and then people just fall out the other end, same as in your product, same as when you’re trying to sell, same as when you’re trying to market.
Edith: Yeah, I think that’s, maybe other founders are smarter than me, but it was kind of this ah-ha moment for me. I remember when I was a younger engineer I sat in on a meeting once and a vendor came and pitched us, and it was one of those awkward meetings where they’re really pitching us and I remember they walked out and I turned to my coworker, like the young engineer and I was like, “I’m really happy that I’m not in sales.”
That moment has come back to me so many times when I’m walking up to some prospect’s office and checking in, I’m going into their room to set up my laptop to do a demo and I’m just like, goddammit.
Paul: But I mean, you’ve probably learned from that and you probably have done it in a much better way that isn’t gonna be so insulting to the engineers or whatever that was.
Edith: Oh, that was just a realization. I believed in the company that John, my co-founder, and I were building. I was like, “Shit. I gotta go do sales. This is my job.”
Paul: So sales is one of the validation techniques. Jessica Livingston, the YC founder, or the founder of Y Combinator, had an article in the Wall Street Journal about this, and it’s like founders need to market less and sell more.
Edith: Interesting, I don’t think I read it.
Paul: So basically the idea is that when you’re pre-product/market fit, a lot of founders are looking to do marketing. Or they try to bring thousands or hundreds of people, leads in the door, and they try to do self-service. But that at the very, very early stage, when you’re validating, the thing that helps you validate is sales, not marketing.
You can’t see from the analytics what you need to do to iterate your product, you can only do that by trying to sell or selling as a proxy for talking to your customers.
Edith: Yeah, it’s funny you say that, because YC has this rep, and maybe it’s undeserved, of wanting everything to be purely marketing because they don’t want salespeople because they don’t like talking to people.
Paul: I think that’s the one of those things that’s demonstrably untrue. Thinking of YC companies that are heavy on sales, like Optimizely, for example. I would say that certainly they have a very strong focus on product/market fit, and the most successful YC companies have had this incredibly strong product/market fit, and I’m thinking Heroku and Stripe.
When you have that strong product/market fit, sales really isn’t that important. But what they do have, and Stripe is really good example here, because the founders of Stripe went out at the very start and they sold. And they might not have called it sales, they might have called it validating, but they literally went to their customer’s house and John and Patrick installed “/dev/payments”, as it was called then, in the software.
Edith: Yeah, I sold. I mean, I remember when CircleCI was considering using us, we went over to your office.
Paul: Yeah, right, right.
Edith: It’s funny because in hindsight it’s like we went over to your office and it was just like, okay, you guys are legit. Nobody even looked at the product, it was just, we’re taking a bet on this risky startup. We want to vet you.
Paul: Probably don’t need to defend Y Combinator too much here, but Paul Graham says “do things that don’t scale”. And so at the very start, the thing that doesn’t scale is fucking validate your product.
Edith: So I feel really relieved now because I’ve been doing all our sales and finally I was just like, “I’m buried.” It’s all inbound.
Paul: That’s great.
Edith: I was actually telling our marketing person to turn it down because I couldn’t keep up.
Edith: We hired a full-time salesperson.
Edith: He is busy all day. He has more than enough to do, and I’m like, okay, I was doing all that and trying to do everything else. I am so happy that we have somebody who’s just full-time on sales now, but I would not have hired him a year ago because at the time I needed to be out there doing it.
Paul: Yep, yep. Because you need that feedback.
Edith: Yeah, I needed that feedback, I needed to be the one out there, talking to people and getting the ground feedback.
Paul: A lot of founders have this idea that it’s gonna market itself and everything’s gonna be top-down or everything’s gonna be bottom-up and it’s gonna be self-serve or maybe have sales later on for other stuff.
It does depend on the situation, the company. You guys are a bit more enterprise-y, we’re a bit more bottom-up. But I remember the, probably shouldn’t say who this founder was, but someone we know and love.
Edith: You know I’m guessing already.
Paul: Yeah, so at one of the Heavybit dinners, one of the Heavybit advisors stood up and said, “We’d like to take this moment to recognize this person who has finally succumbed to having salespeople in their company and seeing the sales go through the roof as a result.”
Edith: I don’t think it’s succumbing at all, it’s that
if you’re a complicated product, if you’re something that people are betting their business on, if they have questions, they want somebody to help them.
Paul: Right, oh yeah. No, I see it, totally.
Edith: I don’t think of it as succumbing, it’s just like people are–
Paul: Well, so the thing is that engineers do see it as succumbing. It’s seductive to only have marketing and seductive to not even have marketing, you know? It’s seductive to only have like–
Edith: An API.
Paul: An API, or at least authentic getting the name out there, which don’t give it that evil word marketing. Marketing is something that marketers do, and marketers are wrong people in some way.
Edith: Yeah, it makes me sad when I hear marketing scoffed at as something less–
Paul: Same with sales.
Edith: Like, oh, if only we didn’t need them. It’s kinda like, how is somebody gonna hear about your product? How is somebody gonna get to know how to use it?
Paul: You see a lot of developers who are very good at marketing sometimes, so they’re really good at getting posts on Hacker News, or they have a big Twitter following, or they’re really good on the conference circuit, or something along those lines. And I find that they, most of all, do not like marketing, despite the fact that they are amazing at marketing, they just don’t have a word for the thing.
Edith: Yeah, and let me loop back.
Good marketing and good sales ties back very closely to your product.
Paul: Right, right, right. This is why you have to do it yourself.
Edith: Yeah, originally John my co-founder and I were working on another idea and I’m like, “I can’t find anybody to use this.” Or like, the only people who are using it are doing it because they’re really good friends.
Paul: Now imagine that you had spent a year building that, waiting for it to come.
Edith: We moved onto another idea, but we built marketing into the very beginning, and it was something we took seriously in terms of, can we get people to use this? Is this interesting? And that definitely informed our product.
I think where people run into trouble is they’re like, well, I’ll build a whole product and then I’ll throw marketing on it.
Paul: Yep. Coming back to the start of this, we have very similar thing, yeah? We’re gonna build this, and then we’re gonna throw fundraising on it.
Edith: The saddest thing is–
Paul: Oh, no. I think I know where you’re going with this.
Edith: We’ll raise money and then we’ll hire marketing and sales to figure out how to sell and market it.
Paul: Oh, even worse. We’ll raise money to get someone to build the product.
Edith: No, no, no, no, no.
Paul: I’ve seen that so many times. It hurts so much. It’s like, you know, do you know anyone to outsource this to, and I’m like, no! No! The reason that I use to explain to people why they shouldn’t outsource is continuous delivery, right?
You need to iterate and validate so quickly that having development cycles that involve shipping it to India or Estonia or something like that. It’s just too long of a product cycle.
Edith: I think you should also be continuously iterating on your marketing message. That’s what I meant, is like, okay, what marketing messages are resonating? What features are people actually interested in? What benefits resonate? And feeding that back very quickly into the product.
Paul: I would say that at that stage that you’re talking about, there isn’t really a major difference between marketing and sales and product. It’s all kind of going on in the founder’s head, so the fact that there’s not necessarily a formal split of like the positioning that we’re doing, really it’s just like what we write on the website.
Edith: Well, yeah, and what I see as the killer is when people are like, oh, we’ll go off and build this product and then leap out of stealth with our fully-formed positioning and this huge launch that will hit everyone, and it’s like, no, you need to do those in parallel.
Paul: You’re right, they all absolutely need to be done in parallel.
Edith: Like the positioning of LaunchDarkly, to be honest, is very different than when we started.
Paul: What were you when you started? I recall that you speculated that bottom-up would be your go-to-market.
Edith: Originally we were much more targeted at product managers. And then I was like, “Hey, I think product managers still love us, but the people who we need to win over are developers.
Paul: Oh, interesting.
Edith: What that meant was just a lot of why we joined Heavybit, is because, okay, this is really something that developers like.
Paul: So as I understood it, you were selling a little bit top-down.
Edith: Originally we wanted to sell bottoms up, for all the reasons you just gave.
The conclusion we came to was that the developers are users, and they love us. To use feature flagging is not something an individual developer can push through.
Paul: Mm-hmm, okay, I can see that. So you’re kind of doing a top-down sell to developers.
Edith: Well, developers are our users. A director, a team lead, a CTO is our buyer.
Paul: So do people buy without talking to you?
Paul: Okay, and you see that increasing?
Edith: It’s funny because I don’t talk to everybody anymore.
Paul: Yeah, well, that’s the sign of product/market fit getting really strong. No, I’m totally serious. The first 50 or 100 customers, I knew all them. I had phone calls where I was like, “What are you using? What aren’t you using? How can I help?” I was debugging people’s builds over the phone.
There came a point, and I felt actually really bad about this, that people would come up to me and say, “Oh, are you CircleCI?” and I wouldn’t know who they were.
Edith: Oh, wow.
Paul: And obviously, right? I mean, Tim Cook doesn’t know the name of someone who buys their MacBook.
Once you get anywhere near success, you have no idea the names of your customers.
But at the start you know everyone. And so as you start to lose track of that, you are obviously on a good path in success, but it feels kinda shitty when someone walks up to you and you’ve never heard of them before.
So pre-product/market fit. So when you’re this company that, you’re looking for fundraising, or you’re looking to build a startup and you think fundraising is on your path, and so you’ve got people but there’s no team. You’re not Steve from Famo.us or something or the Googlers or the Twitterers.
You’ve got an idea that, no one really invests on the idea. So all you’ve got to go by is traction, which you don’t have yet. How do you go from here to there? How do you go from, we’ve got a bit of a product, we’ve gotten a couple people that we talk to maybe, or no customers, and we want to be a venture-backed startup.
Edith: There’s actually a really good book called Traction, which I actually really enjoyed.
Paul: I love that book, yeah.
Edith: Yeah, and the best advice I got out of it was you iterate on your marketing just like you iterate on your product.
Paul: I think there’s much more of a product role to be done before then. Traction, the book, provides you a lot of methods of taking the customers that you have and retaining them or growing them, and finding new customers, all of which can be used to help you validate the existence of product/market fit or the path to product/market fit or how much traction you’re getting. But I think ultimately the only thing that really gets you there is how quickly you can iterate on the feedback that you’re given from those customers.
Edith: Yeah. I agree and I disagree.
I think you can fall into a trap of listening too hard to your very early customers.
Paul: Sure, sure, sure. I guess I’m not being particularly clear on this.
I think the road to product/market fit is listen to your customers. It’s not do exactly what they say.
Edith: I think it’s both listen to your customers and constantly be thinking about how you can get more customers. So one of the saddest stories I heard… well, I’ve heard many sad stories. It was a friend from Alchemist, and he said, “We had two customers who loved us very much and we could not get any more.”
Edith: So I see all the shattered dreams, because I keep in touch with the accelerator, and know all the people from the prior batches. He’s like, “We had two customers who loved us, couldn’t expand beyond that.”
Paul: Wow. That’s really tough. I mean, it’s kind of the same as hitting traction without going through product/market fit, that you’ve got this false sense of security, that people love it, people use it all the time, and actually you’re missing this key component that you should be worrying about, but that maybe you don’t know that product/market fit is your thing.
Fortunately with two people… I mean, imagine they had 10. Imagine they found the only pocket in the world of customers who needed that product, and then they raised money and continued on for two years or something with like, why can’t we get more than 10 customers? Or 20, whatever.
Edith: Yeah, and you could make a very profitable business if those are the right niche. But if you’re charging each of those two customers like $500 a month, then you can’t make anybody else happy. You’re in a hole. The other thing to really watch out for is that you don’t turn into a body shop, into a consulting arm. Because you can be making your customers wildly happy, but you’re actually doing that by being services, which isn’t a scalable business.
Paul: So looking at that via feature flags, I know a bunch of companies that have, if customer ID is our largest customer, do this, else, do this. So I imagine that you can end up with a situation where feature flags are abused to provide this. This customer gets this feature, this customer gets this feature, and no one is using the same product, no one has the same product, and everyone is kind of like a customer you build for.
Edith: Well, first, thanks for mentioning feature flags and thanks for wearing your LaunchDarkly shirt today.
Paul: Did we not do this already in this episode?
Paul: Oh, that was the last one.
Edith: And I had to get it in, ’cause you’re still wearing the shirt.
Paul: Of course.
Edith: I’ll actually give a really nuanced answer to that one. I think that’s fine if you’re using feature flags to act as a license mechanism.
Paul: Okay, that’s not what I was talking about.
Edith: Yeah, but I’m thinking about how some people are using our product, and it’s to enforce licenses where people are kind of doing a Choose Your Own Adventure of different features. That’s fine as long as it’s some standardized bucket of features. If it’s truly that you’re using feature flags as a substitute for branching, and everybody has these one-off products, that’s where you run into a lot of problems, because then you’re basically, like any tree, you have a lot of branches that you can’t maintain.
Paul: So I think the key there is, sometimes you’ll build those to get feedback. You’ll build them to help validate, and you’ll build it for… I remember hearing that Heroku had five different SSL implementations.
Edith: Wow, why?
Paul: I guess there were different properties and they were just like validating or iterating. But there were customers on each of them. And I think they were quite slow to prune, so if you’ve ever been to a Heavybit product discussion that features managers, or PMs, or long-time devs at Heroku, they’ll always talk about sunsetting old features. That’s like a major part of what they do, because they weren’t graded to start. And that came back to bite them.
If you have a feature flag that’s like, if Customer A, do this, if Customer B, do this, if Customer C, do this, that’s often product debt, and it’s a mortgage you take while you’re building it, and it’s good if you pay it back. And it’s bad if you forget to pay it back, or you don’t have time to pay it back, or you’ve repeatedly taken out, keep taking out this kind of mortgage on your product.
Edith: And it’s not even a product, it’s that if you’re