To Be Continuous: Finding Co-Founders, Engineers vs. Artists, Funding Napkin, NPS, Hiring

In the latest episode of To Be Continuous, Edith and Paul discuss 5 topics that are top of mind at most startups. First, they examine approaches for finding a good co-founder. Next, they delve into Michael Dearing’s course on the artistic side of engineering. They then move on to discuss the SaaS Funding Napkin and why 27% of companies that raise series A have no revenue. They also share their experiences using Net Promoter Scores and consider the pros and cons of hiring fast vs slowly. This is episode #32 in the To Be Continuous podcast series all about continuous delivery and software development.

This episode of To Be Continuous, brought to you by Heavybit. To learn more about Heavybit, visit heavybit.com. While you’re there, check out their library, home to great educational talks from other developer company founders and industry leaders.

TRANSCRIPT

Edith Harbaugh: Paul, you found co-founders twice?

Paul Biggar: Three times, actually.

Edith: Three times.

Paul: Yeah, well I guess the first time they found me.

Edith: So tell me about finding co-founders.

Paul: I think that the major thing that I could tell you about finding co-founders is how to do it wrong. And in particular, how to avoid those kind of problems.

Edith: I’ve only found a co-founder once, and it’s worked out so far.

Paul: So, how long did you know John?

Edith: I don’t want to admit how long.

Paul: 15 years?

Edith: Not that long, but a while.

Paul: Okay, I think that’s the major thing about co-founders, you want to find someone who you really really get on well with. That you have great chemistry with, and I think knowing them a long time is probably the best way to do that.

Edith: And I think, even then, I had hesitation. You know, because there are many people that I will go to happy hour with every night, but I would never in a gazillion years start a company with them.

Paul: That’s right, I feel the same way.

Edith: And it doesn’t make them bad people.

Paul: Sometimes they are bad people.

Edith: But they’re still fun to go to happy hour with.

Paul: Yeah, exactly. I have a handful of friends. I know they’re assholes, but they’re still my friends.

Edith: Who’s on that list?

Paul: No one you know.

Edith: Is the top of the list Edith Harbaugh?

Paul: Yeah, exactly. With a dagger drawn in. So the best way to find a co-founder is someone else recommends them. So like all things, if you’re interested in being my co-founder, you should get someone that I know and respect to recommend you.

Edith: So Edith is automatically not on the list then?

Paul: Edith is not on the list. I don’t know anyone who respects you that I also respect.

Edith: Wow.

Paul: That is a complete lie, obviously. I’m sorry if I hurt you, the truth is obviously the opposite.

Edith: Paul’s like, this is funny until …

Paul: Yeah, stepped over that line.

Edith: Does it ask them what’s their favorite 80s movie?

Paul: I think something along those lines, but mostly about startups. So it’s questions like “what kind of startup do you want to build? Do you want to build a big startup, or a small startup? What do you view your role before you find product market fit? What do you want your role to be after we find product market fit?” What are your skills in terms of sales, in terms of marketing, in terms of product validation, in terms of software engineering, all this sort of thing.

So I had around 27 questions that I would expect someone to write about a paragraph about and I also answered them, and so we trade them and then we go and have a chat about them. And we see what are the strengths, do we complement each other, are we able to work through the differences? Basically, those kind of questions, and this is also what OKCupid is really about. Do we have the same shared values?

Edith: And this is fascinating, I mean are you looking for 100% fit, or do they have to be kind of a close match?

Paul: I think the most important thing, really, is the chemistry and being able to resolve differences. Because there are no 100% fits.

Edith: Yeah, it’s funny, so I give a lot of advice to people from my old accelerator, because I try to get back, and the two bits of advice I always give are, make sure you love your space.

Paul: I think love is a good.

Edith: Because there are going to be days which are awful, and if you do not have that, your company will fall apart in an afternoon.

Paul: Right, absolutely.

Edith: Because literally, I’ve seen it happen, I never liked this space, and I never liked you. No, literally, like fuck you I’m quitting.

Paul: So I think Y Combinator says that about 60% to 70% of company failures is just like co-founder relationships.

Edith: Yeah. I don’t want to have you talk about anything you’re not comfortable with, but what do you think didn’t work out with earlier co-founders, what were your lessons learned?

Paul: So, being on the same page about what we’re building, I think is one, and then knowing how to communicate. So knowing how to communicate together. And I think many of my co-founders in the past have been engineers and I was an engineer.

And we all suffered to a certain degree with the issue that engineers don’t necessarily have a great deal of empathy.

Edith: Yeah, I mean there are so many times where you wonder “why did they do this?”

Paul: Exactly. You never want to end up in a situation where you hate each other. Or even where you have the potential to do that. You can’t rescue relationships. By the time you get to the point of trying to rescue it, it’s already dead.

Edith: I was going to make a joke, and then I was like I don’t think this will be funny.

Paul: No, no. Everyone’s thinking back to like the time that the relationship that failed that shouldn’t have failed, and it’s not a joking time.

Edith: I think what was helpful for John and I is that we knew we wanted to start the company, but we said, I said this, I said let’s work on it for five hours a week before we quit our day jobs.

Paul: Yes, great.

Edith: Let’s figure out if we can work together. So it wasn’t what we’re working on now at all, it was an idea like four ideas back. But we’re like, okay, we can work together.

Paul: So one of the interesting things about this, is that I have one idea that I want to work on. And I don’t want to pivot my way to a good idea.

Edith: You want this idea.

Paul: I want this idea. And I mean, there’s variations of how the idea might actually hit the market, but I’m not going to choose between this and a consumer app and some other app.

Edith: Well so John and I were in a different space, because we knew we wanted to work together, we didn’t have an idea.

Paul: Right.

Edith: So we kept thrashing about and finally I said this is not the right idea, let’s just try working on it.

Paul: But you ended up in a space that you both loved.

Edith: Yeah, because we said okay we could work together, this first idea isn’t right, the second idea wasn’t right, the third idea wasn’t right, the fourth idea we were like yes!

Paul: Did you do anything to improve how good co-founders you were to each other. And so what I’m thinking of here is Rap Genius founders go to couple’s therapy.

Edith: We haven’t done that yet.

Paul: Might be an idea. You always want to go to couples therapy before you have a problem. It helps you communicate, it helps you be on the same page. You don’t want to be the nth couple that goes to couple’s therapy saying “we’re about to split up, save us. Should have come in two years ago.”

Edith: This house is almost burnt to the ground, can you save us?

Paul: Yeah.

Edith: No.

Paul: So when you say engineer versus artist, I have no idea what you’re talking about.

Edith: So I just took a management course from Michael Deering which was awesome.

Paul: Yes. Michael Deering is absolutely awesome.

Edith: I love taking courses.

Paul: Okay.

Edith: To me, I think some people think of school as punishment, whereas I love school.

Paul: I think once you’re really into the topic. I went to a Marty Kagan course, which was the best two days I’ve ever spent on product, maybe overall.

Edith: Yeah, I mean I’ve got an engineering degree, and I got an econ degree too, because I like to study. Which makes me sound like a nerd, which I am.

Paul: So you went to Michael Dearing‘s course.

Edith: Yeah. And he asked us to describe the best engineer we knew. And I actually described my co-founder, John.

Paul: Aw, you guys are adorable.

Edith: You know, we’ve known each other a long time. And then he asked us to describe the best artist we knew. And I actually really grappled with that. And I think it’s because I’m such a hard-nosed engineer, that to think of an artist I admired, I just thought of these fluffy, fru-fru people.

Paul: Yeah, of course.

Edith: And I was thinking that this is kind of awful of me. Eventually I decided that my VC, James Sham, was the best artist I knew.

Paul: Okay.

Edith: Because he’s very creative and he sees the best in people.

Paul: Does he do art? Or just to be an artist in some other way?

Edith: I think artist in terms of a free spirit and creator.

Paul: Okay.

Edith: He creates companies. What do you see as the difference between an engineer and an artist?

Paul: I guess there’s quite a lot of overlap. And the person that comes to mind is Paul Graham‘s book, Hackers and Painters. And his point is that, hackers and painters are engineers and artists, right? So I guess he’s saying that they’re kind of the same thing. So I see a lot of art in engineering, like you can build a lot of beautiful abstractions and elegant code, and that kind of thing.

But I guess I don’t put a great deal of value in art itself. Like, sure those pictures are pretty, but I don’t want to speculate about the themes and what it says about life and all that bullshit.

Edith: I though you liked the Sistine Chapel.

Paul: Yeah, the Sistine Chapel’s beautiful. I mean, so that is amazing work. But if you showed me four rectangles and one of them is red, and asked me to say what the world is about through that lens, then fuck off, I’m sorry.

Edith: Do you think that’s the Irish in you, or do you think that’s the Paul in you?

Paul: A lot of it is the Irish in me, like we’re a very cynical, self-deprecating, but that also means deprecating other people, we don’t really like bullshit. Which is ironic, because there are a great deal of Irish artists, they’re just all authors. Like Joyce, and Yeats, and Shaw.

Edith: Yeah. I like Impressionist art, but anything after that kind of misses the mark for me.

Paul: I really don’t like Impressionism.

Edith: What?

Paul: I really don’t like Impressionism.

Edith: So when you go to Rome, you go to the museum, what do you like about the museum?

Paul: I like the big marble statues.

Edith: Oh they’re so cool.

Paul: That usually are of dragons or something.

Edith: So cool.

Paul: Yeah. And in Rome there’s this museum, I think it’s called the La Borghese.

Edith: I tried to go. Oh yeah, I told you about this. But you need a ticket months in advance.

Paul: Really? I just kind of wandered in on a Sunday morning.

Edith: I went on a bad day, because they were like nope.

Paul: Yeah, interesting.

Edith: So you appreciate art?

Paul: I appreciate some art. The less bullshit the better.

Edith: So, I’ll just give away the punchline.

Paul: Okay, yeah, let’s do that.

Edith: Okay, the punchline of this whole thing was that the engineer and the artist are the same person.

Paul: Of course. Was that the punchline in MD’s class? Okay. So you’ve spoiled the punchline for anyone who’s going to do his course.

Edith: Yeah.

Paul: Is there more to his course, or was that it?

Edith: I could give away all the Cliff Notes right now.

Paul: Oh excellent.

Edith: Management is good.

Paul: Management is good. I feel like we just did that podcast.

Edith: Management is good, have a little bit of fun at work.

Paul: Okay, so we’re going to talk today about the SaaS funding napkin, which was a blog post by Christoph Janz, who’s a VC at Point Nine Capital.

Edith: And I thought this was great.

Paul: Yeah, it was amazing. It was funny that we both mentioned it to each other, and we’d both read this separately.

Edith: Well the reason why I thought it was great is, so we just raised our A, and you know what happens as soon as you raise a round.

Paul: People start trying to give you more money, for some reason.

Edith: Yeah, you start thinking about your next round.

Paul: Oh okay, yeah that’s always valuable. So there’s an interesting thing that he says in there, which is that you kind of need to think about the point that you’re at the 300 million revenue, and you need to think about that all the way back to the seed round, nearly.

Edith: Yeah, and it makes a lot of sense. And I was thinking why previous startups I worked at have not succeeded, and I realized how important TAM really is.

Paul: Okay, yeah. Total Addressable Market.

Edith: Yeah. Because you can have a very successful product, but unless you have the ability to get to 300 million, it’s hard to get VC money.

Paul: So it’s interesting that you always need to be thinking about your next round. And then the implication of that is that, at your next round, you need to be thinking about the next round.

Edith: Oh yeah.

Paul: So it recurses forward, basically, and so you need to think about your IPOwhen you’re raising a seed round.

Edith: Yeah. I mean, at least have some notion of how you’ll get there, like a notational roadmap.

Paul: Right. So the napkin talks about, for each of the stages, pre-seed, seed, series A, series B, series C, what do you need to have to raise a good amount? Like what does your team need to be like, what does your tech need to be like, what does your MRR need to be?

Edith: Yeah. And I thought it was great. So I disagreed with portions of it.

Paul: What did you disagree with?

Edith: But I loved his framework. Let’s see. I think he alighted over something, which Thomas Tunks actually wrote about, which is they love to glom on all these metrics, you know like you need to have 100k to 250k MRR to raise an A. And then there’s this huge asterisk.

Paul: Okay, what’s in the asterisk?

Edith:

Paul: Have zero revenue.

Edith: Yeah, which is huge when you think about it.

Paul: I think some much larger number, I don’t think there are very many companies who are raising with 100k to 250k. Circle raised its series A on 70k. Now that was a few years ago, and I’ve heard since that raising a really good round is like done at a quarter of a million.

Edith: But yet the statistics show that.

Paul: Zero. And so, why are they raising with zero?

Edith: Because they can tell a good story about other proof points of traction.

Paul: Right. Yeah, so I think an important part of this is that it’s SaaS.

Edith: And I hate that. I think that if you’re in a breakout business your metrics are going to be terrible until they’re awesome.

Paul: Yeah, but if you’re raising a series A, you’ve had plenty of time to get them to awesome.

Edith: But yet 27% of companies have zero.

Paul: Are these founders who have had an exit before, or something along those lines?

Edith: That’s my assumption.

Paul: Right yeah.

Edith: You know, I have a friend who might get a preemptive A with no metrics. Like he literally formed a company.

Paul: Right, and were they a successful founder, had they had an exit, something like that?

Edith: They’d been in a company with a big exit, but they were not the founder there.

Paul: Okay, so the company, I’m going to pick randomly, is like DropBox and they were like the co-founder of DropBox.

Edith: They were not the co-founder.

Paul: Not the co-founder, but there’s a bunch of companies that were founded by early Facebook employees.

Edith: Yeah.

Paul: Right, that kind of thing.

Edith: Like I guess Path being perhaps one that springs to mind.

Paul: Right, Quora.

Edith: Yeah. So I think as a SaaS founder, you can have obsess over all these metrics, and it’s like well wait a second.

Paul: I think it’s great to obsess about the metrics. But the metrics are like real business metrics, they’re not like fake metrics, or they’re not like eyeballs, or user retention. I mean, user retention is very important. But they’re not like the dotcom era fake metrics.

Edith: Yeah, I think it’s just interesting though, because I think VCs like to claim that they’re guided by science, when really at the end of the day, there are 27% that have zero metrics.

Paul: Well I think that makes sense, because if VCs are just guided by science, then they kind of bring no value to the table. And LPs could invest directly.

Edith: Well there’s the moneyball theory, that why do you have VCs at all, why not just chug all your metrics into like a FICO score type thing and then out pops whatever your valuation is.

Paul: Right, and I think that a lot of the value of VCs is that they do actually know a little something about startups. At least the good VCs do.

Edith: Well yeah. A good value-add VC will see that this is a diamond in the rough.

Paul: Right, exactly. I’m thinking back to the Facebook and what they saw in Facebook, and it was that there wasn’t very much revenue, actually there wasn’t really any ad revenue, there was no validation of the ad business. And this is why a whole bunch of those reporters were saying oh it’s bullshit, they barely make any money, why do you think Facebook’s good?

And I think this is really good actually, because you can contrast how reporters think about things, because reporters know almost nothing about anything that they’re reporting on. Whereas the people like VCs who are deep in it, while many of them also don’t know shit, the ones who really know their stuff, will just be like night and day ahead of anyone else.

Edith: Well, so here’s a counterpoint: Secret.

Paul: Secret, I totally agree. Secret was perfect. So Secret was this company that did a 10 million secondary or something like that.

Edith: Yeah, something huge.

Paul: But I think they raised like 15 million, and six of it was secondary, and so that means that the founders just got that cash themselves.

Edith: Bought a Lamborghini?

Paul: Yup. So I did the numbers on my Twitter a few years ago, on Secret, and it was perfect. When you do the numbers from a VC perspective that was an excellent bet. It was a bet that lost, but you need to keep making those bets.

Edith: Yeah, it’s funny, so Dave McClure invested. A story about myself, is Dave McClure is very metrics-driven, very ARRR. He invested in us when we had no revenue.

Paul: Okay.

Edith: When I saw him, he’s like, well Edith, I knew you.

Paul: Right, exactly, yeah.

Edith: Cause I was like why did you invest in us, you’re the king of ARRR.

Paul: Right, but you’re you. And so that’s worth investing in.

Edith: Yeah, which I thought was funny.

Paul: Well, so for Secret, the metrics that they had was like this really high engagement, really high growth, and there was a strong potential to be the next Facebook. That’s what you’re given, time to make a bet. What are you going to do, you’re going to bet on that. And the VCs are right to bet out of that, because sure, nine times out of 10 it’s not going to turn into Facebook, but the time that it does turn into Facebook, you’re going to have 100x return, or maybe more. And so overall that’s an expected value of 10x for that investment in Secret.

Edith: Well, you’re buying on growth. It’s funny because he has another article on his blog about his hatred of the hockey stick graph.

Paul: Who’s he?

Edith: Christoph.

Paul: Oh yeah.

Edith: About his hatred of the hockey stick graph that you see at every demo day. You know the graph, right?

Paul: Yeah, I know the one, it’s got three months in it.

Edith: Well, and it’s the whole thing is that you’re trying to force investors to buy growth.

Paul: Yeah, but that’s because

Edith: I think what’s interesting though, is that you start to hit your B, and this is what I was looking, is that suddenly the music starts to slow down a little bit.

Paul: Right. Because growth slows down around the B. The VCs who invest in your B only care about the numbers at that point. They’re no longer willing to take a punt because of great founders or a great team. It’s like, you’ve hit your numbers or you haven’t. The chairs being pulled, the music stops, is there a seat for you?

Edith: Yeah, it’s funny. So I gave advice to people in my accelerator, and I say at the pre-seed stage, or even the seed, you’re really pitching team, dream and traction.

Paul: Okay, yeah.

Edith: And you really can get away with just one.

Paul: You’re a bunch of founders who just left Google. Or just left Stanford or something like that. You can probably raise on just team.

Edith: Or like Buoyant, which was some ex-Twitter engineers.

Paul: Yeah.

Edith: It was like, well, who’s not going to want to invest in this?

Paul: Exactly, yeah. Twitter, really?

Edith: Paul. Not everybody’s Mozilla, dude.

Paul: Oh god.

Edith: So then I think you can get away with one out of the three at your seed. I think at your A you have to have two out of three. And I think at your B.

Paul: Above all, I mean, at your A, if you don’t have team, there has to be team.

Edith: Well let me say this. I think you have to have a team, but you don’t have to have such a pedigreed team if you have the metrics.

Paul: Right, but you have to have grown into good founders. Like you can get away in your seed without having a great team because, essentially, they’re looking for social proof. But the team that they’re looking for at series A are like people who are executing well. And you can’t get away without having that.

Edith: Well, what I was trying to say, maybe I’ll just be more blunt is, at your A you can still get a pass if like, oh these are, you know, say it again, these ex-engineers from Twitter, they have this great vision, maybe their metrics are still not great and they have zero revenue.

Paul: Maybe we need to bring in a CEO.

Edith: Well, no, so that’s why I think A companies get funded with zero revenue.

Paul: Right, because the team is good.

Edith: The team is good, the dream is still good.

Paul: So Javier Soltero, right? He has this talk on Heavybit and he says that sometimes it’s better to have no revenue than some revenue.

Edith: Yeah, as soon as you start to have revenue, people start to pick it apart. Because suddenly they can look at what’s your CAC.

Paul: Right, until then you can sell the dream. I think they talked about this in Silicon Valley, right?

Edith: I don’t watch the show.

Paul: Oh you should. So one of the things that I really liked about Christoph’s napkin, is he talks about product-market fit.

Edith: It’s a metaphorical napkin, dude.

Paul: I mean, he makes it look like a napkin, but he has a logo on his napkin, and I’m wondering about that.

Edith: It would be pretty cool if they printed out a bunch and gave them away at a party.

Paul: He talks about product-market fit. In order to raise an A, and anything after that, you need to have clear product-market fit. And one of the areas that I see companies fail, is where they manage to raise an A without having product-market fit. Or a B.

Edith: Well, let’s pick apart Medium. That’s actually pretty relevant right now.

Paul: Okay, yeah.

Edith: Super famous founder, Ev Williams. You know founded, blogger, Twitter.

Paul: Did he take money for it, or is he just putting his own cash in?

Edith: Medium, no they were venture funded. They took I think $100 million.

Paul: Right okay.

Edith: But I mean eventually the music stops. Like you just said, like hey we’re not making money.

Paul: They’re not making money, yes. So they essentially haven’t found product-market fit. They found something.

Edith: They found that people will post essays. I mean you use Medium, I use Medium.

Paul: Yeah, yeah, medium is good.

Edith: But they haven’t found a way to monetize it.

Paul: Right, which is a missing part of a product-market fit. Especially that they refuse to put ads on it. Which I think is going to make making money a challenge.

Edith: I’ve worked at ad-supported businesses, and you have to have a vast volume of ads.

Paul: Right. I mean they have vast volumes of readers. I don’t know, are they top 100 on the internet?

Edith: Yeah, it’s funny that they just totally backed away from it though, maybe they just ran the numbers and it didn’t work.

Paul: Or maybe once you’re Ev Williams your vision matters more than making money. So overall, like I think this is the sort of thing where every founder needs to look at this napkin. And not necessarily memorize it, but understand all the transitions. Like you should be looking at two blocks that are next to each other, and say why are those different? What is it that I can learn about the differences between those two squares?

Edith: Yeah, and that you have to, you do have always think about the future. It’s funny, we got our A, I was happy, the team was happy, and then I’m already thinking about the B.

Paul: And so this is kind of the problem with the venture world. No one is ever happy.

Edith: I was pretty happy for a day or two.

Paul: Yeah but no one is ever happy with your results, because you always have to look to the future. No matter what you do this year, you know the phrase what have you done for me lately?

Edith: Yeah, it’s a treadmill.

Paul: Yeah.

Edith: It’s like I look at our monthly numbers, and a year ago I would have been ecstatic.

Paul: And now you’re like meh.

Edith: Yeah, it’s like okay. It’s what you said before, the more your revenue grows, the harder it is to get growth.

Paul: Right.

Edith: Well, you make it sound like the thermal physics.

Paul: The venture math, it’s just like thermal physics.

Edith: You know what the rules are right?

Paul: Like second law of thermodynamics?

Edith: Yeah, what is it? Matter can not be, you can’t win.

Paul: Yeah, so there’s no winning.

Edith: Well, there’s having some fun along the way.

Paul: There’s having fun along the way, but when you’re the biggest company in the world and your salary gets docked, because you didn’t quite hit numbers.

Edith: Oh, are you talking about Google, or Apple?

Paul: That was Apple. Yeah, Tim Cook had his salary docked, a little slap across the wrist for not making the shareholders their cash this quarter.

Edith: I think he’s still doing quite well.

Paul: I think he’s doing all right.

Edith: I don’t think he’s had to turn down the thermostat to 60.

Paul: Maybe if he takes a look at the napkin, though, he’ll figure his way out.

Edith: Yeah, I think where companies get into trouble is where they think they’ve raised a round and their work is over. Like let’s go frolic.

Paul: Well, and that’s kind of my point, the work is never over.

Edith: NPS.

Paul: So that’s net promoter score.

Edith: We just did a net promoter score at my company Launch Darkly, and I really like it. How have you done at Circle CI?

Paul: So we started doing NPS very recently, after hiring a head of customer success. And I think that

And so we were careful, with the new head of customer success, to make sure that when it was launched it wasn’t in a way that would offend our audience sensibilities.

Edith: Tell me more.

Paul: Well developers are whiny pricks.

Edith: You know, so I have had the opposite experience. So we have a similar audience, we sell to developers. They’re blown away.

Paul: That they’re happy?

Edith: That they’re happy.

Paul: Well they probably love your product.

Edith: They love our product. I’ve been doing NPS surveys for a long time, and this is the best I’ve ever had.

Paul: Wow, what was your score?

Edith: It’s in the 60s so far, which is amazing.

Paul: And so that’s the percentage of people who love it minus the percentage of people who hate it.

Edith: Divided by the total population.

Paul: Okay. And so you get 60.

Edith: Yeah, it’s in the 60s.

Paul: Yeah, that’s phenomenal.

Edith: Yeah, and they’re like developers.

Paul: Wait, your audience is developers?

Edith: Yeah.

Paul: I thought your audience wasn’t developers. Or is it your purchaser isn’t the developer?

Edith: Actually that’s one of the questions on the survey. So developers implement us, and then product managers can use us. So we sent it to everybody that has a Launch Darkly account. And we then ask people roles, because they wanted to be able to segment, to your point, if it turned out the developers hated us and product manager loved us, or vice versa. The issue is there’s nothing to say, because everybody loves us.

Paul: I guess you also get qualitative feedback that comes with it about why they love you.

Edith: Yeah, so we ask them for. So the score is just scalar. And then the interesting stuff usually is, you say like well why did you give us this score, what do you like, what do you dislike, what should we improve on? And it’s just all the stuff I wish they would say.

Paul: Oh nice.

Edith: It makes me really happy.

Paul: So I’m very kind of against sending developers surveys. Because they really don’t like it. Like developers have this aversion to marketing. And I don’t think that’s as true as it was, but there’s certainly still a large element of “oh what is this shit you’re sending me”. I think they think that they can see through the bullshit. I don’t think it really is bullshit, personally.

Edith: I think we’ve had that reaction other times I’ve done surveys, but no, everybody was just saying nice things.

Paul: I wonder is it how, that like NPS is easy and algorithmic, and so people are willing to reply.

Edith: I took over another product once, so this was me throwing the prior people under the bus. I took over a product, they’d never done NPS, everybody claimed that the product was fine. I did NPS and it was awful.

Paul: And you got qualitative about why.

Edith: We got qualitative that this product was a total dud.

Paul: So like no one wanted it, or? Was it the product doesn’t work that great and you had to save it, or was it like the product is useless, there’s no saving?

Edith: People really liked the idea of the product, but it fundamentally did not work.

Paul: Okay, that’s a good thing to learn.

Edith: It fundamentally did not work. And everybody had been like, “it works enough”. So the engineering people inside had been like this is a tolerable level of works enough.

Paul: Did they use it? Cause I feel that this is the sort of thing, and I’m not saying that we shouldn’t do NPS, I’m saying that like there’s a lot of products that if you just go use it, you will feel the pain. And you won’t even need to, I mean you should still ask customers so that you can then measure their change when you fix it, but you can just feel, “oh my god this is awful.”

Edith: I think an important thing for me to do an NPS survey is that you are not your user.

Paul: Well, okay, true true. But, I’m sure with that product, that they could have just, if they had been their own users they could have just, “yeah this is awful.”

Edith: Well there’s this layer of, I don’t know, rose-colored glasses, that people could see that it was a new product, it had a lot of rough edges.

Paul: Right.

Edith: People will be tolerant of our rough edges. And no people were absolutely–

Paul: At the start of Circle I brought my laptop to a meetup and I got people to sign up for Circle, and it was just so painful. Just watching them fail at certain screens, and like oh, just click refresh, oh sorry about that, we haven’t fixed that bug yet. And on and on. I was like, oh, no wonder we’re having problems in our sign-up flow.

Edith: Yeah, I think any new product has rough edges. You know, I agree with Reid Hoffman who founded LinkedIn, he said if you’re not completely embarrassed, you shipped too late. But I was really surprised at how positive people were when I consider us still a bit of an early product.

Paul: Still today you think of yourselves as an early product?

Edith: And I guess maybe that’s just a function that when I wasn’t looking we got old.

Paul: Right. I mean, you’ve raised a series A, that’s not very early.

Edith: Yeah, I mean we’ve been in market for over a year and a half now.

Paul: How many customers do you guys have?

Edith: We don’t really say publicly, but enough.

Paul: Enough. Yeah, that’s good to know. So like, six.

Edith: Circle CI. More than six, less than a million.

Paul: Less than a million, okay. That’s a big range you’ve left there.

Edith: Yeah, well thanks for wearing your Launch Darkly shirt today.

Paul: What proportion of people do you send your NPS out to? So we were testing the send. So I wanted to test that we were asking the right questions. So we sent it to some people and were like okay these are the right questions and now we’re sending it to everyone else.

Paul: So how many did you send the are these the right questions to? Like 10?

Edith: To 25%.

Paul: 25%. Okay. And so you sent 25% of the people that you were sending the NPS to? Did you send the NPS to everyone?

Edith: Not yet.

Paul: But your plan is to send NPS to everyone.

Edith: Yeah.

Paul: Wow.

Edith: Wow?

Paul: Like you’re going to send that again in three months, and three months after that.

Edith: Yeah.

Paul: We keep ours to like a much much lower number.

Edith: Why?

Paul: Well one, because the sample is generally good enough.

Edith: Yeah, well I guess I love reading surveys.

Paul: Yeah, but we don’t need to, I feel that it interferes with people and it gives them a negative perception of things. I don’t know if you’ve seen much of how

Edith: So actually, we didn’t do it in-app because I, I think that in-app distracts people from what they’re trying to do right then.

Paul: Sure.

Edith: And it makes them think of the last thing they did instead of their overall impression.

Paul: Sure, and I’m not saying that you should do it in-app, I’m saying that it is inherently an impersonal interaction with your customers, and you don’t want to do that with all of your customers, nor do you want to do it repeatedly to the same people.

Edith: No I agree on repeatedly.

Paul: I mean, if you’re doing it every three months, that’s a lot.

Edith: I think you could cohort it so you’re not hitting the same people over and over.

Paul: Yeah, that’s kind of my point.

Edith: So cohorts.

Paul: Yeah, but that’s not sending it to 100%, it’s sending it to a cohort.

Edith: It is funny though, so at TripIt we had this luxury of we had 10 million users.

Paul: Right, so you could pick like a million, or something, or even 100,000.

Edith: Well so we had a problem there, if we would get too many results.

Paul: Oh, yeah. So did you come up with a size that was good?

Edith: Yeah, so at TripIt we had to be careful about how are we going to sample this so that we’re getting meaningful data but not having 15,000 surveys to read.

Paul: So at Launch Darkly, how many people have you sent it to?

Edith: Paul, I’ll tell you, just not on the air.

Paul: Okay, my question is, if you’ve sent it to a statistically meaningful number of people, which hits across your demographics and so on, then why continue to send it to the rest of the audience?

Edith: Oh, because it’s been so positive, I’m wondering if there’s some lurking attractors out there that we need to root out.

Paul: Is there really value there?

Edith: Yeah.

Paul: I wouldn’t say so compared to the negative aspects.

Edith: What are the negative?

Paul: Well the negative is that you bother people.

Edith: I don’t think it’s a bother at all.

Paul: They’re in their inbox, they’re trying to do work.

Edith: Yeah, and a little innocent email pops up like give us feedback?

Paul: Yeah, and takes 30 seconds of their time, multiplied by. So 30 seconds of their time multiplied by tens of thousands of users, you’re wasting hundreds of hours of people’s time, wasting productivity, for an answer that you already have.

Edith: You don’t think people want to be heard. You don’t think people want to have their voices heard.

Paul: I think ask them in three months.

Edith: You don’t think people want to have, “hey I do have an opinion and I want people to know about it.”

Paul: No. People fundamentally don’t care about any of your products. Not just your products, anyone’s products.

Edith: I disagree.

Paul: No one cares about Coke the way Coke cares about Coke.

Edith: I think, having read our surveys, that people do care. And they want to be heard.

Paul: Sure, they love your product, it’s wonderful. But, if you hadn’t sent them that survey, they probably wouldn’t have thought, you know what, Launch Darkly is wonderful.

Edith: People want to feel allegiance to a brand.

Paul: True true.

Edith: People want to feel that they’re listened to. So it’s funny, to your point, you have to be careful about doing NPS surveys because you’re basically asking people to be a promoter for you.

Paul: You’re asking, okay, yeah, yeah. So are you saying that by sending it out to everyone, you encourage them to promote it?

Edith: Yeah.

Paul: So is that the benefit of it?

Edith: Well it’s a virtuous cycle.

Paul: It sounds like the benefit that you’re looking for is you just like people saying nice things about you.

Edith: You say that like that’s a horrible thing.

Paul: It’s not a horrible thing, but it’s a little conceited.

Edith: Wait, so you don’t like people saying nice things about you?

Paul: No I do, but when I’ve had 10,000 people say nice things about me, I’m not going to be like okay, that was wonderful, 10,000 people said nice things, what are the other 90,000 going to say?

Edith: Did 10,000 people say nice things about you?

Paul: I’m sure they have.

Paul: So hiring quick versus slow.

Edith: Yes.

Paul: The distinction that we have here. So you’ve, what, doubled in size in three months?

Edith: Yup.

Paul: So we did this after our series B as well, we went from like 40 to 75 or 80 in like three months. And there’s always this tension of, are we going to hire super quickly, or are we going to space it out? So when we were very small we spaced it out. We tried to have no more than one new person a month and tried to scale it out slowly.

Edith: I think it’s a consequence of being at different stages. Like for us it was very clear that we had a screaming need for more customer success people. That we were not doing a good job with our existing customers. So it’s like okay, we have enough work right now for three more customer success people.

And we do have a hiring plan of us bringing more people in at the right time, it was more that…

Paul: That was what you needed right then.

Edith: That was what we needed right then.

Paul: So the thing that you get when you have rapid growth, is that all of your processes break.

Edith: Not so much break, but have an opportunity to evolve.

Paul: An opportunity to evolve, what do you mean by that?

Edith: Well, the stuff that works when you’re a small team will not scale up.

Paul: Absolutely.

Edith: Yeah. And I love what Shaun Burns, our advisor said, he’s like the stuff that gets you to 10,000 in revenue will not get you to 100k in revenue, will not get you to a million, will not get you to 10 million.

Paul: Right yeah, completely true.

Edith: So his belief is that, every time you have a break point you need to rethink what you’re doing.

Paul: Very often the processes that you have when you’re small, and very often you have no process when you’re small, or invisible process. It’s a failure of your company to scale if you don’t address what is the version of this that works when we’ve doubled in size?

Edith: Yeah, that’s a conscious thing that I’m trying to do right now. And it’s not a value judgment at all. It’s like the processes that we’re putting in place now would have made no sense before.

Paul: We talked a little bit about why flat sucks in a previous episode. And I think

Edith: Yeah, the way I try to position it, is that part of scaling up is specializing. And part of specializing is splitting out roles. Like we used to have our designer also handle support. Now we can split the cell.

Paul: Right.

Edith: Like we have this opportunity to concentrate and focus.

Paul: Right. So what you’re saying is that, whenever you have the opportunity to change process, you’re going to do it, or you’re going to look at whether to do it?

Edith: I think you have to look carefully at whether that process is still serving the goals it used to serve.

Paul: It’s not process for process sake, the goal is the important thing.

Edith: Which is, I’m examining my own role, I used to do all the job offers myself, and I’m like do I really need to do this any more, or do I just need to approve the salary?

Paul: Right. Another way to maybe look at it is like, the goal is to close them, and maybe you talking to them helps close them, and so that’s going to factor in to how you look at the process.

Edith: Yeah, but literally, I used to go into our tool and put in everybody’s salary, and address, and I’m like do I need to do this any more? Probably not.