In episode 43 of To Be Continuous, Edith and Paul are joined by Ellen Chisa, Paul’s co-founder and CEO of Dark, as well as John Kodumal CTO and co-founder of LaunchDarkly. The group discusses pitching, what it’s like finding a startup partner and the road to effective management.
This episode of To Be Continuous, brought to you by Heavybit. To learn more about Heavybit, visit heavybit.com. While you’re there, check out their library, home to great educational talks from other developer company founders and industry leaders.
ABOUT THE GUESTS
Ellen Chisa is co-founder and CEO of Dark.
John Kodumal is CTO and co-founder of LaunchDarkly.
TRANSCRIPT
Edith Harbaugh: What made you decide to do a startup?
Ellen Chisa: I did my first startup a little over 10 years ago. It probably should’ve been a reality TV show instead of a startup, where five of my friends and I decided to have a company.
Except four of them were dating each other, two pairs of two. None of us had ever not lived in a dorm, and it went about as well as could be expected. But it was enough to convince me to do it again.
Edith: Now would be a great time for you to introduce yourself.
Ellen: I am Ellen Chisa and I am Paul’s co-founder and the CEO of ellenandpaulsnewstartup.com.
Edith: Were you one of the two pairs, or the fifth person?
Ellen: There were two of us who weren’t in one of those ill-fated relationships. So I guess I was the fifth person or the sixth person.
Edith: So, what made you decide to do a startup?
John Kodumal: I’ve known Edith for a long time. I think it was like around 2007 or so, we were having a Thanksgiving dinner. Edith was at my house. I cook a mean Thanksgiving dinner.
Edith: Turducken.
John: Yeah, it was good. And I think we were just like, “We’re going to start a startup,” right?” Like, “You know that, right?” Took a little while for the timing to shake out, but we knew from that point it was an eventuality.
Paul Biggar: Was that your first startup?
John: My first startup, yeah. This is only my third job ever in life.
Edith: Now would be a great time for you to introduce yourself, since I know you quite well.
John: I’m John Kodumal. I’m Edith’s co-founder and CTO of LaunchDarkly.
Edith: I already have so many questions such as, this is only your third job? I mean, you graded in college.
John: Oh, okay, I suppose that’s true. I graded for a CS AD at Harvey Mudd, that’s logic. I spent two hours a week grading homework assignments.
Edith: And when you were getting your PhD?
John: I was just poor. I got a PhD in computer science, much like Paul. You just get paid very poorly for a number of years. I had a stipend, so I guess theoretically as a grad student you’re both an employee and a student.
But literally, I was in the Bay Area earning 20K a year for six years. So yeah, it’s not fun.
Edith: Were you a TA?
John: You can teach. I taught for one semester and then basically, the way it works out, is that happens instead of your stipend. So you don’t get paid more for teaching, it’s just something you have to do.
Paul: It’s crazy when you go to your first job afterwards, and suddenly you have seven times as much money.
John: Yeah.
Ellen: Why doesn’t everyone drop out immediately and get a job that pays seven times as much money?
Paul: Because if you don’t stay for the PhD, you’ll only get a job that makes five times as much money.
John: Yeah, and then you can walk around and people call you “Dr.”
Paul: Do you actually use “Dr.”?
John: I tried to once and it didn’t go over well. He asked me, I was at a bank and my credit card says “Dr.”
Paul: Yeah, I had that one, too. It was super awkward.
John: Yeah, they’re like, “Dr.?” Oh, what kind of physician are you? And I’m like…
Ellen: Computers.
John: Computers…? And they were extraordinarily disappointed.
Edith: Did they try to show you a wart or something?
John: Yeah, after that point I was like, no.
Paul: Showed you a bug in their software. They’re like, “Can you fix this?”
John: Yeah, it’s like, “It looks like a buffer overflow.”
Edith: Which is funny because your wife is actually a medical doctor.
John: She is, yeah. She’s an ophthalmologist and she has the same sort of insecurities around her title as me. It’s like, I’m a doctor of something useless, but she also gets confused with optometrists. Optometrists are different from ophthalmologists.
Ophthalmologists are medical doctors and surgeons, but if she tells people she’s an ophthalmologist, a lot of them think she’s an optometrist and they think she does the “cover your eye and read the big ‘E'” thing.
Edith: You know, what they both have in common is that nobody can spell them.
John: It’s true.
Edith: Ellen, you had this, first, reality-world startup, and then you decided to do another one.
Ellen: There was more in between the two. I had the first reality-world startup and I learned many things I was bad at. Then I decided to go learn a bunch about all of them. Eventually it would be the right time to do another one. But I didn’t want to do it before I felt like I knew the right set of things.
Edith: How did Paul persuade you?
Ellen: I think Paul had happened to meet me at the exact moment where I knew I was ready. because the job I had before this, I’d been in EIR in an incubator and I started working on a project. Then eventually the incubator went away, we kicked out of the companies we’d invested in, we canceled all the other internal projects and started building it.
I’d just been through this experience of seeing a team scale from just me to 50 people. And go from, basically, where we were at at the beginning, which was, it was a project within an incubator to a serious B stage company.
I’d seen the entire process and it felt like there were no excuses not to do it anymore. If there was time to do it, it was now.
Paul: It was good timing on my part.
Edith: Did you have hesitations, given the way stuff had gone south before?
Ellen: No, there were 15 minutes where Paul walked me through a lot of slides about X and L complexity, and I was like, “Yeah that seems like a problem.”
There were a couple slides in the middle where I was like, “Hm, I don’t think this is going to work.” Then about 20 minutes later, I was like, “Oh no, this is definitely going to work. Yeah, we should do this.” But it was very much a within-the-first-half-hour decision.
Edith: John, it took us 10 years.
John: Yeah, we weren’t even sure what the idea was for a while.
Paul: You were doing consumer stuff or consumer ideas before you decided to do LaunchDarkly?
Edith: We had a joke that we should make like a graveyard of all our ideas that we tried for about a day.
John: Oh, there was the static analysis startup.
Paul: Oh yeah, that would’ve been good.
John: Yeah, it would’ve been really good.
Ellen: Did you have a process for how you tested all of the ideas, or was it just every day something new?
John: Mostly we convinced ourselves that they were terrible ideas as quickly as we could.
Paul: And they took a day?
Edith: The terrible ones would take an hour.
John: Yeah, the static analysis one we worked for six or eight weeks on it, maybe even longer.
Paul: Oh, shit.
Edith: No, like eight months.
John: Oh, eight months. It flew by so fast because we were having so much fun.
Paul: But what was it going to be?
John: I had this observation that DVCS and Git and the way people do development doesn’t work really well with most static analysis tools. because they’re not built to be incremental in any way.
The thing that static analysis doesn’t have is any notion of time, like how the code base is changing over time.
So I had this observation that if you married the types of static analyses, that map easily to graph reachability problems, to get meta data and you layered get meta data on top of that static analysis data, you could basically use data log as a query engine to ask questions about code base and static analysis questions that marry time and commit history with the facts that you’re interested in.
It would be incremental by design. When you made an incremental change, you would just re-derive the necessary facts, based on that change. So it’d be much faster and you could do things with temporal understanding.
Paul: Did you come to the conclusion that this was a terrible idea, or no market? How did it go from, “Oh that sounds pretty interesting,” to…
Edith: We worked on it for a solid year.
John: We worked on it for awhile.
Edith: Because when I look at dates…
Paul: This is ramping up, we’re six weeks and eight months, now it’s a year.
John: We spent like 15 years on this.
Edith: Well this is development estimation versus…
John: It was a combination of two things. The first and most important thing was, Edith was talking about the idea with a bunch of prospective customers and the level of excitement we were hearing was not that great. There were a lot of attempts to commercialize static analysis right around when I was finishing grad school. And they all ran into that hair-on-fire problem.
Paul: You worked at Coverity, right?
John: I worked at Coverity for three years.
Paul: That one got commercialized and it exited.
John: Yeah, it exited. Did a really good exit. It was a great company, some amazing technology. But I think a lot of the stuff that they were doing, it was really applicable to legacy code bases or a lot of like embedded systems and stuff like that.
Then modern software development, like, modern web software development, say as a subclass in there, a lot of techniques there weren’t interesting and Coverity as a tool wasn’t interesting for them.
Paul: So you just talked to all the people, and then…?
Edith: I got kind of a begrudging, “Hey, maybe I’ll look at that as a favor to you.”
Paul: What was the process of going from that to, “We’re not going to do this anymore?”
John: It was a combination of that plus I realized at some point I’d spent nine years of my career working on this stuff and I was done.
Paul: Oh, yeah.
John: It was just too much. I knew how far the rabbit hole went and I desperately wanted to get out of that rabbit hole.
Ellen: At this point was it just the two of you? Just working on ADS together and trying to figure out what to start?
John: Yeah, there’s a bunch of code that I wrote. There’s a prototype of an analysis system.
Paul: Did you release it? Open-source it?
John: No, I really should, it’s like something I’ve been meaning to circle back to because I did a prototype analysis which was like a clone detector based on like some interesting research that had come out of Davis, and it was like really good, actually.
It worked really well, there was a lot of stuff there that would’ve been interesting but taking that interesting research idea and turning it into a real product, was a lot of work.
That was that last-mile problem in static analysis, going from, “Here’s an interesting research problem,” to, “Here, a polished commercial product that a company can use on their code base and derive value out of,” that’s a huge leap.
That was something we learned at Coverity and something that I rediscovered in this process. Those two things together, it was more than enough information to let us know that we needed to work on something else.
Paul: If he has to answer why are we making our own language, Ellen, this is the thing.
Edith: Are you really making your own language?
Paul: Yes we are, but the reason is if we don’t make our own language then we have to deal with someone else’s.
Ellen: I don’t think that’s the only reason.
Paul: It’s not the only reason but like…
Edith: I thought the reason was that Paul likes languages.
Paul: No,
it’s just easier to build a new language that is designed for exactly what you want than to try and prove properties about someone else’s language.
John: I think the other interesting thing is, another reason that it’s hard to commercialize a lot of the research that I did. It’s much easier for people to adopt new languages now. I think the thesis when I was in grad school was like, “Oh, it takes like 30 years for a language to hit mainstream.”
And then you saw Java where I think the increment was like maybe 10 years or something like that. I might have my history a little bit wrong on that. But then new languages like Go, all of LaunchDarkly is written in Go and Go is not that old a language.
Edith: John, was it hard to give up the idea? I mean, this was your research, this was something you spent a lot of time on.
John: I still think it’s an interesting idea, the specific approach we were taking to it, and I hope that somebody pursues it. because I think software development ultimately will be better off if we had that kind of information, the kind of thing I had envisioned.
But it was easy to get rid of it as an idea, because going down those familiar paths of writing those parts of analyses that I’d done before at Coverity and in my thesis work, I was like, there was too much familiarity with it. And I really wanted to do something new and different.
Edith: I thought it would be a harder conversation when we had that conversation, because it was like, “Hey, maybe we should push a couple more months and see if we can productize this, I don’t want to give up too soon.”
Paul: You said that?
Edith: I remember it, because I was like, “I don’t want to. We’ve put so much work on it, I don’t want to give it up too easily. Let’s make sure we’re on board with giving it up.” Do you remember that?
John: Yeah, and actually that was right around when I left my company. And it was really hard because I went for about, I guess it was about a year, I’d forgotten the timing of it, but I went for about a year knowing I was going to leave my last company and feeling like we had something that I was anchored to that we were going to work on together.
And that was going to be the thing, and then I knew I was going to leave. I put my notice in and we made this decision to pivot and do something else. We were going to scrap that code base and it was a little bit hard because
I was leaving and I couldn’t really articulate what I was going to work on next, because I didn’t have a concrete idea. So it felt even riskier to be leaving a good situation.
Ellen: What did you tell people?
John: I had all sorts of hair-brained ideas. I think I told people that I was working on a VR-related idea, which I wasn’t.
Paul: It’s interesting, the thing of not wanting to do what you’ve done for so long that I feel once I’m done with something, I’m done. I’m not going back to it.
I think you were saying the same thing about the travel startups.
Edith: I do not want to do any more travel startups.
Ellen: Me neither.
Edith: I was at Tripit for a long time.
John: Oh, that’s so funny, two static analysis founders and two travel founders.
Edith: By the way, my middle name is Ellen.
Ellen: I did not know that.
John: My middle name is Paul.
Edith: And many people in this room are wearing LaunchDarkly shirts.
Paul: Many.
Edith: Nearly all, and Paul, looks lovely on you.
Paul: Thank you, thank you. It always does.
Edith: Ellen, did you have to make a similar leap of faith to join Paul?
Ellen: Yeah, actually. I don’t think I could’ve picked worse timing to have a startup. At the point in time when I met Paul, actually,
a mutual friend had emailed me saying Paul was looking for a co-founder. I said not to introduce me because I was afraid I would want to do it.
My husband and I had just bought a house in Boston and I had this company and I’d just built this team I was very proud of. We’d just raised a bunch of money, and I was getting married in August.
I think there were some other things going on. It just was not great. But I think when I looked at it I said, “Oh, this is exactly what I wanted to do in three years. I don’t have to wait for three years. I can just do it right now.” And that seems like the better answer.
Edith: I think we did the reverse, we just kept postponing and postponing it.
John: Then we ended up picking a similarly bad set of timing. I’d just had a kid and left a comfortable situation. I think when it’s the right time, it just feels like the right time, regardless of like how it might seem externally.
Paul: Your wife’s a doctor. I feel you’re going to be okay.
John: Yeah, that’s true.
Ellen: The way I did it, I think it was a more slippery slope where I was like, I will regret it forever if I don’t do this. Probably it won’t work. How can I do the things to make it fall apart the fastest?
Paul and I had met on a Friday, and by the next Friday you had already flown to Boston and were living in my house and we didn’t know each other at all.
Edith: Did he bring his LaunchDarkly shirts, though?
Ellen: He did.
– Yeah, like the first day we hung out for, I think, five hours. And that was a lot.
– Yeah.
Paul: I had gone through this process with a bunch of people before and I had not got to the point of five hours with someone actually feels like a pretty good experience.
Edith: I guess we were the reverse. We just kept postponing and postponing it. Some of it was me and some of it was you.
John: Yeah, because we were both in relatively comfortable situations in a lot of ways.
Paul: If you hadn’t postponed it, would you have come up with feature flags?
Edith: I don’t know.
Paul: It feels like feature flags are, not new but starting to catch on around the time that you started building.
John: I think if we’d come earlier it would’ve been too soon, for sure, for the way we built it. I think if we’d started the company five years earlier…
Edith: People were still shifting to the cloud at that point.
John: Yeah, I think we probably would’ve had to build a completely different product that was not SaaS-focused, probably like an open-sourced play or something like that.
Paul: You probably wouldn’t have realized that feature flags was the thing to build, either.
John: No, absolutely not. It was the experiences that Edith and I had both had, the size organizations that we had, it drove us to realize where the need was. And so without that experience, I don’t think we would’ve built as good of a product as we built.
Paul: You’d probably have tried to build “CI as a Service.” It was very popular in 2012.
Edith: It’s funny because we knew we wanted to start a company and we’d always be bouncing ideas off each other. I would say, “All I really how is to build software. I don’t have any deep domain expertise in,” I don’t know, “autonomous vehicles.”
Paul: Something about plants, right?
Edith: I was at a plant company but that was a bad one. But then it turned out that knowing a lot about software was actually really valuable.
John: Yeah, because
not only do I only really know about software, I only know about how to build software for other software developers.
Paul: Edith, did you talk to any other potential co-founders?
Edith: No.
Paul: Ever ever think about it, either?
Ellen: How did you know you wanted to start a company together?
Edith: I was just like, John would be awesome.
Paul: How many years did you know each other by that point?
Edith: A long time.
Paul: Like 10?
John: Ish.
Edith: Ish. Well, we had briefly that other guy who I won’t name.
John: We thought about bringing a third person in around like a specific idea we had. He was a really brilliant person, but at the end of the day he wasn’t invested in the same way that we were. I guess I didn’t talk to anybody else, either.
Edith: It’s weird because I know that, Paul, you interviewed probably 50 people.
Paul: 49 people.
John: Were you literally the 50th person, Ellen?
Paul: Or maybe the 49th, I’m not sure.
Ellen: We can look at the spreadsheet.
Paul: There’s a spreadsheet.
Edith: See, I had the opposite where I had been working with a lot of people for a long time. And I could never find enough conviction to start a company with any of them or any of the people I met socially who I knew who started things.
I had come to a conclusion of, basically the shape of the person I wanted to work with and what I wanted them to be good at or not good at. So then I could pattern match that pretty quickly. Where you had the pattern from having talked to 49 people, and I had it from knowing a lot of people and knowing it fit.
Paul: Well, I first went through the people I knew, and some of them are in the 49 but I realized that either there wasn’t anyone who was ready to go now or there wasn’t anyone who was the right fit.
Ellen: Yeah, I didn’t talk to anybody else.
Paul: That’s the way you’re supposed to do it, right?
You’re supposed to have someone that you’ve known all your life and they’re the perfect co-founder for you.
John: I think Edith is probably the only person that really believed in me as a potential co-founder.
Edith: Well, of course, you were perfect.
John: Well, thank you.
Paul: Aw, you guys.
Ellen: Very cute.
Edith: No, I had this deep conviction. I’m like, “John is the guy.”
John: But you were the person with that deep conviction, so that’s why we’re here.
Edith: I was like, “I will pry him out of his company and he will come do this.”
Paul: So did it take much convincing?
John: No, not really.
Edith: Some.
Paul: It sounds like you were coding on this idea part-time while you still worked. So you didn’t quit everything and then start doing it, by any means.
John: There were timing reasons for that. I think there was a conviction that I was going to leave. I think when I talked to some of the people that Edith talked to afterwards, I realized that she was worried that I wasn’t going to leave.
But there was never any point where I wasn’t going to leave. I was going to leave as soon as… We actually incorporated a little bit earlier, before we really started working on anything. When we signed the incorporation paperwork I knew I was done, I was going to leave. And the timing was just around other external factors and stuff like that.
Edith: He’d just had a baby, so I wanted the baby to be at least six or nine months old.
John: Which is really good because having a two-month-old and a startup, it’s crazy. It’s not fun.
Paul: I like the other thing, the thing with the process where you find the ideal person. Because you do find the ideal person that way. Well, I think most people don’t.
Ellen: Thank you?
Paul: There’s a compliment in there somewhere.
Edith: What has it been since, you’ve been working together for six months now?
Paul: A little over.
Edith: What’s funny is, even after John and I started the company and we’ve known each other for decades, we kept document of stuff we didn’t know about each other.
Paul: Oh, we just found out some stuff ourselves.
Ellen: Just funny stuff.
John: This seems like a really good venue for the two of you to share some of those things.
Paul: My one that we just discovered this afternoon, or that she didn’t know, is that
on a scale of one to ten, I basically live between four and seven. I never hit ten, but I never go below a four, either.
I’m generally just sitting in a five or a six, just generally content.
Edith: Sometimes you seem really morose.
Paul: It’s probably a four.
Ellen: I learned that I’d been interpreting the four as a one, and vastly overreacting.
Edith: Sometimes you seem really down, Paul.
Paul: That’s a four.
Edith: I’ve never seen a seven.
Paul: Yeah, sevens aren’t around that much. I’m usually at a five or a six.
Ellen: Term sheet arriving.
Paul: Oh yeah, that was good one.
Edith: Is is true that you’re in The Commitments?
Paul: Ah, yes.
Edith: I wasn’t sure. Is this just an Irish cliche?
Paul: No, that picture is this thing Ian tweeted. That was actually me at seven years old.
Edith: Here you are slumming it as a developer when you’re a movie star.
Paul: Yeah, I know, my 83 pounds went exceptionally far.
Edith: How did you end up in The Commitments?
Paul: Oh, that’s 83 pounds, that’s what I was paid for the thing. Because we used pounds in Ireland back in those days. Sorry, what was that?
Edith: How?
Paul: How did I end up there? My drama teacher, I think she was married to the casting person for the movie. Obviously it was filmed in Ireland and they needed people, we were at, me and this other guy, were going to be page boys. And then they cut the wedding scene, so they just threw us into some other stuff.
Edith: You should put that in your LinkedIn profile.
Paul: Yeah, “Star of The Commitments,” that would be good.
Edith: I was on Reading Rainbow, which is an American TV show.
Paul: What were you doing on it?
Edith: I was talking about my idea of a perfect person.
Ellen: What was the perfect person?
Edith: I said it was somebody that could eat a taco without spilling anything.
Ellen: Yes.
Paul: John, obviously a question here, can you eat a taco without spilling anything?
John: Not even close, no.
Edith: Actually, no, that wasn’t what I said, it was my friend Ian said, and I found it more memorable than what I actually said. But somewhere there’s a video clip of me. Do you know what Reading Rainbow is?
Ellen: Yes. LeVar Burton visited Kickstarter when I worked there, it was great.
Edith: Do you know what it is?
Paul: Vaguely.
Edith: John, explain what Reading Rainbow is.
John: That’s a kid show, LeVar Burton, it’s great. There’s a catchphrase.
Paul: Did he wear his visor?
John: No it was like pre–
Paul: Oh, pre.
John: Yeah, pre-Geordi LaForge. But I think that it was pretty popular when we were younger. I actually didn’t know that you were on Reading Rainbow.
Edith: I was around seven or eight.
Paul: But what else went in the spreadsheet? What else do you know about each other that you did not before?
Edith: Allergic.
John: Oh yeah, I’m highly allergic to everything. This started in my mid-20s or so. I was not highly allergic to anything and now I’m highly allergic to many things.
Edith: That he actually started a degree in design.
John: Well, in human and computer interaction. I was a PhD student when I went to Berkeley. First I was in human and computer interaction, and then I shifted over to PLs after that.
I spent two years doing HCI stuff. It’s really hard, actually. I think a lot of people have this misconception that HCI is like the softer CS, but it’s incredibly difficult and incredibly hard work.
Edith: John also has a minor in statistics.
John: Yep, you had to do a lot of course work at Berkeley for the PhD. So I ended up having to take a bunch of grad-level statistics in math. And that’s where I learned that CS is very easy compared to math, especially grad-level math.
Paul: It’s interesting because all of the PL papers, there’s no statistical vigor at all. None. We put an average, we put it on a graph, we’re done.
John: Yeah, that’s because when people like you and I go into like those actual math classes, they get blown out of the water by brilliant mathematicians.
Edith: This was handy because I actually didn’t know that he’d gotten this minor in statistics until we’re starting LaunchDarkly and there started to be more harder math with bucketing and putting people… He’s like, “I got this.” I’m like, “You got this?” He’s like, “Yeah, I got this minor in statistics.” I’m like, “That’s pretty cool.”
Paul: I keep forgetting that Ellen went to HBS. We were looking at some finances I don’t remember… Oh, it was the 409A. I was explaining what I had done from my previous 409As, and she’s like, “No, no, I know how to drive this model from scratch. It’s okay.”
Ellen: I believe it was, “This is how you do a discounted cash flow,” or, “This is why there’s a discounted cash flow.” And I was like, “Yes, we can make one right now.”
John: I don’t understand what any of those words mean.
Ellen: I didn’t understand what any of the static analysis was, so it’s fine.
Paul: You just wear the pants, it’ll be fine.
Ellen: I did have a moment at HBS where they decided one year that every single HBS student should have to start a startup as a class, in groups of five, which I feel does not model proper co-founder behavior. But fine.
I ended up having to pitch this fake startup with five people on stage to 1,000 people.I unintentionally made a joke and laughter spread throughout the audience. By the point at which I realized everyone was laughing,
I was talking about unit economics and I had no idea why they were laughing. The only thing I could think to do was to look down and see if I was still wearing pants.
I was like, “Well, I’m still wearing pants. I’m just going to keep talking about unit economics and maybe they’ll stop laughing soon, and it worked out.”
Edith: What was the joke?
Ellen: The startup was an on-demand photography startup, and I was trying to draw to the distinction of you wouldn’t use it to hire a wedding photographer, you would use it in the event that you had just been engaged and you were having your first engagement party, as in the party the Sunday after you got engaged on Friday night.
I think there must have been people who were on engagements that were not the first and it was interpreted as, “Oh, you use this the first time you get engaged, not the third and fourth.” It just took a very long time because it probably started with one person who had been in this situation, and it turns out, unit economics are not very funny.
Edith: I think they’re extremely fascinating though.
Ellen: Fascinating, yes. Worthy of 1,000 people laughing at you on stage, no.
Edith: Well it depends on the conference.
Paul: Edith, why did you decide to do a startup?
Edith: I think
it was the same reason I rode my bike across country or I was an engineer or I ran 100 miles. I just had this burning desire that I wanted to do it.
Even when I was in college I wanted to do it, I had other friends who had done startups, like the DreamHost people.
I remember them buying their first servers, them going to Fry’s and coming back with a server. I was sitting in the courtyard at Mudd. I just never felt like I knew enough, but I always wanted to do it.
It took me awhile to, as John said, to take this leap. I remember it was really hard. One of the reasons we postponed was I pushed it a couple times where I’d be like, “Oh, you know…”
Paul: “Want to learn another thing, and maybe I should get another job to learn this thing.”
John: Yeah, I think we ended up starting at the logical time. There wasn’t really any reason to postpone anymore. If we postponed again it would’ve been another couple years at that point.
Edith: Yeah, what if we had done it two years earlier? Would that have just been too soon?
John: Probably, yeah, I think.
Paul: For feature flags or for yourselves?
John: Both.
Edith: I don’t know, I know I felt completely green. I felt like I knew a lot about some things and other things,
we would literally be Googling how to incorporate, we did our clerky paperwork, we were both like, “Is this the right way to do it? I don’t know.”
John: I think it was the right time for me. My last couple of years at Atlassian, I was still learning a ton of stuff. I think it was all stuff that became really valuable for LaunchDarkly later, so I think the timing was right then. I think if I left two years earlier, there were things that I learned in those two years that I definitely needed.
Edith: It was scary to go without a salary. I remember I was talking to Will who was actually our first angel, and I kept wanting him to fund us before he left. He basically told me in very nice words, because Will is, did you ever meet Will Eldredge?
Paul: Oh, yeah.
Edith: One of the nicest guys.
Paul: Yeah, really nice guy.
Edith: He told me very nicely, “No, you have to quit.” Because I was like, “Can’t you give us some money so we could quit?” He’s like, “no.”
John: I think when we were at Alchemist, when we were in our interview for Alchemist, I think one of the things that really helped was, it was the day after my last day at Atlassian, I left Atlassian, my last day there was a farewell thing.
Then I got a text from Edith saying that we had a meeting with Alchemist Acceleratorthe next day. That ended up going well, so we went one day from me leaving Atlassian to getting at least some funding.
Edith: We’d gotten turned down by YC and 500 and then this old co-worker had gone through Alchemist, this other accelerator, and he’s like, “You should apply.” We applied and we didn’t hear back and I was just like “Oh, another rejection.”
The best thing that could’ve happened was that the organizer Robbie called me, he’s like, “Why haven’t you responded to my e-mails?” I’m like, “What e-mails?” And they had written us offering us an interview.
I know this now, Robbie sends out floods of emails and it had gone to my spam folder. Knowing Robbie now, I’m like, of course it went to spam. And so I’m like, “John, I know that you probably want to hang out with all your friends and have this good time tonight, but we got an interview at 10 a.m. tomorrow at DFJ.” And he’s like, “Really?”
Ellen: What was your reaction?
John: I was a little bit surprised. I was really looking forward to enjoying myself. I mean, I’d been at Atlassian for six years, so leaving was kind of a big deal. Saying farewell to the team with libations was one of the things I was looking forward to doing.
Edith: And I was like, “Ten a.m. Sand Hill Road, we’ve got to be there.” And he’s like…
John: Was that at DFJ?
Edith: Yeah, it was.
John: Wow, full circle.
Edith: Yeah, I know. It was our first meeting ever on Sand Hill Road, was our accelerator. Was your leaving like that? Were they sad to see you go, was it hard?
Ellen: Prior to this I worked for Paul English in my last company and he handled the departure better than I’ve ever seen anyone handle a departure.
It went from immediately it wasn’t about the company, it was immediately about me and what I was trying to do and how he could help. Which was, “I’ll make introductions, I’ll invest. I want to see your deck. Let’s talk about the idea, I’m super excited. I want to meet your co-founder, I want to do all this stuff.”
Then about halfway through the conversation he shifted back to, “I’m sad you’re leaving, I really enjoyed working with you.” But he handled it in this way that was better than anything I could have asked for or expected.
Edith: I think that’s what I try to do. We had a guy I like very much who quit to go to get his MBA. I was like, “I’m happy for you. This is a good opportunity, we’ll miss you.”
I know that the Valley or the world is small and people remember departures.
Ellen: But you were happy for someone to go get an MBA. I feel like normally everyone’s so upset about it.
Edith: Why would people be upset?
Paul: “We failed another one.”
Ellen: When I did it everyone was like, “You’re ruining your life, you’ll be two years behind, this is terrible, why would you ever do it?” It was not done.
Edith: Really, even at Harvard?
Ellen: Yeah.
Edith: Wait, I thought Harvard was like the golden palace?
Ellen: It was certainly interesting. I think people just don’t place much value in management, when you think about how many people say they don’t want to have one-on-ones because it’s a waste of time.
I’ve talked to so many founders who don’t want to do any management practice, don’t want to have any process, don’t want to have anything to do with any of that. And I find it baffling.
Just because you want to do it differently doesn’t mean you have to throw out everything that anyone else ever thought of.
Edith: That’s funny because we were just talking on the way over. we believe very strongly in, not management with a capital “M,” but just good management.
Paul: Little bit of process, not too much.
Edith: We sent our engineers to Dearing’s course why did you encourage them to do that?
John: Well, Dearing’s course, that thing is really good.
Paul: This is the “General Management”?
John: I think when you first hear the idea of, “You’re going to take a management training course, a three day management training course,” you’re like, “Oh my god, what level of bullshit am I in for?”
But Dearing’s course is actually really good. There’s stuff I learned in that class that,when I’m doing well as a manager, it’s because I’m introspecting and looking back at some of the things that I learned there.
You know, there’s a lot of years of experience you could have as a manager that get distilled down to things that you learn in those three days. I don’t know, I thought it was really valuable and I think it’s something we’re encouraging all of our people-leaders to do.
Edith: I think that you have a process, whether you acknowledge it or not.
You can either say we have a process, that some things are working and there’s other things we’re tweaking, but let’s acknowledge this. Or you have an awful process that you don’t even acknowledge how awful it is.
Ellen: If you don’t acknowledge it, people don’t know how to fix it, because they don’t know what to say is wrong they just know it’s wrong somehow.
Paul: I think that there’s, and this comes from someone who had a startup with no process…
Edith: Well, you had a process.
Paul: What you were saying, Ellen, about throwing it all out, that was sort of deliberately the point. We wanted to sort of start from nothing and do the things that were necessary as we came across them.
I think that there’s something to that, but it just doesn’t work out. You end up just doing nothing and then getting trapped under the weight of what you don’t do.
John: It turns out, after millennia, people have gotten really good at managing people, and companies have gotten really, there’s actually lessons that you can learn.
Paul: But there’s also a huge amount of people who are terrible at managing. We all have those anti-patterns that we’re trying to avoid, and maybe in our case, I think we threw out the baby with the bath water.
John: Yeah, because it’s all about standing on the shoulders of giants, right? Having the experience, maybe even being at some of those companies and seeing what worked well and what didn’t, and being able to throw out the things that didn’t work well.
Paul: I think that requires having worked at companies where it worked well, and I think we didn’t have experiences where we could point to this worked really well.
John: I can see that. Atlassian, for example, where I came from, it’s extraordinary. It’s a really well-run company. There’s so many lessons to be learned from there. They’re really good.
Like I said, I’ve had three jobs. I have that experience. Then I had Coverity which had a bunch of other different challenges where things weren’t working so well. I would view Coverity as more like the norm in terms of how a lot of startups are run, and Atlassian being this exceptional outlier. Seeing those two things and having those be my only two data points was kind of interesting.
Paul: I think a lot of the people who are throwing out all this stuff are younger. You do startups younger now, and you know people come right from college and they haven’t worked at a place that is really functional.
The other thing is, especially for engineers, the influence is open source, which is this massively dysfunctional sort of environment. So you take the “open-source policies,” and I put inverted commas around that, and then you apply that. They’re like, “That’s the way it’s supposed to be,” and then you end up, surprise-surprise, in a dysfunctional land.
Edith: How do you feel that Harvard Business School has helped you or not helped you with the actual startup? Because I’d always kind of wished I would get an MBA, it just never worked out.
Ellen: I think the most interesting thing about it, it’s still one of the hardest things I’ve done. At HBS the way it works is you’re in a classroom with 90 people and you read a packet story, you’re supposed to analyze the solution and be able to speak on it on any particular piece of data or on any opinion of what happened in that story, at any moment unexpectedly.
So you’re basically walking to a room with 90 people every day knowing you have to have 10 seconds of great content on something, and you just don’t know what those 10 seconds are and when they’re going to fit into the conversation or if they’ll even happen.
Edith: Much like a podcast.
Ellen: Much like a podcast. Except you don’t get a chance to talk again later, or edit or anything, and you’re graded on 10 of those 10-second chunks.
Edith: Wow.
Ellen: So it’s super high pressure, in contrast to something like fundraising. Fundraising is high pressure because you need the money to keep your company alive. But you’re telling a story you know and believe in. You’re not trying to pick 10 arbitrary seconds that make you sound really smart.
I think the pressure of being on the spot for three to five hours every single day, to sound good at any moment, builds a lot of emotional resilience.
Edith: Something I struggle with with fundraising, and John, I was a terrible fundraiser when I started. I didn’t know how to distill the story at all. I would just start with things that were true. Me and my friend want to start a company, we tried other ideas, now we’re doing this.
Paul: No, no, that’s, I would fund that.
John: I remember one piece of feedback, which was you were talking about failed deployees and stuff like that. I think you said something like I’ve worked on a lot of products that have just utterly failed. This particular investor was like, “I don’t think people really want to hear that you’re a failure.”
Edith: I would say I got better. At the beginning I was just really bad because I was just like, the word I heard once was “word vomit.”
Ellen: HBS, you can not word vomit because they’ll cut you off. So you get good at being concise and thinking on your feet.
Edith: Do you feel like it helped you with just being more able to think on your feet and react?
Ellen: It definitely helped with that. It also helped because you read so many stories over time, you have a lot of examples to draw from. So you can almost always come up with an analogy or parallel or give yourself a new framework to think through a situation.
So it’s no longer you’re in this situation where, “I made this thing, I have to solve this problem as it came up.” Every time you end up in one of those you have five good examples of other ways people have solved the problem.
Edith: I think that was the point I was trying to make about all these failed projects. I have all these things, this toolbox of, “Okay, I try this and this, I could reuse that template in a different way.” But it certainly didn’t come across that way.
John: Yeah that’s like too nuisance of a position for a two-minute pitch.
Edith: In my mind I’m like, “Okay, we try this playbook. Here’s how we could’ve tweaked it. Let’s do this.” Our content strategy is very successful because that took playbooks, but it was a terrible way to pitch.
Paul: The first time I tried to pitch in Silicon Valley, I tried to do it the Irish way. Which is you have conversations with people and you talk about anything other than the actual thing. And then at the end you’re like, “Oh yeah, so, I’m doing this thing.”
Edith: You just kind of slide across a note at the end.
Paul: Yeah, or set up another meeting afterwards. At that meeting you might spend quite a considerable amount of time just small talking as well.
I realized that the way I was pitching, it was sort of self deprecating, because in Ireland you don’t talk things up.
You’re like, “I was talking to these investors and this was in YC Demo Day,” and I’m like, “You know, this idea it’s a pretty good idea. Most startups fail, this one probably will too. But the idea itself, it’s quite reasonable.”
Edith: “I got this horse to sell you. It’s an okay horse.”
Paul: Then literally next to me there was another guy in my batch and he’s saying, “This is going to be a billion-dollar company.” In retrospect I was like, oh yeah that’s how you do it. But at the time, I was thinking, no, it’s not.
I could never sell that, that’d be ridiculous. The chance of this being one-billion-dollar company are minimal. And then when I actually learned how to fundraise, it was like, focus on the things you actually believe and don’t talk yourself down and that kind of thing. But it doesn’t necessarily come naturally for a lot of people.
Edith: It didn’t for us, took a lot of practice.
John: I think for engineers too, it’s pretty hard because you want to be extremely factual. You want to be driven by metrics. You’re very focused on how the world is today and what the state of affairs is today.
I don’t think that that works really well when you’re pitching, of that humbleness around the things that you built so far and all the things that you need to do to get to where you need to go in terms of building new things or whatever. None of that plays very well when you’re pitching.
Paul: Yep.
Edith: It’s funny, somebody uses me now, us as an example because we had somebody who we talked to three weeks after we started full-time. In hindsight we pitched him terribly and he didn’t invest, and now we’re a pretty moderate decent success and every time I see him, he’s just like…
Paul: I feel this is true,
of all the investors that you talk to, at the start they always get the worst version of you, the least refined.
You don’t have messaging, you don’t have positioning, you don’t know what works, you haven’t AB tested your way through 100 investors. They’re obviously going to say no.
Almost everyone I’ve seen, and this is certainly true with me, it’s like at the start you get a ton of no’s. And then you get one yes and then a few more no’s and then another yes, and you start to get it right. Then all of a sudden, something turns and they’re all yeses from there.
The people who really did you a solid were the people who listened to you when yousounded like a fucking idiot, but they didn’t get to invest. The guys who saw you later, and these guys are amazing, those people got to invest.
Edith: Oh yeah.
Ellen: But legitimately everyone always says that, and I always felt like the first one would go really badly and then we get a term sheet when I walked out of the first one I’d ever done.
Paul: They knew you for years.
Ellen: True. But just saying, I don’t think you should be, the first one’s going to go terribly. I think you should walk in and be like, “I want a term sheet,” walking out of the first one.
Paul: Absolutely. I actually talked to a bunch of people who I felt were going to not invest before I talked to the list of investors that I already knew, that I felt would invest, because I wanted to get the bad ones out and people who probably weren’t going to give me money anyway.
Edith: Also, to be honest, Ellen, you have a deep track record in startups. Whereas John had been at two companies. I had been more on the product eng side, so I didn’t know anybody and I didn’t know any single investor. Literally, none.
Paul: This last fundraiser is not an example of how one fundraiser is.
Ellen: Oh no, this was very strange.
Edith: Like when I started, my old boss, I asked him for advice, one of the founders of Tripit. He’s like, “Just call up your associate friends you know from business school and go pitch them for practice.”
Ellen: That’s what I did. I guess that’s why I was comfortable going in and doing one, was I called all my friends first.
Edith: Yeah. And John and I went to a 128-person school.
Paul: All engineers.
Edith: Engineers, CS, bio.
Paul: I like that it’s a 128, nice round number.
Ellen: Oh, we all went to tiny… well yours was probably bigger. Mine was 80 people.
Edith: Yeah, but I didn’t know a single associate. None, we didn’t have anybody, we pitched John’s brother.
John: Yeah, he was not insightful.
John: Awesome guy, but he’s a real-estate lawyer.
Edith: That was literally, I was like, “John, we’ve got to practice on somebody.”
John: Yeah, and he thought we were a horror show.
Paul: How many investors did you talk to in your seed round?
Edith: Probably, all told, I have a spreadsheet somewhere, 40 to 50.
Paul: I remember reading someone’s “we failed a startup” post where they had raised a very small thing and then they went to raise something else and like, they talked to 65 people and they didn’t get it going and they felt like they were done. I was like, “Holy shit, 65 is not…” How many did we, 140?
Ellen: Yes, I will say that as much as I can say the first one went really well, it set things up to feel really badly after that.
Edith: What happened after that?
Ellen: Then there was a lot of no.
Paul: So one is where we’re doing like this incredibly ambitious thing, so lots of people felt this isn’t going to work. And then also we may have priced it a little high, a little too high. And so it took us longer to get that finished.
Edith: What kept you going through 140? I mean, that’s a lot.
Paul: There was progress along the way.
Ellen: Yeah, I think it wasn’t quite that many. It was a lot of people, though.
Paul: It was 140.
Ellen: I think it was also that I was learning every single time we pitched because I hadn’t worked in this space before, so whatever the questions were that anyone would ask even if they were VC questions or they were from people who hadn’t worked in the field, I would just write them all down.
Then I would go home and then I would learn that set of things. So it was basically free education about the market and the space, they gave me more perspectives, besides Paul’s.
Paul: Probably useful to have more perspective beside Paul’s.
Ellen: No, you’re always right.
John: How did you guys split up the pitch between the two of you?
Paul: What thing that I learned during the pitch is apparently I like to talk.
Ellen: Paul did not learn this until we had one that was via telephone and it was, I think it’s Uber Conference that sends out the time spoken after. And Paul I think talked three times as much as everyone else on the phone combined.
Paul: This was the meeting where I was trying to be quiet and let Ellen talk more. So she’d been telling me this for quite some time and it really wasn’t…
John: You’re like, “I have metrics now to prove.”
Ellen: “I have metrics, really, you should let me talk.”
Paul: She didn’t even point it out. I was looking at the thing and it says I spoke this much, I literally thought there was a bug.
Edith: Do you remember when we were at Web Summit and we thought that there was a bug?
Paul: I think there was a bug. I think they cut us off short.
Edith: I think you just…
Paul: Really?
Edith: Yeah, I think we can pull the video.
Paul: Oh, no.
John: So you like to talk?
Paul: Apparently so.
John: How do you deal with the fact that you’re the talkative CTO?
Paul: I talk less.
Ellen: Actually, I like when you talk, I think it’s fine. I have no need to be the person talking all the time.
Paul: I think that you have a lot of very good answers for things I don’t know, and so when I talk I just like I hear a question I don’t know and I’m like, “Oh shit I better start to talk to cover, mask the fact that I don’t know the answer.” Ellen is sitting there going, “I know the exact answer to this, what the fuck are you doing?”
John: We got our strategy down, it took a little while. But I don’t like to talk, so that helps.
Edith: The original story of the podcast is Heavybit wanted to do one with John and me, of our startup struggle or just at every two week check in with a startup.
Paul: Oh yeah, that sounds like a good idea. You should’ve done that.
John: I didn’t think the production values were high enough.
Edith: So we’d come in when we were four people and we were like, “This would be a good thing, you can get some publicity,” and asked John and John’s like…
John: I had software to write. I had things to do, the microphones weren’t high quality at the time. Now they’re fantastic.
Edith: These are the same microphones.
John: Way to sell me out.
Paul: The entire time I’ve been doing this, they’ve been exactly the same.
John: Super high quality, as always.
Edith: We sat out there, it would’ve been like a half hour.
John: Thanks for selling me out.
Ellen: Any minute not spent writing code at the beginning is a wasted minute, you’re never getting it back.
John: That’s true.
Edith: John said he wasn’t going to do it with me and I waited another week because I’m patient and I asked him again and he still said no.
John: Now I couldn’t even, it was years before they let me on this thing.
Edith: I asked you three times because I’m persistent. I’m like, maybe he’ll have changed his mind after the third one. He was like, no. Then I asked Paul.
Paul: And then Fred.
Ellen: What happened to Fred?
Paul: I think I vetoed Fred. It was nothing about Fred, it was just like, two people is the right number of people for a podcast. Three is too many.
Edith: The exact quote was “Fred talks too much.”
John: Really?
Ellen: That’s ironic.
Paul: Thanks, Ellen.
Edith: You said it in an e-mail to him.
Paul: I mean, you have to say things to people’s face.
Ellen: Being honest is good.
Paul: It’s okay, Fred doesn’t listen to this. We can say whatever we like.
John: How did you decide to go to a really tiny school?
Ellen: I went to a tiny engineering school in Needham, Massachusetts called Olin. I originally did not want to go because when I went it had full tuition scholarships and I thought if you’re an engineering school and you have to have everyone go for free, you can’t possibly be a good school. Why would anyone do this?
Then I spent more time with the school and it turned out it had been started out of a foundation that used to build engineering buildings and they wanted to change how we taught engineering and move it away from, “We’re going to fail out one of three of you, and two, we’re going to make engineers who can actually build things for people and not just solve problem sets and then who can actually sell those things to people and not just be like at the whims of MBA types.”
I just loved the curriculum so much that I was like, “I see myself here, I want to go here.” And then we had a two-tiered admit process where you applied and then you went for a weekend and actually got to build stuff and meet people. So by the time I started our incoming class, was ’89, and I think I already knew 60 of the people and they already felt like my best friends. So it’s sort of like I’m going to go be with my best friends now. Even though they were these people I’d met for a weekend.
Edith: That’s pretty cool, because Mudd was pretty much the exemplar of the other one. We started with 169, we finished with 128, that’s why I knew it, because I had counted everybody that got failed out.
Ellen: That was how my high school was. My high school class started with 180 and we finished with 131. And we had an Olin building.
Edith: Yeah.
John: Sound like similar missions.
Ellen: They’re very similar. The Olin curriculum was taken pretty heavily from Mudd and Cooper Union, and MIT.
Paul: I feel like a chump going to a regular university.
Edith: Well we don’t have PhDs. I’m the chump here, everybody else has a graduate degree.
Ellen: No, I dropped out.
Edith: Oh, do you feel like that’s helped you?
Ellen: Well, yes. I got to be at a company from one to 50 people. That was delightful and then I got to meet Paul and start this company, so seems good.
Paul: We still put HBS on the pitch deck.
Ellen: Better half.
Edith: I guess I have a final story. John, has it been what you expected, or what surprised you?
John: I think the surprising is like, and this is super cliche, but like you always feel at some level you’re going to hit a success point where you feel like the stress level will go down. Haven’t hit that yet, have you?
Edith: Oh, yeah. It’s funny, I made a joke in a speech the other day, a talk I gave thatI was not very good at selling, but I felt I had this huge pressure because I had you, we had our first engineer I had to get much better at fundraising and selling so that was really stressful when we had no customers and no money.
Because literally I was like, okay, I talked my good friend into quitting a good job and I talked to Patrick who had a baby, into doing this, too. That was a lot of stress.
John: But it shifts, there’s the same amount of stress, at least from my perspective.
Edith: I think for me, it’s a different level of stress now that I want to do well by the employees, I want to do well by our customers, people are trusting us. But it’s very different than like, “Oh my god, we literally have no money.”
John: I suppose. But now it’s like you literally feel like 25 people are relying on you to get everything right instead of just Patrick or just me, right?
Edith: We have a business now and customers, I mean, it took us a year to get our first paying customer.
John: Yeah, that’s true.
Edith: That was hard.
Paul: Yeah, that sounds difficult.
Edith: We were in this new product and a new market. Like the first person to pay us took a year and that was a long kind of period of, “Is this a going concern?”
Paul: We’re in this place where like our first paying customer is like, not even going to be a year, it’s going to be a while. And that’s a fucking scary hand.
Edith: Oh yeah, it’s like, “Is this just an expensive hobby?”
Paul: Yeah, right. In CircleCI, our first customer is three months and paying every six months.
Edith: So we had customers, it just took us a year to get our first, well…
Paul: It was the first one to give you money.
Edith: Actually I’m making it sound, we deliberately didn’t want to take money for a little bit because I didn’t want to lock our pricing into stone. So we had people using us.
Paul: How long after you started?
Edith: We had our first person use us about two months after we started.
Paul: That’s great.
Edith: Because I was always just like, “We’ve got to get people to try this, bad.”
Paul: Was the product terrible at that point?
John: It was terrible, yeah. I was writing all the frontend codes, so that’s how terrible it was.
Ellen: It’s not terrible, it was too late.
Edith: I was very like vigilant. I was like, “We’ve got to get people to try this, I don’t care if you think it’s broken, we need to have people using this.”
John: I remember looking at one of our dashboards. When that customer went live it was like six requests per second hitting our service. I was like, “How are we going to stay up? This is massive.” Yeah, and I think we do, I don’t know, 50,000 per second now.
Paul: It’s crazy time.
Edith: I was just always very disciplined of “this can’t be a hobby,” because I saw other people in our accelerator who treated it like a hobby. So we’re like, “Look at us, we’re a startup.”
Paul: We try to avoid that.
Ellen: Yeah, definitely try to avoid that. I had one of those, it was not very productive.
Edith: The startup you were in where it’s like it’s a hobby?
Ellen: Yeah, the six of us who none of us knew what we were doing, it definitely was, “We’re playing startup.” We played startup for a year. I mean, I didn’t want to do that. I think it’s fun when you’re 19, and then maybe not after that.
John: How do you guys drive urgency knowing that what you’re building is such a moon shot in a lot of ways? You’re building something so ambitious that you could be expanding the scope over and over again.
I think if I was working on the problem you guys are working on, I’d immediately just keep blowing it up and blowing it up until the scope of it was so large and it’d be hard to focus in. Wonder how you guys manage that?
Ellen: I think it’s about getting the right time horizon. The nice thing about having a moon shot thing is you can be like, “In 15 years it’s going to be everything and it’s going to be amazing and it’s great.” Everyone can kind of feel anchored to the, “I‘m going here in 15 years, it’s going to be awesome.”
Then most days, everyone needs to show up and be like, “What is the most important thing I can build right this second, today, this week?”
Most of the time people don’t even need to be thinking about three months from now, other than in a very fluffy sort of way. Most of the energy is spent on, “What can I do today to make this go faster?”
Paul: We have our Monday meetings. Monday we plan and we say, “Where are we on the plan? Are we achieving what we’re hoping for? Are we getting the validation that we thought we’re going to have? Are the hypotheses being proved?”
But last week on Wednesday we redid the plan a little bit and then we’ll do it again on Thursday and a little bit on Friday, as we got more information and as we figure out more of what we’re doing. So it’s very short time horizons on specific hypotheses. It might be three days, it might be three weeks. It might be three hours on some hypotheses.
But the fact that we’re able to keep those things short mean that we’re you know,
it’s not we have to build everything, it’s like it’s this one very, very specific thing we want to test out and that drives the direction that we’re going.