In the latest episode of To Be Continuous, Edith and Paul examine why traditional hiring methods are failing. They discuss flaws of interviewing such as the interviewer’s bias toward pattern matching, and the impossibility of getting a true representation of a candidate through an interview setting.
They explore various forms of dysfunction observed at tech companies today and consider what it takes to build a cohesive team at a startup. This is episode #30 in the To Be Continuous podcast series all about continuous delivery and software development.
This episode of To Be Continuous, brought to you by Heavybit. To learn more about Heavybit, visit heavybit.com. While you’re there, check out their library, home to great educational talks from other developer company founders and industry leaders.
TRANSCRIPT
Paul Biggar: Okay, so I guess the question then is what’s your favorite thing about hiring?
Edith Harbaugh: Hiring?
Paul: Specifically about interviews.
Edith: Well, okay so I think of interviews as a necessary evil. I think what I like about hiring is that it’s pretty cool to build a team. I think the most important realization I’ve had is no one person can do everything. There’s something really gratifying about building a team that’s bigger than you that can produce far more than you ever could alone.
Paul: Right.
Edith: So I think hiring and building a team is a core competency, I think, of a startup founder.
Paul: So the context for this question was when we spoke to Marianna in the last episode, she talked about how interviews are often so bad that you don’t really know what you’re getting and you miss great people because you’re measuring for something kind of arbitrary.
Edith: Well they’re flawed in so many different ways, like people get nervous.
Paul: So this is a big one that we deal with at Circle, it’s like how do we know this person doesn’t know how to do the thing we ask them to do versus just got nervous in an interview.
Edith: Yeah, I didn’t use to like people watching while I did Excel spreadsheets. I do a lot of spreadsheets, I do projections and some people are perfectly comfortable with somebody standing over their shoulder.
Paul: Is it the fact that they didn’t like people knowing that you could use Excel?
Edith: No, it’s just something about having to do something on demand, under scrutiny, is a little bit paralyzing.
Paul: So I think there’s this bias almost, towards people who enjoy that. I view it almost like playing board games? You’re solving a puzzle with another human. It’s kind of a game, it’s something that you optimize, it’s maybe something you practice, and of course playing this particular kind of board game on a whiteboard with a nerdy interviewer or a stranger, is not anything at all like building software.
Edith: Yeah, I think you can tend to freeze up. I think you can optimize for people that know arcane knowledge. I think you totally suppress the ability to know who’s good at building a team.
Paul: They know who’s building a team.
Edith: Who’s a good building block of a team.
Paul: Oh, I see, okay.
Edith: Who’s going to be able to take constructive criticism, who’s going to be able to provide constructive criticism.
Paul: In college, we did a module which I’ve heard almost no one kind of talk about since, but the elements of a team and the ones that really stick in my mind are kind of the starter and the finisher. The people who can start projects and the people who can drive it over the line.
But there was another one which was the person who made the team work well together. And if you measure their output by number of lines or number of bug fixes or any sort of quantifiable measure, maybe you can’t even tell that maybe they look the worst person in a team, but actually they’re the gel that makes the whole team work.
Edith: Well this kind of goes back to the whole holacracy thing. You know, we don’t need managers. They’re dead weight. Well no, in fact you do.
Paul: Yeah, I mean I have a whole thing about managers.
Edith: Know about holacracy?
Paul: There’s this idea that you want to actually test whether someone’s going to be good at their job, and no one basically knows how to do that.
Edith: Even like basic things. So when I started using Quip, I would be writing something the way I always wrote it and then my co-founder would start making suggestions and I found it excruciatingly painful. This is my first draft, please don’t look at it. And he said, “Well it’s on Quip. It’s ready to be looked at”. After that, I started writing everything in text files.
Paul: I think that there are so many people who have different work flows and one of the things we found at Circle is that the amount of people who had different ways of running the application locally, like everyone had their own way. Some people were running it in VMWare, some people in Docker, some people had like setup all sorts of weird things.
If you catch someone outside of their workflow, are you really getting a true representation of themselves?
Of their ability to deliver code? Actually, let me take that back because it’s not the ability to deliver code that’s important, right? It’s the ability to, as a team, deliver the business objectives. So if you’re measuring, someone can write. Can you write a quicksort or binary tree or something like that on a whiteboard. It’s 17 steps removed.
Edith: Can you take a product specification and digest it, can you coordinate with other people who are in systems that you’re going to have to integrate with? Are you going to take constructive feedback in a code review? Are you going to weigh right what’s important in terms of scalability, security.
Paul: Have you used Feature Flags before?
Edith: Well that’s a new one, but thanks for wearing your T-shirt and talking about it. And all these are not things you’ll get out of an interview. These are behavior things about producing software.
Paul: So a bunch of people have started, and I guess that’s been known for a couple of years, but you see her group, for example, talk about this stuff, bringing engineers on site to try and build something during their time. And I think it’s kind of a closer approximation than a white board interview, but it’s still, I mean you’re not working with people. You’re almost necessarily separated from the team because you want to see what your individual contribution is.
Edith: Yeah, so when I was at TripIt, they did something I liked very much, which was they gave us a homework assignment and they said purposely, don’t spend too long on it. Spend about two to three hours on this, then you’re going to come in and present it. So I was interviewing for a product job.
Paul: Even that is nothing like work with a team to deliver a business objective.
Edith: Well I mean a lot of what a product manager’s job is is to write initial specifications and then present to a bigger team.
Paul: Okay, I see that now.
Edith: I vividly remember what mine was, so they just said, “here’s your problem” and I said, “okay to solve this problem I need to come up with personas for business travel”. So I came up with personas and I presented and I said, “here’s how these personas will tackle this problem”. And it was the nucleus for what I actually did once I joined.
Paul: Okay, nice.
Edith: So the people that we interviewed after that I did not like, were the people who just said, so the issue was basically integrating Yelp reviews into TripIt, and it just got down to this very low level very quickly.
Paul: Right, right. So they already made that decision, they hadn’t really looked at the goals.
Edith: Yes, they said okay we’ll stick this in here, we’ll do this. For me, it was more important that you fit a product feature into how it’s going to be used via a person.
Paul: Right. So I’m looking at interviews where I was on the receiving end of the interview. And so there was one at Dropbox, and I didn’t get this job, so you’re put in a room for four hours and every hour a new set of two people come in, and they’re sitting at the other end of a table from you, and then you’re a little performing monkey.
Designing some sort of algorithm that they specify. And I remember getting into a discussion with one of the interviewers. A discussion is a synonym for an argument. Where basically I had made some assumptions and he didn’t like my assumptions, and we were just kind of talking across purposes and it got into a bit of an argument, and I didn’t get hired.
Edith: Yeah, so when I was interviewed at TripIt, I remember I was interviewing with the founder and VP of Engineering, Andy Denmark. It was the two of us in a room. He said something pretty abrasive and sharp. And I was just like, “whoa”. And I kind of took a breath. Either I can yell at him back or I could just wait. And I was like, “Okay Andy that’s interesting you say that”. And he’s like, “I just wanted to see how’d you react”.
Paul: That’s kind of a real dick move. So I mean that the idea that someone would fuck with you to see how you’d react. It’s an awful thing to do to someone, while at the same time it’s kind of like this is what our culture is and you kind of gotta be able to deal with our culture. I’m not really sure whether it’s a good or bad thing.
Edith: It’s funny because TripIt was an exceedingly polite culture, but it was very direct.
Paul: Okay, I see what you’re saying.
Edith: So nobody ever raised their voices, nobody ever shouted. I don’t think I heard a swear word, except for occasionally, but people were very direct with feedback, like, “That’s not working”.
Paul: Okay, the other interview that I did that was interesting, and your presentation reminded me of it, this was at Lookout, and at Lookout, what they have you do after you do the interview is they have you come in and present to the entire company, although the company might be too big at this stage.But at the time it was like 70 people and about 35 of them would show up, people from all sorts of functions, not just engineers.
And so I was presenting on alias analysis and how it was useful for security. Half the people in the audience are marketing or sales or something that isn’t an engineering function and I’m presenting about this really hardcore, esoteric thing and it’s like, I don’t see the value in this at all. Later, I attended one of these and I saw the value in that someone completely bombed. It was just terrible, and I feel like they could probably have achieved that in a different way without wasting an hour of 35 people’s time, plus expecting the candidate to do that.
Edith: Was is that they wanted to show that you’re comfortable with presenting your work?
Paul: No idea, I mean that wasn’t the part of the job.
Edith: It’s funny, so I got an engineering degree and part of our project was to present our work. We had a clinic project and you worked on it for a semester or the whole year, and then you would go to the auditorium and stand in front of engineer professors who’d been at their jobs for 20 to 50 years and were the most brittle, curmudgeon you could think of. So they knew far more than you ever could as an undergraduate and were eager to tell you, but that was excellent training for the real world.
Paul: I mean maybe, but if you go to a company, if you look at all the different ways that you could present your work to a thing, so you could go up and give a PowerPoint presentation, you could be doing some sort of code review, you could be sitting in a meeting with people like the Jeff Bezos style of 15 minutes of reading at the start of meeting and we don’t talk. There’s all sorts of ways you could present and I’m not sure that it’s necessarily a good idea to interview on one of them.
Edith: I guess I was going on a different tangent that I think being able to present your ideas is actually very valuable, if you’re technical.
Paul: Yeah, absolutely. Communication is sort of a key thing in all elements of life, I’m not going to disagree there.
Edith: I gave a lot of talks about feature flagging and I found interesting that some of the people who asked the toughest questions seem like the harshest critics, and I found later are actually people who are interested in becoming a customer.
Paul: I see, so they’re thinking, This sounds cool, but there’s no way it can possibly work.
Edith: I don’t think you can have this area figured out.
Paul: Right, and then you give them the answer and they’re like. “Oh they clearly know what they’re talking about.”
Edith: Yeah, so it’s people who are like, “well how do you handle this?”, and I think they’re being a little pushy and it’s because they care. This is something that’s bothered them in real life, and so they’re asking questions because they’re engaged. It’s the people who don’t ask any questions who don’t care.
Paul: Well there’s also the kind of people who ask questions to show off what they know. And in conferences, these are the worst people.
Edith: Those were our engineer professors. Like of course, you’d been in the field for 30 years, you know more than me as an undergrad.
Paul: There’s a certain amount of just what kind of person it is who’s asking them. There’s this area, very famous CS professor who invented a whole branch of compiler stuff, who I’m not going to mention because he’s a bit of an asshole, and we were at this conference and he consistently asked really nasty questions. There was one guy he asked a question and the guy kind of froze and he’s like, “Oh, I dunno”. Maybe there was a language barrier as well, then the guy eventually goes, “you know it’s okay, you don’t need to answer, I knew the answer anyway”.
Edith: Ugh, so that reminds me, I think there are people who think if they make other people look bad, they look good. I was talking to, I’m not going to name the company, it’s a gaming company and I worked with the founder before at another job and he’s notorious for screaming fits at his employees.
Just, you know, “you all suck, you all fucked up. This is awful, why do I have such idiots?” And my take on that, now that I’m further along in my career is that you are the senior founder. If you are screaming at everybody that works for you for being incompetent fuckups, you know who’s the incompetent fuckup?
Paul: It’s clearly the founder.
Edith: Yeah, you have, number one, not hired good people and number two, not trained and mentored them to be better people.
Paul: And also you yourself appear to have some deep psychological issues that maybe you should be talking to someone about.
Edith: Yeah, if truly everybody around you is incompetent.
Paul: Right, and you hired them all.
Edith: And you hired them all, and you have not either mentored them to do better.
Paul: So what you’re saying is that he was bad at interviewing?
Edith: It’s funny, because he actually interviewed me. I didn’t get the job, I worked for a related company of his and once we worked together for about six months, he tried very hard to hire me.
Paul: Oh right, interesting.
Edith: He was like, “Wow, Edith’s actually really good. I really want her to work for me”. And I was like, “No”.
Paul: So clearly then, they had an interview process that was failing, right? My sense is that:
Absolutely nobody in the Valley, possibly in the whole world, knows how to do interviewing at all.
I mean perhaps it’s not even possible.
Edith: I was about to say, I’m sure some people are, but we were talking before about Google.
Paul: Right, I think they estimate that 50% of people who they hire were successful within their roles.
Edith: And they did another study actually where they examine the most successful teams and the most successful teams were not the ones with the highest smarts or scores, it was the one where they worked together best as a team.
Paul: The people that I continue to be very impressed by are the people who don’t necessarily have CS pedigree and don’t come across as all knowing, and often don’t know that much algorithms or impressive stuff, but are just able to get stuff done.
And it’s usually through their own personal focus they’re able to sit at a desk and produce code, business API, whatever it is, for like eight hours. That beats the pants off someone who’s got all this pedigree or who knows operating systems in and out, but gets constantly distracted by IRC or Hacker News or a shiny thing over there.
Edith: And also I think somebody who, I keep coming back to this, who can work with other people. And Marianna just talked about this, whether they’d be people in their own community, in other teams or partners.
Paul: It kind of comes back to how teams are structured and I wonder if you can even hire without knowing what team that person is going to be on, or what role they’re going to function in the team.
Edith: You mean more of a Google style of let’s hire a thousand people and let God sort them out?
Paul: Yeah, so I certainly think you can hire a thousand people and it’s like, okay this person is going to be very good at gelling our team together, let us find a team where that works.
Edith: Where team needs gelling?
Paul: Yeah, exactly. I’m not sure if that’s what Google does. The impression I get is no, but people are clearly not replaceable cogs in the way that people would like them to be.
If you’re going to interview people as if they’re replaceable cogs or as if everyone that you interview is the same as the next person, you can’t possibly be successful.
The thing that we talked about that started the episode where someone gets nervous. It’s like are we going to say that all people who get nervous in interviews are bad at their jobs? That couldn’t possibly be true.
Edith: Oh gosh, so I flashed back. I interview at TripIt and I also interviewed Greg Brockaway, the founder, who has a very interesting speaking style that’s very direct. And he asked me a question and I actually froze. And it was the silliest question for me to freeze on. He asked me if I read. And I read a lot.
Paul: Oh, and you froze because there were just too many directions to answer?
Edith: And he’s like, “what do you like to read”. I’m mean you’ve seen all my books. I read a lot.
Paul: It’s an amazing collection of books.
Edith: And finally I stammered out, “books”. And he said, “And what kind of books?” I’m like, “all of them”. And I walked out and I said, “I didn’t get that job”.
Paul: But you got it.
Edith: I got the job. But it’s just sometimes people just freeze.
Paul: I remember one of the things you see where this comes up a lot, in sort of marginalized communities or people who are not well represented in the tech industry.
People like to pattern match. They see another white male engineer and think great, he’ll fit really well in our Starcraft league. And if you’re a kind of nervous engineer then you’re not one of the confident people and therefore you don’t pattern much.
Edith: Yeah, it’s funny. So James Smith gave me some really good advice. He’s about, I’d say a year to two years ahead of Launch Darkley, so I was trying to hire our first marketing person and it was funny because I kept interviewing people and I would look them up on LinkedIn and he was connected and it said I interviewed them a year ago. He gave me very good advice which was marketing people are very good at marketing themselves.
Paul: I see.
Edith: By definition.
Paul: I mean are they?
Edith: Well he said, watch out.
Paul: Oh, okay. Right.
Edith: They’re good at marketing.
Paul: Can I challenge that briefly?
Edith: Yeah.
Paul: So marketing is, there’s brand marketing and there’s what was traditionally marketing, like 10 or 15 years ago, and now a lot of marketing is digital marketing. It’s analytics and it’s funnels and it’s numbers and it’s correlations and all that sort of thing and optimizing all these things.
Paul: It’s really not branding anymore. At least, it’s not the same function that I think most SaaS companies are looking for in marketers. And so
If they’re really good at marketing themselves, perhaps that’s a sign that they’re the brand of marketers that you’re not looking for, whereas maybe the person that you’re looking for is someone who comes in and can’t look you in the eye, but gives you a spreadsheet.
Edith: That’s very meta of you. I think you do need both. You do need somebody who could make copy, you do need somebody who can craft a message.
Paul: Sure.
Edith: You need to pair that with analytics.
Paul: I’m just saying, maybe good marketers are not necessarily good at branding today, for SaaS companies.
Edith: I think you also have to be careful of somebody who’s too good at branding.
Paul:Why?
Edith: Oh, they might be too concerned about their personal brand, rather than the company.
Paul: So this is something that’s kind of creeping into engineering and has been for the last maybe decade or so of people who build projects, open source projects especially. Who are able to create a lot of traction for their open source projects and then who come to your company and are a lot more interested with their own personal brand and promoting themselves, rather than working on the hard problems that your company needs.
Edith: Yeah, I had a long discussion with a friend of mine about this. There are some rockstar developers. And as you put it, maybe they’re very good at building this open source project, but they’re not very good at working on a team.
Paul: Right. That’s kind of the definition of a rockstar, right? That they’re the person that the spotlight is on and maybe you actually need the bassist.
Edith: I in fact despise the phrase rockstar. I think we talked about this.
Paul: I feel like we did. Did we have an episode where we talked about like 10X developers or that sort of thing?
Edith: Yeah, I mean we’ll just recap briefly.
Paul: Yeah, sure.
Edith: As you said,
I think a successful team is more like a chorus or an orchestra. You have your tubist your trombonist, your viola, your person who’s doing the symbols, your little triangle in the corner, and they’re all specials in their little part but they’re all playing together and they’re making a sound that is so much better than any of these individual parts.
Paul: Yes.
Edith: That’s a successful team. And if they’re all playing at different times and not paying attention to where everybody else is playing, you have noise. You have a cacophony.
Paul: I had an interesting experience. I went to see Joshua Bell, who’s a famous violinist, play a couple weeks ago and ignoring when Joshua Bell was on stage, which wasn’t all of it, but there was a conductor and the conductor was very full of himself. He was waving his arms around in a way that sorta indicated that he was not so much communicating, but performing for himself.
Edith: Showboating.
Paul: Showboating, very good word. I think that this is one of the reasons that engineers can often hate working with PM’s or with managers, because you get someone who isn’t really contributing, but who’s like waving their arms around frenetically.
Edith: Yeah.
Paul: I mean, I’m not saying it’s always the case, because a good conductor, no doubt, is exceptionally valuable for making a team, for geling a team, bringing it together, making the sound work just right. But then, if you get a bad one and they’re getting all the credit for the team, for all the team’s effort, then it’s not a fun experience to be on that team.
Edith: Yeah, I mean that goes back to, like I said, the bad managers, the one who yells at his team and tells them they’re all idiots. I worked with a guy who’s notorious for, “everything’s all broken, everybody who works for me is a ninny, only I can save the day”. And he did this three times, so I was like okay. The issue is not that everybody on your team is a ninny, it’s that you let things fall apart.
Edith: The firefighter versus the quiet person who just gets shit done.
Paul: Right, this is something that I struggled with, because certainly in the early days of Circle, I really didn’t have any idea how to sort of put out a product vision and there was all this code that was being delivered where I’d stop it at the last minute. I’d be like, “Oh no, this is all wrong”.
Edith: Oh, the Seagull Style. It is corrosive.
Paul: It really was, it was very toxic and it created all sorts of problems within the team that it took years to fix.
Edith: So I had that happen to me to, from the other end. Why do you think it was toxic?
Paul: Well people put all this effort in and then it’s like literally just before you’re about to ship that you’re told no, like it’s all wrong. I don’t know if you’ve seen James Lindebaum’s, Jamesisaservice.heroku.com, something like that. Yeah, it was the same kind of thing. It’s like, no this is wrong for reasons I’m not quite able to articulate and I certainly should have articulated before you started writing code out.
Edith: Yeah, so this happened to me early in my career from the other end. I was working on a project, and back then, we had UAT’s, user accepting tested, which is basically it’s all done. So I was in engineer at the time. The thing is all done, it’s built, it’s past QA, we’re about to ship it and we show it to our VP of product, no actually he was the co-founder at the time, and it’s mainly for us to walk through it so that he could do his own demos. That is what we thought the purpose was.
Paul: Right, and his purpose?
Edith: He looked at it and is like, “This is not what I want. At all”.
Edith: Not what I want? Wow. At all, and I looked at the product manager I worked with and I said, “this is just what the product manager had”. I did a lot of meetings with the product manager and he was like, “product manager’s wrong”.
Paul: And so was the product manager wrong?
Edith: I don’t know, so I was in engineering and he looked at the product manager and said the product manager who worked for him is an idiot, this isn’t what you should’ve built and he grabbed the mouse and he went to a competitor’s site and he’s like, “this is what you should’ve built”. And I said, “well you know, we built this. This is just supposed to be the UAT where you accept it”. He’s like, “this UAT has failed”.
Yeah. And I think a lot of my career has been in revulsion to that moment. That’s why I become a product manager, because I didn’t want to have projects like that anymore, I was like, “well I’ll be a better product manager”. But really, it was just the whole idea of getting feedback as soon as possible from everybody, internal and external.
Paul: I often give people advice on, you just raised the seed roundor maybe you’re trying to get from product market fit to just after product market fit. Often we talk about company culture and what this structure of your company is going to be like. The thing that I try to convey most of all is how important it is to set your culture and your mission and your product vision and your values.
Because if you don’t have those, then everyone is going to be going in their own direction. I mean they’re going to go in their own direction anyway, but the more closely you can make sure that everyone on your team knows what it is you’re building, why it is you’re building it, who you are as a people together, the less you’re going to have exactly what you talked about, where you get to the final hurdle and someone says, “This UAT has failed.”
Edith: Yeah, and I didn’t see this as a hurdle, I thought it was our opportunity to get the pat on the back, thanks for your work done. And all the engineers and I were looking at each other, shell shocked.
Paul: It was all their first time getting that feedback in front of one of these things?
Edith: I think so, yeah.
Paul: So presumably you quit on the spot, you’re like this company is clearly dysfunctional.
Edith: Well I was a kid. No, we went back, we rebuilt it from scratch. It was such a waste of effort.
Paul: So in the end, was he right?
Edith: I don’t know. I really don’t know. I don’t know if whether what we’d done would’ve been fine or not.
Paul: I mean clearly he was wrong in the sense that he should’ve created a process within his company where you get that at the start.
Edith: Yeah, he should not have said my product manager’s a ninny. And why did you listen to what he was saying. I was like, “dude, you hired him”.
Paul: I get mocked a lot in Circle for referring to things as shit. And I try not to do it about our own stuff, but often you look at something that maybe was built a couple of years ago and that seemed like a good idea at the time, you’re just like how is this still in the product.
Edith: You know, if you’re not completely embarrassed by it.
Paul: Exactly. I know in theory that shipping something that were shit is a necessary element, if only from like an experimentation point of view.
Edith: If only to discover that something that you thought was important, nobody cares about.
Paul: Right, yes. Exactly. And sometimes they’re still there because sometimes it’s more important to build the thing that someone is asking for and no one is using it so it’s kind of very low priority to get rid of it.
Edith: Yeah, it goes back to Dave McClure’s saying, kill a feature.
Paul: Right, right. So then there are lots of things that are shit and I got mocked, I think from a lot of my employees because I would refer to things as shit. This is shit, we need to fix it, this is shit, we need to fix it. And I tried very hard to make it so it wasn’t the thing that you have built is shit, or you as a person are shit. But people are still tied into the things that they built and the ideas that they had, so it can be very hard, especially if you’re a former engineer, as a sort of a first time manager or for some company founder, to figure out careful ways of expressing that you want to make the product better for reasons, comparing it to where the product currently is.
Edith: It’s also a matter of prioritization. Everything can’t be absolutely perfect.
Paul: No, of course not and if it is, you probably failed.
Edith: Yeah, so it’s trade offs. It’s like okay, this is not as strong as we want it to be, but we’re focusing on this other thing. But to tie it back to the original, it’s very hard to interview for that.
Paul: Right. Exactly. What you were saying about the TripIt manager, was testing you for how you handled direct feedback, and it’s like the more I think about it, the more useful that is.
Edith: So you swung around.
Paul: Well yeah, I mean if it’s a culture of direct feedback, then it doesn’t do anyone any good to harp someone who doesn’t work well with direct feedback.
Edith: Yeah, and this was like, “Whoa that’s a little bit harsh, but okay. Take a breath”.
Paul: But at the same time, it looks almost abusive to do to an interviewee.
Edith: I guess I’ve been in very tough engineering cultures and I thought it was just minor compared to what I had heard before.
Paul: Right.
Edith: I mean, I think the opposite is also very painful if your stuff is terrible, but nobody tells you.
Paul: Yeah, oh God that is the worst. There’s this idea of millennials are coming up and into the workforce, and that sort of thing.
Edith: And I heard everything before when I was Gen X, like everybody complains about the new generation.
Paul: Right, and the specific things that people complained about millennials are things like safe spaces and they have no way of taking feedback and that kind of thing. They want everything to be delightful and shiny. I dont know, for some reason the word “My Little Pony” popped into my head.
Edith: Are you a brony?
Paul: No, no.
Edith: There’s nothing wrong with it.
Paul: But still, I am not a brony.
Edith: This is a safespace, Paul. You can let your mane down, and go graze in the pasture.
Paul: So you’ve got people who believe very strongly, and with very good reason that everything should be nice and people should be nice to each other, and then there’s work cultures that are emphatically not that. Not every work culture, but every work culture’s different and they’re not necessarily, but there’s places where people say things directly, there are places where people have different beliefs than other people in the organization.
You kind of need to interview for that, in a certain sense. Is this person going to be successful within this company, and I see a lot of criticism of interviewing and interviews where they’re not as open to the person who is interviewing as they could be. When we hired very early on, we had a focus on work/life separation. So it just sort of happened that a lot of the first engineers that we hired had families, very often had young kids and so it was important to people to have that work life separation.
But when you’re an early stage startup, there are often times where you need to do something outside of the regular nine to five or outside someone’s regular nine to five hours, but the fact that we had created this work/life separation thing made it very difficult for me to feel that it was okay to call people late in an emergency.
Edith: So what we’ve done is we’ve been very flexible about hours, like we have engineers who work half days. But in return, it’s like okay. You might get to work a half day every week on a Wednesday, but in return.
Paul: We’re going to call you at 3 AM.
Edith: So we were explicit about that trade off.
Paul: So with each one of these things, each one of these requirements, you get an increasingly small interception, in the Venn Diagram sense of people who can manage to do this. And if you hire someone who has major objections to being called at 3 AM or who doesn’t like to go for lunch, doesn’t like Starcraft, whatever it is, you can get to real problems. And that’s not even including the legally protected things.
Edith: Well I think in a small company you have to be really ruthless. This person has a huge impact on your culture.
Paul: The issue is not so much whether or not you can find them, the issue is when you interview for these people, there’s now six facets that you need to interview for. And then the interview’s a whiteboard coding challenge.
Edith: I think a startup is creating value out of nothing. It’s not like a big company where you can hire somebody for two years and see if it works out. If somebody doesn’t make a difference at two months in a startup, it’s a lifetime.
Paul: But very often, you have to hire them without that knowledge. Maybe you wait two months and it’s hire fast, fire fast, sort of thing.
Edith: Yeah, well you talked about this in one of your famous posts.
Paul: Oh, which one is this?
Edith: The guy who wouldn’t work on Sundays.
Paul: Oh God, yeah. I mean that was employee one.
Edith: So in hindsight what would you have done differently with that?
Paul: We failed to set expectations.
Edith: That’s what a lot of my question’s about. Support, blogging, they were setting expectations.
Paul: I guess we failed to know what the expectations were ourselves, and to even agree amongst myself and the other co-founder, as to what the expectations were. I didn’t know that I would have this adverse reaction to someone, it being Sunday, they wanted to work on their own stuff. I didn’t realize until it happened that that was a thing.
Edith: Yeah, early startups are hard.
Paul: Yeah, Startups are really hard.
Edith: We’re two years in and sometimes I look back and I’m like, “whoa”. We talked about this with Martine the other day.
Paul: You kind of wonder how anyone gets startups going at all.
Edith: They’re still really fun. Like I get up everyday, excited.
Paul: I mean startups have lots and lots of excitement. There’s a lot of awfulness.
Edith: It’s fun now that we have customers using us. So we just hit a hundred billion flags. That’s a lot!
Paul: A hundred billion flags?
Edith: I mean I’m an engineer, to think that we created this from nothing is pretty super cool.
Paul: I kind of even think how much that represents. Our major measure were how many machines were in use. We had the graph of how many machines were in use.
Edith: And Amazon had that graph too, and it had a little dollar sign next to it.
Paul: I mean one day we’re on 10 machines, we’re like, “holy shit, there are 10 machines in production”. And then a couple of years later there are a thousand machines. I don’t even know how many there are in production now, and I was like, “wow, our little rinky-dinky 10 machines”. Now we don’t even notice.
Edith: Yeah, I don’t celebrate our machines because I pay the Amazon bill every month.
Paul: We pay the Amazon bill as well, that was terrifying.
Edith: So I think you brought up an interesting point though. I think sometimes interviews fail because people don’t know what they’re interviewing for.
Paul: I would say, as an industry, no one has any idea what they’re interviewing for. So actually, let me talk about a really good interview. This was to be an intern at Barclays Capital. It was a two day interview, they flew over a hundred of us in at the same time to do these things. They sat us in groups and they told us to solve a problem, and it was like, I think they’re like, “okay we’re going to stand 10 of you in a row and you have to sort yourselves into ascending order”.
And so we sat around a table and we discussed how we would communicate bubble sort to each other. It was mostly about communication, there was a small bit of technical stuff, they were sorta like natural leaders and natural followers who sort of established themselves in the thing and they needed both. They knew what they where hiring for and it very much wasn’t a technical skill.
Edith: It was teamwork.
Paul: Right, it was teamwork and at Barclays there was no technical skill required at all. There were some people who built impressive distributor systems and that sort of thing, but what I did in my internship was I added a field to a database. It was terrible. As a result, the teamwork was the thing that really mattered.
Edith: Yeah, it reminds me of, I’m not going to name the name of the software company, but it was a big software company. At the time there was this thing for a while that you don’t need product managers. That they get in the way of engineers.
Paul: I mean, they’re not wrong. That’s kind of the point, right?
Edith: So they didn’t hire product managers, but then they realized that the people who are always in demand on their teams were the engineers who were “product managery”.
Paul: Right, right. I had the belief when we started Circle that good engineers were good product managers. I had a good sense of product vision and I considered it an important part of the product that I was building. And so I kind of thought that that was what it was to be an engineer. As a result, we hired a bunch of people by whiteboard challenges and other engineering things and then I was shocked. I actually was shocked that they didn’t know how to build product. And I was like, “Oh maybe that’s just me. Maybe it’s not a thing that a whiteboard challenge tests for”.
Edith: Yeah, it’s funny. I think I’m very ordinary, but I’m not surprised that I have skills that other people don’t have.
Paul: I mean you must, right? Everyone has skills that other people have, that other people don’t have.
Edith: Yeah, I took it for granted. So I was talking to a friend whose company just failed, you probably know who it is, but it was an engineer and a product person and they built a product. What he said is, “I didn’t realize that you really had to go out and market it and sell it”. Which sounds horrendously naive, but he was very earnest about it and I was like, “No, I’ve been managing revenue as part of my product job for the past eight years. I was responsible for growing revenue”. A line that I cared about, I was very ruthless about that. A lot of product people just do not have that as part of their responsibilities.
Paul: This is why it can be so valuable to work as an engineer at a 20 person company.
Edith: I was picturing a rocket. I was picturing Elon Musk.
Paul: You’re going to be in the bit that’s discarded at the start. No, at like an early stage company that crosses milestones, in particular and that crosses milestones from the 10 to the 20 to the 50 to the 100, or any one of those. You will learn a lot in terms of what’s failing and what you need to hire to deal with those failing, and then seeing the before and the after and knowing where the value is.
Edith: Yeah, so we started this off by talking about jobs you didn’t get. I very much wanted a job that I didn’t get.
Paul: Where?
Edith: It was at Flixter. I very much wanted to work at Flixter, I would’ve been their first product manager. Got deep in the interviews and I’d come out of Enterprise Software, they looked at me and said, very nicely, “you seem very competent, but we want somebody who’s been there, done that with consumer software”. That’s fair, didn’t get that job. And I worked instead at Military.com, which was great because she gave me, my boss said we’re X amount of millions in revenue, make it X plus 25% out of the year. So what do I do to achieve that? She said, “Figure it out”.
Paul: Nice.
Edith: Awesome!
Paul: Yeah. So it was the thing that you needed in that stage in your career.
Edith: I figured it out, I got the revenue up and I went later and talked to the guy who had gotten the job and he said, “Yeah, you don’t want that job”.
Paul: And why was that?
Edith: You know, you would’ve just been a product manager, you wouldn’t have gotten the revenue responsibility, you would’ve become the person you are, basically.
Paul: The amount of growth that people have at early stage startups versus much later stage startups, like you see kids who come out of college and they go to Google, and
While I think there’s a great amount of things to be learnt at Google, it’s just incomparable to the amount that you learn as an early engineer at an early stage startup.
Edith: Well I think there are good early stage startups and there are bad early stage startups.
Paul: Even if you work at the bad ones, I think that you learn so much from them.
Edith: What do you think the lessons you learn are?
Paul: I mean you learn what doesn’t work. It’s like yelling at people is a terrible management tactic.
Edith: It’s not a management tactic.
Paul: Well yeah, exactly.
Edith: It’s more like an anti-tactic.
Paul: Right, and it’s like I think there’s so much to learn from failure, and having failed a number of startups at this point, I think a lot of what made Circle successful is what I learned from failing at the other ones.
Edith: I think there’s a lot of value in, not even having failed, but to have gotten that rhythm of this is how, so Marianna talked about this before, this is how we do things successfully.
Paul: Right. Exactly. And part of that is, this is what unsuccessful looks like. So I get asked often:
How do you know when you have product market fit? My answer is generally it’s kind of like being in love. You can think sometimes that you’re in love, but when you’re in love, you know it. And it’s the same with product market fit.
If you’re kind of thinking, maybe, is this product market fit? That’s not product market fit. When you have product market fit it’s like this blindly obvious thing. It’s taking off, you can’t stop customers from using it. It feels nothing like the feeling of not having product market fit.
Edith: Well not to be a curmudgeon, but that’s hormones and then it wears off.
Paul: Okay, maybe it wasn’t the greatest analogy in the world.
Edith: We talked about this before where you think you have it, but you don’t. You’re just at a local maximum. Like Secret thought it had product market fit.
Paul: It had something. I think Secret is this really, really interesting case study, especially around the investment. There’s a big secondary, I think three or four million, six million was taken off the table in a secondary investment. And then when it failed, people were like, “Give back the money!”
Edith: It’s like no.
Paul: It’s like no, of course not. That’s why we took secondary.
Edith: That’s not the bargain.
Paul: I think even when the VC’s wrote a thing about how they should give the money back, and I was like, I don’t know if I’d ever work with that VC, as a result of him saying that.
Edith: I think that’s a thing that we have in the States which allows people to do startups. The idea of limited liability.
Paul: Right, right.
Edith: People are very hesitant to do startups, for example, in Japan because you’re personally liable.
Paul: Phil Greenspun, who made ArsDigita, had a very interesting blog post where, I think he was talking to his mom and he was talking about all the money that investors had put into it and ultimately lost, and she’s like, “and do you have to pay that back? And he’s like, “well no, not at all. And actually, they want to give me more to try again”.
Edith: Yeah, I mean I remember my mom’s quote when I started the company. I was afraid of telling her. My mom’s worked her whole life, so I think she retired after 25 or 30 years at Lockheed Martin. So she’d been there when it was Martin Marietta, they got acquired by Lockheed, same job, actually went to college on a Lockheed scholarship. So for me to tell her I was going to go work at, not only at a risky startup, but my own risky startup, I didn’t really want to tell her.
Paul: Because she had a more risk averse career than you had.
Edith: Yeah and then my grandfather had worked his whole life at Bell, so I really dreaded telling her. Like I didn’t actually tell her until I had left my job. And I started working in a startup and they were coming to visit. I was like, “Oh they’re going to want to see my office”.
Paul: Here’s my office, it’s a closet in someone else’s office.
Edith: It wasn’t even that, I was literally working out of my apartment. John and I didn’t even have an office. Finally, alright I needed to just tell her, so I called her and I told her and she said, “Well Edith, I’ve seen many idiots start startups. You’re smarter than most idiots”. I think we drifted away from interviewing, but I think you do have to kind of be an idiot to start a startup.
Paul: I mean pitching is kind of interviewing in a certain way.
Edith: It is.It’s very artificial.
Paul: And then people are judging you on things that don’t matter, even in the slightest.
Edith: It’s very artificial, it’s very screening for things that might not matter.
Paul: Yes, theres pattern matching. Paul Graham talked about this guy that they funded because he literally looked like Mark Zuckerberg, and then they realized afterwards this was a terrible thing. What kind of idiots are we for doing that, that pattern matching and then they controlled against it. But even someone as smart and experienced as Paul Graham makes that silly mistake.
Edith: Yeah, and I think a lot of it is VC’s are going for really outsized returns. So they’re not going for 5% return.
Paul: I had coffee with Martine a couple weeks ago and he told me about this experiment, I don’t remember where it was done but there were three animals of some kind. Like ducks, let’s say, and one of them was fed on a schedule and one of them was fed randomly and one of them was fed when they pressed the button. The duck that was fed on a schedule would start to arrive a couple of minutes before the schedule for it’s feeding.
The duck that pressed a button, pressed a button when it was hungry and it just got in the habit of that. And then the other duck just went absolutely insane because it had no idea when the food was coming and so it would tap its foot three times and touch its head and quack to get the food. And this feels a lot like what VC’s are doing. They have no idea what works and so they look for some kind of, there possibly is no pattern that works, but they have to discern some sort of pattern amongst the tea leaves and figure it out. So a lot of the times it’s going to look fucking stupid.
Edith: Well yeah, I mean it’s easy in hindsight to pick on why do you fund VO, why do you fund Secret, and I think in a VC’s head, these are going to be the next Instagram or WhatsApp.
Paul: I did the numbers on Secret, and that was “absolutely fund, no questions asked”.
Edith: Well people thought it was going to be the next thing, as I said.
Paul: I mean if you just look at the expected value, if it had been the next thing, and lets say, okay companies that get to the stage that Secret are at, there’s a 10% chance of it becoming the next Facebook, and if it does then that means that our investment right now is worth 10 billion, and I have to put in 10 million to get that 10 billion. And sure there’s a 10%, so you have to discount that thousand time return by 10 to get your expected value, but it was a no brainer for anyone who could do basic math. Obviously VC’s are very good at basic math.
Edith: I think that’s why the dev tool space has struggled a little bit.
Paul: No so much anymore.
Edith: Well not so much anymore.
Paul: I mean once it was established, it was very easy, people were like, “oh yeah dev tools are these great things”. In 2011, people are like, “yeah developers don’t pay for money”. I got an email from Paul Graham saying developers don’t pay for tools.
Edith: I still get that developer tools will never be big. I think people don’t realize, as Mariana said, that tooling has become essential. You don’t have a carpenter show up at a work site and say, “hey, you need to build a hammer and a level before you can start working”.
Paul: Well I think people don’t realize how much space there is in tools. It’s not just hammers.
Edith: There are nails. Screwdrivers!
Paul:We are the hammer that every company uses as a service.
Edith: I think also, it’s funny. So I do diligence now for other VC’s and I always feel a little sheepish about it because I was vetting somebody’s idea the other day, and I was like, “Oh I don’t think that’s a big idea at all”. I heard this echo in my head of people saying that about our idea, because when we came out, we were considered crazy.
Paul: I remember the first time I heard, “this is ridiculous.”
Edith: So what was your reaction when you heard that.
Paul: I wrote a library to do this in like 10 lines and the reaction was “this is ludicrous, who would pay for this?”
Edith: Yeah, and now you’re a customer.
Paul: Yeah, exactly right, because turns out that once you implement it five times in five different ways, you’re like maybe there’s an abstraction here and maybe they’ve got it.
Edith: So I think any idea, when it starts out, is either completely bloody obvious or seems so stupid. And we were certainly in the so stupid camp. Everybody’s like, “what? who needs that, who would pay money for that?”
Paul: So Google acquired Apigee. And I was like, “oh, I forget what Apigee did, but I guess that must’ve failed”. No, Google bought public company Apigee for 625 million, and I still don’t know what Apigee does.
Edith: API management, clearly.
Paul: Well clearly I don’t know why you need API management.
Edith: It’s hilarious, because I think people think of the dev tool space as this uniform thing, and it’s not.
Paul: Remember when Scott Raney came on, and he’s like there are four different areas of dev tools and there are people who are actually making like editors, and there are people who are making soft tools, like Circle, and then there are the people who’re making products that appeal to developers like Stripe and those are completely different categories.
Edith: I think there’s stuff that’s within the stack, like within a data center that’s just, I have no clue about.
Paul: Right, because I’ve never had a data center.
Edith: My VC friend put it best, he’s like, “Yeah, there’s people who invest deep in the stack, I’m not one of them”.
Paul: Right because presumably, they don’t know it. Don’t invest in what you don’t know.
Edith: It’s very funny, he was actually the guy who was at SalesForce and who acquired Heroku. So he was corporate development. It was interesting, we’re in the house that Heroku built.
Paul: Billy. Smart guy. So it’s very much like interviewing, right? So you’re talking to this person and you’re trying to discern some pixie dust about what makes them special in some way.
Edith: And I think if you’re a good company, you’re also interviewing the VC. Like is this a VC I want to work with? I’ve gotten various strong warnings about a couple of firms, just do not work with them.
Paul: Yeah, I could give a couple warnings and have gotten a couple warnings myself about those.
Edith: The best advice I got, it was from my friend, Mike B., who’d actually gone through YC. He said, Trust that you’ll find an investor, but what you’re really trying to do is find the right fit. Because you’re going to be with this person for a long time. Like a long time.
Paul:
For people who want to be company vendors, there’s a lot of lore, a lot of excitement and shininess around becoming a company vendor these days.
Edith: I guess it’s like being a screenwriter in LA.
Paul: And there’s just so many of these things that you have to do and you’re signing up for them. If you’re not signing up for them, don’t fan the company.
Edith: I think people get confused with the successful startup that has raised a B, C, or D. Where you go to all the conferences.
Paul: You sign up on the stage and people are like, “oh my God, that person!”
Edith: And there’s all the grunt work before that.
Paul: You’d probably never reach that level. Most companies fail.
Edith: Yeah, most companies fail and you just have the sheer grunt work to get a company going.
Paul: I find it fascinating, when I’m talking to large companies, I remember I was talking to Symantec, and I was talking to VMWare and I’m talking to some mid-level manager, or something like that. He’s like, you know I have a startup. And I’m like, “what?” And he’s like, “yeah, I sold a startup to said company for a certain amount of money”. There’s just all these companies that you’ve never heard of, where they get acquired for a amount of life changing money and then the person goes and retires quietly to a middle manager at that startup and has a family, whatever it is that fulfills them afterwards.
You never hear of most acquired or exited startups because their founders do not get to stand on stage and do the exciting things and the keynote things and all that jazz. They build a thing, and kind of quietly, they go away.
Edith: Maybe they don’t want to go stand on the conference stage.
Paul: Yeah, exactly.
Edith: I think it’s interesting. So Sean Byrnes came for a prior podcast. He founded a pretty successful startup, Flurry?
Paul: Yeah, I feel like I’ve heard of that.
Edith: I dunno if you’re sarcastic or not.
Paul: No, I’ve heard of it.
Edith: Then he could’ve just stayed at Yahoo and been a middle manager, but no, he started another company from scratch.
Paul: That kind of serial entrepreneur. Some people just want to keep building things.
Edith: I wanted to keep building things. So I’m a mentor for my accelerator, which is not YC, it’s Alchemist and I give a class on Angel fundraising and the phrase I hate most is just “tell us the hack.”
Paul: The hack? I’ve never even heard this.
Edith: The hack to get Angel money.
Paul: Oh no.
Edith: Yeah, so everybody has this idea to write an email, send it to 30 people on Angelist and just cash the convertible.
Paul: Oh my God.
Edith: What’s the hack? There is no hack. Well, here I’ll condense my talk. You write down a list of everyone you know that has money.
Paul: Yeah.
Edith: You correlate that list with Angelist to see if they actually Angel invest.
Paul: Yeah.
Edith: You set up meetings with them, you ask them for money.
Paul: That actually wouldn’t be how I would describe it.
Edith: Well this is Angel fundraising.
Paul: Yeah, no, no. Suppose that you make that list of people you know who have money and then you discover that there are two people on the list. Then how do you do it?
Edith: Well what I say is that usually you have more people than you think. So I do this with people, like I don’t know anybody. Okay, I know you worked at Paypal. Then people come for 101 sessions and they’re like, “Oh, I don’t anybody who has money. I’m like, Okay, you worked at Yelp. Yelp is public.”
Paul: So I came to Silicon Valley with knowing nobody, and I started my company very shortly after. So the list of people who had liquid money was very, very low.
Edith: That’s the people who I feel sorry for.
Paul: But there’s a hack for that as well. It’s that you talk to a bunch of entrepreneurs, you talk to like all the entrepreneurs that you know and then you tell them that you’re raising money and then you ask them for introductions to the people that you know.
Edith: So you make a list of everybody you know that has money.
Paul: So you’re viewing those people who don’t have money but have a connection to money. I understand what you’re saying.
Edith: So the hack of I just want to send out emails that have money coming in.
Paul: I didn’t even want to send out emails.
Edith: The other thing I didn’t like was why does a CEO have to do this? Why can’t I hire somebody to go fundraise?
Paul: It kind of makes sense. There’s a very intuitive logic that’s like, well it’s a simple, repeatable process that other people are good at. Why can’t they do it?
Edith: Well because then that person is your CEO.
Paul: Right, yeah. There’s a tiny, tiny thing missing. The fundraiser’s the CEO. Until series C, probably.
Edith: What people are really investing in is the core team.
Paul: Right and they want someone who can stand up and stand in front of a PowerPoint slide and present something. It’s a core part of being CEO.
Edith: I don’t know if you’re sarcastic or not.
Paul: I’m semi-sarcastic at this point.
Edith: I didn’t use slides when I fundraised, at least at an Angel level.
Paul: Yeah, I mostly had a conversation.
Edith: You had a conversation and you say.
Paul: And then you send in the slides afterwards.
Edith: I never send slides.
Paul: Oh really? So that’s what I should’ve done, but inevitably there’d be someone who says, “Okay send me the slides, I just want to show it to my partners”. And then it’s halfway across the Valley.
Edith: They have all these Angel receipts.
Paul: So at the time that I was fundraising, there was a move for VC’s to invest in seed rounds because they didn’t really understand how the whole dynamic was playing at. They stopped doing it a couple years later because it turned out to be a disaster, but I certainly talked to a ton of VC’s around it.
Edith: Yeah, I think that’s a very different thing though, I think Angels and Seed are different. Why do you think it was a disaster?
Paul: So it was signaling risk. People ended up with like three investors, three VC’s in their rounds, none of them would do the A, and as a result, nobody would do the A.
Edith: Yeah, and for the firms they saw it as a funnel where we’re going to invest in 10 and maybe do the round of two.
Paul: If even.
Edith: But then that basically shot those other 10.
Paul: I remember talking to Bessemer in the seed round and they offered to put in maybe 100K, and I had just had a big schooling on signaling risk. They’re like, “Oh no, no, we put it in under another name. The name on the cap table is not actually Bessemer.”
Edith: That’s hilarious.
Paul: Then when it came to do the A, Bessemer didn’t give us a term sheet.
Edith: Oh, so you took it?
Paul: I didn’t take it because I was afraid of the signaling risk and had enough money anyway, and then when it came time to do the A, if we had had them in the RAND, it would’ve been this awful signal that killed the A.
Edith: Yeah, because then everybody’s like, “well what do they know?”
Paul: Right, so in the end they knew nothing and there were various reasons they didn’t do the A.
Edith: Yeah, I’ve heard about this whole thing of scouts.
Paul: Like venture partners?
Edith: Well it’s this real thing where people would claim they’re in Angel, but they’re actually investing for a VC firm.
Paul: Oh wow, because then they got the information, they pass it to the VC firm and the entrepreneur doesn’t know?
Edith: Right.
Paul: That is shitty.
Edith: Yeah, we gotta call time.
Paul: So okay, finishing thoughts on this: interviewing is hard.
Edith: Interviewing lacks for sometimes the skills that are not the same as success.
Paul: And no one knows how to interview for the skills that are success and maybe even knows what those skills are.
Edith: And I’d say also that it could even select for stuff that is an anti-pattern for success.
Paul: Absolutely, yeah.
