Right Grid
  • Overview
  • Transcript
Trajectory

The Language of Liftoff

Cindy Alvarez Microsoft

The best DevOps intentions and tools can be derailed by... words.  Yes, words! How we frame ideas, ask questions, and identify risks has a huge impact on a team's ability to ship faster, learn faster, and innovate more.  I'll share examples of the good - how to move from bikeshedding to experiments, how to inspire experimentation, how to combat FUD.  I'll share examples of what not to do - how to stamp out individual developer initiative, how to instill fear of failure, how to build learned helplessness.  (I'm sure you'll choose to follow the good and not the anti-patterns.)

Downloads slides

Cindy Alvarez

Cindy helps internal teams at Microsoft get better at continuous learning, customer understanding, DevOps, and culture. She is the author of Lean Customer Development: Build Products Your Customers Will Buy. She also hosts a membership-based community offering toolkits and curated advice and tactics for enterprise organizations working through innovation and change projects at http://www.cindyalvarez.com/foundry. When she’s not thinking about the intersection of tech and teams, she enjoys running half-marathons, cooking, and parenting two small humans. You can follow her on Twitter at @cindyalvarez.

Heidi Waterhouse: I'm here to introduce the next speaker for you. Cindy Alvarez is here, and she wants to talk to you about the language of liftoff. I'm excited to hear her talk about how we're going to do this, how we're going to talk about the way that we change things and the way that we make things go faster. Without any further ado, here's Cindy. Thank you very much.

Cindy Alvarez: Alright. Hello everyone. My goal for this talk is that we all leave it smarter than we were coming into it, and I recognize that might be a high bar, but just putting that out there. I find one of the interesting things when we try to change systems, we all think we know what we're aiming for, and often we don't actually just say it in plain language, write it on the whiteboard, etc. I'm a big fan of that. Whatever it is that you want to happen with your systems and with your teams, tell them because they might not know.

Hey. Logistics to start with, you can get the slides, there's a little automatic thingamajiggy, which we'll send them to you. You're also welcome to take photos, of course. Let's think about what I just said. What does it mean to be smarter? It's really the ability to take some piece of new information and assimilate it into what we already know, and use that to think about problems in a new way or to tackle problems that previously seemed impossible. That's really what we're trying to do with all of DevOps. When we're at when we're instituting feature flags, when we're instituting, automation and practices, what we're really doing is we're taking these incredibly complex systems that previously were pretty much unknowable by almost anyone. We kind of pretended that we understood them, but we really didn't.

We're breaking them down so that they can be knowable by more people. The reason that we're doing that is so that we can be faster so that we can put more ideas into the system, figure out whether they work or not, and kind of get them out. When we think about these systems that we're building, what's the biggest and frankly most expensive part of that. It's the people we have. We can have these really phenomenal tools. We can have things like LaunchDarkly, we can have all the things we're using, but fundamentally those tools don't overcome the fact that we're all a bunch of humans, and we all have human brains, which have a lot of fundamental flaws to them. A lot of good things too, but a lot of things that get in the way of us building great products.

I find that a lot of times, I work at Microsoft, I come from a background of startups, I do dozens and dozens of developer interviews every year, and so I kind of hear all the perspectives. The funny thing is that feature flags, for a lot of enterprises that feels like way more chaos than they're used to, and it's terrifying, and for startups it feels like way too much process, and that's terrifying in its own way. We kind of have both sides where people's mental outlook on things is coloring how they're going to do things.

But fundamentally, whenever we're launching something, every feature flag, every idea is ... it's a hypothesis. It's something that we think might work. Adrian said one third work, one third are neutral, one third are bad. We want to get as many of those through the system as possible. To do that, what we need is the people, that big part of the system, every single person not just needs to be trusted trust but verify, but more importantly needs to feel like they're trusted. There's kind of a weird thing here, Adrian also talked about time to value, which is a great metric that I love. One of the things that we often don't measure in time to value is the time between where someone said,  "I have an idea, or oh I don't feel so good about that decision, " and the time that elapses before they do something about it.

Whether you're in a small startup or a big enterprise, there's often a long period of time where people are running it past their one coworker, they're putting it out in a private slack channel to see what happens. They're kind of hesitating because the last time someone criticized what their boss suggested, that person got eye-rolled or kind of got talked down in a meeting.

I talk about language because it's honestly a thing that we don't have a lot of examples for. We have sample code for just about anything you're going to try out on your systems. We have a lot of great books that kind of talk in theory about the ways we should talk to our teams. But we often don't have the sample code of language, of,  "Here, just try saying this and see if it works. " I'm actually going to give you some of those in this talk.

I talked about our brains and the hardware we have. One of those things is that fundamentally our brains have a bunch of defaults that don't serve us well in building software. They did a really good job of keeping us alive. We didn't get eaten by woolly mammoths, so that part's great. But these are not bad things, but they're things that we all have and kind of have to overcome. When I say defaults, I really do mean it in that way. I mean it's not every person is impacted in the same way, and it means we can overcome them.

But to kind of start with, we're all a little bit egocentric. We like to see the world as it benefits us, and when we see changes, even if we don't say it out loud because we've been well socialized, we're thinking what's in it for me? If we don't know what that is, it's hard for us to think about learning a new behavior and giving up the thing that we already knew how to do.

Loss aversion. We don't like giving things up. Usually, when people talk about loss aversion, they talk about it in the form of gambling, people who will keep making bets on roulette because they're convinced that it's going to pay off the next time. But loss aversion in software is often,  "I knew how to do this thing, I was competent at it and now that you're making me do a new thing, I'm going to have to lose that competence. " Even though it's in the short term that feels pretty awful, and we hate that. We have authority bias, and even in the flattest hierarchy startup, this is something that kicks in as a default.

Again, we might practice really hard on empowering everyone, and when people are tired, or stressed, or something else is going on in their life, they kind of fall back to this authority bias of,  "Oh, the boss said so, the senior engineer on the team said so, this person that everyone respects the most in this area said so, so I'm kind of going to defer to that. " I've found that happen with teams of mine. As much as I tell them that they own decisions and I try and stay out of their way, there'll be a time when people get tired and they're like,  "I was waiting on you to see what you thought. " Please don't do that. That's an incredible bottleneck.

We also have something called reactance, which is basically known as the like,  "Nah, nah, I'm not going to do it. " If we're told to do something that kind of makes us want to do it even less. These are the defaults we're working with, and it means that whenever we're trying to improve systems, or changed systems, we're fighting against these. I think it's one of those things where we tend to pretend that this doesn't happen and we kind of couch it as we should be professional, we should be mature and overcome these, and then we just all default back to this. I think it's like any other constraints in the system, know it's there and learn to work around it.

What these defaults really have an impact on is how we react to the things we hear. The words we use have a big impact on what people are going to do, how willing they are going to be to change and speak up. When we talk about improving processes and changing methods, a lot of the things that come naturally for us to say that feel instinctively like the right thing actually inadvertently derail people or create really misaligned incentives. I always like to bring up Wells Fargo open as many accounts as possible. Obviously, a really terrible incentive because everyone started opening accounts that no one wanted, opening fake accounts, bullying customers into getting this. When you just kind of make these statements, sometimes you end up with that. I'm going to talk about a few things that people say, and a few things that go unsaid and how these can impact your process.

Again, so I said, of all the things that we can do to make ourselves work effectively, refactoring our language is the easiest. When we talk about pushing ideas through a system, you can generally change the things you say and watch the room and watch how people react. That's an experiment you can run very, very quickly. I like to run a lot of them.

I'm going to give you as, like I said before, sample code. I'm going to give you some things that people have said that haven't worked, and some alternatives, and your tone or your level of formality are going to change depending on your company. But fundamentally, no matter how much we think that we're all different, humans are really, really similar. Whether you're in a foreperson, San Francisco startup, or you're in like the Windows Kernel team at Microsoft, if people say the same things, they just use slightly different language, but the same feelings come out in the same hesitations.

The first thing is if you are, and I don't know about this room, if you are the people who are trying to push a change, if you're the directors, the VPs of engineers who are trying to get people to make changes, you might say some of this, or you might be adjacent to your manager who's saying things like this and you're cringing. That's usually the role I play. I'm usually next to the director at Microsoft who's saying something very jargony, and I'm cringing and saying,  "Please don't say that. "

When we talk about changing process, we talk a lot about,  "We're doing this to ship faster, we're doing this to save money, we're doing this to be competitive. " Or my favorite is digital transformation. That's very big among all the enterprises of the world, as far as I can tell. No one knows what it means, and I don't ... not sure that it really means anything. What happens I think is that there are meeting rooms where people are having well-reasoned discussions about trade-offs, and what they're hoping to get, and what they ... why they need to change. They come out of these meeting rooms, and they stand up at all hands, and they use these weird jargon phrases, and everyone's eyes glaze over. They immediately stop listening.

I'm not just going to pick on the corporate jargon folks, because I think things like ship faster too, some people that is like a point of pride to say,  "I went from our build, we used to take 27 minutes and now it takes 24, and yeah, awesome. " That is faster, that's good. But what tends to motivate teams is actually kind of the side effects behind that. It's like what you really want, what really motivates you to change your behaviors, to know that when you push the button, the build is going to go out and you're not going to have to hover over it. When you run that test suite, it's going to pass. Or if it doesn't pass, that means something's broken. It doesn't mean that the test suite is just went crazy today.

Getting rid of those frustrations is generally what's actually motivating people. We care about most about what's in it for me. We care about ourselves and our immediate tribe. When I hear my company's going to be more profitable because of this, I have a very tangential relationship to that, stock ownership or no stock ownership. I don't really care. But I do care if my team is going to get to spend 10% more time working on the cool problems that they like best. I care that my team's not going to get paged because of something that we could have avoided if we had just shipped it behind a feature flag.

These very high level jargony stuff doesn't work very well. If you are a person who is the decision maker, who's the person who's going to be standing at these all hands and delivering things, find out what matters to folks. I would say that the teams around you don't need to agree with your why, and I think that's the death of a lot of San Francisco startups is trying really hard for a complete consensus, which will never happen and shouldn't. But people do need to understand the why.

If you're coming out and you're,  "We need to ship faster, " why is that actually useful? It feels like an obvious question. Of course it should be good. But the reason that shipping faster is better is because we get to try more experiments, or because we get to learn more quickly that the thing we tried works or doesn't work. We get to try something, find out it failed, go to our mentor, ask why, learn how to refactor that, get more of those learning loops in. People need to understand that why, and they want, everyone wants to think that there's something in it for them.

One of the things I find in talking to kind of senior leadership, especially in big companies is they don't actually know what's important to small teams and individual developers. They can't possibly. My manager's always traveling in a bazillion meetings with customers. The amount of time he gets to spend with hands-on code writing developers, it's like this. He's not going to guess correctly. That's where a lot of times I spend time telling him saying,  "Look, it's your job. You do have some one-on-ones. You need to talk to your managers and have them talk to their managers and figure out what's the worst part of these people's lives. For this specific team, what's the worst part of their week? What are they most frustrated about? Your management chain needs to bubble that up to you so that you can reflect it back out. "

I find that when folks try and guess what's the thing that people care about most, they guess wrong and it doesn't land well. Just asking. I feel like that's another thing. If they just have a slide, just ask. I could have that like 20 times in any particular presentation and it would be relevant. If you are the person executing, you're a team lead, you're an engineer, it's on you to kind of speak up about what matters and to let people know when they're not giving you the information they need. I think a lot of times this tends to come out as passive aggressive grumbling, which doesn't lead to better communication even though it does feel very satisfying. I'm a big fan of the, just to be sure what's the problem we're trying to solve? I just want to be clear, and if someone kind of gives you a look like,  "Why are you pushing back? " It's look, we need to know why we're doing this to make sure that we do it correctly. If I don't understand what the end game is, I can't possibly get there.

Bringing forward those blockers then saying,  "Look, this is what's preventing my team from moving fast. This is what's preventing us from doing our best work. How is this solution that you're talking about going to help? " Doing in a very respectful way, but kind of making sure that that conversation happens. What should we see when this starts working? How do we know that it's actually going to?

That kind of segways into the next one, which is a thing that people say when we roll out a product, like a LaunchDarkly or we moved to CICD is we here,  "We're going to be so much faster, we're going to be so much smarter and more productive. " Yeah, eventually. In the short term, you're not going to be, you're going to slow down, it's going to feel terrible. I've never talked to a team that went through a big process change without having this period of terribleness, and it ... we're giving something up. That's why it feels the worst is because we're hearing this is going to be great, and in the short term we're giving up that sense of competence.

I feel like all of the old ways of developing software and in some places at Microsoft, the current ways, sadly, waterfall, and stage gate, and having this person who has to approve, all of that was wrong 40 pages specs, it was all wrong, but it gave us a sense of comfort.

You could come in on Monday and you kind of knew what you were going to be building on Wednesday, and now we don't have that anymore. Even though we've gained something in exchange, people tend to feel bad about this. I think we've all seen the local Maxima graph, right? We tend to kind of gloss over that dip, and I feel like we really need to lean in on that dip, which is we're kind of here. In order to get to the best point, we're going to go through a period of temporary stupidity.

That's okay. I think it's people want to look on the bright side. No, no. Developers like honesty, your product managers like honesty, and it's ... just embrace it. I'm a big fan of saying,  "Look, we're going to be dumb or for the next few weeks. It's going to feel bad, and then we're going to come out of it and be better. " That's actually very comforting to people because what they worry about, if you're ... whether you're a director level person or you're just the person on the engineering team that everyone goes to, if you're just talking about how we're going to be faster and people aren't faster, they're worried about like,  "I'm doing badly at my job. I'm doing badly at my job, I feel like I'm going to do badly at it next week, " and I think you probably haven't planned for this. You were expecting this amount of work and you're going to get this amount, and it's not going to end well for me. Embrace this temporary stupidity, talk about it openly. It helps people feel much better and bring up other things that are bothering them.

Let's put those two together. How can we actually talk about something that we're going to do that's different? These are the three parts in this order that I recommend, which is basically like, here's why the status quo isn't working. Here's why whatever this thing is that we did in the past that read that we're doing today, that you know how to do, that it has a certain level of predictability, here's why it's not going to work. That's important because especially those of you who are in or tangential to enterprises, people will, if they think they can get away with the status quo, they will. They'll keep doing it until someone comes and yells at them.

You have to say,  "This is why this is broken and we can't do it. " Then yes, it's going to be harder in the short term. We're going to be dumber for the next few weeks, in the next few months, and we accept it and we're planning for it. Then here's how this is going to make your job better and easier. That's really where we're talking about, you, the mobile team, that's going to be faster to test. You, the back end team, you're going to rewrite these components and they're going to be faster. It's not about the company is going to make more money, or we're going to be digitally transformed, or we're going to be modern or cloud-native or whatever the term that we all like is.

These are good things to say, and when you say all of these, you're anticipating lot of people's objections. Reduces the grumbling a lot. Let's move to a local level. You're not the person driving the big change, but you're a senior engineer. You're the person that people listen to. You want to help your peers do well and do better. You want your mentees to avoid obvious mistakes, right? That's good. We all want to make different mistakes every week, right? That's the thing that I always say on my team, as long as we're not making the same mistake we're doing well.

There's still some inadvertent things that people say that end up leading to kind of bad situations. It doesn't mean that these are always bad. But I'll tell you a couple stories about things that I've seen multiple times. One of these is someone brings you an idea, your mentee brings you an idea, junior engineer brings you an idea hypothesis and experiment, and you have been around for all, so you immediately have some questions, or you remember the last time the company tried this and it didn't work or at ... in your last job you tried this. Immediately it's well, what about this? Have you thought about this? What about this?

I find that for one thing, a lot of times people are just feeling something out. If you got like a little ping in a slack channel, someone had the glint of an idea and they're thinking,  "I just want to make sure it's not completely terribly unfeasible, and as soon as I get the nod of yeah, that might work, then I'm going to go think about it. " This is incredibly effective and we should encourage this. When you jump on like,  "Have you thought about B yet? " Then the person's like,  "Oh, no I mean I haven't, and now I feel dumb, and now this person that I look up to is kind of not respecting me as much and I feel really bad about this. "

Or maybe you have thought about you know A and B and C, but now I'm thinking like,  "Wow, did he think I wasn't competent enough to think of this? That's kind of crappy. This is not the kind of thing that people complain about. If you're sitting here and thinking no one ever complains about this, this is absolutely on the order of things that people might not even realize that they feel bad about it. It just makes them hesitate a little bit more, and it preserves that kind of power imbalance between senior and junior, or manager and non-manager. If I want everyone around me to work fast, I don't want them to be thinking,  "I want Cindy to approve this idea before I think about it anymore, " because I'm here today. There could be lots of great ideas that my team is executing on right now. In fact, I really hope so. I hope that they're not waiting for me to be back in the office tomorrow because that would be a waste of time.

Instead it's a big fan of kind of,  "What should I be worried about or asking about? Someone brings you an idea and you're like,  "Oh, okay, that could be interesting. If you were me, what should I be worried about? " That really gives kind of the freedom to say,  "Well, I haven't thought yet. I just wanted to check in real quick. " Or,  "I think you'd probably worry about latency, and here's the thinking I did, or here's the googling I'm about to do after we end this conversation. "

It really puts the ownership on that person with the idea. They're like,  "Okay, I need to tell Cindy, I need to think about what Cindy's going to worry about and bring her answers. " That makes you much more confident when you're re presenting that idea. This, by the way, I stole blatantly from a book called Turn the Ship Around!, which is about a navy ... I don't have the great images like Adrian, but great book about this captain who got on a submarine. Everything was disastrous. No one had any ownership, and it kind of walks through some of the specific things he did.

This was one of them. I've tried it on my team, and I've said it and had people literally go through every single rejection I had an answer them, and I'll look, go as you were. Then I was like,  "Wait, wait, no, let's go make it so. " I was like, because I'm not military, we'll go with that, so like,  "Make it so. " Then they go and they do things, and it's amazing.

Relatedly, let's say someone has done something good, they have shown initiative, and you're excited, and you wanted to show how excited you are, and you think,  "Wow, this is so great. They did something good. I want them to do even better next time. " You say,  "Great, oh, next time you should try this, and you should try this, and you should try this. " We think we're being helpful, and we think we're trying to accelerate this person's growth, but what they're perceiving is,  "What you did wasn't good enough and you should listen to me cause I have a lot of ideas about how to make it better. I don't really trust you to figure out the next steps on your own. "

And I have a story from this from just a couple of weeks ago. One of the engineers I work near had figured out something with Azure Pipelines, and he's really pleased about it, and he wrote a blog post, and he published it, and he was sharing it, and I was like,  "This is great. "An engineer in another part of the division was like,  "That's a really excellent blog posts next time. I don't know why you didn't publish it on the Azure DevOps board, and also you should check with marketing because they might want to check and make sure it's okay, and PR can help you. " He just sent this email, it was like a page long.

This poor engineer's like,  "Did I do something wrong? " I said,  "No, no, he's complimenting you. He's just he's trying in a ham-handed way to be helpful. " He's like,  "I don't actually want to write a blog post ever again. " Oh, that's awful. There was nothing in that blog post like,  "I'm sorry, this is Microsoft size company. There's nothing one individual can do to take down the company or even really to create much of a PR nightmare. " There are lots of people doing things, but that's a sound thing. But there's nothing in a blog post that can be harmful. Just tell them good job, stick with that. Or if you have a bunch of advice, if that person had gone to his manager and said,  "Hey, Ethan's blog posts was great, just so you know, here's some extra resources he could use, " that could have been a different conversation too.

But having a manager over here snowball the engineer over here was terrible. What instead?  "Great. I'm curious, what advice would you give to the next person who's going to do this? " This question, the flipping from asking something to what advice would you give, this instead of making people feel dumb about potential mistakes or missed opportunities, it's a way to make them feel smart because now they can help someone else be better. I find if you, any of you who are ever trying to figure out how people are using your products, in my experience, every time I've ever asked an engineer what was challenging about a product, I have never gotten a straight answer. I get like,  "I figured it out. " It was like,  "You shouldn't have to figure it out. You shouldn't have to figure out how to install Visual Studio. It should just work. " They've gotten much better by the way.

But the flip is,  "Okay, if a new person joined your team, and you wanted to give them advice on how to set up whatever tool, what would you tell them? " Then people can talk about all the things that were misleading, or frustrating, or they didn't download this workload, or don't read that blog post, or this tutorials useful. The advise thing really flips us from feeling a little self conscious about admitting that we didn't know something to being in the position of a teacher, which makes us smart. In that position of feeling smart, we're actually a lot better at objectively evaluating our own performance. It's like reverse the Dunning–Kruger effect. That's amazing. It's like giving people a temporary promotion. What advice would you give, really allows people to say,  "Well, nah, now that I think about it, I would like to do this next time. I would tell someone to do this. "

things that didn't work, things that failed, and we all know they do. I know we're supposed to love fail fast, and I've been in the Lean Startup community since its inception, but I would just like to mock this one for a minute because this is terrible. No one actually likes thinking in this way. We've kind of tricked ourselves into accepting it's okay, but it's like that Twilight Zone episode where the kids making terrible things happen to the town, but everyone's terrified of them and say have to kept telling them,  "Oh, it's good that you made it snow. " It's like,  "Oh, it's good that that code I wrote was unusable. It's good that this feature I shipped failed and no one's going to see it. " We can pretend, but it's really kind of awful.

Yes, fail fast. Also here's how we're smarter than we were before because everything that didn't work is a thing like Thomas Edison, I now know 10,000 things that will not make a light bulb. When you're talking about things like,  "Here's how we're smarter than we were before, here's what we know that we didn't know before, here's what we'll try next. " That has a nice little arc of feeling like it wasn't that your time was wasted, we learned something, and we're going to take that learning and directly channel it into another cool thing. I have a story here from myself actually is I'm running a relatively new team, and we were working in a particular way and about six weeks into kind of working with our first internal team, we're like,  "This is not working at all. We're going to need a pivot. We're going to need to pivot entirely away from what I wrote, our whole little strategy paper on what we're going to do. "

I just faced, it was really demoralizing because I feel like everyone had worked really hard. For six weeks we'd work with these other customers. We kind of promise things, and then we're like,  "We're not going to be able to do this after all. " Everyone just felt bad. Even though I told my team,  "I am proud of you, we learned a lot. You worked hard. There's nothing you could've done differently, " and I wrote this all down for a little internal doc. I wrote a, here's how we're smarter than we were when we started, and this was for internal use. I was not in particularly embellishing, trying to show off. I just literally wanted to write stuff down.

One of the women on my team read it and she's like,  "I feel so much better. I know you said we shouldn't feel bad, but I have been feeling bad, and I've been thinking we wasted so much time, and I've been wondering what I could've done differently, and now I read this, I'm like, oh right. We did learn a lot of things. Reading this, I realized it would have been terrible to keep going down that same path, and if I really needed to see in-print a long page of things about what we learned in order to feel good about it. I think sometimes we feel like we, maybe we say it once or twice and we really need to talk way, way more about the things that we do well.

I've given you some things to say. How do you figure out the rest of these, because there'll be a lot of situations? Now, how do you run diagnostics on the things that you're saying? The first most important thing that I always tell people is read it out loud. Whether it's something you're going to say in a meeting, it's just like your quick notes, or it's an email, if you couldn't read it out loud and feel okay about saying it to someone, it's probably good. If you kind of sound like a corporate tool, then you sound like a corporate tool. If you sound like you're making excuses or being defensive, that's exactly how it's going to come off to other people.

I spend a lot of time looking like who knows kind of person motoring emails says,  "Well... " But it works incredibly well. Every time I have skipped it, I have walked into a meeting or a standup and said something, and then I'm looking around the room like,  "I should've read that out loud before I came in here. "

The second is, are there questions? The presence of questions. Engineers, product managers are the most question-asking people on the planet. If you have delivered some pronouncement and there are no questions, then there was something wrong with the delivery, then people are skeptical, or they don't think it's going to work, or they think it's a terrible idea, or they don't trust you. I can't tell you what the thing is, but if there are no questions, that's a bad sign. If I ever am in a meeting, even if it's not my meeting where there are no questions, I will go find people one-on-one and be like,  "What'd you think about that? What are you worried about? You didn't say anything. I know someone should have said something. What are the things that people are having back channel conversations about? " It's just a good indicator canary in the coal mine.

Relatedly, what I call the scowl to comment ratio. If you're looking around, you see people kinda making this face... and they don't say anything, that also means the words are ... the words have not landed right. Because there are some people who won't speak up, but in your group of scowling engineers, product managers, at least one of them is going to speak up if they feel like they understood enough about what you were saying to criticize it. I think when things are flying really high level, that's when you get a very high scowl to comment ratio. Yeah.

Finally just ask. Again, I said I could have 12 sides of that. Just ask,  "How was that? What do you think? What do you not know? I know I missed something, what is it now? " On my team every Friday we send a two things you're still worried about, email. It's the most valuable thing I do. Just our little small team, and they can send them to me or to the whole team. But there's always these things that they're just under the radar. People wouldn't say to them until one-on-ones, but just by asking like,  "What are the two things you're worried about? " There's always something. Every week there is something, and it's often not a big thing, but that's even better because you can fix it before it becomes a big thing.

Again, if you missed the slide information, you can get it, you can follow me on Twitter. I have a book of primarily around talking to customers, but everything in that book works just as well on your internal teams, and I do it as well all the time. I also answer questions. If you have questions that come up, send them to me, and I can answer them.

I think we're over time, so I don't know if I have time for questions from the audience. One question. High bar. Yeah. Sure. How do you get actionable feedback that's not ragey or not vague? One of these is smaller groups. If it's a particularly angry team, I'll do one-on-ones and pick ... and you'll do several of them because it's better to do six one-on-ones than to do a one-on-one than to do a group meeting with six people where no one says anything. He'll do six one-on-ones. Most people are not that ragey face to face because it's much harder to do so. It just is. Then once you've done them, I tend to come back and say,  "I talked to a bunch of you individually, these are the commonalities between what I heard. "

If you have people on the other side that are too polite, which I recognize might be rare, but there are different countries and cultures, then I'll usually say something like ...

This might be rare, but there are different countries and cultures. Then I'll usually say something like Look, there's got to be something. What is... if you had to pick one thing that was the worst thing, what would it be? And that works very well for the polite end of the spectrum as well, because then you flipped it so that they feel like they're being unhelpful if they don't tell you something. All right, thank you.