Tools for the Next Iteration of Modern Development

Adam Zimman: All right, thank you so much, Hattie. All right, folks. So first up, thank you so much for joining us here today at Trajectory. Hopefully you had some great experiences, had some great conversations, seen some great content, and looking forward to having some wonderful panelists here today. The prop that I gave them for this panel in general was really trying to have a conversation around tools for kind of the next iteration of modern development.

And really kind of ... You've heard some of this this morning with Adrian's conversation and talk, where he was talking about some of the motivations that were driving changes, and so in that vein I wanted to welcome individuals that are very active with customers, as well as our ecosystem and community, to give their impression on what's driving some of these changes, and how they and their companies are participating in those changes.

Before I get started I'm going to have each one of the panelist introduce themselves. Tell you which company they're with, as well as a little bit about what their technology focus is on. Preeti.

Preeti Somal: Hi, everyone. I'm Preeti Somal and I am VP Engineering at HashiCorp. HashiCorp has a bunch of open-source and enterprise tooling, and we'll talk more about that as we sort of go through some of the questions. I've been there about a year and a half now. Prior to that I spent four years at Yahoo and I was responsible for running the Yahoo private cloud set of services, so tons of learning and tons of empathy for customers that are trying to move through a transformation at this point in time.

Prior to that I spent a bunch of time at VMware and Oracle as well. My work has been mostly in and around the infrastructure systems and applications management space. My focus at HashiCorp is really how do we scale the products that we have. We have seen a lot of traction with both our open-source and enterprise products, and with that, associated growth, in terms of the team sizes and a huge amount of responsibility around making sure that our products hold up to the standards that our customers expect.

My focus is mainly around getting all of these pieces to work well together so that we have happy customers.

Adam Zimman: Excellent. I love happy customers.

Preeti Somal: Yeah.

Adam Zimman: Kolton.

Kolton Andrus: Thank you. My name's Kolton Andrus, I'm CEO and co-founder of Gremlin. My story kind of starts 10 years ago, I worked for Amazon. I was in charge of making sure the website didn't go down, and when Amazon goes down they lose a lot of money, hundreds of thousands, millions a minute. It's funny, as everyone jokes about they want to be like Amazon. They want to be like Netflix, they're doing microservices, they're going after the latest trends.

One of my favorite jokes internally is, "Well, you're going to inherit their problems." And so the benefit of growing up at Amazon 10/15 years ago, and then afterwards joining Netflix and seeing what it takes for a high performing organization to maintain that velocity and to be able to really be successful. At both places I was focused on operations, on keeping the website for Amazon, and helping keep Netflix up and available.

Part of that was a collator, so I sat in on every incident for eight year. I kind of miss getting paged at 2:00 in the morning, as counter-intuitive as that sounds. I guess I'm an adrenalin junkie. I like to be the hero that fixes everything. Then I had the opportunity to build chaos engineering tooling. I built some tooling for Amazon that was used by hundreds of teams that Dynamo used.

Then when I joined Netflix, we know Netflix is the home of Chaos Monkey, but there was a lot of opportunity to contribute and innovate in that space, and so I had the opportunity to build some platforms there and help them push that practice forward. Now, at Gremlin we're about making chaos engineering simple, safe, and secure, so that everyone can do it, build more resilient systems.

Adam Zimman: Excellent. Thank you, and Rachel.

Rachel Stephens: Hi, everyone. Rachel Stephens. I am with RedMonk. We are an industry analyst firm, and for those maybe less familiar with it, it's kind of looking at the broader technological landscape and understanding adoption trends. And trying, for us in particular, focusing on user-led adoption trends, so understanding developer preferences. Understanding, in particular, cloud and the open-source ecosystem and how those can enable developers to build.

And so, yeah, we do a lot of research. We talk with customers, and we talk with companies that build software as well.

Adam Zimman: Awesome. Thank you all so much. The first question that I wanted to get your commentary on was what is driving the changes in the way that the industry is looking to build software? And the kind of three axes that we'd love to get your feelings on are the engineering challenges, the business drivers, and then also how much does culture play a role in the changes that we're starting to see and hear about? Rachel, would you like to start?

Rachel Stephens: Sure. One of the commonalities that we've seen when we talk to businesses that say they want to do digital transformation, is universally pretty much what they mean is that they want to move faster. They're looking for velocity. They're looking for a faster time to deliver value, and so a lot of the times that's what is the ultimate driver, and it has a lot of different challenges to achieve that velocity. I think we've seen a lot of changes in the technological space. We have a lot of fragmentation around tooling.

So if you think back to like a 3-tier app, and the different choices that you made there, versus all of the various choices that you can make as you start to put together microservices. It's a lot to navigate and choose from. I think the other thing that we're seeing is that there is an evolution of abstraction layers, so I mean you can containers and serverless.

If you want to keep going into new and more experimental things, you've got WebAssembly and isolates, and microVM, so how are people supposed to start to navigate those challenges of choosing the correct architecture? I think it is something that the industry, overall, is struggling with, but the key goal for everyone, as in navigate all of these challenges, is to figure out how to put these things together in a way where they can move fast.

Adam Zimman: Great. Preeti.

Preeti Somal: Yeah. I agree with that. The main business driver, all the time, in talking with customers is how can they move fast, and how do they do that in a responsible way, where security is not compromised, their policies of best practices are maintained, et cetera? And so what's really happened is that we're seeing enterprise customers and tech companies shifting from what used to be like a hardware delivered sort of custom made-to-order, data center contained, ready hardware security, and traffic balancing type application, to, now, this application that could be running everywhere.

Multicloud is a reality, most customers we talk to are going with at least two cloud providers, and so how do you take what was this fairly static world, that now runs across multiple clouds, is composed of microservices? You still need to secure it, and you still need to be able to control the rate of change that goes out where you're getting that velocity of delivery, but you're doing it in a responsible way.

We're seeing that sort of driving that fundamental need to deliver faster, to move faster.

Adam Zimman: Right, so the fundamental need being the business driver of moving faster, and then the change in technology kind of falling from that in some regards.

Preeti Somal: Right. Right.

Adam Zimman: Yeah. Kolton, and I'm interested, when we were talking earlier about this question one of the things that you brought up that was really interesting and intriguing to me was some of the cultural sides you brought first. Could you speak a little bit to that?

Kolton Andrus: Yeah, I think speed of innovation is key, and the one bit I'll add there ... they both did an excellent job speaking to that ... is just think about how customer expectations have changed. I think about when I was younger and you shipped a CD to someone. Maybe installing the software on that CD took an hour. Maybe you had to get on dial-up and download what it came from. Okay, maybe that's too long ago, but let's take 10 years ago.

10 years ago we would wait 10 seconds for a webpage to load. Do you know how many people wait a second for a webpage to load anymore?

Adam Zimman: Right.

Kolton Andrus: So, we as customers and people that have expectations, expect a lot more. We expect things to be always on. On my team I joke that a maintenance window is a dirty word, like that just grates off, or it doesn't have maintenance windows anymore. It's always on. So how do we do that? Well, in part it's been about the shift of allowing engineers to drive more of the show. It's less about this top-down approach of we're going to do the most efficient thing, what we think is the most efficient thing, for the business.

A key aspect of that is the silo gardens, "Oh, we've got this QA team, we're going to throw some stuff over the fence, they'll figure it out. Maybe they hate us, maybe they don't. They get paged, not our problem." That's not the world we live in today, now we live in a world where engineers get very much more say, and we have that culture alignment of instead of silo and instead of throw it over the fence, instead of, "That's not my problem," it's very much a world of, "I build it. I own it. I operate it."

One thing that we always touted about that at Amazon is that if you make engineers feel the pain for a little while, they're going to make it better. If you get paged every night because something stupid, arguably, is happening, something maybe minuscule or not critical, then you're going to fix it, just so you can get your peace of mind. I think that alignment of incentives, that alignment of pain, is a healthy one.

Adam Zimman: Yeah, I think you put ... The quote that caught me was you said it's the DevOps mentality to some degree, or associated with DevOps often, of, "Stop saying whose job it is," and start saying, "What needs to be done?"

Kolton Andrus: Yeah.

Adam Zimman: Right, and it really changes your perspective on the tools that you use, when you stop thinking about who's the one that is in charge of it and instead, how are we just going to get the problem solved?

Preeti Somal: If I may.

Adam Zimman: Yeah.

Preeti Somal: In a lot of places, what customers are sort of really moving from is a point where they were shipping software once a quarter, or once in six months, and how do you get from that once a quarter to continuous delivery?

Adam Zimman: Right.

Preeti Somal: You can't do that if you're not initiating culture change as well, right?

Kolton Andrus: 100%, yeah.

Rachel Stephens: I think a big part of culture change is understanding what you're going to do when things go wrong, up front, so like are you going to have a just culture? How are you going to start implementing post-mortems? How are you going to handle systemic failures? Are you going to blame people? I think that figuring out that piece of your culture is an important part of this transformation.

Adam Zimman: Yeah. And- Kolton Andrus: You were tempting me there. So, yeah, its look, "Ooh, yes, yes." Sorry.

Adam Zimman: No, you're perfectly fine. You can jump up and down if you'd like to. Yeah, I think that along those lines, one of the things that I've also noticed in speaking with customers is that we're seeing, as software has taken a larger role for so many organizations that are not tech companies, that are not ... that is not what they do as their primary deliverable, their mission. Software is not only becoming a necessity for their organizations, but it's also starting to flow into other teams, beyond just IT.

That is driving some of these cultural changes as well. So it's really interesting. Moving to the next aspect of how wanted to give the audience some ideas on how you all are thinking about these challenges and the solutions and the opportunities, looking at the ecosystem for Preeti and Kolton of tooling that is complimentary to the technologies that your teams work on. How do you think about those ecosystems, either specific vendors or parts of the stack that you work with to make sure that the whole kind of end-to-end workflow that your customers need to accomplish is supported?

And then for Rachel, from an analyst perspective, we'd love to hear your perspective from the companies that you've spoken with. How do they look at these workflows? What are the ones that they're actually prioritizing? So Preeti, Kolton, would you like ... P

reeti Somal: I can start. So, yeah, at HashiCorp we have a very, very rich ecosystem. If you think about what we're trying to do, is our focus and sort of a principle that we have internally is workflows not technologies, right?

Adam Zimman: Mm-hmm (affirmative).

Preeti Somal: So, for us, our goal is to make sure that we can deliver a consistent workflow to our customers, as they're looking at a multicloud world where applications are dynamic. We sort of solve that problem for four main layers of the stack. At the foundation is security, what happens with secrets management in a dynamic world. You've got an app that's running in GCP, AWS, or maybe Azure. Each of these have their own kind of distinct identity and access management system, how do you rationalize that?

And so we've got a product there. Clearly, if you think about that product, the ecosystem around it is all the various cloud providers, as well as the sort of access and identity mechanism. Even going back to active directory, right. So there's a very, very rich ecosystem around the product that we have. We also have a product that allows DevOps folks to provision infrastructure. It's a product called Terraform, and it's very rapidly becoming a standard.

That, by the definition of it, has to be able to provision lots of different things. We've got an ecosystem of over 200 different vendors that we integrate with. For us it's really about, how do we provide that consistent workflow and how do we make sure that we are building the ecosystem and partnership so that our customers don't have to go to different vendors for various pieces of that puzzle.

The other piece I'll add is when you think about velocity and agility, we are solving a part of that problem and that part is around the services that applications need, but we are relying on ecosystem partners, including LaunchDarkly, for instance, to be able to solve that problem at that software developer layer. So we, ourselves, HashiCorp, is a customer of LaunchDarkly. We use feature flagging for our Terraform SaaS offering.

If you think about it, it's sort of layers, and each layer is building that agility. And with LaunchDarkly what we're doing is really iterating really quickly on our SaaS tier. So, yeah, our ecosystem is immense, and our approach to ecosystem, given our open-source routes, is really to allow that open-source community to blossom around the core extension points that we have.

Adam Zimman: Okay.

Kolton Andrus: So workflow's an interesting one, I think you hit on a lot of what we've seen in this transformation. We used to shift software quarterly or annually, and we could get all our bug fixes together, we could do a bit QA pass and send it out the door, then take a break, not have to worry about it. Instead, what we have now is, as you mentioned, basically every business has become a technology business. Show me a business whose revenue doesn't rely on being online, and I'll show you one whose will in the next five or ten years. I could almost promise that.

There was a story that was shared with me recently, it was a tax calculation service by a pretty well-known company. They could have all the rules and everything put together, and they could ship it, and someone could run it on site. Well, they move to that being a cloud service offering. Well, now they have to have it up, it has to be available all the time. They've got to make changes. They've got to be able to, LaunchDarkly, they got to be able to have the feature flags, and make sure the things work correctly.

For us, that's been a huge learning about how people operate systems. One of my favorite jokes, which is not really a joke, is the on call training I've received at every company has amounted to, "Here's your pager, good luck." Maybe there's a run book over there, maybe it's up-to-date, maybe it's not.

And truthfully, that's the world we live in, now how do we get to a better world?

Well, imagine the first time you're on call somebody says, "Hey, you know what? We're going to run a fire drill. We're going to help you test that you've been on call. We're going to help you test that you get paged, that you know where your dashboards are. We're going to do this during the day, where we can have a discussion about what our run books look like. What is our process? Who should we escalate? How do we handle these things?"

I think on the workflow part, that's a very important piece of it. As we focus on resilience and reliability of systems, there's a lot of pieces in that workflow. Engagements a key piece. When something breaks, does the right person get notified? And if that person's unavailable, does it get escalated well? Monitoring and observability tracing, I won't get into that holy war, but however you bucket that together, the insuring that it's working correctly.

We sometimes take for granted, "Oh, of course we have monitoring." We can answer all these questions, and the truth is I see a lot of gaps in monitoring, or people that haven't set things up correctly, and so they thing they're protected. They think they'll get notified if things go wrong, and truthfully they may not. Or they may not until it's too late. Then you get into incident management and how do you coordinate with 20/30 engineers while you're fixing something on the fly? How do you triage it affectively?

How do you track that work afterwards? How do you do post-mortems or blameless post-mortems, or incident reviews? However you do that, that process is valuable. We have to learn from it. We have to get better. And so I see all of those pieces. You see a lot of software companies that are known in our space, because those are things that enable engineers to answer those questions, to better understand their systems, and ultimately to innovate faster.

Adam Zimman: Awesome. And so given these responses, Rachel, maybe you could add some colors to what you're hearing from some of the customers that you're talking with on how do they think about approaching some of these problems or solutions of changing their tooling? Are there tools or parts of the workflow that they lump together, that you've seen from a pattern perspective?

Rachel Stephens: Yeah. I think one of kind of the RedMonk overarching thesis is that developers are extraordinarily influential in what tooling actually gets adopted within an enterprise. And so if developers like to use your tool, in a world where you have open-source options and cloud options, it's very likely that if it is a preferred tool for developers, and they can get their hands on it, that tool is an organization somewhere, regardless of whether or not someone gave them permission to use it.

I think that's just a reality of working today, so you need to understand that developer choices are probably going to drive how systems are put together. I think we're starting to see tool sets emerge in various ... I think the CNCF has put together some gravity around some of their projects, where people are starting to place those together in consistent ways. I think there's probably a long ways to go for some of those projects, but I think that's one area we're seeing.

I'm thinking the progressive delivery is an area where we're starting to see routing, and feature flagging, and observability, and like automation all kind of starting to come together as the tool suite. I think that there are, in this space in particular, some emerging patterns, but I think that the pattern that RedMonk would bet on most is to bet on what developers like.Adam Zimman: I think that that's probably a pretty consistent view with this audience. I'm guess. Yeah. Yeah, you mentioned progressive delivery. This is something that has come up in a number of talks today, and something that I've been talking a lot about and thinking a lot about is, how do we start to look at some of the changes that are taking place in our technology stacks to be able to provide more flexibility for the teams that are supporting them, the teams that are running them, the teams that are building them from the infrastructure layer?

Giving them greater control points in their code base to be able to actually impart change without having to go through large deploy processes or things like that, and be able to have more safety in their system, ultimately. So, interesting.

The next question that kind of had talked to you all about and would love to get each one of your takes on this, is given these transformations that are taking place, what is the one thing, or what's a good story, that keeps you motivated, or helps keep your team motivated? The things that you are concerned about? Or the challenges that you're most aggressively looking to solve, because you are concerned about the outcomes if you don't solve them? Makes sense?

Kolton Andrus: No. You want to rephrase that for me?

Adam Zimman: I will rephrase that for you, sorry. Yeah, no, I think that what I'm trying to get at is, what are the things that you see as the reasons, Kolton, for Gremlin? What are the reasons that you started Gremlin? You have obviously a vision for things that you've done in the past, but I'm guessing that there's a motivation there around what is driving you to go and talk with companies and convince them this is a good idea.

Obviously, yes, it would be great to see Gremlin be successful, but I'm guessing there's a motivation there around also enabling the success of those customers, because there's something you see, that if they don't do this, whether it's Gremlin or somebody else, if they don't do this their company is at risk.

Kolton Andrus: That's fair, so you want me to go first?

Adam Zimman: Yeah, go right ahead.

Kolton Andrus: Yeah, in particular, our mission at Gremlin is to build a more resilient Internet, and obviously that dovetails well with what we're doing. But if we think about, again, how much we as a society rely on technology today. My wife's been closely following the Boeing Max 737 debacle. Part software, part hardware, part social drift, part business economics, there's a lot of things at play. And it's been an interesting one for us to debate, because I travel a lot and she's concerned for my safety.

She doesn't want me on one of those planes. Well, that's how a lot of people feel, and when we thing about Wells Fargo has outage, and people can't pull money of an ATM, or they accidentally foreclose on homes because of a bug in their software. We think about our transportation. We think about our government. It's tax day, last year for tax day they had to extend the deadline due to a software failure.

These are increasingly important systems. So that's one part, that's why it's important. That's why we do it. The second part is our systems are just becoming harder, they're more complex. I show these microserver's Death Star pictures of what Amazon looked like 10 years ago, and it's like a ball of yarn. Actually, it looks like, if you zoom out far enough, it looks like the galaxy. There's just so many interconnected pieces, that the number of things that can happen and the number of things that can go wrong are astronomical.

Literally testing that everything couldn't go wrong, the sun would burn out before we completed it. And so we're in this Double Jeopardy mode of it's even more important, and it's gotten much harder for us to do. And so that, trying to solve that, trying to make the world a better place so that when technology needs it, so when we need to catch a flight to visit a loved one, or when we're at a hospital, or we're filing our taxes, or we're putting in a home loan, that things work, and they work well.

Adam Zimman: Yeah, absolutely. That was exactly what I was looking for.

Kolton Andrus: You rephrased it well.

Adam Zimman: Excellent.

Kolton Andrus: Thank you for dumbing it down for me.

Adam Zimman: Not at all. Preeti?

Preeti Somal: I think Kolton really summarized it really, really well. Clearly the implications of issues is really, really high. Right, and what really motivates us is what we're trying to do is help customers get there. It's amazing that despite sort of AWS having been around for years, and cloud being something that people have been talking about for years, the customer maturity around how they actually use cloud in the cloud operational model, in the sense of it being dynamic, is really lacking.

Right, so time and again you meet customers where they've tried a lift and shift and it fails horribly, and it's too expensive, and they can't keep it secure, and, and, and, right. So for us it's really about, how do we help with key pieces of that puzzle and help customers really think about this more as an operating model. In fact, a lot of times when we're working with enterprise customers, what we're talking to them about is start with bringing that model in your existing applications and existing data centers, before you even go out.

Right, because it's that culture. It's that way of thinking. It's that way of operating. And so really helping that be successful is what we're trying to do, right, and one of my favorite lines when I talk with customers is, the power of open-source and developers. What Rachel was saying, developers are making decisions, 10 out of 10 times I'll be in a customer meeting and they will say to me that, "Hey, I'm already using your products."

Recently I met somebody who's a VP of technology at one of the tech companies, and before that meeting he was doing an analysis of which HashiCorp products they're using. They weren't even aware of the fact that they were using one of our products. We have a product called Consul, which is around service discovery. They did not know that product was being used in tens of thousands of servers that they're running, because developers, because open-source, because it was really to use and compelling, right.

When you hear stories like that, it's really, really fulfilling, because it means that the software that you're building is getting used at scale, thanks to open-source and thanks to this community. So, yeah, despite all the sort of tough aspects of being in software, those are the pieces that are really, really fulfilling.

Adam Zimman: Awesome.

Kolton Andrus: If I may, one thing you said in there really resonates with me. One thing we learned, I built tooling for internal use at Amazon and Netflix, before building Gremlin, and you have to build good software or people won't use it.

Preeti Somal: Right.

Kolton Andrus: And you have to make it easy, so the line that we say over and over again is, engineers want to do the right thing, but you need to make it easy to do the right thing. If you do that, you can help people avoid some of the pitfalls and save some of that pain, and get to a better place, and so that's one way we shaping our software and helping people do the right things up front and easily can aid in that.

Adam Zimman: Yeah, absolutely. Rachel?

Rachel Stephens: Yeah, so I obviously come at this from a slightly different perspective. I think one of the things I think is interesting kind of watching the landscape, is people's motivators for change can be very different. I think you kind of think of it like internal versus external locus of control, if you want to go super philosophical. But if you think about I want to change because I'm excited about the opportunities to do better, and I'm excited to pursue something new, and I want to drive these changes, versus I have to change because the industry around me is compelling me to.

And I am afraid, and it is all based on kind of this gloom and doom picture of I have to adapt or die. I think it maybe doesn't make a huge difference on the large level around what maybe you pick as your architecture or something like that, but I think those locus of control can actually have a very large outcome on the actual success of your transformation and your change, in terms of cultural buy-in, and in terms of your teams enthusiasm to pursue the project in terms of what you actually end up building and achieving.

I think one of the things that I am excited to see, is how people are embracing digital transformation, not in a place of fear, but in a place of like, "I can do some really cool stuff." And so that's what I'm excited about.

Adam Zimman: Yeah, no, I mean I think that that's ultimately the mind shift. I think Cindy spoke a bit about this this morning, and kind of the language that we use, especially in internal positioning, right? But I think that the context is the same. You make a great point. If you can, as a company, start to position the technology as a way for you to create new opportunity for your organization and your company, as apposed to this kind of requirement or crutch that you need to lean on so that your company doesn't go away.

It changes the tone of what people are willing to do internally, and the risks they're willing to take to be able to experiment and try new things with technology.

Rachel Stephens: I think one of the things that's really cool about progressive delivery as a concept, is that it creates space for new alignment between IT, and the engineering teams, and their business partners, so that you can work together to deliver business outcomes in a way that is meaningful for everyone. That's one of the trends that I also really like.

Adam Zimman: Yeah.

Kolton Andrus: I hear telling engineers what they have to do works out well as well.

Adam Zimman: Yeah.

Kolton Andrus: It's like that's a recipe for success.

Adam Zimman: Yeah. In telling them exactly how they should do it and-

Preeti Somal: What tools to use.

Adam Zimman: ... what tools to use.

Kolton Andrus: Yeah. Yeah.

Adam Zimman: That's it. Yeah. Yeah, no, I think that one of the things that we've seen in talking with a lot of LaunchDarkly customers is that giving engineers the flexibility to be able to choose the tools that they want, but then make it really easy for them to adopt those, had definitely driven them to use tools in a way that even the tool creators never conceived. Michael gave a great talk this morning, from IBM, talking about using feature flags basically as a service match.

This is something that when LaunchDarkly was being started, was never a use case that we foresaw, but giving the individuals the flexibility to experiment and try new things, and making the architecture open to be able to use it in new ways has led to some really amazing things. Awesome.

Yeah, I mean I really appreciate all of you coming and joining me for the panel today, and having a little conversation about technology, and, yeah, thank you very much. Appreciate it.

Preeti Somal: Thank you. Thank you.

Kolton Andrus: Thank you.

Adam Zimman: All right.

Rachel Stephens: Thank you.