Moving Faster Without Fear: Customer Panel From Software Release Summit

The bad news: you missed our recent Software Release Summit. The good news: you can catch up on all the great talks and panels here. 

One of those insightful sessions was our customer panel, Moving Faster Without Fear. Meet our guests:

• Krista Sappington VP, IT Application Manager at BOK Financial, a top 25 US-based bank supporting customers in the consumer commercial wealth space 
• Aaron Kerwin, Director, Software Engineering at Datasite, a global SaaS provider serving customers in investment banking, private equity firms, and larger corporations  

They were hosted by Alea Abed, Director of Product Marketing at LaunchDarkly. What's great about this discussion though is that Alea wasn't the only one asking questions—Krista and Aaron had some really insightful questions for each other too. We've got highlights for you below, and when you have a chance, it's really worth checking out the full discussion.  

Key business initiatives and concerns in 2023

For Krista's team at BOK Financial, a top priority is investing in modernizing their wealth platform application and its architecture. They're also continuing to invest in building out more robust self-service digital card capabilities. 

"What that means is our customers will be able to activate their debit card in their mobile app, change their pin, set their pin, and then add those bank cards to their Apple Wallet or Google Pay or Samsung Pay," she said. Strengthening their security posture and effectively managing risk is also on the docket.

Delivering software at a high speed, with high velocity, and low risk is top of mind for Aaron's team at Datasite. "We have typically about a hundred changes we like to put in production a quarter," he explained. As the quality assurance group, they're focused on what they can do to support that velocity for all business initiatives from a SDLC process and tooling perspective.

Both customers are looking to mature their software release practice by tracking metrics that help them understand the impact of their deployments.

Using feature management as a safety net for innovation

With BOK being a financial services firm, Krista shared how they think about balancing risk with innovation. "We really have to understand what we're trying to accomplish and what we're going to build, because that's going to set the tone for the risk tolerance and considerations when we're building and architecting a solution," she said. 

In the case of a high risk change or high complexity in the solution, Krista explained that they work with stakeholders, their Scrum team, development team, and operations to understand the environment. That way they know what controls they have and what levers they can pull to either shut things down or redirect user experiences in the event of something going wrong in production.

According to Aaron, the teams at Datasite "will actually use LaunchDarkly anytime they feel that they want to manage risk." If production is down or a key user experience flow is impacted, Aaron explained, they don't want their customers to be impacted. Equally, as part of an on-call rotation, no one wants to be woken up at 3 a.m. 

Now, Aaron described, when engineers are planning to release something, it's practice to talk about risk mitigation. "'What's your deployment strategy going to look like? What's your rollback strategy going to look like?' And if the engineer goes, 'Yeah, we've got it behind a feature flag,' then everyone's like, 'Okay, no worries!'"

Streamlining releases to reduce stress and prioritize speed

Aaron shared a recent success from Datasite: the release of their first website and application refresh in over four years. "We wanted it to be low risk. We didn't want to impact our customers. We did want to have high adoption." 

In addition, they had to keep the old and new look for the product application running in parallel (to enable customers to revert to the old design until they were ready to transition), all of which adds up to what could have been an extremely complex and stressful release. They completed it successfully (with an adoption rate of 85-90%) with a progressive delivery strategy. The rollout went all the way from internal beta release, to internal GA, to a subset of customers, then eventually to all its customers, all using a global feature flag in LaunchDarkly. 

Aaron also gave us some insight into how controlling access to new features and different experiences worked before they used LaunchDarkly: a cumbersome process involving supplying different URLs to customers and asking them to update their bookmarks with a new URL to get an update. Datasite needed a scalable solution for providing targeted experiences for their customers, but had to weigh up the effort involved in standing up and maintaining that solution themselves. "You could totally build a fully fledged solution around feature management and release management," he said. "But then there's the competing factor that then you have to allocate people to that and manage it and make it available. And if you rely on that to reduce your risk, now the uptime for this new feature management solution has to be really high."

"If something goes wrong, you could impact your experience that you were actually trying to de-risk," Aaron explained, getting right to the crux of the Build vs. Buy dilemma (Bingo!). 

Unlocking more release strategies

When asked how the use of LaunchDarkly has impacted developer experience at BOK, Krista said, "It's helped us be more creative in terms of how we want to go about architecting or even designing a solution and how we might want to release it."  

Krista described how when BOK has a particular feature to release, with LaunchDarkly in the mix, they can now consider who really needs to do the heavy lifting vs letting LaunchDarkly do the work. 

Advice for leaders looking to adopt feature management

You need to understand your current release and deployment processes, said Krista. "Do you understand what your team's pain points are, what opportunities there are? Because LaunchDarkly could help solve some of those problems … depending on your situation and needs—if you need control and flexibility and a safety net—this product is really helpful in producing that."

Most of Datasite's problems were around the engineering deployment and release process. Their front end, back end, and feature flags were disconnected, making it hard to deploy to production safely. "It was kind of hidden because the engineers in QA and SREs would just fight through it and make it all work, and everyone outside of engineering just saw the end result," Aaron said. Even if the end result was satisfactory, that disconnect between front and back ends and feature flags amounts to a lot of wasted time and frustration for engineers, so it's worth digging into those pain points to identify where a feature management solution can help.

To bring this discussion full circle back to metrics, Aaron suggested, "What if you could measure it? What if I could go back and say, 'How many errors are we having in production? And what are those outage times in tracking that?' And then start to make improvements to your release process, and be able to show the value as you make those changes. I think that would be really powerful as well."

Of course there's always more than we have space to capture here, so do watch the whole discussion when you have the time. You can also catch up on all the Software Release Summit talks to hear more about the trends and technologies shaping the software development and delivery landscape, as well as get insights and best practices from leaders in the industry.