In digital banking, staying ahead of the curve is challenging but necessary. Ally Financial—the largest all-digital FDIC-insured bank in the United States—recognized this need, and built an ambitious plan to modernize their software release process. Nathan Gray, Ally’s Director of Digital Engineering Operations, and Jeremy Cox, the company’s Director Lead of Software Engineering, spoke at the LaunchDarkly Galaxy Conference in 2024 about how they collaborated with LaunchDarkly to build faster, safer software releases.
Ally's innovative growth
To truly understand Ally’s trajectory, let’s take a quick look at its history.
Ally Financial was founded in 1919 as General Motors Acceptance Corporation (GMAC). Its initial purpose was groundbreaking for its time: to provide auto financing options to help General Motors compete with Ford's growing empire. For decades, GMAC played a key role in the automotive industry, expanding its services and weathering the ups and downs of the American economy.
Ally’s recent transformation began about 15 years ago when the company's leadership saw an opportunity to leverage their strong position in auto financing to create something new: one of the world's first all-digital banks. This bold move transitioned Ally into the organization it is today. The transition wasn't just a name change; it represented a fundamental shift in how banking services could be delivered, moving from traditional to digital.
Ally now stands as a testament to making a successful digital shift. With over 11 million customers, nearly $200 billion in assets, and a comprehensive suite of financial products including deposits, investments, mortgage, and credit cards, Ally is a powerhouse in fintech.
Within this context, Ally took on its next big challenge: modernizing releases in order to stay competitive in the market.
Balancing speed and safety
As Ally’s digital services evolved, the need to deliver change more quickly and safely while allowing teams to work independently (and at their own pace) became clear.
One major issue was the structure of applications. "We had some cumbersome monoliths," Cox noted. "When you release a large application that has lots of changes from lots of teams, it requires a lot of coordination, a lot of planning, a lot of work. It's really difficult to do." This monolithic architecture was a significant bottleneck for Ally’s engineers, making it challenging to implement changes quickly and safely.
Another major pain point was the reliance on monthly release outages to ship releases.
"We had monthly release outages. Not outages because something went wrong necessarily, but we had to plan on taking our applications down so that we could release software." ~ Jeremy Cox, Director Lead of Software Engineering, Ally Financial
This approach was not only disruptive to customers, but it put additional pressure on development teams.
Problems extended beyond just the size of applications. Ally was working with uncoordinated tooling across their web and mobile platforms. Before 2020, they used different feature flagging solutions for their web and mobile applications—making it impossible to turn on a feature for all customers at once. Some of these solutions only worked at build time but not at runtime, leading to events like customers having access to a feature on the web app but having to wait weeks for it to appear on mobile devices.
Furthermore, teams often relied on individual "heroes" who held critical knowledge about their systems. As Cox put it, "There's [going] to be that one person who knows how everything fits together, or that one person who knows the secret of why this one thing is breaking everything… and that's not sustainable."
Ally worked on addressing these challenges, but solutions were still fragmented across different platforms and didn't provide the necessary flexibility and control. The team knew they needed a more comprehensive approach to feature management and release coordination.
Goals for modernization
Ally's leadership issued a clear mandate: figure out how to deliver change faster but also safer, iteratively instead of in big-bang releases, and allow teams to release changes independently. This wasn't just about adopting new tools; it required a fundamental shift in how Ally approached software development and delivery.
To meet this challenge, Ally focused on a few key areas. The goals were to increase release velocity, implement API versioning and backwards compatibility, and enable daytime releases. Gray spoke about the practice of issuing overnight releases:
“We can’t deliver more change, more value, while also working forty to fifty hours a week and asking teams to then work overnights and to give up their weekends. That’s… burning out your best engineers.” ~ Nathan Gray, Director of Digital Engineering Operations, Ally Financial
The team also wanted to introduce feature-level rollbacks, implement testing in production, and achieve parity between their web and mobile platforms. Gray recalled, “in our previous state, web and API might deliver a feature a month before mobile, or vice versa. We had to have them releasing at the same time.”
Collaborating with LaunchDarkly
In 2020, Ally brought in LaunchDarkly to help with modernization efforts. The first use case demonstrated the power of feature flagging during a critical time in the COVID-19 pandemic. Cox shared this important moment: "The first time we used it, we had a feature that was about to go out to production for customers to verify their employment status. Well, we all remember 2020. March was a time when everything went into chaos. A lot of folks lost their jobs. It's not [the] right time to ask folks to tell us who their employer is."
Using the LaunchDarkly feature flagging capability, Ally was able to simply turn off the feature without rebuilding the entire application or going through new cycles of testing. This experience sold the team on the benefits of separating deploy from release, setting the stage for broader improvements to their release process.
Transforming the release process
With LaunchDarkly as an enabler, Ally implemented several big changes. The first was decoupling the UI from the API. This let teams work in parallel, releasing APIs and gradually turning on features without tight dependencies. This shift towards a more modular architecture was complemented by a move toward microservices and a micro-frontend architecture. By breaking down monoliths into smaller, independently deployable services and frontend components, Ally gained the flexibility to release and roll back features individually.
Feature flagging became central to Ally's new architecture, with LaunchDarkly allowing them to control the rollout of features across the banking frontend with a new level of granularity. This enabled Ally to implement a more sophisticated approach to testing in production. The team developed a gradual rollout process, starting with internal pilots and progressing to canary releases for customers, significantly reducing the risk associated with new deployments.
Alongside technical changes, Ally also revamped release management. The team created clear, automated paths for safely deploying changes to production, giving application developers more autonomy while maintaining necessary controls. Gray says he was “talking about release management as a service provider to the engineering teams that we support.” He remarked that his team is showing engineers “the easiest, least treacherous path, and not just a series of toll gates that they must pass.” This shared responsibility across engineering and release teams helped Ally strike a balance between freedom and responsibility.
Results and impact
The results of Ally's modernization efforts were impressive. From 2020 to 2023, they achieved a 97% reduction in overnight and weekend releases. At the same time, they saw a 300% increase in production deployments. Gray summarized the impact:
"Did we get ourselves out of nights and weekends? Largely, yes. But this is reality, and we're in technology. The occasional night, the occasional weekend has popped up. But looking at the data over the past four years from 2020 to 2023: a massive 97% reduction in overnight and weekend releases, while enjoying a 300% increase in production deployments." ~ Nathan Gray
Lessons learned and next steps
Through this process, Ally developed several guiding principles for release management. These principles emphasize the importance of providing a well-defined, easy-to-follow path for safe production deployments, granting application teams autonomy to move at their own speed, and ensuring that the scope of each release is independently manageable through feature flagging and API versioning. This encourages engineering teams to operate like they're running a company, balancing customer service with risk management, and prioritizing automation and continuous improvement.
Ally's success with LaunchDarkly has sparked interest across the organization. Gray reflects: "the secret of those guys over in digital has gotten out at Ally, and we now have half a dozen departments that have created their own LaunchDarkly projects and are starting to implement it into their own features." Looking to the future, Ally plans to increase federation and team independence, implement release and rollback automation, and explore experimentation and personalization features in LaunchDarkly.
Ally improved its software delivery process and enhanced the work-life balance of its engineering teams by embracing new methods. As digital banking grows, the team at Ally is poised to keep pace with the demands of modern software development while continuing to reach new goals.
.webp)
