Relay Proxy deployment strategies | LaunchDarkly

This topic describes two recommended deployment strategies for the Relay Proxy based on common use cases.

Different Relay Proxy use cases call for different deployment strategies. There are two Relay Proxy deployment strategies we recommend: a Relay Proxy service layer or a node-level relay.

Relay Proxy service layer

A Relay Proxy service layer is a dedicated tier of Relay Proxy instances that SDKs connect to as a service, separate from your application workloads. We recommend a Relay Proxy service layer for the following scenarios:

LaunchDarkly outage mitigation
Proxying from a strict network, such as a “demilitarized zone” (DMZ) or domain allowlist
Point of presence in unreliable networks
Offline or air-gapped point of presence

For simple, low-volume scenarios, a single instance may be sufficient. If you need more than one instance, the service layer consists of two components: the array and the hydration relay.

The array

The array is a set of dynamically scaled Relay Proxy instances behind a load balancer. Each array instance should run on its own dedicated node, such as Amazon Elastic Compute Cloud (EC2) small-medium, and should be able to scale down to zero instances to minimize costs. The scale is based on system resource usage and Relay Proxy metrics.

Connect the array to a shared persistent store so it can cold start during an outage.

The hydration relay

The Hydration Relay is a non-scaling relay instance, or a few instances when environment sharding or isolation is required. It keeps the shared persistent store populated so the array can recover during an outage.

At low to moderate scale, small EC2 instances are sufficient, or you can distribute the hydration relay with your application pods. At higher numbers of proxied environments or higher flag update volume, dedicated nodes become necessary.

Scaling options

This table lists the scaling options for a Relay Proxy service layer:

Scaling strategy	Notes
Horizontal scaling	Easy and automatic by design. Add more array instances behind the load balancer.
Vertical scaling	Increasing node resource capacity resolves resource limit issues quickly but has a cost ceiling. Use as a short-term mitigation. To address resource usage spikes, fix SDK singleton patterns and reduce environment payload size.
Environment sharding	Isolate non-production SDK traffic to its own service layer. Use a dedicated service layer for your production environment so issues in lower environments don’t affect production.
Client/server SDK isolation	Isolate client-side SDK traffic to its own service layer for greater stability of the server-side layer.

Node-level relay

Node-level relay is a single Relay Proxy instance per node, deployed as a DaemonSet or equivalent. SDKs on that node connect to the local relay instance to minimize egress. We recommend a Relay Proxy service layer for optimizing nodes with many LaunchDarkly connections.

If using a Relay Proxy service layer, we recommend the following:

Using one relay per node. Multiple relay instances are allowed as long as there is no overlap in the environments they proxy.
Connect all SDKs on the node to the node-level relay.
As the number of proxied environments increases, you may need to silo applications that use the same environments together to facilitate environment sharding.

The node-level relay can connect to a Relay Proxy service layer or directly to LaunchDarkly.

Scaling options

This table lists the scaling options for a node-level relay:

Scaling strategy	Notes
Vertical scaling	Increasing node resource capacity resolves spike-related resource limit issues. Use as a short-term mitigation. Fix SDK singleton patterns and reduce environment payload size to address the root cause.
Environment siloing	Scope each Relay to only the environments used by workloads on that host. This usually means dedicating specific hosts or host groups to specific applications. The mechanism varies by platform: node labels and DaemonSet in Kubernetes, placement constraints and agent in ECS, and so on.
Horizontal scaling	Not applicable. Because the purpose is to reduce egress, adding another instance on another node provides no benefit.
Client/server SDK isolation	Not applicable. There is no client-side use case for node-level Relay.

Patterns to avoid

The following patterns either introduce unnecessary risk or are unsupported by LaunchDarkly.

Avoid persistent store variants

Persistent store variants replace the Relay Proxy with a persistent store that SDKs connect to directly. A persistent store variant can be node-level, centralized, or distributed.

We don’t recommend using persistent store variants for the following reasons:

Variants offer no additional benefit over a node-level Relay Proxy.
Variants introduce an additional point of failure.
LaunchDarkly does not support the persistent store itself, or connections to it.
LaunchDarkly does not offer scaling assistance because it doesn’t support the persistent store.

Avoid a distributed relay pool

Unsupported pattern

LaunchDarkly does not support a distributed relay pool at any scale. This pattern causes resource utilization feedback loops.

In a distributed relay pool, Relay Proxy pods are distributed among application nodes using container orchestration, such as EKS, ECS, or Kubernetes. However, two or more Relay Proxies connected to the same environments can end up on the same node, defeating the purpose of scaling. In this situation, SDKs have no guarantee of connecting to a relay on the same node. All SDKs connect through a load balancer that routes back to the same nodes already hosting the Relay Proxies. This wastes resources and creates a feedback loop that requires more and more capacity.