Self-hosting an OpenTelemetry Collector

This topic explains how to set up an OpenTelemetry Collector to forward telemetry data to LaunchDarkly. This is useful if you want to:

Buffer and retry telemetry data closer to your application, reducing data loss during network interruptions
Filter or transform data before it leaves your network
Consolidate telemetry from multiple services through a single egress point
Reduce direct internet dependencies from your application pods
Dual-export data to LaunchDarkly and another observability backend simultaneously

An OpenTelemetry Collector does not replace the Relay ProxyThe OpenTelemetry Collector handles telemetry data (traces, logs, metrics) only. It does not proxy flag evaluations or SDK configuration. If you need to proxy flag evaluation traffic, use the Relay Proxy instead. You can use both a Relay Proxy and an OpenTelemetry Collector together.

Prerequisites

To implement an OpenTelemetry Collector you require:

An OpenTelemetry Collector binary or container image. Use the OpenTelemetry Collector Contrib distribution if you need additional exporters or processors.
Your LaunchDarkly SDK key, available from your project’s Settings page.

Collector configuration

Create a file named otel-collector-config.yaml with the following configuration. This sets up the Collector to receive OpenTelemetry Protocol (OTLP) data from your SDKs and forward it to LaunchDarkly:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  # Batch telemetry data to reduce the number of export requests.
  batch:
    send_batch_size: 512
    timeout: 5s

  # Limit memory usage to prevent the Collector from consuming
  # excessive resources under high load.
  memory_limiter:
    # 75% of the maximum memory you want the Collector to use.
    limit_mib: 1536
    # 25% of limit_mib, allows temporary spikes.
    spike_limit_mib: 512
    check_interval: 5s

exporters:
  # Export telemetry data to LaunchDarkly.
  otlphttp/launchdarkly:
    endpoint: https://otel.observability.app.launchdarkly.com
    compression: gzip
    headers:
      # No authorization header is needed. LaunchDarkly identifies
      # your project by the SDK key embedded in the telemetry data
      # resource attributes.

  # Optional: export to a second backend for debugging or migration.
  # debug:
  #   verbosity: detailed

extensions:
  health_check:
    endpoint: 0.0.0.0:13133

service:
  extensions: [health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/launchdarkly]
    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/launchdarkly]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/launchdarkly]

Place memory_limiter before batchThe memory_limiter processor must appear before batch in the processors list. This ensures that the Collector sheds load before buffering more data, and prevents out-of-memory crashes.

Running the Collector

Docker

docker run -d \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 13133:13133 \
  -v $(pwd)/otel-collector-config.yaml:/etc/otelcol/config.yaml \
  otel/opentelemetry-collector-contrib:latest

Docker Compose

services:
  otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    ports:
      - "4317:4317"   # gRPC receiver
      - "4318:4318"   # HTTP receiver
      - "13133:13133" # Health check
    volumes:
      - ./otel-collector-config.yaml:/etc/otelcol/config.yaml
    restart: unless-stopped

Kubernetes

Deploy the Collector as a Deployment or DaemonSet. The OpenTelemetry Operator for Kubernetes simplifies this.

Configuring the SDK to use the Collector

Point your LaunchDarkly observability SDK at the Collector instead of the default LaunchDarkly endpoint. Set the OTLP endpoint to your Collector’s address.

The public endpoint cannot be proxied through the CollectorThe Collector only handles OTLP telemetry data. The SDK still communicates directly with pub.observability.app.launchdarkly.com to fetch sampling configuration. Make sure your application can reach this endpoint.

const client = init('YOUR_SDK_KEY', {
  plugins: [
    new Observability({
      serviceName: 'example-service',
      otel: {
        otlpEndpoint: 'http://localhost:4318',
      },
    }),
  ],
});

config = ObservabilityConfig(
  service_name="example-service",
  otlp_endpoint="http://localhost:4317",
)

client, _ := ld.MakeCustomClient("YOUR_SDK_KEY",
  ld.Config{
    Plugins: []ldplugins.Plugin{
      ldobserve.NewObservabilityPlugin(
        ldobserve.WithServiceName("example-service"),
        ldobserve.WithOTLPEndpoint("http://localhost:4318"),
      ),
    },
  }, 5*time.Second)

plugin = LaunchDarklyObservability::Plugin.new(
  service_name: 'example-service',
  otlp_endpoint: 'http://localhost:4318'
)

var config = Configuration.Builder("YOUR_SDK_KEY")
    .Plugins(new PluginConfigurationBuilder()
        .Add(ObservabilityPlugin.Builder(builder.Services)
            .WithServiceName("example-service")
            .WithOtlpEndpoint("http://localhost:4318")
            .Build())).Build();

const options = {
  plugins: [
    new Observability({
      otel: {
        otlpEndpoint: 'https://otel-collector.your-domain.com',
      },
    }),
  ],
};

You can also set the endpoint using the OTEL_EXPORTER_OTLP_ENDPOINT environment variable for SDKs that support it (Node.js, Python, .NET, Ruby).

Dual-export to LaunchDarkly and another backend

A common use case is exporting telemetry to both LaunchDarkly and another observability provider. Add a second exporter and include it in your pipelines:

exporters:
  otlphttp/launchdarkly:
    endpoint: https://otel.observability.app.launchdarkly.com
    compression: gzip

  otlphttp/other-provider:
    endpoint: https://otel.your-other-provider.com
    headers:
      authorization: "Bearer ${env:OTHER_PROVIDER_API_KEY}"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/launchdarkly, otlphttp/other-provider]
    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/launchdarkly, otlphttp/other-provider]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/launchdarkly, otlphttp/other-provider]

Handle rate limits

LaunchDarkly limits how much OpenTelemetry data it accepts from each project and environment. When a request exceeds the limit, LaunchDarkly responds with an HTTP 429 status or a gRPC RESOURCE_EXHAUSTED status, along with a retry delay. To learn more, read Ingestion rate limits.The Collector’s otlp and otlphttp exporters retry these responses and wait for the retry delay by default. They hold data that is waiting to be sent in a sending queue. If you send large volumes or sudden spikes of data, you can tune the retry and queue settings to hold data through a short period of rate limiting instead of dropping it.Here is an example:

exporters:
  otlphttp/launchdarkly:
    endpoint: https://otel.observability.app.launchdarkly.com
    compression: gzip
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s
    sending_queue:
      enabled: true
      queue_size: 1000

max_elapsed_time caps how long the Collector retries a batch before it drops it. queue_size caps how many batches the Collector holds in memory. To learn more about these settings, read the OpenTelemetry exporter helper documentation.

Filtering sensitive data

You can use the Collector’s filter and attributes processors to remove sensitive data before it leaves your network:

processors:
  filter/drop-health-checks:
    traces:
      span:
        - 'attributes["http.route"] == "/healthz"'
        - 'attributes["http.route"] == "/ready"'

  attributes/redact:
    actions:
      - key: "http.request.header.authorization"
        action: delete
      - key: "db.statement"
        action: delete

To learn more about available processors, read the OpenTelemetry Collector documentation.

​Prerequisites

​Collector configuration

​Running the Collector

​Docker

​Docker Compose

​Kubernetes

​Configuring the SDK to use the Collector

​Dual-export to LaunchDarkly and another backend

​Handle rate limits

​Filtering sensitive data

Prerequisites

Collector configuration

Running the Collector

Docker

Docker Compose

Kubernetes

Configuring the SDK to use the Collector

Dual-export to LaunchDarkly and another backend

Handle rate limits

Filtering sensitive data