Self-hosting an OpenTelemetry Collector

Overview

This topic explains how to set up an OpenTelemetry Collector to forward telemetry data to LaunchDarkly. This is useful if you want to:

• Buffer and retry telemetry data closer to your application, reducing data loss during network interruptions
• Filter or transform data before it leaves your network
• Consolidate telemetry from multiple services through a single egress point
• Reduce direct internet dependencies from your application pods
• Dual-export data to LaunchDarkly and another observability backend simultaneously

Prerequisites

To implement an OpenTelemetry Collector you require:

• An OpenTelemetry Collector binary or container image. Use the OpenTelemetry Collector Contrib distribution if you need additional exporters or processors.
• Your LaunchDarkly SDK key, available from your project’s Settings page.

Collector configuration

Create a file named otel-collector-config.yaml with the following configuration. This sets up the Collector to receive OTLP data from your SDKs and forward it to LaunchDarkly:

1
receivers:
2
  otlp:
3
    protocols:
4
      grpc:
5
        endpoint: 0.0.0.0:4317
6
      http:
7
        endpoint: 0.0.0.0:4318
8
9
processors:
10
  # Batch telemetry data to reduce the number of export requests.
11
  batch:
12
    send_batch_size: 512
13
    timeout: 5s
14
15
  # Limit memory usage to prevent the Collector from consuming
16
  # excessive resources under high load.
17
  memory_limiter:
18
    # 75% of the maximum memory you want the Collector to use.
19
    limit_mib: 1536
20
    # 25% of limit_mib, allows temporary spikes.
21
    spike_limit_mib: 512
22
    check_interval: 5s
23
24
exporters:
25
  # Export telemetry data to LaunchDarkly.
26
  otlphttp/launchdarkly:
27
    endpoint: https://otel.observability.app.launchdarkly.com
28
    compression: gzip
29
    headers:
30
      # No authorization header is needed. LaunchDarkly identifies
31
      # your project by the SDK key embedded in the telemetry data
32
      # resource attributes.
33
34
  # Optional: export to a second backend for debugging or migration.
35
  # debug:
36
  #   verbosity: detailed
37
38
extensions:
39
  health_check:
40
    endpoint: 0.0.0.0:13133
41
42
service:
43
  extensions: [health_check]
44
  pipelines:
45
    traces:
46
      receivers: [otlp]
47
      processors: [memory_limiter, batch]
48
      exporters: [otlphttp/launchdarkly]
49
    logs:
50
      receivers: [otlp]
51
      processors: [memory_limiter, batch]
52
      exporters: [otlphttp/launchdarkly]
53
    metrics:
54
      receivers: [otlp]
55
      processors: [memory_limiter, batch]
56
      exporters: [otlphttp/launchdarkly]

Running the Collector

Docker

$
docker run -d \
>
  -p 4317:4317 \
>
  -p 4318:4318 \
>
  -p 13133:13133 \
>
  -v $(pwd)/otel-collector-config.yaml:/etc/otelcol/config.yaml \
>
  otel/opentelemetry-collector-contrib:latest

Docker Compose

1
services:
2
  otel-collector:
3
    image: otel/opentelemetry-collector-contrib:latest
4
    ports:
5
      - "4317:4317"   # gRPC receiver
6
      - "4318:4318"   # HTTP receiver
7
      - "13133:13133" # Health check
8
    volumes:
9
      - ./otel-collector-config.yaml:/etc/otelcol/config.yaml
10
    restart: unless-stopped

Kubernetes

Deploy the Collector as a Deployment or DaemonSet. The OpenTelemetry Operator for Kubernetes simplifies this.

Configuring the SDK to use the Collector

Point your LaunchDarkly observability SDK at the Collector instead of the default LaunchDarkly endpoint. Set the OTLP endpoint to your Collector’s address.

1
const client = init('YOUR_SDK_KEY', {
2
  plugins: [
3
    new Observability({
4
      serviceName: 'example-service',
5
      otel: {
6
        otlpEndpoint: 'http://localhost:4318',
7
      },
8
    }),
9
  ],
10
});

You can also set the endpoint using the OTEL_EXPORTER_OTLP_ENDPOINT environment variable for SDKs that support it (Node.js, Python, .NET, Ruby).

Dual-export to LaunchDarkly and another backend

A common use case is exporting telemetry to both LaunchDarkly and another observability provider. Add a second exporter and include it in your pipelines:

1
exporters:
2
  otlphttp/launchdarkly:
3
    endpoint: https://otel.observability.app.launchdarkly.com
4
    compression: gzip
5
6
  otlphttp/other-provider:
7
    endpoint: https://otel.your-other-provider.com
8
    headers:
9
      authorization: "Bearer ${env:OTHER_PROVIDER_API_KEY}"
10
11
service:
12
  pipelines:
13
    traces:
14
      receivers: [otlp]
15
      processors: [memory_limiter, batch]
16
      exporters: [otlphttp/launchdarkly, otlphttp/other-provider]
17
    logs:
18
      receivers: [otlp]
19
      processors: [memory_limiter, batch]
20
      exporters: [otlphttp/launchdarkly, otlphttp/other-provider]
21
    metrics:
22
      receivers: [otlp]
23
      processors: [memory_limiter, batch]
24
      exporters: [otlphttp/launchdarkly, otlphttp/other-provider]

Filtering sensitive data

You can use the Collector’s filter and attributes processors to remove sensitive data before it leaves your network:

1
processors:
2
  filter/drop-health-checks:
3
    traces:
4
      span:
5
        - 'attributes["http.route"] == "/healthz"'
6
        - 'attributes["http.route"] == "/ready"'
7
8
  attributes/redact:
9
    actions:
10
      - key: "http.request.header.authorization"
11
        action: delete
12
      - key: "db.statement"
13
        action: delete

To learn more about available processors, read the OpenTelemetry Collector documentation.