Skip to content

OTLP APM metrics from ingress-nginx missing after upgrading to chart version 3.118.3 #1930

@afer4152

Description

@afer4152

Describe the bug

After upgrading the Datadog Helm chart from 3.117.3 to 3.118.3, we observed that APM traces from ingress-nginx (version 4.11.5, which uses OpenTelemetry) are no longer being reported correctly to Datadog APM. Specifically:

With chart 3.117.3: traces were received using the nginx.server span name via OpenTelemetry.

With chart 3.118.3: those traces are no longer received. Instead, we are only getting old-style http.server.request metrics (as seen with older ingress-nginx versions like 4.9.0).

We suspect a new version breaking change in the chart version 3.118.3 that affects OTLP receiver or span name translation logic.

To Reproduce

Steps to reproduce the behavior:

  1. Deploy ingress-nginx version 4.11.5 with OpenTelemetry enabled and properly configured OTEL_RESOURCE_ATTRIBUTES.
  2. Deploy Datadog Helm chart 3.117.3 with datadog.apm.enabled=true and OTLP receiver enabled (gRPC port 4317).
  3. Verify APM traces with span name nginx.server are visible in Datadog APM.
  4. Upgrade to chart version 3.118.3.
  5. Observe that the nginx.server traces disappear, and only http.server.request style metrics are visible.

Expected behavior

OpenTelemetry-based APM traces from ingress-nginx (v4.11.5) using the nginx.server span name should continue to be received and displayed in Datadog APM after chart upgrade.

Versions

  • Ingress-nginx: 4.11.5
  • Datadog Helm chart: 3.118.3 (regression), 3.117.3 (working)
  • Datadog Agent image: default for Helm chart versions
  • Kubernetes: v1.28

Image

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions