Skip to content

Conversation

r0mdau
Copy link
Contributor

@r0mdau r0mdau commented Jun 15, 2025

Description

When moving datapoints in the routing connector, copy

  • isMonotonic and AggregationTemporality for Sum
  • AggregationTemporality for Histogram
  • AggregationTemporality for ExponentialHistogram

Link to tracking issue

Fixes #37273

Testing

Unit tests refactored to add those properties.

@r0mdau r0mdau requested review from a team and mwear as code owners June 15, 2025 03:05
@r0mdau
Copy link
Contributor Author

r0mdau commented Jun 15, 2025

Before merging, I need to know if the proposed unit tests structure used for Sum metric type is okay so I move forward with Histogram and ExponentialHistogram.

Edit: I updated unit tests for the 3 metric types.

@r0mdau
Copy link
Contributor Author

r0mdau commented Jun 15, 2025

Configurations used for tests

Click to toggle contents of `config.yaml`
receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318

exporters:
  debug:
    verbosity: detailed

connectors:
  routing/stage:
    table:

    # outbound metrics have undefined/default IsMonotonic and AggregationTemporality
    - condition: 'attributes["stage"] == "production"'
      context: datapoint
      pipelines: [metrics/prod]

    # outbound metrics have proper IsMonotonic and AggregationTemporality
    # - statement: 'route()'
    #   pipelines: [metrics/prod]

service:
  telemetry:
    logs:
      level: debug
  pipelines:
    metrics/in:
      receivers: [otlp]
      exporters: [routing/stage]
    metrics/prod:
      receivers: [routing/stage]
      exporters: [debug]
Click to toggle contents of `JSON payload` used with curl
{
  "resourceMetrics": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my.service"
            }
          }
        ]
      },
      "scopeMetrics": [
        {
          "scope": {
            "name": "my.library",
            "version": "1.0.0",
            "attributes": [
              {
                "key": "my.scope.attribute",
                "value": {
                  "stringValue": "some scope attribute"
                }
              }
            ]
          },
          "metrics": [
            {
              "name": "my.counter",
              "unit": "1",
              "description": "I am a Counter",
              "sum": {
                "aggregationTemporality": 1,
                "isMonotonic": true,
                "dataPoints": [
                  {
                    "asDouble": 5,
                    "startTimeUnixNano": "1544712660300000000",
                    "timeUnixNano": "1544712660300000000",
                    "attributes": [
                      {
                        "key": "my.counter.attr",
                        "value": {
                          "stringValue": "some value"
                        }
                      },
                      {
                        "key": "stage",
                        "value": {
                          "stringValue": "production"
                        }
                      }
                    ]
                  }
                ]
              }
            }
          ]
        }
      ]
    }
  ]
}

Curl command

curl -X POST localhost:4318/v1/metrics -H "Content-Type: application/json" -d @metrics.json

Outputs

# before:
Resource SchemaURL: 
Resource attributes:
     -> service.name: Str(my.service)
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope my.library 1.0.0
InstrumentationScope attributes:
     -> my.scope.attribute: Str(some scope attribute)
Metric #0
Descriptor:
     -> Name: my.counter
     -> Description: I am a Counter
     -> Unit: 1
     -> DataType: Sum
     -> IsMonotonic: false
     -> AggregationTemporality: Unspecified
NumberDataPoints #0
Data point attributes:
     -> my.counter.attr: Str(some value)
     -> stage: Str(production)
StartTimestamp: 2018-12-13 14:51:00.3 +0000 UTC
Timestamp: 2018-12-13 14:51:00.3 +0000 UTC
Value: 5.000000

# after:
Resource SchemaURL: 
Resource attributes:
     -> service.name: Str(my.service)
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope my.library 1.0.0
InstrumentationScope attributes:
     -> my.scope.attribute: Str(some scope attribute)
Metric #0
Descriptor:
     -> Name: my.counter
     -> Description: I am a Counter
     -> Unit: 1
     -> DataType: Sum
     -> IsMonotonic: true
     -> AggregationTemporality: Delta
NumberDataPoints #0
Data point attributes:
     -> my.counter.attr: Str(some value)
     -> stage: Str(production)
StartTimestamp: 2018-12-13 14:51:00.3 +0000 UTC
Timestamp: 2018-12-13 14:51:00.3 +0000 UTC
Value: 5.000000

@vincentfree
Copy link
Contributor

I ran into this problem while designing a vendor agnostic way to do short-term and long-term storage for metrics(but also logs). This seems to fix the problem which has been plaguing me for a few days with datapoints. Thanks @r0mdau for this fix. I hope a code owner could review this in time for the next release.

@r0mdau r0mdau force-pushed the routing-datapoint-copy branch from 4695009 to 865f629 Compare June 23, 2025 16:56
Copy link
Member

@mwear mwear left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @r0mdau!

@songy23 songy23 merged commit 7056cd9 into open-telemetry:main Jun 25, 2025
186 checks passed
@github-actions github-actions bot added this to the next release milestone Jun 25, 2025
@r0mdau r0mdau deleted the routing-datapoint-copy branch June 25, 2025 11:45
Dylan-M pushed a commit to Dylan-M/opentelemetry-collector-contrib that referenced this pull request Aug 5, 2025
open-telemetry#40729)

#### Description

When moving datapoints in the routing connector, copy

- isMonotonic and AggregationTemporality for Sum
- AggregationTemporality for Histogram
- AggregationTemporality for ExponentialHistogram

#### Link to tracking issue
Fixes open-telemetry#37273

#### Testing
Unit tests refactored to add those properties.

---------

Signed-off-by: Romain Dauby <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Metrics routed using routing connector with datapoint context are missing AggregationTemporality

5 participants