Skip to content

Conversation

iblancasa
Copy link
Contributor

Description

Create a merge_histogram_buckets OTTL function that merges two buckets.

In #40280, I proposed creating a drop_bucket function. Dropping the bucket directly would lead to the loss of data (as it was raised in #40281).

This PR implements this new function avoiding losing data and providing a more meaningful name.

Link to tracking issue

Fixes #40280

Testing

  • Added unit tests
  • Manual test:
  1. Start the OpenTelemetry Collector with this config:
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

processors:
  transform:
    error_mode: ignore
    metric_statements:
      - context: datapoint
        statements:
          - merge_histogram_buckets(3.0) where metric.name == "gen"

exporters:
  debug:
    verbosity: detailed
  prometheus:
    endpoint: "0.0.0.0:8889"   # curl this to inspect metrics

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [transform]
      exporters: [debug, prometheus]

Started this program:

package main

import (
	"context"
	"log"
	"time"

	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc"
	"go.opentelemetry.io/otel/metric"
	sdkmetric "go.opentelemetry.io/otel/sdk/metric"
	"go.opentelemetry.io/otel/sdk/resource"
	semconv "go.opentelemetry.io/otel/semconv/v1.24.0"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials/insecure"
)

func main() {
	ctx := context.Background()

	conn, err := grpc.NewClient("localhost:4317", grpc.WithTransportCredentials(insecure.NewCredentials()))
	if err != nil {
		log.Fatalf("Failed to create gRPC connection: %v", err)
	}
	defer conn.Close()

	exporter, err := otlpmetricgrpc.New(ctx, otlpmetricgrpc.WithGRPCConn(conn))
	if err != nil {
		log.Fatalf("Failed to create OTLP exporter: %v", err)
	}

	res, err := resource.New(ctx, resource.WithAttributes(
		semconv.ServiceNameKey.String("histogram-test"),
		semconv.ServiceVersionKey.String("1.0.0"),
	))
	if err != nil {
		log.Fatalf("Failed to create resource: %v", err)
	}

	meterProvider := sdkmetric.NewMeterProvider(
		sdkmetric.WithReader(sdkmetric.NewPeriodicReader(exporter, sdkmetric.WithInterval(5*time.Second))),
		sdkmetric.WithResource(res),
	)
	defer func() {
		if err := meterProvider.Shutdown(ctx); err != nil {
			log.Printf("Error shutting down meter provider: %v", err)
		}
	}()

	otel.SetMeterProvider(meterProvider)

	meter := otel.Meter("test-meter")
	histogram, err := meter.Float64Histogram("gen",
		metric.WithDescription("Test histogram for merge_histogram_buckets function"),
		metric.WithExplicitBucketBoundaries(1, 2, 3, 4),
	)
	if err != nil {
		log.Fatalf("Failed to create histogram: %v", err)
	}

	// Record 1 value in (-inf,1] bucket
	histogram.Record(ctx, 0.5)

	// Record 2 values in (1,2] bucket
	histogram.Record(ctx, 1.5)
	histogram.Record(ctx, 1.8)

	// Record 3 values in (2,3] bucket
	histogram.Record(ctx, 2.1)
	histogram.Record(ctx, 2.5)
	histogram.Record(ctx, 2.9)

	// Record 4 values in (3,4] bucket
	histogram.Record(ctx, 3.1)
	histogram.Record(ctx, 3.3)
	histogram.Record(ctx, 3.7)
	histogram.Record(ctx, 3.9)

	select {}
}

Checked Prometheus endpoint:

gen_bucket{job="histogram-test",otel_scope_name="test-meter",otel_scope_schema_url="",otel_scope_version="",le="1"} 1
gen_bucket{job="histogram-test",otel_scope_name="test-meter",otel_scope_schema_url="",otel_scope_version="",le="2"} 3
gen_bucket{job="histogram-test",otel_scope_name="test-meter",otel_scope_schema_url="",otel_scope_version="",le="4"} 10
gen_bucket{job="histogram-test",otel_scope_name="test-meter",otel_scope_schema_url="",otel_scope_version="",le="+Inf"} 10
gen_sum{job="histogram-test",otel_scope_name="test-meter",otel_scope_schema_url="",otel_scope_version=""} 25.299999999999997
gen_count{job="histogram-test",otel_scope_name="test-meter",otel_scope_schema_url="",otel_scope_version=""} 10

And debug exporter output:

Count: 10
Sum: 25.300000
Min: 0.500000
Max: 3.900000
ExplicitBounds #0: 1.000000
ExplicitBounds #1: 2.000000
ExplicitBounds #2: 4.000000
Buckets #0, Count: 1
Buckets #1, Count: 2
Buckets #2, Count: 7
Buckets #3, Count: 0

Copy link
Contributor

@edmocosta edmocosta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but before moving forward, let's wait for a second review.
@TylerHelmuth @evan-bradley @jsuereth, your review would be much appreciated here. Thanks.

Copy link
Contributor

@evan-bradley evan-bradley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me, just a couple questions/suggestions.

@iblancasa
Copy link
Contributor Author

@open-telemetry/collector-contrib-maintainers can we merge this? Thanks!

@evan-bradley
Copy link
Contributor

@edmocosta Based on your "LGTM" comment, I'm going to take that as an approval and will merge this. Feel free to comment here if there's additional feedback you have.

@evan-bradley evan-bradley merged commit 109a78f into open-telemetry:main Sep 3, 2025
184 checks passed
@github-actions github-actions bot added this to the next release milestone Sep 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[processor/transform] Add drop_bucket OTTL function

3 participants