Skip to content

[exporter/clickhouseexporter] Support JSON type for logs and traces #40547

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 53 commits into from
Jun 27, 2025
Merged

[exporter/clickhouseexporter] Support JSON type for logs and traces #40547

merged 53 commits into from
Jun 27, 2025

Conversation

SpencerTorres
Copy link
Member

@SpencerTorres SpencerTorres commented Jun 9, 2025 •
edited
Loading

Description

Depends on refactoring from #40536

Adds JSON support to the exporter.

Summary:

  • Added feature gate (clickhouse.json, alpha stability) for enabling the JSON type pipeline. The Map based pipeline is still functional and is the default. (How to use feature gates)
  • The JSON type replaces all Map columns with the new JSON type.
  • JSON is sent to the server as a string.
  • Due to how the server stores JSON paths, OTel standard paths such as a.b.c are returned as objects a { b { c } } by the server. This can be changed via server setting in the future to return all data as flattened paths a.b.c.

Let me know if you have any questions, suggestions, etc.

Thanks!

Testing

  • Added unit tests for related helper functions
  • Added integration tests for JSON logs/traces

Documentation

  • Updated README, changelog

@SpencerTorres
refactor checkpoint
ff7a946
@SpencerTorres
use fully qualified names for table creation/insertion
cd1a250
@SpencerTorres
remove comment
068cfe3
@SpencerTorres
update func call
53579ed
@SpencerTorres
move connection init to Start function, fix some unit tests
c453e43
@SpencerTorres
use new batch.Close func
200fe05
@SpencerTorres
createSchema todo
b83af98
@SpencerTorres
fix summary table ddl
44c14d7
@SpencerTorres
convert exporter tests to proper integration tests
99d3edc
@SpencerTorres
remove duplicate tests, add integration build flags, goleak check
27cb389
@SpencerTorres
fix lint
61540ce
@SpencerTorres
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
c4ff013 
…tor-contrib into clickhouse_full_reorganize
@SpencerTorres
remove default_ddl, use sqltemplates folder
3b1a708
@SpencerTorres
rename file to original generated
8d81900
@SpencerTorres
remove test build tag
63278cd
@SpencerTorres
package test, rename context
26a044c
@SpencerTorres
add tests
339c0c3
@SpencerTorres
run goporto
64079aa
@SpencerTorres
run generate
d77e451
@SpencerTorres
revert lifecylce tests
d592be3
@SpencerTorres
remove TestMain dependency for integration tests
452578c
@SpencerTorres
make fmt
5b36389
@SpencerTorres
re-use existing scope variable
48d8428
@SpencerTorres
collapse if
8d4a4e5
@SpencerTorres
export json logs
57edec2
@SpencerTorres
traces json
c1640ea
@SpencerTorres
json integration tests
2295904
@SpencerTorres
add tests, changelog, fmt/lint
91106bd
@SpencerTorres
readme
a702d61
@SpencerTorres SpencerTorres requested a review from dmitryax as a code owner June 9, 2025 06:46
@SpencerTorres
Copy link
Member Author

SpencerTorres commented Jun 17, 2025 •
edited
Loading

This PR includes a driver update that fixes the HTTP pipelines, if we don't get this merged before the next release I can submit a separate PR to update the module.

clickhouse-go v2.37.1 includes the fix, main is currently on v2.37.0

Without the fix, HTTP pipelines risk starving connections and inconsistent HTTP/400 errors.

Resolved in #40766