Skip to content

Conversation

Caleb-Hurshman
Copy link
Collaborator

Description

Refactor the Couchdb receiver logging to be sub-agent agnostic.
Note: this is the first receiver converted with multiline rules

Related issue

How has this been tested?

Updated transformation_test and confgenerator_test, confirmed the output of output_fluentbit.yaml was unchanged after refactoring the receiver.

Checklist:

  • Unit tests
    • Unit tests do not apply.
    • Unit tests have been added/modified and passed for this PR.
  • Integration tests
    • Integration tests do not apply.
    • Integration tests have been added/modified and passed for this PR.
  • Documentation
    • This PR introduces no user visible changes.
    • This PR introduces user visible changes and the corresponding documentation change has been made.
  • Minor version bump
    • This PR introduces no new features.
    • This PR introduces new features, and there is a separate PR to bump the minor version since the last release already.
    • This PR bumps the version.

@Caleb-Hurshman Caleb-Hurshman force-pushed the feat/couchdb-otel-logging branch from 1064f75 to bedc912 Compare August 19, 2025 17:34
@Caleb-Hurshman Caleb-Hurshman added the kokoro:force-run Forces kokoro to run integration tests on a CL label Aug 19, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Aug 19, 2025
@Caleb-Hurshman Caleb-Hurshman force-pushed the feat/couchdb-otel-logging branch from bedc912 to 97ff6c5 Compare August 19, 2025 19:23
Copy link
Contributor

@franciscovalentecastro franciscovalentecastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added one comment.

[notice] 2021-12-02T23:36:42.555157Z nonode@nohost <0.17165.1> a5f585a0d3 localhost:5984 127.0.0.1 otelu PUT /oteld 201 ok 16
[info] 2022-01-12T16:52:56.998128Z nonode@nohost <0.216.0> -------- Apache CouchDB has started. Time to relax.
[error] 2022-01-12T16:53:03.094488Z nonode@nohost emulator -------- Error in process <0.463.0> with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,399}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,374}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,96}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,39}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,198}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,145}]}]}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a known bug that fluent-bit doesn't parse correctly the last line of the logs when there is a MultilineRule definition (summary : The processor is still looking for another line of the log and it is not flushed/exported correctly when fluent-bit stops).

Workaround : Please add two or more log sample lines to input.log so we can verify the multiline-log is being parsed correctly. This is workaround while we fix/address the bug in parallel.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I noticed this and discussed with Dylan, I'll update accordingly

@Caleb-Hurshman Caleb-Hurshman added the kokoro:force-run Forces kokoro to run integration tests on a CL label Aug 19, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Aug 19, 2025
Copy link
Contributor

@franciscovalentecastro franciscovalentecastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

We will address separately the bug with missing last log in fluent-bit transformation tests.

@franciscovalentecastro franciscovalentecastro merged commit c6005d9 into GoogleCloudPlatform:master Aug 25, 2025
56 checks passed
@Caleb-Hurshman Caleb-Hurshman deleted the feat/couchdb-otel-logging branch August 25, 2025 15:16
rafaelwestphal pushed a commit that referenced this pull request Aug 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants