Skip to content

Conversation

franciscovalentecastro
Copy link
Contributor

@franciscovalentecastro franciscovalentecastro commented Sep 19, 2025

Description

Upgrade fluent-bit submodule from v3.1.6 to v4.0.7 to bring the following bug fixes :

Some details about the upgrade :

  • Revamped Dockerfile.windows to follow : v4.0.7/dockerfiles/Dockerfile.windows.
  • Setting shutdown Grace period to 0s to avoid any unreliable logs to be sent after Exit_On_Eof.
  • Created a submodules/go.mod file to encapsulate any Go code from submodules from the top level module.
  • SLES-12 requires a newer GCC compiler (chose gcc8) to build fluent-bit 4.0.7.
  • CMake 3.20 or higher is required to build fluent-bit 4.0.7.
  • DFLB_KAFKA=OFF is a newly added flag that disables all kafka libraries which fail to build and are not needed.

Related issue

b/443678958
b/446168784
b/446655640

How has this been tested?

Checklist:

  • Unit tests
    • Unit tests do not apply.
    • Unit tests have been added/modified and passed for this PR.
  • Integration tests
    • Integration tests do not apply.
    • Integration tests have been added/modified and passed for this PR.
  • Documentation
    • This PR introduces no user visible changes.
    • This PR introduces user visible changes and the corresponding documentation change has been made.
  • Minor version bump
    • This PR introduces no new features.
    • This PR introduces new features, and there is a separate PR to bump the minor version since the last release already.
    • This PR bumps the version.

@franciscovalentecastro franciscovalentecastro added the kokoro:force-run Forces kokoro to run integration tests on a CL label Sep 19, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Sep 19, 2025
@franciscovalentecastro franciscovalentecastro added the kokoro:force-run Forces kokoro to run integration tests on a CL label Sep 22, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Sep 22, 2025
@franciscovalentecastro franciscovalentecastro changed the title Update fluent-bit to v3.2.2. Update fluent-bit to v3.2.3. Sep 22, 2025
@franciscovalentecastro franciscovalentecastro marked this pull request as ready for review September 22, 2025 17:56
@franciscovalentecastro franciscovalentecastro requested review from a team, XuechunHou and jefferbrecht and removed request for a team September 22, 2025 19:07
@franciscovalentecastro franciscovalentecastro changed the title Update fluent-bit to v3.2.3. Update fluent-bit to v4.0.6. Sep 24, 2025
@franciscovalentecastro
Copy link
Contributor Author

Current Status - Fluent-bit 4.0.6 has a couple of small updates that break some of our tests :

  • Ubuntu / Debian packages provide cmake 3.18.x versions, but fluent-bit now requires cmake 3.20.x to build some dependencies (e.g. onigmo for parse_regex).
  • The following update lib: nghttp2: upgrade to v1.65.0 : fluent/fluent-bit@a237287 adds some Golang code to one of the libraries subfolders that messes up our go test ./... since there is no go.mod to separate the submodule as another codebase.

-DFLB_METRICS=On `
-DFLB_WASM=Off `
-DFLB_WAMRC=Off `
-DFLB_KAFKA=OFF `
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we don't need -DFLB_(IN|OUT)_KAFKA=Off anymore?

Also, nit: OFF -> Off

Copy link
Contributor Author

@franciscovalentecastro franciscovalentecastro Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we don't need -DFLB_(IN|OUT)_KAFKA=Off anymore?

Yes, from the implementation 1 we could remove IN and OUT from this list. I didn't remove them to be overly descriptive. I'll remove them to see how it looks.

# Kafka support (library in general)
if(FLB_KAFKA)
  if(FLB_IN_KAFKA OR FLB_OUT_KAFKA)
  ...
...  

Footnotes

  1. https://github.com/fluent/fluent-bit/blob/v4.0.7/CMakeLists.txt#L757-L771

build-essential cmake bison flex file libsystemd-dev \
devscripts cdbs pkg-config openjdk-${OPENJDK_MAJOR_VERSION}-jdk zip`,
build-essential bison flex file libsystemd-dev \
devscripts cdbs pkg-config openjdk-${OPENJDK_MAJOR_VERSION}-jdk zip` + installCMake,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we only need installCMake for distros that ship < 3.20.0? Bookworm ships 3.25.1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we should only use installCmake when we need to. I conflated all Ubuntu/Debian failures to this reason initially. Removed installCmake. Done!

build-essential cmake bison flex file libsystemd-dev tzdata \
devscripts cdbs pkg-config openjdk-${OPENJDK_MAJOR_VERSION}-jdk zip`,
build-essential bison flex file libsystemd-dev tzdata \
devscripts cdbs pkg-config openjdk-${OPENJDK_MAJOR_VERSION}-jdk zip` + installCMake,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise, Jammy ships CMake 3.22.1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

build-essential cmake bison flex file libsystemd-dev tzdata \
devscripts cdbs pkg-config openjdk-${OPENJDK_MAJOR_VERSION}-jdk zip debhelper`,
build-essential bison flex file libsystemd-dev tzdata \
devscripts cdbs pkg-config openjdk-${OPENJDK_MAJOR_VERSION}-jdk zip debhelper` + installCMake,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noble ships CMake 3.28.3

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

build-essential cmake bison flex file systemd-dev debhelper libsystemd-dev tzdata \
devscripts cdbs pkg-config openjdk-${OPENJDK_MAJOR_VERSION}-jdk zip`,
build-essential bison flex file systemd-dev debhelper libsystemd-dev tzdata \
devscripts cdbs pkg-config openjdk-${OPENJDK_MAJOR_VERSION}-jdk zip` + installCMake,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plucky ships CMake 3.31.6

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@franciscovalentecastro franciscovalentecastro changed the title Update fluent-bit to v4.0.6. Update fluent-bit to v4.0.7. Oct 1, 2025
@franciscovalentecastro franciscovalentecastro added the kokoro:force-run Forces kokoro to run integration tests on a CL label Oct 3, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Oct 3, 2025
@franciscovalentecastro franciscovalentecastro added the kokoro:force-run Forces kokoro to run integration tests on a CL label Oct 3, 2025
@stackdriver-instrumentation-release stackdriver-instrumentation-release removed the kokoro:force-run Forces kokoro to run integration tests on a CL label Oct 3, 2025
@franciscovalentecastro
Copy link
Contributor Author

All tests passed. Merging.

@franciscovalentecastro franciscovalentecastro merged commit 656dd8d into master Oct 4, 2025
62 checks passed
@franciscovalentecastro franciscovalentecastro deleted the fcovalente-fluent-bit-v3.2.2 branch October 4, 2025 01:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants