Skip to content

Initial commit for setting up a new component: k8slog receiver #24439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Mar 18, 2025
Merged

Initial commit for setting up a new component: k8slog receiver #24439

merged 7 commits into from
Mar 18, 2025

Conversation

h0cheung
Copy link
Contributor

Description:

k8slog receiver provides full functionality to collect logs in k8s containers. It is an all-in-one solution, which supports to:

  • collect logs from files inside k8s containers via daemonset
  • support docker and cri-containerd
  • support many docker graph drivers / snapshotters
  • collect logs from stdout of k8s containers via daemonset
  • filter containers by metadata
  • extract metadata of containers
  • collect file logs in k8s containers via sidecar

If we want to collect logs from k8s, we can use this component.

Link to tracking Issue:

#23339

Testing:

Tests for unmarshaling configuration have been added.

Documentation:

Docs for configurations and overall design has been added.

@h0cheung h0cheung requested review from a team and kovrus July 24, 2023 08:05
@h0cheung h0cheung force-pushed the feat/k8slog-setup branch from 4858200 to 3e05fa1 Compare July 24, 2023 10:02
@github-actions
Copy link
Contributor

github-actions bot commented Aug 8, 2023

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Aug 8, 2023
@h0cheung
Copy link
Contributor Author

h0cheung commented Aug 8, 2023

This PR was marked stale due to lack of activity. It will be closed in 14 days.

remove stale

@h0cheung h0cheung force-pushed the feat/k8slog-setup branch from 3e05fa1 to ea3a841 Compare August 8, 2023 13:42
@github-actions github-actions bot removed the Stale label Aug 9, 2023
@h0cheung h0cheung force-pushed the feat/k8slog-setup branch 6 times, most recently from 951c26c to 04d4b40 Compare August 10, 2023 06:39
@dmitryax dmitryax mentioned this pull request Aug 14, 2023
Closed
dmitryax
dmitryax reviewed Aug 14, 2023
View reviewed changes
receiver/k8slogreceiver/config.go Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all of the configuration options required for the initial implementation? Can you remove them for now and add them along with the functionality using them going forward?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, starting small will make things easier to review and get started and we can always add them in the future.

TylerHelmuth
TylerHelmuth reviewed Aug 14, 2023
View reviewed changes
Copy link
Member

@TylerHelmuth TylerHelmuth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@h0cheung I am very excited about this component, sorry I didn't see your proposal sooner. @dmitryax thanks for pointing it out.

receiver/k8slogreceiver/config.go Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets not introduce expr filtering to start. The other filter options you've provided are quite extensive. For expression language filtering I'd prefer we use OTTL.

receiver/k8slogreceiver/metadata.yaml Outdated
Copy link
Member

@TylerHelmuth