Skip to content

[receiver/awsfirehosereceiver] Support logs from AWS services sent directly to Firehose #36379

@kaiyan-sheng

Description

@kaiyan-sheng

Component(s)

receiver/awsfirehose

Is your feature request related to a problem? Please describe.

The current AWS Firehose receiver only supports cwlogs record type which is ingesting JSON format logs from CloudWatch. We should also add support for logs sent from services directly to Firehose. For example VPC flow logs can be sent directly to Firehose and our awsfirehosereceiver should be able to process these requests as well.

384160617-bcb3efed-6f4a-45bc-8730-4b96252a0b5d

Describe the solution you'd like

We can introduce a new record_type: firehoselogs to separate the log type from record_type: cwlogs. Since there can be logs from different AWS services send directly to Firehose, the format will be different too. This record_type: firehoselogs will process the request, store the whole record data into message field, store the timestamp and firehose ARN.

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions