Skip to content

Support complete file ingestion on update #39854

Open
@atoulme

Description

@atoulme

Component(s)

No response

Is your feature request related to a problem? Please describe.

We want to watch a file, and ingest all its contents on any modification of the file. The receiver should always keep the modification stamp of the file and compare it regularly to the file mstamp, or watch for file changes. Whenever this happens, the receiver should then read the entirety of the file and send it as logs.

Files watched would be configuration files of critical services, where all information must be logged for audit purposes.

Describe the solution you'd like

This is an exploratory issue that may result in a new component or an addition to an existing component.

The filelogreceiver may look like a possible approach to tackling this problem, however, it is not at all geared towards this use case. It is likely the stanza library has all the elements necessary to perform the work.

It is unclear if you actually need to persist modification timestamps - a first version can just assume on start that reading all files and send their contents is ok.

We should also set some sane limits, such that past a max size, the receiver should raise an error instead of accepting to read the file. That limit can be set by the user. A reasonable limit is 10K bytes.

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions