[receiver/filelog] File log receiver not parsing the log when restarted #29609
Description
Component(s)
pkg/stanza, receiver/filelog
What happened?
Description
The file log receiver with storage extension is parsing the log data from a file "A". the collector is stopped, more log lines are to a file A. when the collector is restarted the loglines that were written to log A when the collector is in dead state are missing.
Certainly! This test case appears to be testing the behavior of a collector when it is restarted, specifically in the context of a storage extension.
Steps to Reproduce
Test Case reproducible summary, log lines that are written to the file when collector is stopped are not parsed when the collector is restarted
Setup:
Create a unique log stream name.
Start the OpenTelemetry collector with specific configuration and the log stream name.
Create a temporary log file.
Logging:
Write three log messages to the file.
Validation - First Round:
Validate the logs collected by the collector.
Collector Stop:
Stop the collector.
Logging After Stop:
Add more log messages to the file.
Collector Start:
Restart the collector.
Validation - Second Round:
Validate the logs again.
Collector Stop (Cleanup):
Stop the collector.
Expected Result
2023-12-01T09:56:50.594-08:00 | {"body":"First Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
-- | --
| 2023-12-01T09:56:50.862-08:00 | {"body":"Second Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
| 2023-12-01T09:56:50.862-08:00 | {"body":"Third Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
| 2023-12-01T09:56:52.618-08:00 | {"body":"First Message after collector is stopped","attributes":{"log.file.name":"storageExtension.log"}}
| 2023-12-01T09:56:52.769-08:00 | {"body":"Second Message after the collector is stopped","attributes":{"log.file.name":"storageExtension.log"}}
| 2023-12-01T09:56:52.769-08:00 | {"body":"Third Message after the collector is stopped","attributes":{"log.file.name":"storageExtension.log"}}
Actual Result
2023-11-30T12:02:08.228-08:00 | {"body":"First Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
-- | --
| 2023-11-30T12:02:08.476-08:00 | {"body":"Second Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
| 2023-11-30T12:02:08.476-08:00 | {"body":"Third Message, collector is running","attributes":{"log.file.name":"storageExtension.log"}}
Collector version
v0.89.0
Environment information
Environment
OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")
OpenTelemetry Collector configuration
extensions:
file_storage:
directory: /logs/
receivers:
filelog:
include: [/tempLogs/storageExtension.log]
storage: file_storage
exporters:
awscloudwatchlogs:
log_group_name: "adot-testbed/logs-component-testing/logs"
log_stream_name: adot-test-bed
log_retention: 7
service:
extensions: [ file_storage ]
pipelines:
logs:
receivers: [filelog]
exporters: [awscloudwatchlogs]Log output
No response
Additional context
This Behavior is breaking in v0.89.0, it was fine till v0.88.0