Description
Component(s)
processor/transform
What happened?
Description
I'm using filelog to collect several logs using the pattern F:\path\\LOGS\\*.log as there are different folders and log files.
Path can be for eg F:\path\AA11BB\ or F:\path\BB22CC\ etc
In the processing I'm using transform trying to set some attributes based on the log file path
set(log.attributes["folder"], ExtractPatterns(log.attributes["log.file.path"], "(?P[a-zA-Z]{2}\d{2}[a-zA-Z]{2})"))
set(resource.attributes["service.name"], Concat(["my-logs", ExtractPatterns(log.attributes["log.file.path"], "(?P[a-zA-Z]{2}\d{2}[a-zA-Z]{2})")["folder"]], "-"))
This works but it also produces some wrong results.
2025-04-02 01:01:42.566
{
"body": "10:23:49.528946|00054-Service started",
"severity": "INFO",
"attributes": {
"log.file.name": “log1_14568_24-03.25.log",
"log.file.path": “F:\path\AA11BB\LOGS\24-03.25_10.30.11_Pid_8824\log1_14568_24-03.25.log",
"folder": {
"folder": "AA11BB"
}
},
"resources": {
"code": "909090",
"service.environment": "dev",
"service.name": “my-logs-BB22CC”
}
}
the log.file.path is f:\path\AA11BB... but the service name is using the BB22CC value. It looks like it's taking an old value of the folder group name. However the folder attribute is allways correct which makes me think it's the concat operation that does something wrong.
I can see this behavior repeating many times in row... and then some correct values .. and then again a lot of wrong values
Collector version
2.12.0.0
Environment information
Environment
Windows 2019
OpenTelemetry Collector configuration
extensions:
health_check:
# Receivers
receivers:
filelog:
include:
- F:\\path\\*\\LOGS\\*\\*.log
include_file_path: true
# Processors
processors:
batch:
timeout: 1s
resource/filelog:
attributes:
- key: code
value: 909090
action: insert
- key: service.environment
value: dev
action: insert
transform/an:
error_mode: ignore
log_statements:
- set(log.attributes["folder"], ExtractPatterns(log.attributes["log.file.path"], "(?P<folder>[a-zA-Z]{2}\\d{2}[a-zA-Z]{2})"))
- set(log.severity_text, "INFO")
- set(resource.attributes["service.name"], Concat(["my-logs", ExtractPatterns(log.attributes["log.file.path"], "(?P<folder>[a-zA-Z]{2}\\d{2}[a-zA-Z]{2})")["folder"]], "-"))
exporters:
debug:
verbosity: detailed
kafka:
protocol_version: 2.0.0
encoda: otlp_proto
topic: "my_topic"
producer:
compression: lz4
brokers:
- fsa:ddd
service:
pipelines:
logs/filelog:
receivers: [ filelog ]
processors: [ resource/filelog,transform/an,batch ]
exporters: [ kafka ]
extensions: [ health_check ]Log output
Additional context
No response