syslog_parser with syslog exporter results in logs like 2023-08-09T06:34:49.490942+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:34:48Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>) #25114
Description
Component(s)
No response
What happened?
Description
I am following the advice at:
- https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/syslogexporter
- https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/stanza/docs/operators/syslog_parser.md
- https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/syslogexporter/examples/config_with_filelog_receiver.yaml
All I want to do, is send a copy of all my logs, to a syslog endpoint. I am using the following config. I am a bit confused by the documentation how to properly configure the syslog_parser, where to configure it, and how to prevent it affecting my logs when going to sumologic.
Note also I am using the chart from https://github.com/SumoLogic/sumologic-kubernetes-collection for sending logs to sumologic, and using values.yml to add additional config:
logs:
enabled: true
logLevel: info
config:
merge:
extensions:
file_storage/syslog:
directory: /var/lib/storage/otc
timeout: 10s
exporters:
#### syslog is in development - https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/syslogexporter/README.md
syslog:
protocol: tcp
port: 515
tls:
insecure: true
endpoint: log.internal
retry_on_failure:
enabled: true
initial_interval: 10s
max_interval: 40s
max_elapsed_time: 150s
sending_queue:
enabled: true
num_consumers: 20
storage: file_storage/syslog
queue_size: 10000
receivers:
otlp/extrafiles:
protocols:
http:
endpoint: 0.0.0.0:4319
#### tried adding this here
operators:
- type: syslog_parser
protocol: rfc5424
service:
extensions:
- health_check
- file_storage
- pprof
- file_storage/syslog
pipelines:
logs/otlp/containers:
exporters:
- sumologic/containers
- syslog
logs/otlp/kubelet:
exporters:
- sumologic/systemd
logs/otlp/systemd:
exporters:
- sumologic/systemd
logs/extrafiles:
receivers: [otlp/extrafiles]
processors:
- memory_limiter
- batch
exporters:
- sumologic/containers
- syslog
This results in my syslog server receiving logs like this:
023-08-09T06:40:15.764305+00:00 ip-10-82-116-123.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:13Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:15.772269+00:00 ip-10-82-116-123.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:14Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:16.223312+00:00 ip-10-82-116-123.ap-southeast-2.compute.internal %!d(<nil>) 1970-01-01T00:00:00Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:16.231277+00:00 ip-10-82-116-123.ap-southeast-2.compute.internal %!d(<nil>) 1970-01-01T00:00:00Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:16.743542+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:13Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:16.748156+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:15Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:16.753788+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:15Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:16.758331+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:15Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:16.765639+00:00 ip-10-82-116-123.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:14Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:17.224420+00:00 ip-10-82-116-123.ap-southeast-2.compute.internal %!d(<nil>) 1970-01-01T00:00:00Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:17.232145+00:00 ip-10-82-116-123.ap-southeast-2.compute.internal %!d(<nil>) 1970-01-01T00:00:00Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:17.240228+00:00 ip-10-82-116-123.ap-southeast-2.compute.internal %!d(<nil>) 1970-01-01T00:00:00Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:17.484981+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 1970-01-01T00:00:00Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:17.490019+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 1970-01-01T00:00:00Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:17.742741+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:14Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:17.747759+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:16Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:17.753258+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:16Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:17.758243+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:16Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
2023-08-09T06:40:17.763719+00:00 ip-10-82-103-213.ap-southeast-2.compute.internal %!d(<nil>) 2023-08-09T06:40:16Z %!s(<nil>) %!s(<nil>) %!s(<nil>) %!s(<nil>) - %!s(<nil>)
And I actually see an error in one of the pods like:
error decoding 'receivers': error reading configuration for "otlp/extrafiles": 1 error(s) decoding:
* '' has invalid keys: operators
Expected Result
Logs are received with proper content,
Actual Result
As above.
Collector version
0.79.0
Environment information
Environment
Kubernetes running on AWS
OpenTelemetry Collector configuration
As aboveLog output
No response
Additional context
No response