[exporter/splunkhec] Use ObservedTimestamp if Timestamp is empty

[KevinThompsonYCRX]

Component(s)

exporter/splunkhec

Is your feature request related to a problem? Please describe.

Currently the Splunk HEC exporter takes the Timestamp field from each log record. If the timestamp field is not provided on the LogRecord, then this is passing 0 today. According to the OTel Spec, the ObservedTimestamp is recommended to be used when the Timestamp field is not provided: https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-observedtimestamp

Splunk adds the ingestion time as a timestamp, but if the service is interrupted or delayed, this can cause issues trying to queue & re-ingest logs.

Describe the solution you'd like

I'd recommend either passing both the Timestamp and ObservedTimestamp, or add the recommended logic to pass the ObservedTimestamp as the Timestamp if Timestamp is not present.

opentelemetry-collector-contrib/exporter/splunkhecexporter/logdata_to_splunk.go

Line 122 in d1f2818

Time: nanoTimestampToEpochMilliseconds(lr.Timestamp()),

Describe alternatives you've considered

Can use a transform processor to get around this for now

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/splunkhec: @atoulme @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[JaredTan95]

Yes, we can following elasticsearchexporter did.

opentelemetry-collector-contrib/exporter/elasticsearchexporter/model.go

Line 564 in 2766731

if record.Timestamp() != 0 {