Skip to content

[exporter/prometheusremotewrite] Prometheus remote write exporter retry on HTTP 429 #31032

Open
@Bruno-DaSilva

Description

@Bruno-DaSilva

Component(s)

exporter/prometheusremotewrite

Is your feature request related to a problem? Please describe.

Prometheus remote write exporter does not currently retry on HTTP 429s TooManyRequests. Many remote write destinations implement 429 and it's not great if we're dropping metrics because we're being temporarily rate limited.

For context, the remotewrite spec says:

Prometheus Remote Write compatible senders MUST retry write requests on HTTP 5xx responses and MUST use a backoff algorithm to prevent overwhelming the server. They MUST NOT retry write requests on HTTP 2xx and 4xx responses other than 429. They MAY retry on HTTP 429 responses, which could result in senders "falling behind" if the server cannot keep up. This is done to ensure data is not lost when there are server side errors, and progress is made when there are client side errors.

Describe the solution you'd like

The prometheus reference implementation here retries on 429:
https://github.com/prometheus/prometheus/blob/main/storage/remote/client.go#L239-L242
It also uses the 'retry after duration' header to modulate the retry backoff -- not sure how best to handle that in the otel implementation.

The current implementation retries on 500, so I figure we could just add a 429 check to this same if statement, at minimum.
https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/prometheusremotewriteexporter/exporter.go#L270-L273

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions