Emit outcome: failure
in obsconsumer
#13234
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The last remaining part of #12676 is to implement the
outcome: failure
part of the Pipeline Component Telemetry RFC (see here). This is done by introducing adownstream
error wrapper struct, to distinguish between errors coming from the next component from errors bubbled from further downstream.Important note
This PR implements things slightly differently from what the text of the RFC describes.
If a pipeline contains components
A → B
and an error occurs in B, this PR records:otelcol.component.outcome = failure
in theotelcol.*.consumed.*
metric for Botelcol.component.outcome = refused
in theotelcol.*.produced.*
metric for Awhereas the RFC would set both
outcome
s tofailure
.This is programmatically simpler — no need to have different behavior between the
obsconsumer
around the output of A and the one around the input of B — but more importantly, I think it is clearer for users as well:outcome = failure
only occurs on metrics associated with the component where the failure actually occurred.This subtlety wasn't discussed in-depth in #11956 which introduced
outcome = refused
, so I took the liberty to make this change. If necessary, I can file another RFC amendment to match, or, if there are objections, implement the RFC as-is instead.Link to tracking issue
Fixes #12676
Testing
I've updated the existing tests in
obsconsumer
to expect adownstream
-wrapped error to exit theobsconsumer
layer. I may add more tests later.Documentation
None.