Missing attributes in internal Collector logs

[jade-guiton-dd]

Description

In versions 0.123.0 and 0.124.0 of the Collector, there has been a regression in the attributes included in internal Collector logs. Specifically, the otelcol. attributes defined in the Pipeline Component Telemetry RFC, which have been included in said logs since version 0.120.0 of the Collector, are now missing by default.

Reproduction

This can be reproduced by starting a Collector at version 0.122.1 with a debug exporter in one of the pipelines.

You should see a log like this:

2025-04-16T11:35:48.035+0200    info    builders/builders.go:26 Development component. May change in the future.        {"otelcol.component.id": "debug", "otelcol.component.kind": "Exporter", "otelcol.signal": "logs"}

By contrast, in versions 0.123.0 or 0.124.0, with no feature gates set, you will see the following:

2025-04-16T11:36:08.767+0200    info    builders/builders.go:26 Development component. May change in the future.

In this case, information is missing to identify which component is responsible for the log.

The attributes are missing no matter how the logs are emitted; both in standard error output like above, and when exporting the Collector's logs using service::telemetry::logs::processors, as described in the documentation.

Cause

This regression was introduced by PR #12617, whose primary goal was to extend these component attributes to internal metrics and traces.

In cases where logs are exported through service::telemetry::logs::processors, it also switched the component attributes in internal logs from datapoint attributes to instrumentation scope attributes, to fit with the other two signals and reduce data redundancy. This is a breaking change, but note that at this time, we make no stability guarantees on the format of the Collector's internal logs, so please avoid relying on them.

The issue is that the changes in this PR were hastily put behind an alpha (off by default) feature gate, telemetry.newPipelineTelemetry. The result was that the "off" behavior was to omit the component attributes altogether in all circumstances.

Workarounds

• If you are gathering the Collector's logs from terminal output / standard error:

  Enabling the telemetry.newPipelineTelemetry feature gate should restore the missing attributes to the log output. This can be done by running the Collector with the --feature-gates telemetry.newPipelineTelemetry command-line argument.

• If you are exporting the Collector's logs through OTLP using service::telemetry::logs::processors:

  • If you export to an endpoint which supports instrumentation scope attributes:

    Enabling the feature gate will restore the missing attributes, but as instrumentation scope attributes instead of standard log attributes.
    Please check with your observability vendor on whether they support ingestion of instrumentation scope attributes.

  • Otherwise:

    We unfortunately do not have a good workaround for this case. Here are some options:

    • Downgrading the Collector to version 0.122.1;
    • Switching to a different log gathering method, for example collecting logs from standard error, which is a feature of many observability vendors;
    • For some use cases, the file and line of code (code.filepath, code.function, and code.lineno attributes) which are still present on the logs may be sufficient to identify which component a log originates from.

    Note that the behavior behind the feature gate, ie. using instrumentation scope attributes in internal logs to identify components, will eventually become the default, so exporting to an endpoint with no support for scope attributes will become problematic.

[jade-guiton-dd]

Because of this problematic "off" behavior of the feature gate, @djaglowski and I plan to remove it, and the changes to internal telemetry will be enabled permanently; see PR #12856.

[sfc-gh-bdrutu]

The attributes are also missing when exporting the Collector's logs using service::telemetry::logs::processors, as described in the documentation.

Please rephrase this to: attributes are missing all the time.

Downgrading the Collector to version 0.122.1, or upgrading it once the regression has been fixed;

Upgrading to what?

[jade-guiton-dd]

Please rephrase this to: attributes are missing all the time.

I tried to make it clearer that the attributes are missing no matter how they are emitted.

Upgrading to what?

I removed the mention of upgrading, since it seems to have been decided that we will move forward with enabling the feature by default, which will not help with the "export through otlp but no scope attribute support" case.

[jade-guiton-dd]

This bug was fixed in #12856, closing.

For any future readers: the current state after v0.125.0 is that the component-identifying attributes are present in terminal output, as well as in OTLP-exported logs, regardless of the state of the telemetry.newPipelineTelemetry feature gate. However, in the latter case, they are exposed as instrumentation scope attributes, so users using an exporter/endpoint with no support for such attributes may not see them. We consider this a missing feature in the exporter/endpoint, which can be worked around by gathering the Collector's terminal output instead, or by further processing the logs to turn the instrumentation scope attributes into log record attributes.