Dynamic exporter config

[lopezchr]

Component(s)

No response

Describe the issue you're reporting

I need to config an exporter dynamically using some variables created on previews pipeline step. Specifically, I'm working on passing logs that have to be distribute on different streams on AWS cloudwatch. the exporter needs to configure the stream that will be used but, in my case, it has to be dynamic because it depends to the pod information from the source. I've tried with attributes processor to generate the variables and then use these on the exporter, but it didn't work:

processors:
  attributes/logs:
      actions:
        -  key: log_stream_name
            value: "${service.version}"
            action: upsert

exporters:
  awscloudwatchlogs:
    region: us-east-1
    log_group_name: "/eks-logs/default-group"
    log_stream_name: "${log_stream_name}"

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [attributes/logs:]
      exporters: [awscloudwatchlogs]

At the moment of initialization, the service uses this configuration as literal but not dynamically. What can I do?

[mx-psi]

@lopezchr You may be interested in checking the routing connector and having multiple exporters for the different streams

[lopezchr]

In my case, the stream has to be variable and depends on the source (podId). I need a stream for each pod that will be generated over time.

[mx-psi]

Alright, this is something that I would try first to solve then by enhancing the awscloudwatchlogs exporter, so I am going to transfer this to opentelemetry-collector-contrib and label it appropriately

[github-actions]

Pinging code owners for exporter/awscloudwatchlogs: @boostchicken @rapphil. See Adding Labels via Comments if you do not have permissions to add labels yourself. For example, comment '/label priority:p2 -needs-triaged' to set the priority and remove the needs-triaged label.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/awscloudwatchlogsexporter: @boostchicken @rapphil

See Adding Labels via Comments if you do not have permissions to add labels yourself.