New component: OpenSearch Exporter

[MitchellGale]

The purpose and use-cases of the new component

OpenSearch is used for real-time application monitoring, log analytics, website search and more. The OpenSearch exporter would convert OpenTelemetry data directly into OpenSearch allowing logs, trace, metrics to be ingested and searched using OpenSearch. Exporting OpenTelemetry data into OpenSearch will help improve observability.

Many users are currently using OpenSearch as store repository for logs and need a common way and a standard protocol to consolidate both logs and trace signals.

Example configuration for the component

exporters:
  opensearch/trace:
    keyspace: "otel"
    endpoints: [https://localhost:9200]
    traces_index: trace_index
  opensearch/log:
    endpoints: [http://localhost:9200]
    logs_index: my_log_index
    sending_queue:
      enabled: true
      num_consumers: 20
      queue_size: 1000
······
service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [opensearch/log]
    traces:
      receivers: [otlp]
      exporters: [opensearch/trace]
      processors: [batch]

Telemetry data types supported

• Traces
• Logs

Is this a vendor-specific component?

• This is a vendor-specific component
• If this is a vendor-specific component, I am proposing to contribute this as a representative of the vendor.

Sponsor (optional)

Anthony Mirabella (Aneurysm9)

Additional context

Will start contributing with traces initially then follow up with logs.

OpenSearch has recently forked and showed the usage of OpenSearch as a backend store and exploration tools on top of the open telemetry astronomer shop demo using simple schema as the mapping physical store of the OTEL signals

https://github.com/opensearch-project/opentelemetry-demo
https://github.com/opensearch-project/opensearch-catalog/tree/main/schema/observability
https://github.com/opensearch-project/opensearch-catalog/tree/main/docs/schema

I will be directly contributing to this. I am open to becoming a member of OpenTelemetry as progress on this exporter progresses.

[atoulme]

@Aneurysm9 please comment if you are accepting to sponsor, and we can then label this issue as "Accepted component".

[Aneurysm9]

Yes, I am sponsoring this component.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[jaehnri]

@MitchellGale @Aneurysm9, is log support already in progress?

If not, in rancher/opni, we have been using a forked opensearch log exporter for a while now. I'd be glad to contribute it (given the necessary changes, of course).