Prometheus translation modes and content negotiation #4533

ArthurSens · 2025-06-02T21:44:39Z

Related to #4494

Changes

This change focuses on three things:

Flexibilization of underscore scaping
Flexibilization of unit and type suffix additions
Add content negotiation

The goal is to mimic Prometheus' existing configuration that controls the translation of metric names when receiving OTLP metrics via its /otlp endpoint. Configuration alignment will help users migrate in both directions (Prometheus/Collector) when switching their metrics collection systems.

Additional Information

Some further reading might be helpful for reviewers:

Our guide for receiving metrics via OTLP, specifically the UTF-8 section.
Prometheus content-negotiation for UTF-8.

Related issues Update Prometheus spec based on new updates in upstream Prometheus #4494
Links to the prototypes (when adding or changing features)
CHANGELOG.md file updated for non-trivial changes
spec-compliance-matrix.md updated if necessary

…tion Signed-off-by: Arthur Silva Sens <[email protected]>

…ddition Signed-off-by: Arthur Silva Sens <[email protected]>

Signed-off-by: Arthur Silva Sens <[email protected]>

dashpole · 2025-06-05T16:49:45Z

specification/compatibility/prometheus_and_openmetrics.md

-in the metric name MUST be replaced with the `_` character. Multiple
-consecutive `_` characters MUST be replaced with a single `_` character.
+[Prometheus Metric Name](https://prometheus.io/docs/instrumenting/exposition_formats/#comments-help-text-and-type-information).
+Prometheus naming conventions require metric names to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Invalid characters


Suggested change

Prometheus naming conventions require metric names to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Invalid characters

Prometheus naming conventions encourage metric names to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Discouraged characters

dashpole · 2025-06-05T16:50:46Z

specification/compatibility/prometheus_and_openmetrics.md

-consecutive `_` characters MUST be replaced with a single `_` character.
+[Prometheus Metric Name](https://prometheus.io/docs/instrumenting/exposition_formats/#comments-help-text-and-type-information).
+Prometheus naming conventions require metric names to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Invalid characters
+in the metric name SHOULD be replaced with the `_` character, aiming for compatibility with Prometheus conventions. Multiple


Suggested change

in the metric name SHOULD be replaced with the `_` character, aiming for compatibility with Prometheus conventions. Multiple

in the metric name SHOULD be replaced with the `_` character by default, aiming for compatibility with Prometheus conventions. Multiple

dashpole · 2025-06-05T16:51:54Z

specification/compatibility/prometheus_and_openmetrics.md

+[Prometheus Metric Name](https://prometheus.io/docs/instrumenting/exposition_formats/#comments-help-text-and-type-information).
+Prometheus naming conventions require metric names to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Invalid characters
+in the metric name SHOULD be replaced with the `_` character, aiming for compatibility with Prometheus conventions. Multiple
+consecutive `_` characters SHOULD be replaced with a single `_` character.


@ywwg do we want to keep this in the spec? Is this what the prometheus/common escaping library does?

I believe this is not what the prom/common escaping library does, but it's important to note that that code is newer and may not actually be correct. I think the best option is to figure out what Most People Are Doing and write the spec to match that

see: open-telemetry/opentelemetry-collector-contrib#35244

dashpole · 2025-06-05T16:53:44Z

specification/compatibility/prometheus_and_openmetrics.md

-and as a suffix to the metric name unless the metric name already ends with the
-unit (before type-specific suffixes), or the unit metadata MUST be omitted. The
+[UNIT metadata](https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#metricfamily).
+A suffix to the metric name MAY be added unless the metric name already ends with the


Given we are doing escaping by default above, should we make this a SHOULD? Or
"SHOULD, by default,"?

I agree, in general the vibe of the spec is that people SHOULD do these things, but MAY do something else.

ywwg · 2025-06-10T17:40:17Z

specification/metrics/sdk_exporters/prometheus.md

+- `UnderscoreEscapingWithSuffixes`, the default. This fully escapes metric names for classic Prometheus metric name compatibility, and includes appending type and unit suffixes.
+- `NoUTF8EscapingWithSuffixes` will disable changing special characters to `_`. Special suffixes like units and `_total` for counters will be attached.
+- `NoTranslation`. This strategy bypasses all metric and label name translation, passing them through unaltered.


Prometheus already also has UTF8 with suffixes, we should list that here for permutation completeness

ywwg · 2025-06-10T17:41:12Z

specification/metrics/sdk_exporters/prometheus.md

-A Prometheus Exporter MAY support a configuration option to produce metrics without a [unit suffix](../../compatibility/prometheus_and_openmetrics.md#metric-metadata)
-or UNIT metadata. The option MAY be named `without_units`, and MUST be `false` by default.
+A Prometheus Exporter MAY support a configuration option that controls the translation of metric names from OpenTelemetry Naming Conventions to [Prometheus Naming conventions](https://prometheus.io/docs/practices/naming/).
+If the Prometheus exporter supports such configuration it MUST be named `translation_strategy`, and the translation options MUST be:


I don't think we should enforce the configuration name with MUST, esp because the without_units configuration option MAY be named that as indicated below.

ywwg · 2025-06-10T17:42:52Z

specification/metrics/sdk_exporters/prometheus.md

+- `allow-utf8` - Allow UTF-8 characters in metric and label names
+- `underscores` - Replace invalid characters with underscores  
+- `dots` - Replace invalid characters with dots
+- `values` - Escape invalid characters in values only


Suggested change

- `allow-utf8` - Allow UTF-8 characters in metric and label names

- `underscores` - Replace invalid characters with underscores

- `dots` - Replace invalid characters with dots

- `values` - Escape invalid characters in values only

- `allow-utf8` - Allow UTF-8 characters in metric and label names

- `underscores` - Replace invalid characters with underscores

- `dots` - Replace dots with `_dot_` and invalid characters with underscores

- `values` - Escape invalid characters with UTF-8 code points as defined in: https://github.com/prometheus/docs/blob/main/docs/instrumenting/escaping_schemes.md#value-encoding-escaping-values

or some other summary of the correct escaping schemes

ywwg · 2025-06-10T17:45:53Z

specification/metrics/sdk_exporters/prometheus.md

+For example:
+
+- If configured with `NoTranslation` but the client requests `escaping=underscores`, the exporter MUST apply underscore escaping
+- If configured with `UnderscoreEscapingWithSuffixes` but the client `escaping=allow-utf8`, there's no need to revert what has been translated since the SDK will continue to be compliant.


love this clarification!

ArthurSens · 2025-06-11T13:04:15Z

Folks, I'm out until June 25th. @ywwg is picking this up while I'm gone :)

I've removed the draft mode to help Owen with his notification settings

github-actions · 2025-06-19T03:33:18Z

This PR was marked stale due to lack of activity. It will be closed in 7 days.

ywwg · 2025-06-20T14:03:21Z

not stale

ywwg · 2025-06-20T14:03:38Z

(it won't let me remove the stale label)

ArthurSens added 4 commits June 2, 2025 18:22

sdk_exporters/prometheus: Add 'translation_strategy' configuration op…

588bc25

…tion Signed-off-by: Arthur Silva Sens <[email protected]>

compatibility/prometheus: Flexibilize underscore scaping and suffix a…

91a7c40

…ddition Signed-off-by: Arthur Silva Sens <[email protected]>

sdk_exporters/prometheus: Add content negotiation

2798220

Signed-off-by: Arthur Silva Sens <[email protected]>

Lint

f2a4b6e

Signed-off-by: Arthur Silva Sens <[email protected]>

ArthurSens mentioned this pull request Jun 4, 2025

sdk_exporters/prometheus: Clarify Content Negotiation #4543

Open

4 tasks

dashpole reviewed Jun 5, 2025

View reviewed changes

ywwg reviewed Jun 10, 2025

View reviewed changes

ArthurSens marked this pull request as ready for review June 11, 2025 13:03

ArthurSens requested review from a team as code owners June 11, 2025 13:03

github-actions bot added the Stale label Jun 19, 2025

	Prometheus naming conventions require metric names to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Invalid characters
	Prometheus naming conventions encourage metric names to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Discouraged characters

	in the metric name SHOULD be replaced with the `_` character, aiming for compatibility with Prometheus conventions. Multiple
	in the metric name SHOULD be replaced with the `_` character by default, aiming for compatibility with Prometheus conventions. Multiple

Prometheus translation modes and content negotiation #4533

Are you sure you want to change the base?

Prometheus translation modes and content negotiation #4533

Conversation

ArthurSens commented Jun 2, 2025

Changes

Additional Information

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArthurSens commented Jun 11, 2025

Uh oh!

github-actions bot commented Jun 19, 2025

Uh oh!

ywwg commented Jun 20, 2025

Uh oh!

ywwg commented Jun 20, 2025

Uh oh!

Uh oh!