Skip to content

Prometheus translation modes and content negotiation #4533

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 12 additions & 13 deletions specification/compatibility/prometheus_and_openmetrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,23 +252,22 @@ comments (but not metric points) SHOULD be dropped. If dropping a comment or
metric points, the exporter SHOULD warn the user through error logging.

The Name of an OTLP metric MUST be added as the
[Prometheus Metric Name](https://prometheus.io/docs/instrumenting/exposition_formats/#comments-help-text-and-type-information),
with unit and type suffixes added as described below. The metric name is
required to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Invalid characters
in the metric name MUST be replaced with the `_` character. Multiple
consecutive `_` characters MUST be replaced with a single `_` character.
[Prometheus Metric Name](https://prometheus.io/docs/instrumenting/exposition_formats/#comments-help-text-and-type-information).
Prometheus naming conventions require metric names to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Invalid characters
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Prometheus naming conventions require metric names to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Invalid characters
Prometheus naming conventions encourage metric names to match the regex: `[a-zA-Z_:]([a-zA-Z0-9_:])*`. Discouraged characters

in the metric name SHOULD be replaced with the `_` character, aiming for compatibility with Prometheus conventions. Multiple
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
in the metric name SHOULD be replaced with the `_` character, aiming for compatibility with Prometheus conventions. Multiple
in the metric name SHOULD be replaced with the `_` character by default, aiming for compatibility with Prometheus conventions. Multiple

consecutive `_` characters SHOULD be replaced with a single `_` character.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ywwg do we want to keep this in the spec? Is this what the prometheus/common escaping library does?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is not what the prom/common escaping library does, but it's important to note that that code is newer and may not actually be correct. I think the best option is to figure out what Most People Are Doing and write the spec to match that

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


The Unit of an OTLP metric point SHOULD be converted to the equivalent unit in Prometheus when possible. This includes:
The Unit of an OTLP metric point SHOULD be converted to the equivalent unit in Prometheus when possible. This includes:

* Converting from abbreviations to full words (e.g. "ms" to "milliseconds").
* Dropping the portions of the Unit within brackets (e.g. {packet}). Brackets MUST NOT be included in the resulting unit. A "count of foo" is considered unitless in Prometheus.
* Special case: Converting "1" to "ratio".
* Converting "foo/bar" to "foo_per_bar".

The resulting unit SHOULD be added to the metric as
[UNIT metadata](https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#metricfamily)
and as a suffix to the metric name unless the metric name already ends with the
unit (before type-specific suffixes), or the unit metadata MUST be omitted. The
[UNIT metadata](https://github.com/prometheus/OpenMetrics/blob/v1.0.0/specification/OpenMetrics.md#metricfamily).
A suffix to the metric name MAY be added unless the metric name already ends with the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given we are doing escaping by default above, should we make this a SHOULD? Or
"SHOULD, by default,"?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, in general the vibe of the spec is that people SHOULD do these things, but MAY do something else.

unit (before type-specific suffixes). The
unit suffix comes before any type-specific suffixes.

The description of an OTLP metrics point MUST be added as
Expand Down Expand Up @@ -316,7 +315,7 @@ to a Prometheus Gauge.
- The new data point's start time must match the time of the accumulated data point. If not, see [detecting alignment issues](../metrics/data-model.md#sums-detecting-alignment-issues).
- Otherwise, it MUST be dropped.

If the metric name for monotonic Sum metric points does not end in a suffix of `_total` a suffix of `_total` MUST be added by default, otherwise the name MUST remain unchanged. Exporters SHOULD provide a configuration option to disable the addition of `_total` suffixes.
If the metric name for monotonic Sum metric points does not end in a suffix of `_total` a suffix of `_total` SHOULD be added by default, otherwise the name MUST remain unchanged. Exporters SHOULD provide a configuration option to disable the addition of `_total` suffixes.
Monotonic Sum metric points with `StartTimeUnixNano` should export the `{name}_created` metric as well.

### Histograms
Expand Down Expand Up @@ -389,10 +388,10 @@ OpenTelemetry Metric Attributes MUST be converted to
String Attribute values are converted directly to Metric Attributes, and
non-string Attribute values MUST be converted to string attributes following
the [attribute specification](../common/README.md#attribute). Prometheus
metric label keys are required to match the following regex:
naming conventions required to match the following regex:
`[a-zA-Z_]([a-zA-Z0-9_])*`. Metrics from OpenTelemetry with unsupported
Attribute names MUST replace invalid characters with the `_` character.
Multiple consecutive `_` characters MUST be replaced with a single `_`
Attribute names SHOULD replace invalid characters with the `_` character.
Multiple consecutive `_` characters SHOULD be replaced with a single `_`
character. This may cause ambiguity in scenarios where multiple similar-named
attributes share invalid characters at the same location. In such unlikely
cases, if multiple key-value pairs are converted to have the same Prometheus
Expand Down
55 changes: 51 additions & 4 deletions specification/metrics/sdk_exporters/prometheus.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,14 +63,61 @@ The configuration SHOULD allow the user to select which resource attributes to c
include / exclude or regular expression based). Copied Resource attributes MUST NOT be
excluded from the `target` info metric. The option MAY be named `with_resource_constant_labels`.

A Prometheus Exporter MAY support a configuration option to produce metrics without a [unit suffix](../../compatibility/prometheus_and_openmetrics.md#metric-metadata)
or UNIT metadata. The option MAY be named `without_units`, and MUST be `false` by default.
A Prometheus Exporter MAY support a configuration option that controls the translation of metric names from OpenTelemetry Naming Conventions to [Prometheus Naming conventions](https://prometheus.io/docs/practices/naming/).
If the Prometheus exporter supports such configuration it MUST be named `translation_strategy`, and the translation options MUST be:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should enforce the configuration name with MUST, esp because the without_units configuration option MAY be named that as indicated below.


A Prometheus Exporter MAY support a configuration option to produce metrics without a [type suffix](../../compatibility/prometheus_and_openmetrics.md#metric-metadata).
The option MAY be named `without_type_suffix`, and MUST be `false` by default.
- `UnderscoreEscapingWithSuffixes`, the default. This fully escapes metric names for classic Prometheus metric name compatibility, and includes appending type and unit suffixes.
- `NoUTF8EscapingWithSuffixes` will disable changing special characters to `_`. Special suffixes like units and `_total` for counters will be attached.
- `NoTranslation`. This strategy bypasses all metric and label name translation, passing them through unaltered.
Comment on lines +69 to +71
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prometheus already also has UTF8 with suffixes, we should list that here for permutation completeness


A Prometheus Exporter MAY support a configuration option to produce metrics without the UNIT metadata. The option MAY be named `without_units`, and MUST be `false` by default. Attention: this option isn't related to unit suffixes.

A Prometheus Exporter MAY support a configuration option to produce metrics without a [scope info](../../compatibility/prometheus_and_openmetrics.md#instrumentation-scope-1)
metric, or scope labels. The option MAY be named `without_scope_info`, and MUST be `false` by default.

A Prometheus Exporter MAY support a configuration option to produce metrics without a [target info](../../compatibility/prometheus_and_openmetrics.md#resource-attributes-1)
metric. The option MAY be named `without_target_info`, and MUST be `false` by default.

# Content Negotiation

A Prometheus Exporter MUST support [content negotiation](https://prometheus.io/docs/instrumenting/content_negotiation/) to allow clients to request metrics in different formats based on the `Accept` header in HTTP requests.

## Supported Formats

A Prometheus Exporter SHOULD support the following formats in order of preference, as defined by the [Prometheus content negotiation specification](https://prometheus.io/docs/instrumenting/content_negotiation/):

1. **OpenMetrics Text 1.0.0** (`application/openmetrics-text; version=1.0.0`)
2. **OpenMetrics Text 0.0.1** (`application/openmetrics-text; version=0.0.1`)
3. **Prometheus Text 1.0.0** (`text/plain; version=1.0.0`)
4. **Prometheus Text 0.0.4** (`text/plain; version=0.0.4`) - MUST be supported as the minimum requirement

A Prometheus Exporter MAY additionally support:

- **Prometheus Protobuf** (`application/vnd.google.protobuf; proto=io.prometheus.client.MetricFamily; encoding=delimited`)

## Escaping Schemes

For text formats version 1.0.0 and above, the exporter MUST support [escaping schemes](https://prometheus.io/docs/instrumenting/content_negotiation/#escaping-scheme) as specified in the `Accept` header:

- `allow-utf8` - Allow UTF-8 characters in metric and label names
- `underscores` - Replace invalid characters with underscores
- `dots` - Replace invalid characters with dots
- `values` - Escape invalid characters in values only
Comment on lines +102 to +105
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `allow-utf8` - Allow UTF-8 characters in metric and label names
- `underscores` - Replace invalid characters with underscores
- `dots` - Replace invalid characters with dots
- `values` - Escape invalid characters in values only
- `allow-utf8` - Allow UTF-8 characters in metric and label names
- `underscores` - Replace invalid characters with underscores
- `dots` - Replace dots with `_dot_` and invalid characters with underscores
- `values` - Escape invalid characters with UTF-8 code points as defined in: https://github.com/prometheus/docs/blob/main/docs/instrumenting/escaping_schemes.md#value-encoding-escaping-values

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or some other summary of the correct escaping schemes


## Format Selection

When responding to requests, a Prometheus Exporter MUST:

1. Parse the `Accept` header to determine the client's preferred format and escaping scheme
2. Select the highest-weighted format that the exporter supports
3. Apply the requested escaping scheme if specified for the selected format
4. Respond with the appropriate `Content-Type` header indicating the selected format and escaping scheme

## Interaction with Translation Strategy

Although a Prometheus Exporter MAY be configured with a `translation_strategy` for internal metric processing, the final output format and character escaping MUST follow what the content negotiation process determines based on the client's `Accept` header. The content negotiation requirements take precedence over the configured translation strategy when determining the final output format.

For example:

- If configured with `NoTranslation` but the client requests `escaping=underscores`, the exporter MUST apply underscore escaping
- If configured with `UnderscoreEscapingWithSuffixes` but the client `escaping=allow-utf8`, there's no need to revert what has been translated since the SDK will continue to be compliant.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

love this clarification!

Loading