Skip to content

[processor/k8sattributes] Deprecate FieldExtractConfig.Regex config option #33411

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Jul 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .chloggen/chore_25128_regex.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: deprecation

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: k8sattributesprocessor

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Deprecate `extract.annotations.regex` and `extract.labels.regex` config fields in favor of the `ExtractPatterns` function in the transform processor. The `FieldExtractConfig.Regex` parameter will be removed in version v0.111.0.

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [25128]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext: Deprecating of FieldExtractConfig.Regex parameter means that it is recommended to use the `ExtractPatterns` function from the transform processor instead. To convert your current configuration please check the `ExtractPatterns` function [documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs#extractpatterns). You should use the `pattern` parameter of `ExtractPatterns` instead of using the `FieldExtractConfig.Regex` parameter.

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: [user]
48 changes: 48 additions & 0 deletions processor/k8sattributesprocessor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -332,3 +332,51 @@ as tags.

By default, the `k8s.pod.start_time` uses [Time.MarshalText()](https://pkg.go.dev/time#Time.MarshalText) to format the
timestamp value as an RFC3339 compliant timestamp.

## Feature Gate

### `k8sattr.fieldExtractConfigRegex.disallow`

The `k8sattr.fieldExtractConfigRegex.disallow` [feature gate](https://github.com/open-telemetry/opentelemetry-collector/blob/main/featuregate/README.md#collector-feature-gates) disallows the usage of the `extract.annotations.regex` and `extract.labels.regex` fields.
The validation performed on the configuration will fail, if at least one of the parameters is set (non-empty) and `k8sattr.fieldExtractConfigRegex.disallow` is set to `true` (default `false`).

#### Example Usage

The following config with the feature gate set will lead to validation error:

`config.yaml`:

```yaml
extract:
labels:
regex: <my-regex1>
annotations:
regex: <my-regex2>
```

Run collector: `./otelcol --config config.yaml --feature-gates=k8sattr.fieldExtractConfigRegex.disallow`

#### Migration

Deprecation of the `extract.annotations.regex` and `extract.labels.regex` fields means that it is recommended to use the `ExtractPatterns` function from the transform processor instead. To convert your current configuration please check the `ExtractPatterns` function [documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs#extractpatterns). You should use the `pattern` parameter of `ExtractPatterns` instead of using the the `extract.annotations.regex` and `extract.labels.regex` fields.

##### Example

The following configuration of `k8sattributes processor`:

`config.yaml`:

```yaml
annotations:
- tag_name: a2 # extracts value of annotation with key `annotation2` with regexp and inserts it as a tag with key `a2`
key: annotation2
regex: field=(?P<value>.+)
from: pod
```
can be converted with the usage of `ExtractPatterns` function:

```yaml
- set(cache["annotations"], ExtractPatterns(attributes["k8s.pod.annotations["annotation2"], "field=(?P<value>.+))")
- set(k8s.pod.annotations["a2"], cache["annotations"]["value"])
```
13 changes: 13 additions & 0 deletions processor/k8sattributesprocessor/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,20 @@ import (
"fmt"
"regexp"

"go.opentelemetry.io/collector/featuregate"
conventions "go.opentelemetry.io/collector/semconv/v1.6.1"

"github.com/open-telemetry/opentelemetry-collector-contrib/internal/k8sconfig"
"github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor/internal/kube"
)

var disallowFieldExtractConfigRegex = featuregate.GlobalRegistry().MustRegister(
"k8sattr.fieldExtractConfigRegex.disallow",
featuregate.StageAlpha,
featuregate.WithRegisterDescription("When enabled, usage of the FieldExtractConfig.Regex field is disallowed"),
featuregate.WithRegisterFromVersion("v0.106.0"),
)

// Config defines configuration for k8s attributes processor.
type Config struct {
k8sconfig.APIConfig `mapstructure:",squash"`
Expand Down Expand Up @@ -63,6 +71,9 @@ func (cfg *Config) Validate() error {
}

if f.Regex != "" {
if disallowFieldExtractConfigRegex.IsEnabled() {
return fmt.Errorf("the extract.annotations.regex and extract.labels.regex fields have been deprecated, please use the `ExtractPatterns` function in the transform processor instead")
}
r, err := regexp.Compile(f.Regex)
if err != nil {
return err
Expand Down Expand Up @@ -213,6 +224,8 @@ type FieldExtractConfig struct {
// regex: JENKINS=(?P<value>[\w]+)
//
// this will add the `git.sha` and `ci.build` resource attributes.
// Deprecated: [v0.106.0] Use the `ExtractPatterns` function in the transform processor instead.
// More information about how to replace the regex field can be found in the k8sattributes processor readme.
Regex string `mapstructure:"regex"`

// From represents the source of the labels/annotations.
Expand Down
38 changes: 36 additions & 2 deletions processor/k8sattributesprocessor/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/confmap/confmaptest"
"go.opentelemetry.io/collector/featuregate"

"github.com/open-telemetry/opentelemetry-collector-contrib/internal/k8sconfig"
"github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor/internal/kube"
Expand All @@ -21,8 +22,9 @@ func TestLoadConfig(t *testing.T) {
t.Parallel()

tests := []struct {
id component.ID
expected component.Config
id component.ID
expected component.Config
disallowRegex bool
}{
{
id: component.NewID(metadata.Type),
Expand Down Expand Up @@ -127,9 +129,35 @@ func TestLoadConfig(t *testing.T) {
},
},
},
{
id: component.NewIDWithName(metadata.Type, "deprecated-regex"),
expected: &Config{
APIConfig: k8sconfig.APIConfig{AuthType: k8sconfig.AuthTypeKubeConfig},
Passthrough: false,
Extract: ExtractConfig{
Metadata: enabledAttributes(),
Annotations: []FieldExtractConfig{
{Regex: "field=(?P<value>.+)", From: "pod"},
},
Labels: []FieldExtractConfig{
{Regex: "field=(?P<value>.+)", From: "pod"},
},
},
Exclude: ExcludeConfig{
Pods: []ExcludePodConfig{
{Name: "jaeger-agent"},
{Name: "jaeger-collector"},
},
},
},
},
{
id: component.NewIDWithName(metadata.Type, "too_many_sources"),
},
{
id: component.NewIDWithName(metadata.Type, "deprecated-regex"),
disallowRegex: true,
},
{
id: component.NewIDWithName(metadata.Type, "bad_keys_labels"),
},
Expand Down Expand Up @@ -176,6 +204,12 @@ func TestLoadConfig(t *testing.T) {

for _, tt := range tests {
t.Run(tt.id.String(), func(t *testing.T) {
if tt.disallowRegex {
require.Nil(t, featuregate.GlobalRegistry().Set(disallowFieldExtractConfigRegex.ID(), true))
t.Cleanup(func() {
require.Nil(t, featuregate.GlobalRegistry().Set(disallowFieldExtractConfigRegex.ID(), false))
})
}
cm, err := confmaptest.LoadConf(filepath.Join("testdata", "config.yaml"))
require.NoError(t, err)

Expand Down
11 changes: 11 additions & 0 deletions processor/k8sattributesprocessor/testdata/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,17 @@ k8sattributes/too_many_sources:
- from: connection
name: ip

k8sattributes/deprecated-regex:
passthrough: false
auth_type: "kubeConfig"
extract:
labels:
- regex: field=(?P<value>.+)
from: pod
annotations:
- regex: field=(?P<value>.+)
from: pod

k8sattributes/bad_keys_labels:
extract:
labels:
Expand Down