[pkg/ottl] Allow functions to identify literal value getters

[edmocosta]

Component(s)

pkg/ottl

Is your feature request related to a problem? Please describe.

It's currently not possible for functions to identify whether a Getter argument is holding a literal value (ottl.literal), or any other getter type that does not depends on the transformation context to be evaluated.

Not knowing if the argument is a literal forces expensive functions like the IsMatch to only accept static string patterns, so it does not suffer the performance penalty compiling the regex for every single execution. This is a good standard that we probably want to keep, but some functions lacks dynamicity because of that.

Describe the solution you'd like

Providing that information to functions would allow expensive operations such as compiling a regex to continue to be executed once (if it's a literal), but would also allow functions to offer dynamic values support.

Describe alternatives you've considered

No response

Additional context

#39997

[github-actions]

Pinging code owners:

• pkg/ottl: @TylerHelmuth @evan-bradley @edmocosta

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[bacherfl]

(Triage): Removing needs-triage as the issue is formulated clearly and was written by code owner.

[bacherfl]

If this is still available, I would like to look into it, as this might also be useful to have for #37280

[bacherfl]

@edmocosta I was looking into this to see how this can be done, and would like to get your feedback before I continue with the implementation.

My thought was that, in the Parser.buildArgs() function we could check for a new argType, e.g. called LiteralGetter - if we have that, we can use a newLiteralGetter to create the getter, and check if we can retrieve the value with an empty TransformContext. If that is the case, we can return the getter, otherwise we error:

// ...
case strings.HasPrefix(name, "LiteralGetter"):
		getter, err := p.newLiteralGetter(argVal)
		if err != nil {
			return nil, err
		}
		var zero K // please see note below about this
		v, err := getter.Get(context.Background(), zero)
		if err != nil {
			return nil, err
		}
		return StandardLiteralGetter[K]{literal[K]{value: v}}, nil
// ...

The newLiteralGetter function would work similar as the newGetter function , with the difference that it will return an error when it detects the given value to contain a Path, Converter, or MathExpression:

func (p *Parser[K]) newLiteralGetter(val value) (Getter[K], error) {
	if val.Literal != nil {
		if val.Literal.Path != nil {
			return nil, errors.New("literal getter must not use path expressions")
		}
		if val.Literal.Converter != nil {
			return nil, errors.New("literal getter must not use converter expressions")
		}
	}
	if val.MathExpression != nil {
		return nil, errors.New("literal getter must not use math expressions")
	}
        // the other getter types are created in the same way as in newGetter 
       // for slice and map elements we again call newLiteralGetter to make sure we do not include nested path/math/converter getters
}

Note: even though this is avoided by returning only ottl.literal based getters in the newLiteralGetter function, we need to be careful to not introduce any case where we end up trying to call a function of a TransformContext instance when invoking the getter.Get() function with a zero value, as this can lead to panics

[edmocosta]

Hi @bacherfl, I'm wondering if we can find a solution that would allow us identifying whether an existing getter holds a literal value, without necessarily having to implement a custom getter for that, neither brute forcing it by trying to execute the Get without a transformation context. The reason behind that is to continue leveraging the existing custom type implementations, but with some additional literal support.

An idea for that would be have something similar to the ottl.Optional, but for adding literals support to existing getters, without having to break the getters API, or introducing many exported interfaces for that, something like:

type IsMatchArguments[K any] struct {
	Target  ottl.StringLikeGetter[K]
	Pattern ottl.LiteralGetterExtension[ottl.StringLikeGetter[K]]
}

So we could use it within the function as:

var literalRegex *regexp.Regex
if arg.Pattern.IsLiteral() {
   if pattern,   err := arg.Pattern.GetLiteral() {
      literalRegex = regexp.Compile(pattern)
   }
}

return func(...) {
    var regex *regexp.Regex
    if (literalRegex == nil) {
         pattern, _ := arg.Pattern.Getter.Get(ctx, tCtx)
         regex = ...
    } else {
         regex = literalRegex
    }
}

To make it possible, we would need to have/implement some private interface for getters that support literals, something like the following. Custom types could leverage those underlying getters to also add support to it.

type literalGetter interface {
	isLiteral() bool
	getLiteral() (any, error)
}

That's just an additional idea to discuss, if folks think it's a good solution, we can do a proof of concept for that, but I think everything I mentioned here is feasible.

@TylerHelmuth @evan-bradley, do you have any suggestions?

[bacherfl]

Thank you for the feedback, @edmocosta - from a usage perspective, having something like a LiteralGetterExtension makes sense. However I'm not quite sure how the implementation of IsLiteral() for something like a mapGetter could look like - Do you have some ideas or pointers towards how the implementation for this could look like?

[edmocosta]

Thank you for the feedback, @edmocosta - from a usage perspective, having something like a LiteralGetterExtension makes sense. However I'm not quite sure how the implementation of IsLiteral() for something like a mapGetter could look like - Do you have some ideas or pointers towards how the implementation for this could look like?

Yes, I do have some ideas for that, but I'm still need to validate them. Let's see what Tyler and Evan says about this suggestion. If they agree, I can test them out and share some rough diffs here with you, so you don't waste your time trying to implement something that might only work in my head :)