[processor/filter] Register programatically custom OTTL funcs in the filter processor. #40948
Description
Component(s)
processor/filter
Is your feature request related to a problem? Please describe.
The Ops Agent (OA in short, https://github.com/GoogleCloudPlatform/ops-agent) uses the filter processor to exclude logs or metrics for user provided features ( e.g. exclude_logs,exclude_metrics) and also for its internal use of open-telemetry pipelines.
To progressively port more OA features to use the Opentelemetry Collector we would need custom "filter" functionality that wouldn't be appropriate (too specific for OA and Google Cloud ) to be contributed in the opentelemetry-collector-contrib repository.
One example of this is a IsMatch ottl func that uses a Ruby Regex engine instead of the Go RE2 engine. This is required to preserve backwards compatibility in the OA.
Describe the solution you'd like
Add factory options With*Functions (e.g. WithLogFunctions) to the NewFactory function ( https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/filterprocessor/factory.go ) that can be used to create a new processor with custom OTTL functions. Also adding export functions Default*Functions (e.g. DefaultLogFunctions) would let users decide which "subset" of the default functions to use. This enables users of the filter processor to customize their OTTL functions as needed.
A very similar solution has already been accepted and implemented in the transformprocessor (see #39698 , #39966).
The factory options would be used like the following example :
NewFactoryWithOptions(
WithLogFunctions(transformprocessor.DefaultLogFunctions()),
WithLogFunctions([]ottl.Factory[ottllog.TransformContext]{FunctionOne, FunctionTwo, ...}),
WithLogFunctions([]ottl.Factory[ottllog.TransformContext]{FunctionThree, FunctionFour, ...}),
)
PR with implemented solution :
Describe alternatives you've considered
1. Adding new fields to filterprocessor Config struct to pass new OTTL functions programmatically
Note : This solution was rejected after the discussion in this thread #39641 (comment).
Add a property to the filter processor Config which enables the user to programatically add new custom OTTL funcs for logs, metrics or traces to the resulting processor.
Some details about the solution :
- This Config properties can only be set programatically.
- This Config properties can not be set through a config file.
2. Adding OTTL funcs through config during collector runtime
After exploring several [1] [2] threads related to this solution, it doesn't seem feasible.
[1] https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/16098
[2] https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/28573
3. Adding all required features to pkg/ottl
Some very specific telemetry proccesing logic may not necessarily be appropriate to be contributed to opentelemetry-collector-contrib.
Additional context
No response