Decision logging currently disabled

[maltesander]

In the OPA server config.yaml we can activate logging decisions to the console (see https://www.openpolicyagent.org/docs/latest/configuration/#decision-logs), which in turn would be picked up by vector.

services:
  - name: stackable
    url: http://localhost:3030/opa/v1

bundles:
  stackable:
    service: stackable
    resource: opa/bundle.tar.gz
    persist: true
    polling:
      min_delay_seconds: 10
      max_delay_seconds: 20

decision_logs:
    console: true

We must make this configurable via the CRD.

Proposal

Handling decision logging configuration with the current logging solution described here.

• There is already a decision logger implemented for decision logs, which can be utilized here to specify the log level for the decision logs. Log level NONE for no decision logs at all and any other level for enabling them.
• Specifying the log level of the appenders (console or file) can control whether the decision logs would be visible on console, file, none or both.

Example:

logging:                   
    enableVectorAgent: true  
    containers:              
      main-container:
        console:             
          level: INFO # Default: INFO
        file:
          level: INFO # Default: INFO
        loggers:             
          ROOT:
            level: INFO # Default: INFO
          decision:
            level: INFO # Default: NONE
          server:
            level: WARN # Default: null

This would add the decision logs both to console and file. Additionally server logs are printed there as well.

The difference to the prior solution suggestion is that the different logging configurations (decisionLogging and logging), which might affect each other (for example setting decisionLogging.console to true but logging.main-container.console.level to NONE), are less apart and part of the same overall concept, which might reduce confusion when configuring logging. Furthermore, there would be no CRD change in the scope of this issue necessary.

Implementation Considerations

• OPA is printing logs in a json format including a level field for log level. Filtering logs only with grep is not sufficient. It would be better to parse and filter with jq. For this we would add a script/tool to the OPA image (docker-images/opa/stackable/bin) -> jq needs to be added to the docker image
• Default values suggestion: Everything set to INFO and decision logging to NONE

Workaround

1. Stop reconciling the OpaCluster using `spec.clusterOperation.reconciliationPaused: true
2. Manually edit the Configmap containing the OPA config and add

decision_logs:
    console: true

3. Restart OPA Pods

• The end-to-end-security demo is updated (in case it makes sense)
• Adjust Trino integration-test

### Tasks
- [ ] https://github.com/stackabletech/docker-images/pull/695
- [ ] https://github.com/stackabletech/opa-operator/pull/555

[sbernauer]

Update: All the consumers (druid and trino opa authorizer) can now handle this.
Question is if we want to log audit (decision) logs to stdout or if we want to consider this sensitive information. In this case we e.g. want to log to a file and collect that using vector.
On the other hand when developing rego rules you need that log, stdout would be convenient in that case

[fhennig]

Maybe there could be a switch for file/stdout? I think both use cases are very valid.

[xeniape]

Refinement Notes

---

Suggested Steps:

• Add configuration parameter to CRD
• The customer needs to know, that sensible information might be printed to the console, when choosing console as output. Add a warning in the documentation.
• Extend commons-operator to restart DaemonSets on ConfigMap changes. Set according label (restarter.stackable.tech/enabled) on DaemonSet of OpaCluster.

CRD-proposal:

spec:
  servers:
    config: # also in .spec.servers.roleGroups.*.config.decisionLogging
      decisionLogging: # additional field since logging is used for all products
        console: true/false
        url: ... # not part of the ticket, see 'insights into alternatives' for more details

Insights into alternatives:

• Currently it seems not possible to route the decision logs to a file, see related Issue here: Add support for file-based decision logger open-policy-agent/opa#2605
• It should be possible to forward the decision logs to a remote server (using the service field). It probably should also be possible to forward them to the vector agent, but then there would be no output to stderr using this method and a pipe to a syslog-Logger would not work. Syslog-Logger-Piping is out of scope for this issue.
• One can set both console and service (remote server) output for decision logging. That's why the CRD fields were chosen like above for possible future extensions.

Security Considerations:

• Username, group, and decision result are part of the response -> If the response returns "result: true", one could infer user permissions from the logs.

Other:

• The request id req_id is continuously incremented across server and decision logs. Meaning, it wouldn't be enough to just send decision logs to SIEM to check for deleted logs.