sentinel_one: persist cursor state across restarts in threat_event and application_risk data streams

[efd6]

Proposed commit message

sentinel_one: persist cursor state across restarts in threat_event and application_risk data streams

The threat_event CEL program stores its two-level worklist
(threats list, page cursors, fetch_more flag) in plain state
keys that are discarded on agent restart. This causes the
program to re-walk the entire SentinelOne threats catalog from
scratch, producing ~390x event duplication on a customer system
running for 29 days after an upgrade restart.

Move worklist state under state.cursor.* so the CEL input
persists it to the registry. Also expose max_executions in the
threat_event manifest so large estates can raise the execution
budget beyond the default 1000.

Apply the same cursor persistence fix to application_risk,
which stores its pagination cursor in state.next.page instead
of state.cursor.*.

This mirrors the fix applied to the application data stream in
v2.7.0 (PR #18976).

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Screenshots

[github-actions]

Elastic Docs Style Checker (Vale)

Summary: 2 suggestions found

💡 Suggestions (2): Optional style improvements. Apply when helpful.

File	Line	Rule	Message
packages/sentinel_one/data_stream/application/manifest.yml	30	Elastic.WordChoice	Consider using 'can, might' instead of 'may', unless the term is in the UI.
packages/sentinel_one/data_stream/threat_event/manifest.yml	31	Elastic.WordChoice	Consider using 'can, might' instead of 'may', unless the term is in the UI.

---

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

Package sentinel_one 👍(11) 💚(1) 💔(6)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
activity	5524.86	3344.48	-2180.38 (-39.46%)	💔
alert	3731.34	3164.56	-566.78 (-15.19%)	💔
application_risk	11961.72	5263.16	-6698.56 (-56%)	💔
threat_event	9746.59	6535.95	-3210.64 (-32.94%)	💔
activity	5524.86	3668.38	-1856.48 (-33.6%)	💔
application	24630.54	18484.29	-6146.25 (-24.95%)	💔

To see the full report comment with /test benchmark fullreport

[infra-vault-gh-plugin-prod]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[vera-review-bot]

Severity: 🟡 Medium confidence: high path: packages/sentinel_one/data_stream/threat_event/manifest.yml:31

max_executions description tells users to 'Set to 0 for unlimited', but the CEL input rejects 0 at config validation (invalid maximum number of executions: 0 <= 0); drop that sentence or point users to a large finite value.

Details

The new max_executions variable description ends with 'Set to 0 for unlimited.' The CEL input's config validation in beats rejects any non-positive value: if c.MaxExecutions != nil && *c.MaxExecutions <= 0 { return fmt.Errorf("invalid maximum number of executions: %d <= 0", ...) }. There is no 'unlimited' sentinel — 0 causes a hard config validation failure, so the input would fail to start rather than run without a cap. A user following this guidance to remove the execution ceiling would break collection for this data stream. The default of 1000 ships fine; the defect is the misleading instruction in the shipped manifest text.

Recommendation:

Remove the false 'unlimited' claim and describe raising the cap to a large finite value instead:

      - name: max_executions
        type: integer
        title: Maximum Executions
        description: >-
          Maximum number of CEL program re-evaluations per collection interval. The threat event stream makes two API calls per worklist entry (threats + events), so large estates may need a higher budget than the input default of 1000. Must be greater than 0; raise it (for example to 10000) if collection is being cut short before a full interval completes.
        default: 1000
        multi: false
        required: true
        show_user: false

---

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

[elastic-vault-github-plugin-prod]

✅ All changelog entries have the correct PR link.

[vera-review-bot]

No issues across the latest commits c8a6c35.

Review summary

Issues found across earlier commits 5bf878c — 1 medium

• 🟡 max_executions description tells users to 'Set to 0 for unlimited', but the CEL input rejects 0 at config validation (invalid maximum number of executions: 0 <= 0) (link) (Unresolved)

A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

[infra-vault-gh-plugin-prod]

💚 Build Succeeded

• Buildkite Build
• Commit: c8a6c35

History

• 💚 Build #45662 succeeded 5bf878c

cc @efd6