[Osquery_manager] Add pack_name and query_name fields for scheduled queries#20017
[Osquery_manager] Add pack_name and query_name fields for scheduled queries#20017marc-gr wants to merge 3 commits into
Conversation
…ueries Add pack_name and query_name keyword fields to the result and action_responses data streams. Osquerybeat now emits these on scheduled pack query result and response documents (pack_name from the pack config, query_name from the pack's queries config map key). The Osquery Manager dashboards need them to group and label scheduled query results by pack and query name.
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
…_name/query_name Update the Compliance pack and OSSEC rootkit pack dashboards and their saved searches to match scheduled queries by action_id OR (pack_name AND query_name), so they keep working for both legacy documents and documents produced after osquerybeat stopped setting action_id for scheduled queries.
|
✅ All changelog entries have the correct PR link. |
💚 Build Succeeded
History
|
tomsonpl
left a comment
There was a problem hiding this comment.
Reviewed alongside the Kibana (elastic/kibana#276690) and osquerybeat (elastic/beats#51781) PRs. The pack_name / query_name mappings and the action_id OR (pack_name AND query_name) filter migration match the fields osquerybeat now emits, and keeping the legacy action_id clause means old documents still resolve — that looks like the right backwards-compatible approach to me. Two small things I wanted to confirm, inline.
| - name: pack_id | ||
| type: keyword | ||
| description: Pack identifier for scheduled queries that belong to a pack. | ||
| - name: pack_name |
There was a problem hiding this comment.
Small consistency check: here in the result data stream the new fields are keyword without ignore_above, while the same two fields in data_stream/action_responses/fields/action-responses.yml set ignore_above: 1024. Is that difference intentional, or would it be worth aligning the two definitions?
| name: osquery_manager | ||
| title: Osquery Manager | ||
| version: 1.30.0 | ||
| version: 1.30.1 |
There was a problem hiding this comment.
The checklist notes the Kibana version constraints weren't verified. Since these fields only populate once the companion Kibana (elastic/kibana#276690) and osquerybeat (elastic/beats#51781) changes ship, is the current conditions.kibana.version still appropriate for 1.30.1, or should it be bumped to align with the stack version that starts emitting pack_name / query_name?
|
Tick the box to add this pull request to the merge queue (same as
|
What does this PR do?
Adds two keyword fields to the
osquery_managerpackage data streams:pack_name— human-readable pack name for scheduled queries that belong to a pack.query_name— the query name (the pack'squeriesconfig map key) for scheduled queries that belong to a pack.Both fields are added to:
data_stream/result(primary-fields.yml) — osquery result documents.data_stream/action_responses(action-responses.yml) — synthetic scheduled response documents.The legacy pack dashboards and saved searches (Compliance pack, OSSEC rootkit pack) previously matched scheduled queries by
action_idonly. They are updated to matchaction_idOR (pack_nameANDquery_name) so they keep working for both legacy documents and documents produced after the beats change.Package version bumped
1.30.0→1.30.1withbugfixchangelog entries.Why is it important?
Osquerybeat now emits
pack_nameandquery_nameon scheduled pack query result and response documents (companion change on the beats side), and no longer setsaction_idfor scheduled queries. The Osquery Manager dashboards group and label scheduled query results by pack and query name; without mappings for these fields and updated filters, the dashboards cannot render those results correctly.Checklist
changelog.ymlfile.How to test this PR locally
elastic-package build) and boot the stack (elastic-package stack up -d).elastic-package install).logs-osquery_manager.result-default:action_id: "pack_<pack>_<query>"pack_name,query_name,schedule_id(noaction_id), with realisticosquery.*fields per query.Verification performed
Verified against a local
elastic-packagestack (Elasticsearch/Kibana9.5.0-SNAPSHOT) with package1.30.1installed. Realistic per-query documents were ingested in both shapes (legacyaction_idand newpack_name+query_namewithaction_idabsent), then the dashboards and saved searches were driven headlessly with Playwright.Saved searches (Discover) — each returns both shapes with the expected columns populated:
bool.shouldbool.shouldbool.shouldDashboards — 0 "No results" panels and 0 panel errors on both:
action_id = pack_ossec-rootkit) and new rows (action_id = (null), matched viapack_name).New documents show
action_id = (null)in Discover yet still appear, confirming they match onpack_name+query_name, while legacy documents continue to match onaction_id.Related issues