[AWS Firehose] Improve Firehose documentation

[Kavindu-Dodan]

Description

Improves AWS Firehose documentation, aligning with the update done on AWS Firehose Integration documentation at elastic/integrations#12732

Further, updates all Firehose related documentations to have the same or similar recommended configurations.

Documentation sets edited in this PR

Check all that apply.

• Stateful (docs/en/observability/*)
• Serverless (docs/en/serverless/*)
• Integrations Developer Guide (docs/en/integrations/*)
• [] None of the above

Related issue

Related : elastic/integrations#12150
Related : #4784

Checklist

• Product/Engineering Review
• Writer Review

Follow-up tasks

Select one.

• This PR does not need to be ported to another doc set because:
  • The concepts in this PR only apply to one doc set (serverless or stateful)
  • The PR contains edits to both doc sets (serverless and stateful)
• This PR needs to be ported to another doc set:
  • Port to stateful docs: <link to PR or tracking issue>
  • Port to serverless docs: <link to PR or tracking issue>

[github-actions]

A documentation preview will be available soon.

• 🔨 Buildkite builds
• 📚 HTML diff
• 📙 Preview page

Request a new doc build by commenting

• Rebuild this PR: run docs-build
• Rebuild this PR and all Elastic docs: run docs-build rebuild

_{run docs-build is much faster than run docs-build rebuild. A rebuild should only be needed in rare situations.}

_{If your PR continues to fail for an unknown reason, the doc build pipeline may be broken. Elastic employees can check the pipeline status here.}

[github-actions]

Warning

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

[Kavindu-Dodan]

Warning

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

Not an urgent change, hence can wait till the freeze is lifted.

[github-actions]

Warning

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

[zmoog]

@alaudazzi, this is a change that we need to ship on 8.x.

Should we:

• open a new PR targeting the 8.x branch with the update in asciidoc and
• open a new PR targeting main with the exact change in markdown when the freeze lifts

?

[zmoog]

The current changes looks good.

We also mention the es_datastream_name parameter in other Firehose focused guides:

• monitor-aws-cloudtrail-firehose.asciidoc
• monitor-aws-cloudwatch-firehose.asciidoc
• monitor-aws-firewall-firehose.asciidoc
• monitor-aws-waf-firehose.asciidoc

where we suggest to use a log-specific data stream name instead of the one with routing.

I see the following possible enhancements and alignments with the changes already in this PR:

• align the endpoint URL format (https://<cluster-id>.es.<region>.<csp>.elastic-cloud.com instead of https://my-deployment.es.us-east-1.aws.elastic-cloud.com).
• suggesting to use logs-awsfirehose-default instead of the log specific one for better flexibility
• mention that logs-awsfirehose-default is for logs only and the routing data stream is different; these are logs focused guide, but users new to the service may use the wrong one

WDYT?

[alaudazzi]

@zmoog

as per email sent on Jan 17

We are freezing only the main/master public-facing Docs. You can continue to make changes to the 8.x Docs by creating PRs in the 8.x branches.
Before 29-Jan, you can create and merge 9.0.0 Docs PRs using the main/master branches.
During the documentation freeze, open issues for 9.0.0 Docs changes and we’ll add the Docs post migration.
For 9.0.0 beta1, rc1, and rc2, we will temporarily publish release notes in the AsciiDoc system.
We’ll open changes to the 9.0.0 Docs to everyone on 21-Feb.

For questions, post in the #docs Slack channel.

[Kavindu-Dodan]

@zmoog

align the endpoint URL format (https://.es...elastic-cloud.com instead of https://my-deployment.es.us-east-1.aws.elastic-cloud.com).

Yes, I updated all relevant documents to have this URL format. And I had to remove Important notice as I had to add new section mentioning S3 backups. This was missing in some docs and IMO this is important

suggesting to use logs-awsfirehose-default instead of the log specific one for better flexibility

Since es_datastream_name optional, I removed this from all docs except for the Firehose core document ingest-aws-firehose.asciidoc. This IMO avoids confusion for end users.

@alaudazzi thank you for the input. This PR can wait but appreciate a review when you have time 🙏

[zmoog]

Yes, I updated all relevant documents to have this URL format. And I had to remove Important notice as I had to add new section mentioning S3 backups. This was missing in some docs and IMO this is important

LGTM, thanks.

This PR can wait but appreciate a review when you have time 🙏

If I'm getting this right, from Feb 21, the docs in main will be in markdown only, so:

• main: we need to convert this PR in markdown
• 8.x: we need to create another PR targeting this branch with these changes in asciidoc

@alaudazzi ?

[alaudazzi]

I left some minor editing suggestions, otherwise LGTM.

[github-actions]

Warning

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

[zmoog]

LGTM

[github-actions]

Warning

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

[Kavindu-Dodan]

@alaudazzi will it be possible to get a review ?

Besides, the freeze is lift on doc changes. However, it seems observability-docs is still in asciidoc format. So I assume changes are still allowed here ?

[Kavindu-Dodan]

I have opened this PR for Docs-Content repository - elastic/docs-content#578

[alaudazzi]

Left a minor editing suggestion, otherwise LGTM.

[github-actions]

Warning

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

[alaudazzi]

LGTM