Define dataflows for Datadog-owned integrations (batch 1)

[philjlee]

Jira: TXP-277

Adds assets/dataflows.yaml to 74 integrations, declaring 84 dataflows. Mechanical change: no code, no metrics, no manifests touched.

Coverage goes from 14/260 (5.4%) to 88/260 (33.8%).

Precedent: #2925 "Define dataflows for saas-integrations", which created 13 of the 14 existing files in this repo.

Selection criteria

A directory is in this batch only if all of the following hold:

1. It has no assets/dataflows.yaml today. (14 already do; untouched.)
2. It has a parseable manifest.json. The dataflows validator hard-requires one — dataflows_validation_handler.go:56. 5 directories in this repo lack one and cannot carry the asset at all.
3. The correct data_type is mechanically derivable from a committed artifact, with no judgement call:
   • metadata.csv with at least one data row → metrics
   • a log pipeline under assets/logs/*.yaml → logs
   • both → both entries
4. .github/CODEOWNERS resolves <dir>/assets/dataflows.yaml (last-match-wins) to an owner set containing at least one @DataDog/… team.

Every directory in this batch has a Datadog team CODEOWNER, so no external maintainer approval is required to merge. The teams involved are @DataDog/ecosystems-review, @DataDog/agent-integrations, @DataDog/container-integrations, and @DataDog/saas-integrations. External co-owners are auto-requested on many of these paths, but their approval is not a gate.

Field values

provides:
  - id: <app_id>-<data_type>
    always_on: true
    granular: false
    data_type: <metrics|logs>
    direction: inbound

always_on: true / granular: false / direction: inbound matches 14 of the 17 dataflow entries already in the repo. The only deviation in the repo is vercel, which is always_on: false for a Serverless product-enablement reason that does not apply here.

Dataflow IDs are <app_id>-<data_type>, taking app_id from manifest.json rather than the directory name. This is what all 17 existing entries do — e.g. logzio/ → logz-io-events, zscaler/ → z-scaler-logs, cockroachdb_dedicated/ → cockroach-cloud-metrics. All 84 new IDs were checked for collisions against every existing ID in the repo, and against each other.

Validation

These files were validated by executing the real validator, DataflowsValidationHandler from dd-source/domains/integrationscatalog/libs/catalogassetslib/dataflows_validation_handler.go, at ddoghq/dd-source@main, against all 88 dataflows.yaml files in the repo working tree.

Result: 88 files, 101 dataflow IDs, 0 failures, covering per-file unmarshalling, proto constraint validation, and the cross-file HandleLibrary ID-uniqueness check.

The harness was negative-tested first and confirmed to reject: a missing always_on; a data_type outside validDataTypes; an id breaking ^[a-z0-9-]+$; an id under 3 characters; a file with neither provides nor uses; a .yml extension; a missing manifest.json; and the same dataflow ID provided by two apps.

This matters because APW does not post validator comments on integrations-extras PRs (enable_validator_comments is set only for pub-platform-staging and publishing-platform). A malformed dataflows.yaml merges cleanly here and only fails afterwards, in the shared asset pipeline.

Deferred, and why

Category	Count	Reason
RUM SDK tiles	17	There is no rum value in validDataTypes, and no defensible substitute. Needs a schema decision.
Service-check-only	5	cfssl, cybersixgill_actionable_alerts, go_pprof_scraper, isdown, reboot_required. Service checks map to no data_type.
No local data signal	86	Mostly outbound notification/webhook tiles and UI Extensions. Determining the right data_type needs a README read per integration, plus a ruling on whether outbound tiles are events + direction: outbound at all.
Mechanical but externally owned	81	Same derivation as this batch, but no Datadog team CODEOWNER. Batch 2, to be split by owner cluster with a heads-up in #ecosystems-review first.
No parseable manifest.json	5	aerospike_enterprise, scamalytics, warpstream, plus fluxcd and traefik, which are README-only and already in APW's disabled_collections.

No data_type was guessed anywhere. Any integration whose correct value was not unambiguous is deferred rather than approximated.

Contents

Metrics only (51)

Integration	Dataflow IDs
1e	one-e-metrics
ably	ably-metrics
agora_analytics	agora-analytics-metrics
akeyless_gateway	akeyless-gateway-metrics
algorithmia	algorithmia-metrics
authzed_cloud	authzed-cloud-metrics
buoyant_cloud	buoyant-cloud-metrics
census	census-metrics
cloudsmith	cloudsmith-metrics
concourse_ci	concourse-ci-metrics
cribl_stream	cribl-stream-metrics
cyral	cyral-metrics
embrace_mobile	embrace-mobile-metrics
emqx	emqx-metrics
fiddler	fiddler-metrics
gatling_enterprise	gatling-enterprise-metrics
hasura_cloud	hasura-cloud-metrics
inngest	inngest-metrics
lighthouse	lighthouse-metrics
mendix	mendix-metrics
n2ws	n2ws-metrics
neo4j	neo4j-metrics
neoload	neoload-metrics
nn_sdwan	nn-sdwan-metrics
ns1	ns1-metrics
oceanbasecloud	oceanbase-cloud-metrics
packetfabric	packetfabric-metrics
perfectscale	perfectscale-metrics
postman	postman-metrics
purefa	purefa-metrics
purefb	purefb-metrics
redis_sentinel	redis-sentinel-metrics
redisenterprise	redisenterprise-metrics
scalr	scalr-metrics
scaphandre	scaphandre-metrics
singlestoredb_cloud	singlestoredb-cloud-metrics
sofy_sofy	sofy-metrics
sosivio	sosivio-metrics
starburst_galaxy	starburst-galaxy-metrics
statsig	statsig-metrics
stormforge	stormforge-metrics
superwise	superwise-metrics
tidb	tidb-metrics
tidb_cloud	tidb-cloud-metrics
unitq	unitq-metrics
upbound_uxp	upbound-uxp-metrics
upstash	upstash-metrics
vns3	vns3-metrics
yugabytedb_managed	yugabytedb-managed-metrics
zebrium	zebrium-metrics
zilliz_cloud_zilliz_cloud	zilliz-cloud-zilliz-cloud-metrics

Logs only (13)

Integration	Dataflow IDs
appomni	appomni-logs
contrast_security_adr	contrast-security-adr-logs
dagster	dagster-plus-logs
doppler	doppler-logs
fauna	fauna-logs
invary	invary-logs
jfrog_platform_cloud	jfrog-platform-cloud-logs
lacework	lacework-logs
lambdatest	lambdatest-logs
ngrok	ngrok-logs
sqreen	sqreen-logs
twingate	twingate-logs
typingdna_activelock	typingdna-activelock-logs

Metrics and logs (10)

Integration	Dataflow IDs
aimon	aimon-metrics, aimon-logs
anecdote	anecdote-metrics, anecdote-logs
aqua	aqua-metrics, aqua-logs
auth0	auth0-metrics, auth0-logs
bind9	bind9-metrics, bind9-logs
bottomline_recordandreplay	bottomline-recordandreplay-metrics, bottomline-recordandreplay-logs
celerdata	celerdata-metrics, celerdata-logs
gravitee	gravitee-metrics, gravitee-logs
jfrog_platform_self_hosted	jfrog-platform-metrics, jfrog-platform-logs
watchtower_ziris	ziris-metrics, ziris-logs

🤖 Generated with Claude Code

CI status — the red checks are pre-existing, not from this PR

Check	Verdict
validate-dataflows (APW)	pass — "The dataflows assets are valid." This is the check that matters for this PR.
run / Validate	pre-existing. Fails on ddev validate config / ddev validate models drift in 11 integrations.
validate-assets, validate-docs_gitlab	infrastructure. APW reports verbatim: "A system error occurred while validating docs_gitlab. This is not a problem with your assets."
review-check	non-blocking, and self-described as such. 72 of 74 reviewable; purefa and tidb_cloud are excluded because other in-flight PRs touch them.

On run / Validate: this PR adds only assets/dataflows.yaml files and touches no conf.yaml.example, spec.yaml, or pydantic model. Extras CI validates only changed integrations, so adding a file to these directories pulled them into scope and surfaced drift that was already there.

Evidence: the Validate repository workflow has failed on every one of the last 10 runs on master, including 9c197fe — this PR's base commit. That master run flags 39 integrations, and the 11 flagged here are a strict subset of those 39:

celerdata, emqx, fiddler, jfrog_platform_self_hosted, neo4j, nn_sdwan, purefa, purefb, scalr, scaphandre, upbound_uxp

Fixing that drift is a separate change with a different blast radius and different reviewers, so it is deliberately not bundled here.