Skip to content

fix(attribution)!: partition author facts by declared audit route#263

Merged
sunib merged 2 commits into
mainfrom
fix/attribution-audit-route
Jul 22, 2026
Merged

fix(attribution)!: partition author facts by declared audit route#263
sunib merged 2 commits into
mainfrom
fix/attribution-audit-route

Conversation

@sunib

@sunib sunib commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

The bug

Reported by the gitops-api team: every object mirrored through a dedicated in-cluster ClusterProvider was committed as unknown (attribution unresolved), while the same actor's writes through default attributed correctly. Mirroring stayed correct, no condition changed, and no reconcile failed, so the loss was visible only in the commit author.

Their measurement, reproduced twice: 5 objects absent, all having waited the full grace window, with written=66 / matched=12 and zero expiries. Facts were being written the whole time, under a key nobody read.

Why

A kube-apiserver takes one --audit-webhook-config-file, and that file uses the kubeconfig format to specify the remote address of the service, singular. So one physical cluster posts audit under exactly one route.

Facts were partitioned by the ClusterProvider's name, so every other provider naming that same cluster read a partition nothing writes.

That shape is one the product encourages. allowedNamespaces and allowSourceNamespaceOverride are provider-wide, so a platform admin granting the override to one tenant and withholding it from another has no choice but to create two providers on one cluster — which is exactly what the source-namespace e2e does.

The fix

ClusterProvider.spec.attribution.auditRoute declares the route a cluster's events arrive on, and that value partitions the facts.

metadata:
  name: tenant-acme-delegating   # the name humans read and GitTargets reference
spec:
  attribution:
    auditRoute: prod-eu-1        # the route this cluster's audit events arrive on
  • Defaults to metadata.name, so an install that sets nothing resolves exactly what it resolved before.
  • Several providers share one cluster's facts by declaring one route.
  • Providers on different clusters keep separate partitions. That is what still separates an etcd-snapshot clone from its origin: a clone reproduces every metadata.uid, so a design keying facts by object identity alone could not tell them apart, while a human-chosen route can.

Ingestion gets simpler, and loses its last Kubernetes read

The /audit-webhook/<route> existence gate, AuditProviderResolver, its cmd/main.go adapter, and the annotation-routing startup rule are all deleted. Parse the route, record the fact — no API call.

A route is a partition name, not a claim about an object, so a fact for a route no provider declares is stored and expires unread. That also stops dropping audit batches in flight while a provider is being created or recreated, which the API server does not retry after a 404.

The failure is now loud

A route that has never resolved and produces five unresolved events in a row logs once, naming the fix rather than the symptom. The threshold is not one because a lone miss is ordinary under audit-batch delay; a run with nothing ever matched is the signature of a route nobody writes to.

Also removed

PurgeClusterFacts and its decision record. It was unwired, and a shared route makes it unsafe: purging on one provider's deletion would drop the facts of every other provider on that route, and of the operator's own cluster, which was never torn down. The reasoning that still binds — no finalizer, because helm uninstall would strand the object in Terminating — moved onto LegacyClusterProviderFinalizer, where someone about to re-add one will actually read it.

Breaking changes

No deprecated aliases and no dual-read, deliberately: this path has no users to protect.

Change From To
flag --author-attribution-cluster-annotation-key --author-attribution-audit-route-annotation-key
chart value attribution.clusterAnnotationKey attribution.auditRouteAnnotationKey
fact-key infix cluster:<name> route:<auditRoute>

An operator still passing the old flag fails to start, which is the loud outcome. The key-infix change gives a rolling upgrade a TTL-bounded window (minutes) where events commit as the configured committer — the ordinary degradation, and the same rollout behaviour the v2v3 key bump had.

Tests

  • e2e (test/e2e/audit_route_attribution_e2e_test.go): a dedicated in-cluster ClusterProvider whose name is not default, declaring auditRoute: default, must commit as the impersonated OIDC identity and specifically not the unresolved placeholder. This fails on main. A second case reaches an object through a rules[].sourceNamespace override, pinning the report's claim that the override was innocent.
  • unit: AuditRoute() defaulting; two providers sharing a route joining one fact while a non-declaring one still misses; the Declare-time capture and its teardown; the warning's four cases; the handler storing a fact for a route no provider declares.

task lint, task test, and task test-e2e all pass (Ran 69 of 91 — 69 Passed, 0 Failed).

Design: docs/design/attribution-fact-identity.md

🤖 Generated with Claude Code

sunib and others added 2 commits July 22, 2026 05:35
Every object mirrored through a dedicated in-cluster ClusterProvider was
committed as "unknown (attribution unresolved)", while the same actor's writes
through "default" attributed correctly. Mirroring stayed correct, so the loss
was visible only in the commit author.

A kube-apiserver takes one --audit-webhook-config-file, and that file names one
server URL, so one physical cluster posts audit under exactly ONE route. Facts
were partitioned by the ClusterProvider's name, so every other provider naming
that same cluster read a partition nothing writes. The source-namespace feature
encourages exactly that shape: allowedNamespaces and allowSourceNamespaceOverride
are provider-wide, so two delegation stances on one cluster need two providers.

ClusterProvider.spec.attribution.auditRoute now declares the route a cluster's
events arrive on, and that value partitions the facts. It defaults to the
object's own name, so an install that sets nothing resolves exactly what it
resolved before. Several providers share one cluster's facts by declaring one
route; providers on different clusters keep separate partitions, which is what
still separates an etcd-snapshot clone from its origin (every identity readable
from inside a cloned cluster was cloned along with it).

Ingestion gets simpler rather than harder, and loses its only Kubernetes read:
the /audit-webhook/<route> existence gate, AuditProviderResolver, and the
annotation-routing startup rule are gone. A route is a partition name, not a
claim about an object, so a fact for a route no provider declares is stored and
expires unread. That also stops dropping audit batches in flight while a
provider is being created, which the API server does not retry after a 404.

The failure is now loud: a route that has never resolved and produces five
unresolved events in a row logs once, naming the fix rather than the symptom.

PurgeClusterFacts is deleted along with its decision record. It was unwired, and
a shared route makes it unsafe: purging on one provider's deletion would drop
the facts of every other provider on that route. The reasoning that still binds
(no finalizer, because helm uninstall would strand the object in Terminating)
moved onto LegacyClusterProviderFinalizer.

BREAKING CHANGE: --author-attribution-cluster-annotation-key is renamed to
--author-attribution-audit-route-annotation-key, and the chart value
attribution.clusterAnnotationKey to attribution.auditRouteAnnotationKey. There
is no deprecated alias: an operator still passing the old flag fails to start.
The fact-key infix changes from cluster: to route: with no dual-read, so a
rolling upgrade has a TTL-bounded window where events commit as the configured
committer.

Design: docs/design/attribution-fact-identity.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Takes auditRouteFor from 0% to 100% (including the fallback that resolves the
provider name when the ClusterProvider cannot be read, which is what keeps a
transient read failure from keying the lookup to a partition nothing writes),
covers the never-resolves warning through the real resolver rather than the
counter alone, and pins which keys each fact shape writes: the rv-only hatch
exists only for a fact with no UID, because a UID-bearing fact's rv key would
be dead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@sunib, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 37 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b7f55758-1493-42de-b536-b11b58e4ee47

📥 Commits

Reviewing files that changed from the base of the PR and between f103ca8 and f15b010.

📒 Files selected for processing (43)
  • api/v1alpha3/audit_route_test.go
  • api/v1alpha3/clusterprovider_types.go
  • api/v1alpha3/gittarget_types.go
  • api/v1alpha3/zz_generated.deepcopy.go
  • charts/gitops-reverser/README.md
  • charts/gitops-reverser/templates/deployment.yaml
  • charts/gitops-reverser/values.schema.json
  • charts/gitops-reverser/values.yaml
  • cmd/main.go
  • config/crd/bases/configbutler.ai_clusterproviders.yaml
  • docs/INDEX.md
  • docs/architecture.md
  • docs/attribution-setup-guide.md
  • docs/bug-report.md
  • docs/config-flag-conventions.md
  • docs/configuration.md
  • docs/design/attribution-fact-identity.md
  • docs/design/multi-source-audit-ingress-hardening.md
  • docs/facts/audit-webhook-api-server-connectivity.md
  • docs/finished/clusterprovider-fact-purge.md
  • docs/finished/multi-cluster-author-attribution.md
  • internal/audit/outcome/outcome.go
  • internal/audit/outcome/outcome_test.go
  • internal/controller/clusterprovider_controller.go
  • internal/controller/clusterprovider_controller_unit_test.go
  • internal/controller/gittarget_controller.go
  • internal/controller/gittarget_source_cluster.go
  • internal/controller/gittarget_source_cluster_test.go
  • internal/queue/attribution_index.go
  • internal/queue/attribution_index_test.go
  • internal/queue/key_prefix_test.go
  • internal/watch/audit_route_test.go
  • internal/watch/author_resolver.go
  • internal/watch/author_resolver_test.go
  • internal/watch/cluster_context.go
  • internal/watch/manager.go
  • internal/watch/materialization.go
  • internal/watch/target_watch.go
  • internal/webhook/audit_handler.go
  • internal/webhook/audit_handler_test.go
  • internal/webhook/audit_metrics_test.go
  • test/e2e/audit_route_attribution_e2e_test.go
  • test/e2e/cluster/audit/webhook-config.yaml
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/attribution-audit-route

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sunib
sunib merged commit 0d61128 into main Jul 22, 2026
11 of 12 checks passed
@sunib
sunib deleted the fix/attribution-audit-route branch July 22, 2026 05:40
sunib added a commit that referenced this pull request Jul 22, 2026
PR #263 edited docs/architecture.md and docs/configuration.md, which this
branch had rewritten against the style guide, so both files conflicted.

architecture.md auto-merged. configuration.md had one hunk where #263 renamed
`/audit-webhook/<name>` to `/audit-webhook/<audit-route>` on the same line this
branch had re-punctuated; the resolution keeps both.

Verified after resolving: every line that differs from main in those two files
is a bullet, a heading's capitalization, or a punctuation mark, and #263's
`audit-route` and `auditRoute` terms are all present. Both files still pass
markdownlint and Vale with zero errors.

Merged rather than rebased on purpose: PR #262 carries 16 inline review
comments that a force-push would orphan, and this repository squash-merges, so
the merge commit does not survive into main anyway.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.09091% with 12 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cmd/main.go 53.8% 5 Missing and 1 partial ⚠️
internal/queue/attribution_index.go 82.6% 0 Missing and 4 partials ⚠️
internal/controller/gittarget_controller.go 0.0% 1 Missing ⚠️
internal/watch/materialization.go 0.0% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant