Add Anthropic provider (apache-airflow-providers-anthropic)#69003
Merged
Conversation
kaxil
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jason810496,
jscheffl and
potiuk
as code owners
June 26, 2026 00:04
kaxil
force-pushed
the
add-anthropic-provider
branch
2 times, most recently
from
June 26, 2026 00:32
3b8adcc to
b64bf04
Compare
amoghrajesh
reviewed
Jun 26, 2026
amoghrajesh
left a comment
Contributor
There was a problem hiding this comment.
From a static code review, I am taking it for a spin now to see how it goes.
Lee-W
reviewed
Jun 26, 2026
kaxil
force-pushed
the
add-anthropic-provider
branch
from
June 26, 2026 12:10
b64bf04 to
260a157
Compare
Add a dedicated provider for the Anthropic Claude API so Dag authors can use Claude through the official Anthropic Python SDK instead of hand-writing SDK calls in tasks. It offers first-class connection management (first-party API, Amazon Bedrock, Google Vertex AI, Claude Platform on AWS, Microsoft Foundry, and keyless Workload Identity Federation) with labelled connection form fields, plus deferrable Message Batches and Managed Agents session operators, a batch sensor, and matching triggers. Targets Airflow 3+ and ships as not-ready / incubation while the Managed Agents beta surface stabilises.
kaxil
force-pushed
the
add-anthropic-provider
branch
from
June 26, 2026 14:25
260a157 to
6cbb964
Compare
kaxil
requested review from
choo121600,
ephraimbuddy,
jedcunningham and
vatsrahul1001
as code owners
June 26, 2026 14:25
jason810496
reviewed
Jul 3, 2026
jason810496
left a comment
Member
There was a problem hiding this comment.
Thanks for adding the anthropic provider!
- Pass aws_region to the AnthropicAWS (Claude Platform on AWS) client - Cancel batch / archive session on the deferred error event, and on any post-submission failure in the synchronous path (billing/resource leak) - Add async on_kill to both triggers to clean up killed deferred tasks (3.3+) - Tolerate transient poll errors in the synchronous wait helpers - Require both description and rubric in the agent outcome - Rename the batch operators/sensors/triggers modules anthropic.py -> batch.py (and their tests) for agent.py/batch.py symmetry
vatsrahul1001
approved these changes
Jul 3, 2026
This was referenced Jul 3, 2026
1 task
75 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New provider
apache-airflow-providers-anthropicfor the Anthropic Claude API, so Dag authors call Claude through the official Anthropic Python SDK instead of embedding SDK calls in task code.It ships:
AnthropicHook— builds the right client for the configured platform: first-party API, Amazon Bedrock, Google Vertex AI, Claude Platform on AWS, and Microsoft Foundry. Supports keyless Workload Identity Federation (configured on the connection or resolved from the environment).AnthropicBatchOperator(deferrable) +AnthropicBatchSensor+AnthropicBatchTrigger— the Message Batches API (submit → defer → collect).AnthropicAgentSessionOperator(deferrable) +AnthropicAgentSessionTrigger— Managed Agents sessions (message and outcome runs).conn-fields, not raw Extra JSON.Design rationale
common.ai.common.aiis a provider-agnostic abstraction; this provider exposes Anthropic-specific surfaces (Message Batches, Managed Agents, token counting, the platform clients) that do not fit a neutral interface. Users who want a provider-agnostic layer keep usingcommon.ai; users who want direct Claude/SDK features use this one.deferrable=Falsemirrors the same cancel-on-timeout (batch) / archive-on-timeout (session) teardown so neither mode leaks a server-side resource.modelon the connection. The default model is read fromextra['model']so it changes without editing DAGs; it falls back toclaude-opus-4-8.Notes / gotchas
platformin {anthropic, aws}; calling them on Bedrock/Vertex/Foundry raises a clear error rather than a raw 404.state: not-ready/lifecycle: incubationwhile the Managed Agents beta surface stabilises.anthropic>=0.101.0(introducedAnthropicAWS); verified against 0.109.x.Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.