Skip to content

Bulkdelete dagrun backend#62399

Draft
Pei-Cheng-Yu wants to merge 15 commits intoapache:mainfrom
Pei-Cheng-Yu:bulkdelete-dagrun-backed
Draft

Bulkdelete dagrun backend#62399
Pei-Cheng-Yu wants to merge 15 commits intoapache:mainfrom
Pei-Cheng-Yu:bulkdelete-dagrun-backed

Conversation

@Pei-Cheng-Yu
Copy link
Copy Markdown
Contributor

@Pei-Cheng-Yu Pei-Cheng-Yu commented Feb 24, 2026

why

related #52439
according disscussion in PR
we need a bulkdelete endpoint for dag run

how

I implemented this by referencing the existing bulk action implementations in:

  • task_instance.py
  • connection.py
  • variable.py

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: [OpenAI Codex / GPT-5 ] following the guidelines

for ideation and partial code generation. I reviewed and adjusted the changes to align with existing implementations of corresponding bulk actions elsewhere, and verified they pass prek.
I make sure that codex only modify the right part of code, and any thing about endpoint that requiring codegen were regenerated with prek and then pass the local prek test


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR.

Nice. Overall looking good.

Just a few suggestions/nits

@Pei-Cheng-Yu Pei-Cheng-Yu force-pushed the bulkdelete-dagrun-backed branch from 8a6c6a7 to 2714195 Compare March 1, 2026 14:36
@Pei-Cheng-Yu
Copy link
Copy Markdown
Contributor Author

Thanks for the review!

Update

I think that I just misread the router pattern and used the wrong bulk body type, I just change to the correct one mention in the suggestion
as for the requires_access_dag_run_bulk part, the implementation follows the same bulk auth pattern, but because this endpoint is DAG-scoped (/dags/{dag_id}/dagRuns), if i understand it correctly, the authorization is resolved from path dag_id instead of per-entity keys.

Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the way how this is currently implemented. It doesn't seem possible to delete multiple runs from multiple dag_ids at the same time. Which kind of defeat the purpose.

If you take a look at the 'bulk' operation on TIs bulk_task_instances you'll get an idea of how we can rework this to achieve this.

Also feel free to 'mark as resolved' conversation you have addressed.

@potiuk
Copy link
Copy Markdown
Member

potiuk commented Mar 11, 2026

@Pei-Cheng-Yu This PR has been converted to draft because it does not yet meet our Pull Request quality criteria.

Issues found:

  • Merge conflicts: This PR has merge conflicts with the main branch. Your branch is 209 commits behind main. Please rebase your branch (git fetch origin && git rebase origin/main), resolve the conflicts, and push again. See contributing quick start.
  • ⚠️ Unresolved review comments: This PR has 7 unresolved review threads from maintainers. Please review and resolve all inline review comments before requesting another review. You can resolve a conversation by clicking 'Resolve conversation' on each thread after addressing the feedback. See pull request guidelines.

Note: Your branch is 209 commits behind main. Some check failures may be caused by changes in the base branch rather than by your PR. Please rebase your branch and push again to get up-to-date CI results.

What to do next:

  • The comment informs you what you need to do.
  • Fix each issue, then mark the PR as "Ready for review" in the GitHub UI - but only after making sure that all the issues are fixed.
  • Maintainers will then proceed with a normal review.

Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. If you have questions, feel free to ask on the Airflow Slack.

@potiuk potiuk marked this pull request as draft March 11, 2026 12:32
@Pei-Cheng-Yu Pei-Cheng-Yu force-pushed the bulkdelete-dagrun-backed branch from 57ce9e9 to b16ac42 Compare March 13, 2026 14:54
@Pei-Cheng-Yu
Copy link
Copy Markdown
Contributor Author

Thanks, I understand now.

I followed the earlier suggestion and interpreted it in a way using BulkBody[DAGRunPatchBody] that kept the implementation effectively single-DAG, just with dag_id available from the payload for auth.
After working through it, I realized that does not actually achieve the intended cross-DAG bulk delete behavior.

I should have stopped and clarified the direction when that became ambiguous. I just rework it to align with the bulk_task_instances pattern.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:airflow-ctl area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants