Skip to content

add bulk delete for dagruns#60718

Open
Pei-Cheng-Yu wants to merge 5 commits intoapache:mainfrom
Pei-Cheng-Yu:delete-failed-tasks-on-runs-page
Open

add bulk delete for dagruns#60718
Pei-Cheng-Yu wants to merge 5 commits intoapache:mainfrom
Pei-Cheng-Yu:delete-failed-tasks-on-runs-page

Conversation

@Pei-Cheng-Yu
Copy link
Copy Markdown
Contributor

@Pei-Cheng-Yu Pei-Cheng-Yu commented Jan 17, 2026

Why

related #52439

How

  • add BulkDeleteRunsButton and useBulkDeleteDagRuns for collectively delete runs

video

Clipchamp.7.mp4

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@boring-cyborg boring-cyborg bot added the area:UI Related to UI/UX. For Frontend Developers. label Jan 17, 2026
@Pei-Cheng-Yu Pei-Cheng-Yu changed the title add bulk delete for dagruns draft: add bulk delete for dagruns Jan 17, 2026
@Pei-Cheng-Yu Pei-Cheng-Yu changed the title draft: add bulk delete for dagruns add bulk delete for dagruns Jan 17, 2026
@Pei-Cheng-Yu Pei-Cheng-Yu marked this pull request as draft January 17, 2026 16:49
@Pei-Cheng-Yu Pei-Cheng-Yu marked this pull request as ready for review January 17, 2026 16:52
@pierrejeambrun pierrejeambrun added this to the Airflow 3.2.0 milestone Jan 19, 2026
Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 🎉 thanks for the pull request.

Can you take a look at the DeleteVariablesButton and DeleteConnectionsButton and implement the same for TI.

Image Image

@Pei-Cheng-Yu Pei-Cheng-Yu force-pushed the delete-failed-tasks-on-runs-page branch from 238658d to 5d2e210 Compare January 23, 2026 22:34
@Pei-Cheng-Yu
Copy link
Copy Markdown
Contributor Author

Can you take a look at the DeleteVariablesButton and DeleteConnectionsButton and implement the same for TI.

Thanks for reviewing, I just updated it!

Copy link
Copy Markdown
Member

@guan404ming guan404ming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! Left some comments.

@guan404ming
Copy link
Copy Markdown
Member

guan404ming commented Jan 27, 2026

Can you take a look at the DeleteVariablesButton and DeleteConnectionsButton and implement the same for TI.

I think maybe we could handle that in another separated pr since I think that one is more complicated.

Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we need another PR to update the backend to add the bulk delete feature similarly to variables and connections before we can merge this.

This PR is looking good to me, but we probably don't want to make N calls. (if someone tries to delete 500 dagruns this way, it's probably going to be a problem)

@Pei-Cheng-Yu Would you be interested in contributing that backend feature to enable this PR first?

@Pei-Cheng-Yu
Copy link
Copy Markdown
Contributor Author

Thanks for the reviews !
Sure! I just make the hook's return value align with the pattern in useBulkDeleteVariables.ts, and now i can investigate into the backend part!

@guan404ming
Copy link
Copy Markdown
Member

Nice! Feel free to open a new pr to handle the backend part~

@guan404ming
Copy link
Copy Markdown
Member

Hi @Pei-Cheng-Yu, I think this would be a great feature to include in 3.2. Just wanted to check if you're still working on this? If you're tied up at the moment, I’m happy to take it over and help handle it. Thanks!

@Pei-Cheng-Yu
Copy link
Copy Markdown
Contributor Author

Hi @guan404ming , thanks for checking and apologies for the delay.

The past few weeks have been quite busy on my end, so I’ve been working on this incrementally. The backend implementation is now completed, and only the unit tests remain. I expect to have them finished within the next two days.

Really appreciate the offer to help!

@guan404ming
Copy link
Copy Markdown
Member

No need to apologize at all! I completely understand how busy things can get. It’s great to hear that the backend implementation is finished. If you run into any blockers or have questions while wrapping things up, feel free to reach out anytime. Looking forward to the update!

@Pei-Cheng-Yu
Copy link
Copy Markdown
Contributor Author

Thanks a lot for the understanding and support!

Copy link
Copy Markdown
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me this is looking good. Yes, at the moment this makes N calls to the backend but it is not possible to make more batch calls than the list has elements, so 500 is not possible. I think it is more important to have the feature then making it pretty.

Therefore I think we can also merge this PR and have a follow-up to improve with a batch backend. But leaving final merge to an UI expert.

Tested manaully besides code check and works well.

Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we can deliver value incrementally. Having the feature 'non optimal' is still better than not having the feature at all.

We've done this in the past for other parts of the UI.

I'm happy to move forward with this and refactor it later when scalability becomes a problem.

A few suggestions though before we can move forward.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Rename file + component to DeleteRunsButton to be consistant with other DeleteVariablesButton and `

>
<FiTrash2 />{" "}
<Text as="span" fontWeight="bold">
{translate("common:delete")}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The confirmation modal isn't consistent with other confirmation modal:

Image Image Image

Comment on lines +52 to +54
<FiTrash2 />
{translate("dags:runAndTaskActions.delete.button", { type: translate("dagRun_other") })}
</Button>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not consistent. Here it's just delete:

Image Image

Comment on lines +77 to +82
type SelectionConfig = {
allRowsSelected: boolean;
onRowSelect: (key: string, selected: boolean) => void;
onSelectAll: (selected: boolean) => void;
selectedRows: Map<string, boolean>;
};
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is alreaady defined in useRowSelection.ts we should reuse it.

Comment on lines +71 to +75
const parseRunKey = (key: string): SelectedRun => {
const [dagId = "", ...rest] = key.split("__");

return { dagId, dagRunId: rest.join("__") };
};
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the dag_id / dag_run_id holds a __ ?

getKey: (run) => runKey({ dagId: run.dag_id, dagRunId: run.dag_run_id }),
});

const selectedRuns = useMemo<Array<SelectedRun>>(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

React compiler is taking care of useMemo I belive. This isn't needed anymore.

Comment on lines +65 to +75
if (failed.length > 0) {
const [first] = failed;

if (first) {
setError(first.reason);
}
toaster.create({
description: `${failed.length}/${runs.length} failed`,
title: translate("dags:runAndTaskActions.delete.error", { type: translate("dagRun_one") }),
type: "error",
});
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When multiple deletions fail, only failed[0].reason is stored in error state. The toast says ${failed.length}/${runs.length} failed but doesn't say which ones or why. Consider showing all failed run IDs in the error alert.

@kaxil kaxil requested a review from Copilot April 10, 2026 19:55
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a bulk-delete workflow to the DAG Runs listing, enabling users to select multiple runs and delete them in one action (addressing the “select many runs and collectively remove…” workflow referenced in #52439).

Changes:

  • Introduces a BulkDeleteRunsButton confirmation dialog for deleting selected DAG runs.
  • Adds useBulkDeleteDagRuns to perform multi-run deletion and invalidate relevant React Query caches.
  • Updates DagRuns page to support row selection via checkboxes and show an action bar when rows are selected.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File Description
airflow-core/src/airflow/ui/src/queries/useBulkDeleteDagRuns.ts New hook implementing multi-run deletion and cache invalidation with success/error toasts.
airflow-core/src/airflow/ui/src/pages/DagRuns.tsx Adds checkbox-based row selection and an ActionBar entry point for bulk deletion.
airflow-core/src/airflow/ui/src/pages/BulkDeleteRunsButton.tsx New UI button + dialog to confirm and execute bulk deletion.

{translate("dags:runAndTaskActions.delete.dialog.warning", {
type: translate("dagRun_other"),
})}{" "}
({count} {translate("dagRun", { count })})
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The translation call translate("dagRun", { count }) is likely incorrect: in public/i18n/locales/*/common.json, dagRun is a nested object (e.g. for column labels), not a pluralizable string key. This will render as the key name (or [object Object]) instead of “Dag Run(s)”. Use the existing plural keys like dagRun_one / dagRun_other (or an existing pluralized resourceName pattern) for the count label.

Suggested change
({count} {translate("dagRun", { count })})
({count} {count === 1 ? translate("dagRun_one") : translate("dagRun_other")})

Copilot uses AI. Check for mistakes.
Comment on lines +91 to +116
{
cell: ({ row: { original } }: DagRunRow) => {
const key = runKey({ dagId: original.dag_id, dagRunId: original.dag_run_id });

return (
<Checkbox
borderWidth={1}
checked={selection.selectedRows.get(key)}
colorPalette="brand"
onCheckedChange={(event) => selection.onRowSelect(key, Boolean(event.checked))}
/>
);
},
enableSorting: false,
header: () => (
<Checkbox
borderWidth={1}
checked={selection.allRowsSelected}
colorPalette="brand"
onCheckedChange={(event) => selection.onSelectAll(Boolean(event.checked))}
/>
),
id: "select",
meta: { skeletonWidth: 2 },
} satisfies ColumnDef<DAGRunResponse>,
]
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The selection checkbox column is missing enableHiding: false (and uses id instead of the established accessorKey: "select" pattern). Since DataTable enables column hiding globally, users can hide this column and lose the ability to change selections. Consider matching the existing selection-column pattern used in e.g. pages/Connections/Connections.tsx and pages/Variables/Variables.tsx by setting accessorKey: "select", enableHiding: false, and enableSorting: false.

Copilot uses AI. Check for mistakes.
Comment on lines +70 to +74
const runKey = (run: SelectedRun) => `${run.dagId}__${run.dagRunId}`;
const parseRunKey = (key: string): SelectedRun => {
const [dagId = "", ...rest] = key.split("__");

return { dagId, dagRunId: rest.join("__") };
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runKey/parseRunKey rely on a "__" delimiter to round-trip { dagId, dagRunId }. This breaks if dagId itself contains "__" (DAG IDs are user-defined strings), leading to deletes being issued against the wrong DAG. Prefer an encoding that can’t collide (e.g. JSON.stringify/JSON.parse, or base64/URL-encoding of each part with an unambiguous separator).

Suggested change
const runKey = (run: SelectedRun) => `${run.dagId}__${run.dagRunId}`;
const parseRunKey = (key: string): SelectedRun => {
const [dagId = "", ...rest] = key.split("__");
return { dagId, dagRunId: rest.join("__") };
const runKey = (run: SelectedRun) => JSON.stringify({ dagId: run.dagId, dagRunId: run.dagRunId });
const parseRunKey = (key: string): SelectedRun => {
try {
const parsed = JSON.parse(key) as Partial<SelectedRun>;
return {
dagId: typeof parsed.dagId === "string" ? parsed.dagId : "",
dagRunId: typeof parsed.dagRunId === "string" ? parsed.dagRunId : "",
};
} catch {
return { dagId: "", dagRunId: "" };
}

Copilot uses AI. Check for mistakes.
Comment on lines +44 to +94

const mutate = async (runs: Array<SelectedRun>): Promise<void> => {
if (runs.length === 0) {
return;
}
setError(undefined);

const results = await Promise.allSettled(
runs.map(({ dagId, dagRunId }) => deleteMutation.mutateAsync({ dagId, dagRunId })),
);

const failed = results.filter((result): result is PromiseRejectedResult => result.status === "rejected");
const queryKeys = [
[useDagRunServiceGetDagRunsKey],
[useTaskInstanceServiceGetTaskInstancesKey],
[useTaskInstanceServiceGetHitlDetailsKey],
...runs.map(({ dagId, dagRunId }) => UseDagRunServiceGetDagRunKeyFn({ dagId, dagRunId })),
];

await Promise.all(queryKeys.map((key) => queryClient.invalidateQueries({ queryKey: key })));

if (failed.length > 0) {
const [first] = failed;

if (first) {
setError(first.reason);
}
toaster.create({
description: `${failed.length}/${runs.length} failed`,
title: translate("dags:runAndTaskActions.delete.error", { type: translate("dagRun_one") }),
type: "error",
});

return;
}

toaster.create({
description: translate("dags:runAndTaskActions.delete.success.description", {
type: translate("dagRun_one"),
}),
title: translate("dags:runAndTaskActions.delete.success.title", { type: translate("dagRun_one") }),
type: "success",
});

clearSelections?.();
onSuccessConfirm?.();
};

return {
error,
isPending: deleteMutation.isPending,
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

useDagRunServiceDeleteDagRun() returns a single React Query mutation instance, but mutateAsync is invoked concurrently for every selected run. Mutation state (notably isPending) only tracks the most recently started mutation, so the UI loading state can flip to false while earlier deletions are still in flight. Consider implementing this as one mutation that accepts the array of runs (and internally runs deletes with a concurrency limit or sequentially), and manage a dedicated isPending state for the bulk operation.

Suggested change
const mutate = async (runs: Array<SelectedRun>): Promise<void> => {
if (runs.length === 0) {
return;
}
setError(undefined);
const results = await Promise.allSettled(
runs.map(({ dagId, dagRunId }) => deleteMutation.mutateAsync({ dagId, dagRunId })),
);
const failed = results.filter((result): result is PromiseRejectedResult => result.status === "rejected");
const queryKeys = [
[useDagRunServiceGetDagRunsKey],
[useTaskInstanceServiceGetTaskInstancesKey],
[useTaskInstanceServiceGetHitlDetailsKey],
...runs.map(({ dagId, dagRunId }) => UseDagRunServiceGetDagRunKeyFn({ dagId, dagRunId })),
];
await Promise.all(queryKeys.map((key) => queryClient.invalidateQueries({ queryKey: key })));
if (failed.length > 0) {
const [first] = failed;
if (first) {
setError(first.reason);
}
toaster.create({
description: `${failed.length}/${runs.length} failed`,
title: translate("dags:runAndTaskActions.delete.error", { type: translate("dagRun_one") }),
type: "error",
});
return;
}
toaster.create({
description: translate("dags:runAndTaskActions.delete.success.description", {
type: translate("dagRun_one"),
}),
title: translate("dags:runAndTaskActions.delete.success.title", { type: translate("dagRun_one") }),
type: "success",
});
clearSelections?.();
onSuccessConfirm?.();
};
return {
error,
isPending: deleteMutation.isPending,
const [isPending, setIsPending] = useState(false);
const mutate = async (runs: Array<SelectedRun>): Promise<void> => {
if (runs.length === 0) {
return;
}
setError(undefined);
setIsPending(true);
try {
const failed: Array<PromiseRejectedResult> = [];
for (const { dagId, dagRunId } of runs) {
try {
await deleteMutation.mutateAsync({ dagId, dagRunId });
} catch (reason) {
failed.push({ reason, status: "rejected" });
}
}
const queryKeys = [
[useDagRunServiceGetDagRunsKey],
[useTaskInstanceServiceGetTaskInstancesKey],
[useTaskInstanceServiceGetHitlDetailsKey],
...runs.map(({ dagId, dagRunId }) => UseDagRunServiceGetDagRunKeyFn({ dagId, dagRunId })),
];
await Promise.all(queryKeys.map((key) => queryClient.invalidateQueries({ queryKey: key })));
if (failed.length > 0) {
const [first] = failed;
if (first) {
setError(first.reason);
}
toaster.create({
description: `${failed.length}/${runs.length} failed`,
title: translate("dags:runAndTaskActions.delete.error", { type: translate("dagRun_one") }),
type: "error",
});
return;
}
toaster.create({
description: translate("dags:runAndTaskActions.delete.success.description", {
type: translate("dagRun_one"),
}),
title: translate("dags:runAndTaskActions.delete.success.title", { type: translate("dagRun_one") }),
type: "success",
});
clearSelections?.();
onSuccessConfirm?.();
} finally {
setIsPending(false);
}
};
return {
error,
isPending,

Copilot uses AI. Check for mistakes.
Comment on lines +71 to +85
toaster.create({
description: `${failed.length}/${runs.length} failed`,
title: translate("dags:runAndTaskActions.delete.error", { type: translate("dagRun_one") }),
type: "error",
});

return;
}

toaster.create({
description: translate("dags:runAndTaskActions.delete.success.description", {
type: translate("dagRun_one"),
}),
title: translate("dags:runAndTaskActions.delete.success.title", { type: translate("dagRun_one") }),
type: "success",
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bulk-failure toast text is not localized ("${failed.length}/${runs.length} failed") and the toast titles/descriptions use dagRun_one even though multiple runs may be deleted. Please switch to existing i18n keys (e.g. common:toaster.bulkDelete.* or a new dags:* bulk-delete key) and use the plural resource name (dagRun_other) and/or include the counts so the message is accurate.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UI Related to UI/UX. For Frontend Developers. ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants