Skip to content

Add TaskFail entries to Gantt chart#37918

Merged
bbovenzi merged 2 commits intoapache:mainfrom
astronomer:add-task-fails-to-gantt-chart
Mar 7, 2024
Merged

Add TaskFail entries to Gantt chart#37918
bbovenzi merged 2 commits intoapache:mainfrom
astronomer:add-task-fails-to-gantt-chart

Conversation

@bbovenzi
Copy link
Copy Markdown
Contributor

@bbovenzi bbovenzi commented Mar 5, 2024

If we're going to have a TaskFail table. Then let's use it. Before it was only used in the old Task Duration table, but that didn't give a user a good sense of their Task Instance. Instead, using it in the Gantt chart can tell more about how a dag run occurred.

Before. Can only see the final task instance try:
Screenshot 2024-03-05 at 5 20 45 PM

After. Can see all previous task instances failures. Notice how TryNumber = 6 and there are 6 bars in the chart:
Screenshot 2024-03-05 at 5 20 25 PM


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@bbovenzi bbovenzi added this to the Airflow 2.9.0 milestone Mar 5, 2024
@boring-cyborg boring-cyborg bot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Mar 5, 2024
Copy link
Copy Markdown
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really cool!

I am not sure whethere the current implementation has a small "refresh problem", tested with my firefox and a task with multiple retires. While running and waiting and taking a look while it was running looked like this:

image

...but my test had 5 attempts/4 fails:
image

..then flipping back still th same, but after hitting F5 then finally screen is updated with all attempts:

image

Is this a local browser caching problem? Or a small glitch/bug in refresh?
Noticed that also flipping gantt to another dag run and back does not help. I really need to hit F5 to get the right/final update.

Another glitch that I found - but this might be data related? If I "clear" a task in UI forcing a re-run, I do not see previous runs of the task in the gantt - might it be that that is not captured in task_triestable when clearing? (Alongside with the scheduling delay which is always shown for last execution only.

Bye the way as being on the Gantt view I noticed that probably the bug report in #37935 is correct - whenever using the example_bash_operatoras example DAG to test I needed to reload the gantt to get the new DAG run published after trigger. If I am starting to trigger from Grid view this is always automatic.

Copy link
Copy Markdown
Contributor

@eladkal eladkal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it!

@bbovenzi bbovenzi force-pushed the add-task-fails-to-gantt-chart branch from c640b39 to febee38 Compare March 7, 2024 21:31
@bbovenzi
Copy link
Copy Markdown
Contributor Author

bbovenzi commented Mar 7, 2024

Added autorefresh to fetching Task Fails.
Also added a small disclaimer in the Task Fail tooltip to remark that we only record task fails and not other kinds of retries.

I noticed that redirect bug and I'll work on that separately.

@bbovenzi bbovenzi merged commit 35b6163 into apache:main Mar 7, 2024
@bbovenzi bbovenzi deleted the add-task-fails-to-gantt-chart branch March 7, 2024 22:09
@potiuk
Copy link
Copy Markdown
Member

potiuk commented Mar 7, 2024

Nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues type:new-feature Changelog: New Features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants