Issue with Grid view in Airflow 2.3.1rc1

[MatthieuBlais]

We are facing an issue with the Grid view in Airflow 2.3. We would like to know if this issue can be reproduced by others or not.

• We tried to upgrade our development environment from Airflow 2.4.0 to 2.3.0. We have 200+ Dags and 6M+ records in the task_instance table. We run on K8s, and deployed Airflow with a Helm chart.
• We faced some issues while upgrading the db. It was likely related to this issue.
• To work around, we upgraded to version 2.3.1rc1, and we managed to complete the upgrade.
• While testing changes, we noticed some grid views cannot be loaded. When we open the DAG page, the area remains blank

• Looking at the JS console, we can see some errors. These errors are shown only for DAGs that can't load the grid view.

• Looking at grid.js, the error happens while filtering an array of instances based on a runId. Some of these instances are null, hence the error.

• We initially thought it could be related to our issues while upgrading the db. We could have corrupted some records.
• After more investigation, we noticed the DAG we were looking at had been modified recently (1 task had been added). Looking at our DB, we had 8 records in the task_instance for this new task and 25 records for the other task of this DAG. This matches with what we could see in the array of instances before the filtering.
• To confirm that, we decided to update another DAG with a grid view shown on the UI. We added a task and deployed it. The error could be reproduced, and the grid view wasn't displayed anymore.

If this bug can be reproduced by others, I believe there are 2 ways to fix this:

• In the JS file, we check the instance is not null before checking the runId value.
• We can fetch the last 25 instances for a given task, but if there are less than 25 records, we should not have null values to fill up the array.

[zachliu]

i can reproduce this in 2.3.1

also i had problems on the db migration when i tried to upgrade from Airflow 2.2.5 to 2.3.0

had to remove quite a lot of "dangling" items after the db migration

i think you should create a github issue on this

[zachliu]

ah someone already created one #23588
it seems they have problems reproducing it, this discussion should help them

[jedcunningham]

I think that is a different issue. This one, however, is really easy to reproduce:

from datetime import datetime

from airflow import DAG
from airflow.operators.bash import BashOperator

with DAG(
    dag_id="simple", start_date=datetime(2022, 4, 1), schedule_interval=None
) as dag:
    BashOperator(task_id="hello", bash_command="echo hello")
    # BashOperator(task_id="world", bash_command="echo world")

Run it once with the second task commented, then uncomment it, wait for the DAG to be parsed again, and it'll blow up.

This was not an issue in 2.3.0, so this is a regression in 2.3.1.

[jedcunningham]

I've confirmed this is the PR that caused the regression: #23813

[jedcunningham]

I've got a draft PR with a fix open: #23924

[zachliu]

simple fix indeed 👍

[jedcunningham]

This is a better fix (#23947) and also is much faster.

[jedcunningham]

And #23947, which will be in 2.3.2, fixes this. Thanks everyone.