Incorrect logic when reading from local log files

[zachliu]

Apache Airflow version

2.6.0

What happened

The logic of this line

airflow/airflow/utils/log/file_task_handler.py

Line 323 in ab54c63

if not (remote_logs and ti.state not in State.unfinished):

is incorrect.

We are trying to achieve:

• when finished, if we don't have remote logs, then read from local

But the this line does the opposite:

• when not not unfinished (not finished), if we don't have remote logs, then read from local

Also, the "triple negation logic" is really hard to read 😂 It should be

if ti.state in State.finished and not remote_logs:

What you think should happen instead

No response

How to reproduce

Run 2.6.0 with local logs

Operating System

Irrelevant

Versions of Apache Airflow Providers

No response

Deployment

Other

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

cc: @dstandish -> if that's correct assesment, then likely we need 2.6.1 quickly (the triple not logic makes it hard to confirm it 🤣

[zachliu]

my bad, the logic is correct 😅

case#	remote_logs	task state	triple not logic final output
1	no remote logs	unfinished	True (read local)
2	got remote logs	finished	False (don't read local)
3	no remote logs	finished	True (read local)
4	got remote logs	unfinished	True (read local)

but i'm not 100% sure why we still need to read local logs in case 1 and 4 here

[yamrzou]

This issue causes the logs to disappear from the UI once a task is finished..

[potiuk]

This issue causes the logs to disappear from the UI once a task is finished..

@yamrzou I think if you see an issue like that - you should separately report it as an issue and explain in what circumstances/which version/what is your configuration. Adding statement like that without all this information adds noise and does not make anyone any closer to diagnosing and fixing any issue. I am not even sure if you have any issue now or after or before the migrate - such statements are awfully confusing and misleading so either provide the context, or stop making useless comments please.

[yamrzou]

@potiuk I thought the comment was self-explanatory. Here is more context:

I deployed Airflow 2.6.0 on two VMs, one for the webserver and one for the scheduler/worker. When a task is running, the logs are shown correctly in the UI. As soon as the task is finished (either success or failure), the logs tab only shows one line containing the hostname of the worker, as can be seen in the screenshot below:

I tried to track down the cause of the issue, by manually running the following line on the webserver VM:

airflow/airflow/utils/log/file_task_handler.py

Line 500 in 91e18bf

response = _fetch_logs_from_service(url, rel_path)

from airflow.utils.log.file_task_handler import _fetch_logs_from_service
_fetch_logs_from_service('http://10.0.3.36:8793/log/dag_id=DAG_ID/run_id=RUN_ID/task_id=TASK_ID/attempt=ATTEMPT_NUMBER.log', 'dag_id=DAG_ID/run_id=RUN_ID/task_id=TASK_ID/attempt=ATTEMPT_NUMBER.log')

It fetched the complete log successfully. So I suspected that the issue I'm encountering is the same as this one. If not, I'll open a separate issue.

Edit: Looks like someone else opened a similar issue: #31059

[potiuk]

@potiuk I thought the comment was self-explanatory. Here is more context:

Not without explaining your configuration - commenting on a closed PR without providing your context and configuration does not help. Airflow logging is super configurable and we have no idea to read inside your head and find out which configuration you use when you make such comment.

Can yo please comment in the #31059 and explain the logging configuration you have (pretty please). That will help to investigate the root cause.