[AIRFLOW-1486] Unexpected S3 writing log error

[skudriashev]

Dear Airflow maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

• My PR addresses the following AIRFLOW-1486 issue and references them in the PR title.

Description

• Here are some details about my PR, including screenshots of any UI changes:

Current:

ERROR:root:Could not read logs from s3://airflow-log/my_dag/task1/2017-08-04T16:03:00/1.log
INFO:root:The key my_dag/task1/2017-08-04T16:03:00/1.log now contains 4648 bytes

Expected:

INFO:root:The key my_dag/task1/2017-08-04T16:03:00/1.log now contains 4648 bytes

Tests

• My PR adds the following unit tests:
  • tests.utils.test_logging.TestS3Log

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

[mention-bot]

@skudriashev, thanks for your PR! By analyzing the history of the files in this pull request, we identified @artwr, @criccomini and @allisonwang to be potential reviewers.

[codecov-io]

Codecov Report

Merging #2499 into master will increase coverage by 0.24%.
The diff coverage is 53.84%.

@@            Coverage Diff             @@
##           master    #2499      +/-   ##
==========================================
+ Coverage   69.71%   69.95%   +0.24%     
==========================================
  Files         146      146              
  Lines       11444    11446       +2     
==========================================
+ Hits         7978     8007      +29     
+ Misses       3466     3439      -27

Impacted Files	Coverage Δ
airflow/utils/logging.py	58.82% <53.84%> (+27.19%)	⬆️
airflow/jobs.py	77.67% <0%> (-0.37%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ddc5026...4e30c73. Read the comment docs.

[skudriashev]

@bolkedebruin, @asfgit please have a look. Thanks ;)

[allisonwang]

@skudriashev Thanks for writing tests for S3 logs! Using moto is a better choice as we don't need to mock the behavior of S3. Could you point out how does this change fix the error?

[skudriashev]

@allisonwang when S3hook writes log first time - it tries to read existing log first and since it doesn't exist - error is printed, but doesn't make sense. So I fixed that behavior.

[allisonwang]

Get it. The log doesn't show up in UI but as a debug message. LGTM. Thanks for fixing the issue. I was trying to test S3 log in PR #2464 but moto doesn't work well with current structure. It requires passing mock S3 connection into S3Hook. This test can also be used to test the new S3 log handler.

[aoen]

LGTM, mocking is at least better than nothing so I'll merge this for now and we can follow up with better tests later if we desire. Thanks for your contribution!