Skip to content

Remove unnecessary logs while retrieving connections / variables#51786

Merged
amoghrajesh merged 3 commits intoapache:mainfrom
astronomer:do-not-log-for-backends
Jun 17, 2025
Merged

Remove unnecessary logs while retrieving connections / variables#51786
amoghrajesh merged 3 commits intoapache:mainfrom
astronomer:do-not-log-for-backends

Conversation

@amoghrajesh
Copy link
Copy Markdown
Contributor

When connection / secrets backend isnt setup, we should not log that those entities aren't found there.
It leads to wrong information and confusion.

DAG used:

from __future__ import annotations

from airflow.models.baseoperator import BaseOperator
from airflow import DAG
from airflow.sdk.definitions.connection import Connection


class CustomOperator(BaseOperator):
    def execute(self, context):
        gc = Connection.get("athena_default")
        print("The conn is", gc)
        # print("Conn uri is", gc.get_uri())


with DAG("example_get_connection_from_defn", schedule=None, catchup=False) as dag:
    CustomOperator(task_id="print_conn")

No backend configured, logs show:

[2025-06-16, 17:55:11] INFO - 2025-06-16 12:25:11 [debug    ] DAG file parsed                [task] file=example-metadatadb-backend.py: chan="stdout": source="task"
[2025-06-16, 17:55:11] INFO - 2025-06-16 12:25:11 [debug    ] Connection not found in any of the configured Secrets Backends. Trying to retrieve from API server [task] conn_id=athena_default: chan="stdout": source="task"
[2025-06-16, 17:55:[11](http://localhost:28080/dags/example_get_connection_from_defn/runs/manual__2025-06-16T12:25:06.782393+00:00/tasks/print_conn?try_number=1#11)] INFO - The conn is Connection(conn_id='athena_default', conn_type='athena', description=None, host=None, schema=None, login=None, password=None, port=None, extra=None): chan="stdout": source="task"

After fix:

DAG bundles loaded: dags-folder: source="airflow.dag_processing.bundles.manager.DagBundlesManager"
[2025-06-16, 18:06:[3](http://localhost:28080/dags/example_get_connection_from_defn/runs/manual__2025-06-16T12:36:33.447212+00:00/tasks/print_conn?try_number=1#3)3] INFO - Filling up the DagBag from /files/dags/example-metadatadb-backend.py: source="airflow.models.dagbag.DagBag"
[2025-06-16, 18:06:33] INFO - The conn is Connection(conn_id='athena_default', conn_type='athena', description=None, host=None, schema=None, login=None, password=None, port=None, extra=None): chan="stdout": source="task"

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Comment thread task-sdk/src/airflow/sdk/execution_time/context.py Outdated
Comment thread task-sdk/src/airflow/sdk/execution_time/context.py Outdated
Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>
@amoghrajesh amoghrajesh merged commit 23ecee8 into apache:main Jun 17, 2025
71 checks passed
@amoghrajesh amoghrajesh deleted the do-not-log-for-backends branch June 17, 2025 08:32
github-actions bot pushed a commit that referenced this pull request Jun 17, 2025
…riables (#51786)

(cherry picked from commit 23ecee8)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

Backport successfully created: v3-0-test

Status Branch Result
v3-0-test PR Link

RoyLee1224 pushed a commit to RoyLee1224/airflow that referenced this pull request Jun 21, 2025
kaxil pushed a commit that referenced this pull request Jun 30, 2025
…riables (#51786) (#51826)

(cherry picked from commit 23ecee8)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants