Skip to content

GCSToBigQueryOperator max_id_key Feature Throws Error #26279

Description

@patricker

Apache Airflow Provider(s)

google

Versions of Apache Airflow Providers

8.3.0

Apache Airflow version

2.3.4

Operating System

OSX

Deployment

Virtualenv installation

Deployment details

No response

What happened

When using the max_id_key feature of GCSToBigQueryOperator it fails with the error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/airflow/providers/google/cloud/transfers/gcs_to_bigquery.py", line 312, in execute
    row = list(bq_hook.get_job(job_id).result())
  File "/usr/local/lib/python3.9/site-packages/airflow/providers/google/common/hooks/base_google.py", line 444, in inner_wrapper
    raise AirflowException(
airflow.exceptions.AirflowException: You must use keyword arguments in this methods rather than positional

What you think should happen instead

The max id value for the key should be returned.

How to reproduce

Any use of this column fails, since the error is related to retrieving the job result.

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions