datetime in parameters cause "TypeError: Object of type set is not JSON serializable"

[ajbosco]

Apache Airflow version

2.2.2rc1

Operating System

Debian Buster (docker)

Versions of Apache Airflow Providers

n/a

Deployment

Other 3rd-party Helm chart

Deployment details

No response

What happened

The DAG parses and runs fine on 2.1.3, but when we upgrade to the latest version it fails to import:

Broken DAG: [/usr/local/airflow/dags/test_dag.py] Traceback (most recent call last):
  File "/usr/local/lib/python3.8/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type datetime is not JSON serializable

What you expected to happen

No response

How to reproduce

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[jedcunningham]

This should be fixed in 2.2.2 via #19267. Can you test in 2.2.2rc1 and report your results in #19515? Thanks.

[jedcunningham]

@ajbosco, can you provide a DAG that reproduces this? The DAGs I have handy that use set and datetime params work.

[dstandish]

@dag.task(params={"a": {1, 2, 3}, "b": datetime.now()})
def datetime_param(value):
    print(value)

datetime_param("{{ params.a }} | {{ params.b }}")

could not repro with this task

[potiuk]

Try pendulum datetime @dstandish

[jedcunningham]

pendulum.datetime worked for me 🤷‍♂️

[dstandish]

no luck

@dag.task(params={"a": {1, 2, 3}, "b": pendulum.now()})
def datetime_param(value):
    print(value)


datetime_param("{{ params.a }} | {{ params.b }}")

[potiuk]

We need example then @ajbosco :)

[kaxil]

Yup, let's just wait until then.

Also please double-check with the Airflow version in the Footer of the webserver or airflow CLI to confirm that you are using 2.2.2rc1

[ajbosco]

Welp....I think there was some issue in the database I was using. I started over from scratch and cannot reproduce this error either. My bad!