Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion airflow-core/docs/authoring-and-scheduling/deferring.rst
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,11 @@ A *Trigger* is written as a class that inherits from ``BaseTrigger``, and implem
* ``run``: An asynchronous method that runs its logic and yields one or more ``TriggerEvent`` instances as an asynchronous generator.
* ``serialize``: Returns the information needed to re-construct this trigger, as a tuple of the classpath, and keyword arguments to pass to ``__init__``.

Two optional lifecycle hooks are also available:

* ``cleanup``: Called after ``run`` exits for any reason (success, timeout, triggerer shutdown, or user kill). Use this to release local resources held by the trigger instance, such as open connections or temporary files.
* ``on_kill``: Called only when a user explicitly kills the deferred task (mark-failed, clear, or mark-succeeded). Use this to cancel external work — for example, cancelling a BigQuery job or terminating a Databricks run — that you do not want to keep running after the user acts on the task. Unlike ``cleanup``, ``on_kill`` is **not** called on triggerer restart or redistribution, so you can safely put external cancellation logic here without risk of cancelling in-flight work during a rolling deploy.

This example shows the structure of a basic trigger, a very simplified version of Airflow's ``DateTimeTrigger``:

.. code-block:: python
Expand Down Expand Up @@ -155,7 +160,7 @@ There's some design constraints to be aware of when writing your own trigger:
* ``run`` must ``yield`` its TriggerEvents, not return them. If it returns before yielding at least one event, Airflow will consider this an error and fail any Task Instances waiting on it. If it throws an exception, Airflow will also fail any dependent task instances.
* You should assume that a trigger instance can run *more than once*. This can happen if a network partition occurs and Airflow re-launches a trigger on a separated machine. So, you must be mindful about side effects. For example you might not want to use a trigger to insert database rows.
* If your trigger is designed to emit more than one event (not currently supported), then each emitted event *must* contain a payload that can be used to deduplicate events if the trigger is running in multiple places. If you only fire one event and don't need to pass information back to the operator, you can just set the payload to ``None``.
* A trigger can suddenly be removed from one triggerer service and started on a new one. For example, if subnets are changed and a network partition results or if there is a deployment. If desired, you can implement the ``cleanup`` method, which is always called after ``run``, whether the trigger exits cleanly or otherwise.
* A trigger can suddenly be removed from one triggerer service and started on a new one. For example, if subnets are changed and a network partition results or if there is a deployment. If desired, you can implement the ``cleanup`` method, which is always called after ``run``, whether the trigger exits cleanly or otherwise. If you need to cancel external work when a user explicitly kills the task, implement ``on_kill`` instead — it is only called for user-initiated cancellations, not on restart or redistribution.
* In order for any changes to a trigger to be reflected, the *triggerer* needs to be restarted whenever the trigger is modified.
* Your trigger must not come from a Dag bundle - anywhere else on ``sys.path`` is fine. The triggerer does not initialize any bundles when running a trigger.

Expand Down
1 change: 1 addition & 0 deletions docs/spelling_wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@ camelCase
Cancelled
cancelled
Cancelling
cancelling
cas
Cassandra
cassandra
Expand Down
Loading