Add parameter deferrable to AzureContainerInstanceOperator

[tsanton]

Description

AzureContainerInstancesOperator currently blocks a worker slot while it polls Azure Container Instances (ACI) for container group state and fetches logs in a tight loop. For long-running containers, this ties up a regular worker even though the task is mostly waiting on an external service.

Similar to the recent work adding deferrable support to AzureBatchOperator, it would be useful to add a deferrable mode to AzureContainerInstancesOperator so it can suspend itself and resume via a triggerer.

Proposed change:

• Add a deferrable: bool = False parameter to AzureContainerInstancesOperator that enables async/deferrable execution.

• Add a remove_on_success: bool = True parameter to AzureContainerInstancesOperator. Current behavior deletes the container group in finally; deferrable mode should preserve this.

• Logs + XCom: Today the operator streams logs into task logs during execution and optionally pushes logs to XCom. Deferrable mode cannot “live stream” logs in the same way without extra complexity, so we skip live log streaming in deferrable mode and fetch all in execute_complete

Use case/motivation

Adding this parameter and changing the operator to use asynchronous calls would reduce resource consumption when using the Azure Container Instance Operator.

Related issues

Add parameter deferrable to AzureBatchOperator: #59779
Add deferrable support to AzureBatchOperator: #59798

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[SameerMesiah97]

If everyone else is okay, I can take this.

[cruseakshay]

@SameerMesiah97 I gave this a try. When you get a chance, please take a look at #62772.