Track a future SandboxExecutor for isolated Task SDK workloads

[zozo123]

Scope

This issue tracks a future SandboxExecutor that would run a complete Airflow Task SDK ExecuteTask workload in an isolated, ephemeral runtime.

It does not track the narrower agent code-execution tool proposed by #68847, and #68847 does not close this issue.

Sandboxed-execution surfaces

These are separate execution boundaries, not one API and not a monotonic security ranking:

Surface	What moves into the sandbox	Airflow role	Status
Restricted code orchestration	Monty-compatible glue code only	AgentOperator(code_mode=True) orchestrates registered worker-side tools	Implemented by #68407
Sandboxed code execution as a tool	Model-generated full Python	The credentialed agent remains outside and calls a sandbox as a tool	Proposed by #68847
Sandboxed command/job execution	One author-defined command or job	A future resumable/deferrable SandboxOperator could manage files, artifacts, cancellation, and reconnection	Tracked by #69862
Sandboxed TaskInstance execution	The complete Task SDK ExecuteTask workload	A SandboxExecutor becomes responsible for remote task lifecycle	Tracked by this issue

Moving more of the task into a sandbox is not automatically safer for every use case. A whole-task executor normally provides the Task SDK workload with scoped Airflow API access, while sandbox-as-tool can deliberately keep the agent's credentials outside the generated-code environment.

Future SandboxExecutor responsibilities

A sandbox runtime's create/run/destroy operations are not, by themselves, an Airflow executor contract. A production executor must additionally handle:

• deterministic workload identity and idempotent submission;
• capacity, queueing, heartbeat, and terminal-state reconciliation;
• Task SDK token and DAG-bundle delivery;
• durable task logs and result state;
• cancellation and forced termination;
• scheduler-restart adoption and external resource lookup;
• sandbox disappearance, ambiguous provider responses, and retry safety;
• leases, TTLs, and leaked-resource reaping;
• files, artifacts, secrets, network policy, and provider trust boundaries.

The implementation should use Airflow's existing public BaseExecutor and ExecuteTask interfaces rather than treating the agent-focused backend in #68847 as the executor SPI.

Motivation

Potential use cases include:

• isolated per-task dependencies and runtime images;
• bursty or heavyweight jobs without a shared worker image;
• stronger infrastructure boundaries for sensitive workloads;
• ephemeral remote compute without requiring every installation to operate Celery or Kubernetes workers.

A whole-task executor is not the same security solution as isolating only model-generated code. Both may eventually be useful and composable.

Proposed progression

1. Harden and evaluate the narrowly scoped SandboxToolset in Add SandboxToolset for isolated agent code execution in common.ai #68847.
2. Establish backend conformance tests and clear vendor ownership.
3. Prove files, artifacts, cancellation, reconnection, and lifecycle semantics through an operator-level integration.
4. Develop a third-party SandboxExecutor against BaseExecutor and ExecuteTask.
5. Extract broader shared runtime types only after the tool, operator, and executor consumers demonstrate genuinely common semantics.

Related work

• Add code mode (Monty sandbox) to common.ai AgentOperator #68407 — merged Monty-based restricted code orchestration.
• Add SandboxToolset for isolated agent code execution in common.ai #68847 — proposed sandbox-as-tool integration in common.ai.
• https://github.com/zozo123/airflow-provider-sandbox — experimental whole-task executor prototype; not the implementation of Add SandboxToolset for isolated agent code execution in common.ai #68847.

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[zozo123]

Scope update after deeper review and maintainer feedback:

PR #68847 no longer implements the SandboxExecutor originally described by this issue. It now proposes a narrower SandboxToolset in common.ai for executing model-generated Python in an isolated sandbox.

We now see four separate layers:

• Monty code mode: restricted orchestration code invoking registered tools.
• SandboxToolset: full model-generated Python executed in a separate sandbox.
• Future SandboxOperator: an author-defined remote command/job with files and artifacts.
• Future SandboxExecutor: the complete Airflow ExecuteTask workload running remotely.

These are complementary, but they should not share one underspecified API. In particular, the current create/run/destroy backend is a code-sandbox contract, not an Airflow executor contract. A real executor must additionally support workload submission, logs, durable status, cancellation, scheduler-restart adoption, DAG bundles, scoped Execution API access, and leaked-resource reconciliation.

Proposed progression:

1. Harden and land the narrowly scoped SandboxToolset.
2. Add backend conformance tests and vendor-owned adapters.
3. Prove artifacts, cancellation, and reconnection with a resumable/deferrable SandboxOperator.
4. Explore a separate third-party SandboxExecutor using Airflow's existing BaseExecutor and ExecuteTask interfaces.

Therefore this issue remains the future executor tracker; PR #68847 should not be presented as implementing or defining that executor.