[WIP] End-to-end exportable pipelines (object detection)

[yonigozlan]

Add end-to-end pipeline export for object detection

This PR adds the ability to export entire object detection pipelines (including preprocessing and postprocessing) to ONNX and torch.export formats. This is made possible by leveraging the torch/torchvision-based fast image processors that support two-stage preprocessing:

1. First stage (preprocess with intermediate_return=True): Converts PIL images to raw tensor format
2. Second stage (_preprocess): Applies pure torch/torchvision operations (normalization, resizing, etc.) that are fully exportable and compilable

This separation allows the entire preprocessing pipeline to be traced and included in the exported model, while the first stage (PIL → tensor conversion) happens outside the exported graph.

Usage

from transformers import pipeline

# Create pipeline
pipe = pipeline("object-detection", model="facebook/detr-resnet-50", device="cuda")

# Export to torch.export (E2E)
exported = pipe.export(
    example_image="image.jpg",
    format="torch",
    dynamic_shapes=True,
    threshold=0.3,
)

# Export to ONNX (E2E)
pipe.export(
    example_image="image.jpg",
    format="onnx",
    save_path="model.onnx",
    dynamic_shapes=True,
    threshold=0.3,
)

Benchmarks

Tested on CUDA A10G. Benchmarking code available at: https://github.com/yonigozlan/E2E-vision-transformers/blob/main/benchmark_pipeline_export.py

Each configuration represents a different export strategy:

• Normal pipeline: Baseline - no export, standard pipeline inference
• torch.export model-only (fast/slow): Only the model is exported; preprocessing/postprocessing in Python using fast or slow image processor
• torch.export E2E: Full pipeline export (preprocessing + model + postprocessing) using torch.export
• ONNX model-only (fast/slow): Only the model is exported to ONNX; preprocessing/postprocessing in Python using fast or slow image processor
• ONNX E2E: Full pipeline export (preprocessing + model + postprocessing) to ONNX

facebook/detr-resnet-50

Configuration	Inference FPS	Total FPS	Speedup vs Normal	Speedup vs ONNX model-only (fast/slow)
Normal pipeline	12.32	11.54	1.00x
torch.export model-only (fast)	21.32	18.99	1.73x
torch.export model-only (slow)	12.36	11.55	1.00x
torch.export E2E	25.21	21.92	2.05x
ONNX model-only (fast)	28.32	24.19	2.30x
ONNX model-only (slow)	15.84	14.53	1.29x
ONNX E2E	35.50	30.18	2.88x	1.25x / 2.24x

PekingU/rtdetr_v2_r101vd

Configuration	Inference FPS	Total FPS	Speedup vs Normal	Speedup vs ONNX model-only (fast/slow)
Normal pipeline	19.82	17.81	1.00x
torch.export model-only (fast)	20.40	18.19	1.03x
torch.export model-only (slow)	18.20	16.45	0.92x
torch.export E2E	20.97	18.57	1.06x
ONNX model-only (fast)	32.32	27.25	1.63x
ONNX model-only (slow)	26.70	23.15	1.35x
ONNX E2E	36.57	30.81	1.84x	1.13x / 1.37x

PekingU/rtdetr_r50vd_coco_o365

Configuration	Inference FPS	Total FPS	Speedup vs Normal	Speedup vs ONNX model-only (fast/slow)
Normal pipeline	25.21	22.08	1.00x
torch.export model-only (fast)	27.12	23.42	1.08x
torch.export model-only (slow)	23.08	20.34	0.92x
torch.export E2E	28.05	24.00	1.11x
ONNX model-only (fast)	43.79	35.06	1.74x
ONNX model-only (slow)	33.59	28.21	1.33x
ONNX E2E	49.97	40.04	1.98x	1.14x / 1.49x

ustc-community/dfine-large-obj365

Configuration	Inference FPS	Total FPS	Speedup vs Normal	Speedup vs ONNX model-only (fast/slow)
Normal pipeline	17.93	14.72	1.00x
torch.export model-only (fast)	19.27	15.45	1.07x
torch.export model-only (slow)	17.47	14.29	0.97x
torch.export E2E	19.83	12.46	1.11x
ONNX model-only (fast)	45.37	28.77	2.53x
ONNX model-only (slow)	35.39	24.30	1.97x
ONNX E2E	56.56	34.00	3.15x	1.25x / 1.60x

• ONNX E2E 1.13x - 1.25x speedup over ONNX model-only (fast processing) and 1.37x - 2.24x additional speedup over ONNX model-only (slow processing)

This implementation currently supports the object detection pipeline, with plans to extend to all vision tasks in the future.

System config:

CUDA Version: 12.8

onnx                      1.20.0
onnx-ir                   0.1.12
onnxruntime-gpu           1.23.2
onnxscript                0.5.7.dev20251211 (issues with 0.5.6!)
torch                     2.8.0

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: detr, rt_detr, rt_detr_v2

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[github-actions]

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42829&sha=6e96c9

[stevhliu]

looks very cool! let me know if i can help add some docs for this. i think we can have a section for it in Inference > Pipeline API