feat: Use threading in the standard pipeline and move old behavior to legacy

[dolfim-ibm]

This PR is promoting the replacing the StandardPdfPipeline with the new multi-threads version (which was only experimental). The old behavior is still available as legacy.

More details on the new pipeline design and performance in #1951.

Parameters controlling the pipeline:

# batch size in the given stage
pipeline_options.ocr_batch_size = 4
pipeline_options.layout_batch_size = 4
pipeline_options.table_batch_size = 4

# maximum number of pages put in a stage queue
# potentially so many pages opened at the same time (watch for memory)
pipeline_options.queue_max_size = 100

Checklist:

• Documentation has been updated, if necessary.
• Examples have been added, if necessary.
• Tests have been added, if necessary.

[github-actions]

✅ DCO Check Passed

Thanks @dolfim-ibm, all your commits are properly signed off. 🎉

[dosubot]

Related Documentation

Checked 3 published document(s) in 1 knowledge base(s). No updates required.

^{How did I do? Any feedback?}  

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[codecov]

Codecov Report

❌ Patch coverage is 65.91928% with 152 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling/pipeline/legacy_standard_pdf_pipeline.py	0.00%	102 Missing ⚠️
docling/pipeline/standard_pdf_pipeline.py	84.93%	50 Missing ⚠️

📢 Thoughts on this report? Let us know!

[PeterStaar-IBM]

lgtm!

[PeterStaar-IBM]

wonderful!