Support DataFusion PartitionedFile extensions for external index
#5481
haohuaijin
started this conversation in
Feature Requests
Replies: 1 comment 1 reply
-
|
That would be cool, I have to loop deeper to understand the missing pieces here, and if things are done in a format-agnostic ways on the DF side. |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Current datafusion use
PartitionedFile's extensions filed to pass aParquetAccessPlanwhen open a parquet file inParquetOpener::open()to support external index, the related pr apache/datafusion#10813.And i check the vortex document that vortex have the ScanBuilder::with_selection that can do the same thing like
ParquetAccessPlan, butVortexOpener::opencurrent do not use thePartitionedFile's extensions.maybe related to #2254
Beta Was this translation helpful? Give feedback.
All reactions