[C++][Parquet] Proof-of-concept: Trying to using FlatBuffer as Parquet Footer

[mapleFU]

Describe the enhancement requested

Background: Parquet Metadata evolution

Should we just do a POC for this one?

Component(s)

C++, Parquet

[mapleFU]

@alkis Do we have some POC or code with Scrub? I think we may first do a POC with some Metadata API here?

[alkis]

I have one but I need to port it to arrow style.

[mapleFU]

Great! Thanks for the effort!

I would also glad to help with metadata api in current parquet C++, since it's also a bit weird. If any problem on adapting, just ping me

[alkis]

My prototype is focused on converting thrift to flatbuf while preserving most of the semantics. I haven't looked at the metadata API in detail, I assume that will remain largely the same.

[mapleFU]

This is appreciated but I think the problem is currently the metadata highly relies or thrift impl 🤔. Glad to see the patch!

[alkis]

I am trying to add the benchmark under src/parquet/metadata3_benchmark.cc but I have trouble with cmake. How do I write the rule such that it links absl in?

I added this to src/parquet/CMakeLists.txt:

add_parquet_benchmark(metadata3_benchmark)

But the absl headers are not found.

[mapleFU]

Seems flight-rpc includes absl, maybe target_link... for absl would work

Also cc @kou for help

[kou]

How about add_parquet_benchmark(metadata3_benchmark EXTRA_LINK_LIBS absl::XXX)?

[kou]

BTW, can we link #43793 to this?

[alkis]

BTW, can we link #43793 to this?

Done.