Pin supported upper version of fsspec

[albertvillanova]

Pin upper version of fsspec to avoid disruptions introduced by breaking changes (and the need of urgent patch releases with hotfixes) on each release on their side. See:

• Temporarily pin fsspec < 2023.10.0 #6331
• Temporarily pin fsspec < 2023.9.0 #6210
• Temporarily pin fsspec #5731
• Fix CI by temporarily pinning fsspec < 2023.3.0 #5617
• Fix CI by temporarily pinning fsspec < 2023.1.0 #5447

I propose that we explicitly test, introduce fixes and support each new fsspec version release.

CC: @LysandreJik

[github-actions]

Show benchmarks

PyArrow==8.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.006915 / 0.011353 (-0.004438)	0.004110 / 0.011008 (-0.006898)	0.084392 / 0.038508 (0.045884)	0.079649 / 0.023109 (0.056540)	0.305760 / 0.275898 (0.029862)	0.343968 / 0.323480 (0.020488)	0.005402 / 0.007986 (-0.002584)	0.003342 / 0.004328 (-0.000986)	0.064774 / 0.004250 (0.060523)	0.055919 / 0.037052 (0.018866)	0.315194 / 0.258489 (0.056705)	0.355014 / 0.293841 (0.061173)	0.032140 / 0.128546 (-0.096406)	0.008865 / 0.075646 (-0.066781)	0.287684 / 0.419271 (-0.131588)	0.053504 / 0.043533 (0.009971)	0.306852 / 0.255139 (0.051713)	0.331125 / 0.283200 (0.047925)	0.023476 / 0.141683 (-0.118207)	1.506590 / 1.452155 (0.054435)	1.574508 / 1.492716 (0.081792)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.239987 / 0.018006 (0.221981)	0.459144 / 0.000490 (0.458654)	0.008509 / 0.000200 (0.008309)	0.000335 / 0.000054 (0.000280)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.028353 / 0.037411 (-0.009058)	0.082345 / 0.014526 (0.067819)	0.499524 / 0.176557 (0.322967)	0.152896 / 0.737135 (-0.584239)	0.096978 / 0.296338 (-0.199360)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.404855 / 0.215209 (0.189646)	4.053103 / 2.077655 (1.975448)	2.069638 / 1.504120 (0.565518)	1.917354 / 1.541195 (0.376159)	2.035816 / 1.468490 (0.567326)	0.480358 / 4.584777 (-4.104419)	3.594316 / 3.745712 (-0.151396)	3.582952 / 5.269862 (-1.686910)	2.101142 / 4.565676 (-2.464535)	0.057004 / 0.424275 (-0.367271)	0.007715 / 0.007607 (0.000108)	0.487417 / 0.226044 (0.261372)	4.863100 / 2.268929 (2.594172)	2.569038 / 55.444624 (-52.875587)	2.187167 / 6.876477 (-4.689310)	2.270034 / 2.142072 (0.127962)	0.578095 / 4.805227 (-4.227132)	0.133283 / 6.500664 (-6.367381)	0.060164 / 0.075469 (-0.015305)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	1.269120 / 1.841788 (-0.572667)	19.493072 / 8.074308 (11.418764)	14.560576 / 10.191392 (4.369184)	0.167440 / 0.680424 (-0.512984)	0.018493 / 0.534201 (-0.515708)	0.392774 / 0.579283 (-0.186509)	0.420903 / 0.434364 (-0.013461)	0.461904 / 0.540337 (-0.078433)	0.643104 / 1.386936 (-0.743832)

PyArrow==latest

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.006985 / 0.011353 (-0.004368)	0.004246 / 0.011008 (-0.006762)	0.066246 / 0.038508 (0.027738)	0.080757 / 0.023109 (0.057648)	0.391774 / 0.275898 (0.115876)	0.424957 / 0.323480 (0.101478)	0.005575 / 0.007986 (-0.002411)	0.003447 / 0.004328 (-0.000881)	0.066565 / 0.004250 (0.062315)	0.057597 / 0.037052 (0.020544)	0.394663 / 0.258489 (0.136174)	0.430310 / 0.293841 (0.136469)	0.032746 / 0.128546 (-0.095800)	0.008783 / 0.075646 (-0.066863)	0.071940 / 0.419271 (-0.347331)	0.048877 / 0.043533 (0.005344)	0.390269 / 0.255139 (0.135130)	0.411867 / 0.283200 (0.128668)	0.024101 / 0.141683 (-0.117582)	1.507370 / 1.452155 (0.055215)	1.585810 / 1.492716 (0.093093)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.222796 / 0.018006 (0.204790)	0.459035 / 0.000490 (0.458546)	0.005322 / 0.000200 (0.005122)	0.000099 / 0.000054 (0.000045)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.033237 / 0.037411 (-0.004174)	0.098244 / 0.014526 (0.083718)	0.106654 / 0.176557 (-0.069903)	0.159675 / 0.737135 (-0.577460)	0.108470 / 0.296338 (-0.187869)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.429085 / 0.215209 (0.213876)	4.281206 / 2.077655 (2.203551)	2.320492 / 1.504120 (0.816372)	2.153218 / 1.541195 (0.612024)	2.287122 / 1.468490 (0.818632)	0.497307 / 4.584777 (-4.087470)	3.799541 / 3.745712 (0.053828)	3.380053 / 5.269862 (-1.889809)	2.100009 / 4.565676 (-2.465667)	0.057988 / 0.424275 (-0.366287)	0.007381 / 0.007607 (-0.000226)	0.506843 / 0.226044 (0.280798)	5.071286 / 2.268929 (2.802357)	2.750487 / 55.444624 (-52.694137)	2.415613 / 6.876477 (-4.460864)	2.667144 / 2.142072 (0.525072)	0.624889 / 4.805227 (-4.180338)	0.134191 / 6.500664 (-6.366473)	0.060704 / 0.075469 (-0.014765)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	1.353074 / 1.841788 (-0.488714)	20.507074 / 8.074308 (12.432766)	14.911788 / 10.191392 (4.720396)	0.149248 / 0.680424 (-0.531176)	0.020593 / 0.534201 (-0.513608)	0.398458 / 0.579283 (-0.180825)	0.434846 / 0.434364 (0.000482)	0.478853 / 0.540337 (-0.061484)	0.648072 / 1.386936 (-0.738864)

[lhoestq]

In particular I expect fsspec to do another breaking change in the next release (switch to glob.glob)

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[lhoestq]

see #6338

[albertvillanova]

Yes, unfortunately breaking changes are quite usual from their part.

[github-actions]

Show benchmarks

PyArrow==8.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.006099 / 0.011353 (-0.005253)	0.003672 / 0.011008 (-0.007336)	0.083095 / 0.038508 (0.044587)	0.059607 / 0.023109 (0.036498)	0.319591 / 0.275898 (0.043693)	0.351945 / 0.323480 (0.028465)	0.004785 / 0.007986 (-0.003201)	0.002965 / 0.004328 (-0.001364)	0.062907 / 0.004250 (0.058657)	0.049122 / 0.037052 (0.012070)	0.344641 / 0.258489 (0.086152)	0.361519 / 0.293841 (0.067678)	0.027254 / 0.128546 (-0.101292)	0.008081 / 0.075646 (-0.067565)	0.261569 / 0.419271 (-0.157702)	0.045101 / 0.043533 (0.001568)	0.313645 / 0.255139 (0.058506)	0.337843 / 0.283200 (0.054644)	0.020968 / 0.141683 (-0.120715)	1.438450 / 1.452155 (-0.013705)	1.507567 / 1.492716 (0.014850)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.230826 / 0.018006 (0.212820)	0.434363 / 0.000490 (0.433873)	0.008210 / 0.000200 (0.008010)	0.000212 / 0.000054 (0.000157)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.025278 / 0.037411 (-0.012133)	0.073659 / 0.014526 (0.059133)	0.085147 / 0.176557 (-0.091409)	0.145451 / 0.737135 (-0.591684)	0.086400 / 0.296338 (-0.209939)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.429887 / 0.215209 (0.214678)	4.292626 / 2.077655 (2.214971)	2.266824 / 1.504120 (0.762704)	2.090472 / 1.541195 (0.549277)	2.186477 / 1.468490 (0.717987)	0.503684 / 4.584777 (-4.081093)	3.100791 / 3.745712 (-0.644921)	3.008938 / 5.269862 (-2.260923)	1.885559 / 4.565676 (-2.680118)	0.057434 / 0.424275 (-0.366841)	0.006639 / 0.007607 (-0.000969)	0.506579 / 0.226044 (0.280535)	5.058905 / 2.268929 (2.789977)	2.708321 / 55.444624 (-52.736304)	2.367388 / 6.876477 (-4.509089)	2.422660 / 2.142072 (0.280587)	0.587562 / 4.805227 (-4.217665)	0.125260 / 6.500664 (-6.375404)	0.061856 / 0.075469 (-0.013613)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	1.280495 / 1.841788 (-0.561292)	17.968873 / 8.074308 (9.894565)	13.922838 / 10.191392 (3.731446)	0.149907 / 0.680424 (-0.530517)	0.016736 / 0.534201 (-0.517465)	0.333417 / 0.579283 (-0.245866)	0.367710 / 0.434364 (-0.066654)	0.389648 / 0.540337 (-0.150690)	0.535625 / 1.386936 (-0.851311)

PyArrow==latest

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.006237 / 0.011353 (-0.005116)	0.003787 / 0.011008 (-0.007221)	0.062536 / 0.038508 (0.024028)	0.062335 / 0.023109 (0.039226)	0.455209 / 0.275898 (0.179311)	0.488961 / 0.323480 (0.165482)	0.004875 / 0.007986 (-0.003111)	0.002961 / 0.004328 (-0.001368)	0.063045 / 0.004250 (0.058795)	0.048624 / 0.037052 (0.011571)	0.455743 / 0.258489 (0.197254)	0.494024 / 0.293841 (0.200183)	0.028690 / 0.128546 (-0.099856)	0.008147 / 0.075646 (-0.067499)	0.069479 / 0.419271 (-0.349792)	0.041613 / 0.043533 (-0.001919)	0.460472 / 0.255139 (0.205333)	0.475606 / 0.283200 (0.192406)	0.020600 / 0.141683 (-0.121083)	1.464960 / 1.452155 (0.012805)	1.540942 / 1.492716 (0.048226)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.214558 / 0.018006 (0.196552)	0.410482 / 0.000490 (0.409992)	0.005539 / 0.000200 (0.005339)	0.000076 / 0.000054 (0.000021)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.027044 / 0.037411 (-0.010367)	0.081512 / 0.014526 (0.066986)	0.101963 / 0.176557 (-0.074593)	0.146686 / 0.737135 (-0.590449)	0.092676 / 0.296338 (-0.203663)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.468766 / 0.215209 (0.253557)	4.680514 / 2.077655 (2.602859)	2.562454 / 1.504120 (1.058334)	2.383692 / 1.541195 (0.842497)	2.481820 / 1.468490 (1.013330)	0.509122 / 4.584777 (-4.075655)	3.201597 / 3.745712 (-0.544115)	2.853539 / 5.269862 (-2.416323)	1.891535 / 4.565676 (-2.674141)	0.058594 / 0.424275 (-0.365681)	0.006448 / 0.007607 (-0.001159)	0.535950 / 0.226044 (0.309906)	5.388239 / 2.268929 (3.119311)	2.999986 / 55.444624 (-52.444638)	2.733291 / 6.876477 (-4.143186)	2.841548 / 2.142072 (0.699475)	0.602388 / 4.805227 (-4.202840)	0.126369 / 6.500664 (-6.374295)	0.061519 / 0.075469 (-0.013951)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	1.322746 / 1.841788 (-0.519042)	17.940825 / 8.074308 (9.866517)	14.679559 / 10.191392 (4.488167)	0.146481 / 0.680424 (-0.533943)	0.018060 / 0.534201 (-0.516141)	0.334924 / 0.579283 (-0.244359)	0.384735 / 0.434364 (-0.049629)	0.391834 / 0.540337 (-0.148503)	0.540011 / 1.386936 (-0.846925)