Skip to content

[GH-3181] GeoPandas: Implement distributed dissolve and tools.collect - #3185

Open
jiayuasu wants to merge 10 commits into
apache:masterfrom
jiayuasu:feature/geopandas-geometry-aggregation
Open

[GH-3181] GeoPandas: Implement distributed dissolve and tools.collect#3185
jiayuasu wants to merge 10 commits into
apache:masterfrom
jiayuasu:feature/geopandas-geometry-aggregation

Conversation

@jiayuasu

@jiayuasu jiayuasu commented Jul 27, 2026

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Is this PR related to a ticket?

What changes were proposed in this PR?

This PR implements distributed GeoPandas-compatible GeoDataFrame.dissolve and geopandas.tools.collect.

dissolve uses a native Spark groupBy with Sedona ST_Union_Aggr and native Spark attribute aggregations. It supports GeoPandas grouping, index, ordering, aggregation, null, empty, CRS, and SRID semantics without a Python row UDF.

tools.collect performs geometry and validation-metadata aggregation in one Spark action with ST_Collect_Aggr on executors, then materializes only the single scalar result required by the API. It does not collect the input geometry rows to the driver.

English and Chinese GeoPandas API documentation are updated for the new APIs.

How was this patch tested?

  • python/tests/geopandas/test_geometry_aggregation.py (14 tests)
  • Repository pre-commit hooks

The tests cover grouped and ungrouped dissolve, Series and MultiIndex groupers, grouping-column dictionaries, as_index, sorting, dropna, supported aggregation forms, source-order-sensitive aggregations, null/empty/all-null geometries, CRS/SRID preservation, native execution plans without Python UDF nodes, validation, and single-action distributed tools.collect.

Did this PR include necessary documentation updates?

  • Yes, I am adding new APIs. I am using the current SNAPSHOT version number v1.9.1.
  • Yes, I have updated the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GeoPandas: implement distributed dissolve and tools.collect

1 participant