Skip to content

[GH-3182] GeoPandas: Implement distributed sjoin_nearest - #3184

Draft
jiayuasu wants to merge 8 commits into
apache:masterfrom
jiayuasu:feature/geopandas-sjoin-nearest
Draft

[GH-3182] GeoPandas: Implement distributed sjoin_nearest#3184
jiayuasu wants to merge 8 commits into
apache:masterfrom
jiayuasu:feature/geopandas-sjoin-nearest

Conversation

@jiayuasu

@jiayuasu jiayuasu commented Jul 27, 2026

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Is this PR related to a ticket?

What changes were proposed in this PR?

This PR implements distributed GeoPandas-compatible sjoin_nearest for the top-level API and GeoDataFrame.

The implementation uses Sedona's distributed KNN join rather than a Cartesian product, Python geometry UDF, or driver-side geometry collection. It supports inner, left, and right joins; equidistant matches; max_distance; distance_col; exclusive; suffixing; named and MultiIndex inputs; null/empty geometries; active geometry; CRS; and SRID preservation.

The SQL KNN predicate gains five- and six-argument forms for query-local include_ties and exclusive controls. The regular planar path now routes non-point queries by their full geometry and reconciles partition-local candidates with bounded per-query state and stable row identities. Legacy three/four-argument and geography tie behavior remain unchanged.

Candidate selection without ties is deterministic across regular and both broadcast plans. max_distance is currently an output post-filter and does not prune the distributed KNN search.

English and Chinese SQL and GeoPandas documentation are updated.

How was this patch tested?

  • Direct KNN iterator and execution-option tests (12 tests)
  • KnnJoinSuite (41 tests)
  • GeoPandas nearest-join and Python ST_KNN plan/API/type tests
  • Spark 3.4 / Scala 2.12 shaded package build
  • Spotless and repository pre-commit hooks

The tests force regular and both broadcast KNN plans and cover non-point queries, duplicate rows, ties, exclusive ranking, outer reconstruction, indexes, column collisions, null/empty geometries, CRS/SRID metadata, and legacy/geography compatibility.

Did this PR include necessary documentation updates?

  • Yes, I am adding new APIs. I am using the current SNAPSHOT version number v1.9.1.
  • Yes, I have updated the documentation.

@jiayuasu
jiayuasu marked this pull request as draft July 28, 2026 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GeoPandas: implement distributed sjoin_nearest

1 participant