RS_ raster predicates: reconsider assuming WGS84 for a missing CRS

[james-willis]

RS_Intersects/RS_Contains/RS_Within default a missing CRS on either operand to WGS84. Now that the GeoParquet reader attaches CRSes, CRS-less-but-actually-WGS84 values are much rarer, so this default may do more harm than good — a one-sided-CRS error would surface the mismatch instead of silently assuming a frame.

This tracks the Sedona (Spark) side. The SedonaDB side: apache/sedona-db#1075.

[jiayuasu]

@james-willis

After exploring the implementation in #3194, I think the alignment direction should be reversed: Sedona Spark should retain its established behavior, and SedonaDB should align with it.

Sedona Spark currently applies a simple rule when raster CRS or geometry SRID metadata is missing:

• A missing raster CRS is interpreted as WGS84.
• Geometry SRID 0 is interpreted as WGS84.
• Operands with known, different CRSs are transformed to a common CRS before comparison.

This rule is imperfect because missing metadata may describe data that is not actually WGS84. However, it gives every operand an independent coordinate interpretation. That property is particularly important for optimized spatial joins.

Raster CRS is stored per value in Spark, so one raster column may contain rows using EPSG:4326, EPSG:3857, custom non-EPSG CRSs, and no CRS. A strict “exactly one operand has a CRS must error” rule cannot generally be validated once at planning time. If validation occurs only after envelope filtering, the error becomes data-dependent because the index may discard the incompatible pair before the scalar predicate evaluates it. Guaranteeing the error would require a preflight scan and classification of both inputs.

We also considered treating an SRID-0 geometry as native raster coordinates when the raster has a custom non-EPSG CRS. This keeps custom rasters usable, but introduces pair-dependent semantics: the same geometry row can be interpreted in a different CRS for every raster row. It also prevents the geometry from having one independently computable envelope for the spatial index. Supporting this correctly requires conservative filtering, multiple join paths, or significant loss of pruning.

The existing WGS84 default avoids these problems:

• Every operand can be normalized independently.
• Mixed per-row raster CRSs are supported naturally.
• Scalar predicates, range joins, and broadcast-index joins use the same interpretation.
• No preflight validation job is required.
• Join optimization does not change whether a query returns a result or raises a CRS error.

The strict one-sided-CRS behavior and optimized raster–vector join are relatively recent in SedonaDB, including the work in apache/sedona-db#1073. Sedona Spark’s behavior is older and has a much larger compatibility surface across existing raster functions and applications. Changing Spark would therefore introduce substantially more risk than aligning the newer SedonaDB behavior with Spark.

I propose that both engines use the established Sedona Spark rule:

1. Missing CRS or SRID metadata is interpreted as WGS84.
2. Known CRSs are respected and transformed when necessary.
3. Scalar predicates and optimized joins must use the same interpretation.
4. Documentation should strongly recommend assigning explicit CRS/SRID metadata whenever it is known.

This intentionally favors backward compatibility and consistent query execution over strict detection of missing metadata. A strict mode could be reconsidered later as an opt-in or coordinated breaking change, ideally with schema-level CRS information and a migration path.

[james-willis]

@paleolimbot

[paleolimbot]

I don't see this as an alignment issue...Sedona users migrating to SedonaDB may see an error and can make their CRS explicit with ST_SetCrs() or ST_SetSRID(). They should be doing this anyway, particular for raster inputs. The resulting code will run on both engines.

We could also adopt type-level CRS tracking for Rasters, too (which would allow this error to be caught at planning time). The issue with the silent assumption is that it can produce silently incorrect results rather than an error.

As a migration path for Sedona Spark, you could have an opt in warning, followed by an opt out warning on the first instance of the encountered behaviour, followed by a behaviour change. If you don't get any feedback by the second stage, we will know this was behaviour that wasn't used anyway (do we have evidence that it's being used today?)