Skip to content

feat: deliver provisional Yaoundé urban expansion dataset v2 - #2

Open
Omer-alt wants to merge 12 commits into
mainfrom
feature/data-pipeline
Open

feat: deliver provisional Yaoundé urban expansion dataset v2#2
Omer-alt wants to merge 12 commits into
mainfrom
feature/data-pipeline

Conversation

@Omer-alt

@Omer-alt Omer-alt commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

This PR delivers the provisional Yaoundé urban-expansion dataset v2 at 30 m resolution for 2000–2025. It replaces the former common-window Landsat logic
with exact per-epoch QA-valid window selection and rebuilds the downstream
composites, built-up candidates, states, transitions and model-ready table.

The release contains:

  • 6 built-up states and 5 consecutive five-year transitions;
  • 720,605 eligible cell-time rows across 210,842 unique cells;
  • transition-level demand and support diagnostics;
  • built area, PBA, NUMP and MPS metrics;
  • schemas, data dictionary, dataset card and reproduction documentation.

NDBI is the
operational mapping method, while the generated 270-cell blind validation
sample still requires annotation
.

Main changes

Landsat catalogue and composites

  • Evaluates 3-, 4-, 6- and 12-month windows independently for each epoch.
  • Calculates support from the exact union of QA-valid pixel masks.
  • Achieves at least 95% one-observation core coverage for all six epochs.
  • Adds a usable 2000 epoch and rebuilds all v2 composite/count assets.

Built-up mapping

  • Exports continuous NDBI, IBI, IBUI, VbSWIR1-BI and NDBSUI diagnostics.
  • Compares binary NDBI, IBUI, VbSWIR1-BI and NDBSUI candidates.
  • Keeps IBI continuous-only because of unstable ratio-denominator values.
  • Uses epoch-specific Otsu thresholds on valid core pixels.

Final dataset

  • Applies a provisional NDBI protocol with absorbing built-state persistence on
    valid observations.
  • Constructs transitions only on pairwise common-valid support and for cells
    non-built at forecast origin.
  • Measures predictors at forecast origin to prevent target leakage.
  • Adds final release metadata, checksums, v2 exploration and handover guidance.

Landsat support

Epoch Selected window Scenes Coverage ≥1 obs. Previously (Coverage)
2000 Mar–May, 3 months 6 98.110% 6.191%
2005 Oct–Dec, 3 months 6 95.525% 16.035%
2010 Feb–Jan, 12 months 12 98.648% 57.978%
2015 Mar–May, 3 months 16 98.079% 74.585%
2020 Dec–Feb, 3 months 12 99.888% 99.536%
2025 Nov–Jan, 3 months 19 99.888% 99.870%

All raster products use the frozen EPSG:32632 grid with 30 m cells,
946 × 1278 dimensions and transform [30, 0, 762570, 0, -30, 444150].

Reproduction and storage

The pipeline is configuration-driven and phased because Earth Engine exports
are asynchronous and five cell-time CSVs are downloaded from Google Drive.
Full instructions are in docs/reproduction_and_handover.md.

Large raw, processed and staging files remain outside Git (See on the shared Drive). Git tracks code,
configuration, compact metadata, manifests, checksums, tests and documentation.

Limitations

  • Manual/AI Agent validation is pending/required; NDBI remains provisional.
  • 2000 and 2005 have limited observation depth despite high spatial coverage.
  • DIS, WUP and Sprawl per New Dweller are deferred.
  • 1990 and 1995 are outside the active v2 scope because no usable catalogue
    products were retained under the current protocol.

@Omer-alt
Omer-alt requested a review from rapsoj August 4, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant