Review fixes: deploy/onboarding, schema & doc drift, CI hardening

.github/dependabot.yml

@@ -0,0 +1,34 @@
+# Dependabot — automated dependency PRs. Monthly + grouped to keep noise low
+# on a single-maintainer project. Python deps are managed by uv (uv.lock); the
+# pip ecosystem is intentionally omitted because Dependabot doesn't update
+# uv.lock — bump those with `uv lock --upgrade`. The ruff version pinned in
+# ci.yml / .pre-commit-config.yaml is likewise a manual bump (keep the two in
+# sync). See docs review / PLAN_pre_release_followup.md.
+version: 2
+updates:
+  - package-ecosystem: github-actions
+    directory: "/"
+    schedule:
+      interval: monthly
+    groups:
+      actions:
+        patterns: ["*"]
+    open-pull-requests-limit: 3
+
+  - package-ecosystem: npm
+    directory: "/"
+    schedule:
+      interval: monthly
+    groups:
+      npm:
+        patterns: ["*"]
+    open-pull-requests-limit: 3
+
+  - package-ecosystem: composer
+    directory: "/"
+    schedule:
+      interval: monthly
+    groups:
+      composer:
+        patterns: ["*"]
+    open-pull-requests-limit: 3

.github/workflows/ci.yml

@@ -34,7 +34,9 @@ jobs:
           python-version: "3.13"
           cache: 'pip'
           cache-dependency-path: pyproject.toml
-      - run: pip install ruff
+      # Pin to the version in uv.lock / .pre-commit-config.yaml so the format
+      # gate can't drift when ruff ships a release (the churn behind df6e422).
+      - run: pip install ruff==0.15.11
       - run: ruff check src/ tests/
       - run: ruff format --check src/ tests/
 
@@ -127,7 +129,7 @@ jobs:
         run: FLOOR_PERCENT=5 scripts/check-php-coverage.sh coverage.xml
 
       - name: PHPStan
-        # Level 7 with phpstan-baseline.neon grandfathering the 123
+        # Level 8 with phpstan-baseline.neon grandfathering the 79
         # pre-existing issues. Per docs/PLAN_php_layer_split.md, the
         # baseline shrinks tier-by-tier as files are split.
         # --memory-limit bumped because TypeSpecifier on the larger

.pre-commit-config.yaml

@@ -62,7 +62,7 @@ repos:
         stages: [pre-commit]
       - id: phpstan
         name: PHPStan
-        # Same gate CI runs in lint-misc (level 7 with phpstan-baseline.neon
+        # Same gate CI runs in lint-misc (level 8 with phpstan-baseline.neon
         # grandfathering). Surfaces typed-PHP regressions before push
         # instead of waiting for the CI cycle. Per PLAN_pre_release_followup.md T2.6.
         entry: vendor/bin/phpstan analyse --memory-limit=1G --no-progress

CHANGELOG.md

@@ -8,6 +8,26 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 
+### Added
+- **Gradient profile**: per-reach `max_gradient` plus a statistically-binned
+  gradient chart on reach-detail pages (DEM-sampled; canyon-trace artifacts
+  flagged via `reach.gradient_unreliable`).
+
+### Fixed
+- **rDNS resolver bounded** so `kayak-status.service` no longer times out on
+  slow reverse lookups (wall-clock budget + negative-cache backoff).
+- **Deploy path**: the documented quick-start now loads the metadata snapshot
+  so a fresh DB renders data; `systemd/install.service.sh` installs all 15
+  timers; `deploy/SETUP.md` deploys the live `conf/sites/` split instead of the
+  retired `deploy/levels`; migrations 0052/0054/0055/0056 no longer carry stray
+  `BEGIN/COMMIT` that broke the runner's transaction wrapper.
+- **Docs drift**: schema-doc table count corrected to 24/25 and the dropped
+  `maintainer_credential` table removed; hardware specs corrected to the
+  Hetzner CPX11 (2 vCPU / 2 GB / 40 GB).
+
+### Changed
+- Pinned `ruff` in pre-commit/CI to match `uv.lock` and stop formatter drift.
+
 ## [1.1.1] - 2026-05-21
 
 ### Added

CLAUDE.md

@@ -28,11 +28,18 @@ POSIX ACLs grant `www-data` access: execute-only on `/home/pat` and `/home/pat/k
 ```bash
 python3 -m venv /home/pat/.venv
 /home/pat/.venv/bin/pip install -e ".[dev]"
-/home/pat/.venv/bin/levels init-db       # Creates schema, seeds states/sources/fetch_urls
-/home/pat/.venv/bin/levels pipeline      # Fetch live data and generate HTML
+/home/pat/.venv/bin/levels init-db --no-seed            # Empty schema + stamp migrations
+/home/pat/.venv/bin/python scripts/import_metadata.py   # Load gauges/reaches/sources from data/db/*.csv
+/home/pat/.venv/bin/levels pipeline                     # Fetch live data and generate HTML
 ```
 
-`init-db` seeds states, sources, and fetch URLs from `data/sources.yaml`.
+`init-db` creates the schema and stamps migrations; `--no-seed` skips the
+`sources.yaml` state/source seed so the canonical rows from `data/db/*.csv`
+(loaded by `import_metadata.py`) import without duplicate-by-name sources.
+Without the metadata load every source is an orphan with no `gauge_source`
+link, so `levels pipeline` fails at `orphan-check` and the site renders empty.
+A plain `levels init-db` (seeded from `data/sources.yaml`) is enough for a
+fetch-only smoke test.
 
 ## Build and Development Commands
 
@@ -43,7 +50,7 @@ pip install -e ".[dev]"              # Install in editable mode with dev deps (p
 levels --help                        # CLI entry point (registered in pyproject.toml)
 levels init-db                       # Create tables, seed states/sources, stamp migrations
 levels migrate                       # Apply any pending data/db/migrations/*.sql files
-levels pipeline                      # fetch → fetch-usgs-ogc → calc-rating → update-gauge-cache → calculator → build → orphan-check
+levels pipeline                      # fetch → fetch-usgs-ogc → calc-rating → update-gauge-cache → calculator → build → orphan-check → check-reaches
 levels build                         # Generate static HTML/CSV/text to public_html/
 
 # Less-common subcommands (see `levels <cmd> --help` for details)
@@ -140,6 +147,7 @@ Runs these steps in order:
 5. **calculator** — evaluates `CalcExpression` formulas referencing `LatestObservation` values
 6. **build** — generates per-state HTML pages, CSV, and text files to `public_html/`; inlines CSS and SVG sparklines
 7. **orphan-check** — soft-fails the run (after build) if any fetch-active source lacks a `gauge_source` link; the existing systemd `OnFailure` chain emails + ntfys on the non-zero exit. See `docs/done/PLAN_orphan_sources.md`.
+8. **check-reaches** — soft-fails the run (after build) if any `reach.geom` fails the format / endpoint validator (`kayak.cli.check_reaches.scan_for_issues`); raises so the same `OnFailure` chain fires.
 
 Multi-source gauges aggregate across all linked sources directly: `update-gauge-cache` reads MAX across `gauge_source`, and PHP queries JOIN through `gauge_source` rather than picking a primary source.
 
@@ -151,15 +159,15 @@ Multi-source gauges aggregate across all linked sources directly: `update-gauge-
 
 ### Database
 
-Single normalized SQLite database (`kayak.db`). Schema defined in `src/kayak/db/models.py` (SQLAlchemy 2.x ORM, 25 tables; live DB adds `schema_migrations` for 26 total). Key tables:
+Single normalized SQLite database (`kayak.db`). Schema defined in `src/kayak/db/models.py` (SQLAlchemy 2.x ORM, 24 tables; live DB adds `schema_migrations` for 25 total). Key tables:
 
 - `source` / `gauge` / `gauge_source` — data sources and physical gauge stations. `source.timezone` is an IANA TZ name (populated from `sources.yaml` → `stations:`) used by `BaseParser.dump_to_db` to localize naive timestamps from feeds that publish local time (USBR's per-station local TZ; wa.gov PST year-round). NULL = treat naive as UTC.
 - `observation` — time-series data (source_id, observed_at, data_type, value)
 - `latest_observation` / `latest_gauge_observation` — cached most-recent reading with delta_per_hour
 - `reach` / `reach_state` / `reach_class` / `reach_guidebook` — paddleable runs with state, class, and guidebook relationships
 - `fetch_url` / `calc_expression` — how to obtain data (fetch vs. calculate)
 - `rating` / `rating_data` — gage height ↔ flow conversion tables (dormant — reserved for per-gauge rating curves)
-- `editor` / `editor_session` / `editor_magic_link` / `maintainer_credential` — Phase 1 editor accounts + session cookies
+- `editor` / `editor_session` / `editor_magic_link` — Phase 1 editor accounts + session cookies
 - `change_request` / `change_request_attachment` / `edit_history` — proposal queue + audit trail
 - `huc_name` — WBD HUC2/4/6/8/10/12 name lookup populated by `levels assign-huc`
 - `schema_migrations` — tracks applied `data/db/migrations/*.sql` versions

README.md

@@ -50,13 +50,20 @@ python3 -m venv .venv
 # 2. Activate it — every subsequent step assumes `levels` resolves to .venv/bin/levels.
 source .venv/bin/activate
 
-# 3. Initialize database (creates tables, seeds states/sources from YAML)
-levels init-db
+# 3. Create the schema (empty tables + stamped migrations)
+levels init-db --no-seed
 
-# 4. Run the full pipeline (fetch live data, generate HTML)
+# 4. Load gauge/reach/source metadata from the tracked CSV snapshots.
+#    init-db alone seeds only states + sources/fetch_urls from sources.yaml;
+#    with no gauge_source links the pipeline's orphan-check fails and the site
+#    renders empty. This loads the real gauges, reaches, and source -> gauge
+#    links that make the pipeline produce a populated site.
+python scripts/import_metadata.py
+
+# 5. Run the full pipeline (fetch live data, generate HTML)
 levels pipeline
 
-# 5. Serve locally
+# 6. Serve locally
 php -S localhost:8000 -t public_html
 ```
 
@@ -130,20 +137,23 @@ Key systemd timers:
 - **kayak-cert-expiry** — daily at 06:30 (Let's Encrypt cert health probe; pages on <21 days remaining)
 - **kayak-editor-retention** — daily at 03:45 (prune expired editor sessions + magic links)
 - **kayak-metadata-snapshot** — daily at 04:30 (commit metadata-table drift to `data/db/*.csv`)
+- **kayak-status** — daily at 03:30 (renders the `/_internal/status` operator dashboard to `var/status.html`)
+- **kayak-fetch-osmb** — daily at 03:30 (fetches Oregon State Marine Board hazard/access GeoJSON overlays)
 - **kayak-cert-renewal-test** — weekly Monday 04:15 (`certbot renew --dry-run`)
 - **kayak-backup-weekly** — weekly Sunday 03:15 (4-copy retention; chains to off-site upload via `OnSuccess=`)
 - **kayak-audit-gauges** — weekly Sunday 03:29 (orphan-gauge + reach-mapping audit, emails on drift)
 - **kayak-config-drift** — weekly Sunday 05:30 (diffs repo `conf/`/`deploy/`/`systemd/` against `/etc/`, alerts on drift)
 - **kayak-heartbeat** — weekly Sunday 06:00 (confirms alert pipeline)
+- **kayak-recap** — weekly Monday 07:00 (pipeline-activity recap email from journald events)
 
 ## Documentation
 
 | Document | Contents |
 |----------|----------|
 | [CLAUDE.md](CLAUDE.md) | Architecture, dev setup, conventions, key patterns |
 | [CONTRIBUTING.md](CONTRIBUTING.md) | Development workflow, testing, adding parsers |
-| [deploy/SETUP.md](deploy/SETUP.md) | Production deployment (Hetzner/Oracle Cloud) |
-| [docs/database-schema.md](docs/database-schema.md) | Full schema reference (25 ORM tables + `schema_migrations`) |
+| [deploy/SETUP.md](deploy/SETUP.md) | Production deployment (Hetzner CPX11) |
+| [docs/database-schema.md](docs/database-schema.md) | Full schema reference (24 ORM tables + `schema_migrations`) |
 | [docs/schema-overview.svg](docs/schema-overview.svg) | ER diagram |
 | [docs/security/posture.md](docs/security/posture.md) | Current security posture (controls, accepted findings, operator obligations) |
 

biome.json

@@ -2,17 +2,11 @@
   "$schema": "https://biomejs.dev/schemas/2.4.11/schema.json",
   "files": {
     "includes": [
-      "static/sw.js",
-      "static/map.js",
-      "static/picker.js",
-      "static/reach-map.js",
-      "static/search-map.js",
-      "static/feature-map.js",
-      "static/gauge_picker.js",
-      "static/plot-hover.js",
-      "src/kayak/web/static/levels.js",
-      "src/kayak/web/static/filters.js",
-      "src/kayak/web/static/style.css"
+      "static/**/*.js",
+      "static/**/*.css",
+      "src/kayak/web/static/**/*.js",
+      "src/kayak/web/static/**/*.css",
+      "!static/leaflet.*"
     ]
   },
   "formatter": {