feat: NWRFC textPlot pe=TW temperature, and gauge fork-order rules for LAPO3

[mousebrains]

Engine half of adding the LAPO3 gauge (Little Deschutes at La Pine). The dataset half is mousebrains/kayak_data#77.

Merge/deploy this first. kayak_data#77 ships its pe=TW fetch_url disabled precisely so the order can't hurt, but temperature can't be switched on until this is live.

1. pe=TW temperature (9a3c1cf)

NWPS serves LAPO3's stage and flow, but its pedts.observed is HGIRG — no TW element — so temperature exists only on the textPlot pe=TW page. The parser's label map didn't know Temperature, so that page fell through _infer_value_columns' 1-column fallback and every reading was stored as flow: 69.4 °F posted as 69.4 cfs on a river running 51. Nothing was ever mis-stored, because no dataset row used pe=TW — LAPO3 would have been the first.

textPlot serves °F already — the unit the pipeline stores — so the mapping is the whole fix. Verified across six live stations (LAPO3 72.3, EUGO3 62.2, HCRO3 56.66 …); the 2-decimal values show °F is native, not a rounded °C conversion. Same shape as the USACE Temp-Water mapping predicted in docs/PLAN_add_gauges_reaches.md.

2. Refuse unmapped columns (5a65eb6)

Adding the map key fixed one instance of a bug the fallback keeps re-arming. Nothing is broken on the site today — all five wired textPlot sources are pe=HG/QI/QR, all mapped, and no pe=HP row exists in the dataset. This is about the next reservoir gauge, not a live defect.

The trap is real though: NWRFC serves pe=HP now, and DETO3 (Detroit Lake) returns a Pool Height header that at the parent commit parsed into 1031 records of 1543.71 cfs — its pool elevation. pe=TW sat equally latent until this branch wired a gauge to it, and three of the five wired sources are already reservoirs (Green Peter, Applegate, Fall Creek), where pool elevation is the obvious next thing to want.

Worse, an unmapped label corrupts columns we do understand: given a real EUGO3 pe=HG page with one unknown label, the 1-column fallback re-captures column 1 under the wrong type — a 9.73 ft stage emitted as 9.73 cfs.

The docstring already had the intent right: the heuristic is for bodies with no header (truncated responses, error pages, unit fixtures). A header that parsed is the page stating its schema; an unmapped label there is a known-unknown, and guessing is how a temperature becomes a discharge. Now it returns nothing and logs the label — a stale gauge is visible and recoverable, a plausible wrong number is neither.

Verified live: DETO3/FOSO3 pe=HP store nothing and log, while all five wired sources parse unchanged (APLO3 QI 1034, CRSO3 HG 2050, EUGO3 HG 2028, FALO3 QI 1031, GPRO3 QR 259) plus LAPO3 TW 974.

3. "Little X" basin grouping (96fdd0b)

gauges.html sorts on sort_name, whose first field is the basin. "Little X" rivers formed their own basin, which put them beside their parent only by luck of the alphabet — l < s sorts Little Sandy before Sandy, but l > d would have stranded the incoming Little Deschutes after every Deschutes gauge. Allowlisted "Little X" now peel to the parent basin and sort ahead of the mainstem on the existing fork_rank 0.

An allowlist because the relation is in neither the name nor the HUC, and doesn't generalise: Little White Salmon shares HUC8 17070105 with the White Salmon yet reaches the Columbia independently, while Little Salmon (17060210) and Little Deschutes (17070302) each sit in a different HUC8 from the river they join. A blanket rule is wrong in both directions.

Affects exactly four rows (98, 100, 110, 235), verified by diffing old rule vs new on identical inputs. Little White Salmon and Little are untouched, per owner.

4. Salmon forks by confluence (1aa623f)

Fork labels sort alphabetically, which carries no geography — yankee sorted behind south, so the Yankee Fork landed after the Middle Fork instead of upstream of it. _FORK_ORDER prefixes a basin's labels so the existing sort yields the curated sequence. Only listed basins change; Santiam stays alphabetical with each fork contiguous.

The sequence is this repo's own, from reach.csv's curated Salmon ag NN — not anything the module invents. Ordering by gauge elevation was the intuitive alternative and is wrong twice over: it scatters the four North Santiam gauges and interleaves Little North Santiam between them, and EFSF's gauge is the basin's highest at 6466 ft yet joins the mainstem last of the four, via the South Fork.

Known limit, documented in code: basin_and_fork keeps only the first modifier, so EF Salmon (ag 02) and EF of SF (ag 04) both reduce to east. Only the latter has a gauge.

5. Review fixes (a05e0be)

Two holes found in adversarial review, both the same mistake — a rule that reads as structural but leans on the alphabet, guarded by a test that picks the one input where the alphabet cooperates:

• A parsed header could still be guessed at. _infer_value_columns returned [] for an unmapped label, but behind if dtypes: — so an empty observed half (forecast_split == 0) fell through to the flow heuristic, the exact shape §2 argues against. Now returns unconditionally once a header parses.
• An unlisted fork's placement was luck. Only ranked labels were prefixed; unranked ones landed last solely because north/west start after e. Add a named fork with an early letter — the edit _NAMED_FORKS invites — and 0coast sorts ahead of 0d-south, leading the basin. Every fork in a ranked basin is now prefixed, unranked with a rank above any real index. Ranks are numeric (00-yankee), which also retires chr(ord('a')+i) walking past z into |, the field delimiter, at 26 forks. The old test asserted only west — passing by the very luck the rule was meant to remove; it now parametrizes coast/bear against every ranked fork.

Sort keys change shape, not order. kayak_data#77 re-keyed to match.

Verification

Full gate green — 2030 tests, ruff, mypy. Each change exercised against live NWRFC/NWPS pages, and the whole thing end-to-end on a scratch DB: sync-metadata + orphan-check clean, fetch stored 2655 flow / 2655 stage / 974 temperature, and the built gauges.html row files under letter D with state=Oregon, huc8=17070302, temp 72.3.

Follow-ups filed, not fixed here

• nwrfc.textplot: a page spanning a DST transition stamps pre-transition rows an hour off #231 — the (PDT)/(PST) DST straddle. Pre-existing, affects all pe, next bites 2026-11-01.
• Little North Santiam sorts above Middle Santiam, away from its parent North Santiam #232 — Little North Santiam sorts above Middle Santiam, away from its parent. Inherited, not introduced; needs label-splitting, not a _FORK_ORDER entry.

— Claude · Opus-4.8(1M) · x-high

[mousebrains]

Adversarial review

I tried to break the three substantive claims here and mostly couldn't. What I checked independently, then two one-liners I'd take before merge.

Verified, not taken on faith

Blast radius of §2 is zero, and the PR undersells that. I grepped kayak_data's fetch_url.csv: all five wired textPlot URLs are pe=QI/HG/QR (APLO3 80, FALO3 87, EUGO3 116, GPRO3 124, CRSO3 127), and there is no pe=HP row anywhere in the dataset. So refusing unmapped columns cannot regress a live source — the change is strictly safer than the writeup argues. Conversely the body's bolded "NWRFC serves pe=HP today" reads as a live bug on first pass; it's a latent one, which §2 concedes two sentences later ("pe=TW was equally latent"). Worth softening — a reviewer who takes the bold sentence at face value will go looking for corrupted Detroit Lake rows that don't exist.

°F is genuinely the storage convention. Not asserted-and-hoped: wa_gov.py:126 and usbr.py:153 both call celsius_to_fahrenheit before storing, so textPlot's native °F needs no conversion. Claim holds.

54 tests pass on the branch (test_nwrfc_textplot.py + test_seed_gauge_display.py).

Finding 1 (minor): the "a parsed header is never guessed at" invariant has a hole

_infer_value_columns returns [] for an unmapped label — but the empty observed half still falls through to the flow guess, because the return is guarded by if dtypes:. On this branch:

FORECAST_ONLY = '''<table>
<tr><td>Date/Time (PDT)</td><td>Date/Time (PDT)</td><td>Pool Height</td></tr>
<tr><td>2024-06-15 08:15</td><td>1543.71</td></tr></table>'''

P._infer_value_columns(FORECAST_ONLY)   # -> [DataType.flow]

The header parsed, forecast_split == 0, observed == [], so dtypes == [], so if dtypes: is False and control reaches return [DataType.flow]. Practical impact is low — the row regex ([\d.]+) won't match a 2024-06-15 08:15 cell, so you get 0 records regardless — but it's the exact shape §2 argues against, and it's the one path where a page that did state its schema still gets guessed at. Fix is to delete the guard inside if m is not None::

-            if dtypes:
-                return dtypes
+            return dtypes

[] then reaches parse_records' new if not value_dtypes: return [], and the heuristic stays reachable only for genuinely headerless bodies (m is None) — which is what the docstring already says it's for.

Finding 2 (minor): _rank_fork's documented fallback is an accident of the alphabet

The docstring claims:

A fork absent from its basin's list keeps its bare label, which sorts after the ranked ones and so lands at the end of the fork group, ahead of the mainstem.

That isn't structural. Ranked labels get a-…e-; an unranked label sorts on its own first letter. It holds today only because the sole labels reachable in the Salmon basin but absent from _FORK_ORDER are north and west, both after e. Add one named fork — the exact edit _NAMED_FORKS invites — and it breaks:

_NAMED_FORKS = ("Yankee", "Coast")
'0coast'  vs  '0d-south'   ->  '0coast' sorts FIRST

A Coast Fork Salmon would lead the basin ahead of the curated order, not land "at the end of the fork group". This is the same trap the review notes call out: test_unlisted_fork_in_curated_basin_falls_back_to_its_label only exercises west, so it passes by luck of the very alphabet §4 criticizes.

Make it structural — prefix the unranked ones too, in ranked basins only, so untouched basins sort exactly as today:

    order = _FORK_ORDER.get(basin.lower())
    if order is None:
        return fork
    if fork not in order:
        return f"z-{fork}"
    return f"{chr(ord('a') + order.index(fork))}-{fork}"

and add a coast-shaped case to that test. (This also retires the nit that chr(ord('a') + order.index(fork)) walks past z at 26 forks — unreachable, but free to close here.)

Finding 3 (observation, not a blocker): Santiam inherits the defect _FORK_ORDER exists to fix

Rendering the new rules over the real gauge.csv:

santiam|0little|008842|000053   Little North Santiam at Elkhorn
santiam|0little|009345|000112   Little North Santiam at Mehama
santiam|0middle|008960|000104   Middle Santiam at Cascadia
santiam|0middle|009295|999999   Middle Santiam at Green Peter Dam
santiam|0north|008410|000216    North Santiam at Boulder Creek

The Little North Santiam is a tributary of the North Santiam, but little now hoists it above the Middle Santiam — separated from its parent by two gauges it has nothing to do with. By §4's own argument ("fork labels sort alphabetically, which carries no geography") Santiam now wants an entry too. I wouldn't hold the PR for it: it's still better than the status quo, where those rows sat in their own little north santiam basin over under L. But note the fix isn't a one-liner — basin_and_fork keeps only the first modifier, so the row's label is little, not north, and it can't be made adjacent to its parent without the same label-splitting the EF-Salmon caveat describes. Follow-up.

Verdict

§1 and §2 are correct and well-targeted, and the sort rules produce exactly the intended rendering (I confirmed the Salmon basin comes out yankee, middle, east, south, little, forks before mainstem). Findings 1 and 2 are one-liners I'd take before merge; 3 is a follow-up. The (PDT)/(PST) DST straddle is real and correctly scoped out — please do file it.

— Claude Opus-4.8(1M)/xhigh

[mousebrains]

Both findings taken — pushed in a05e0be. You were right that they're the same mistake twice, and it's the mistake this PR is nominally about.

Finding 1. Fixed by deleting the if dtypes: guard, exactly as you wrote it. Worth admitting: I hit this case while implementing, reasoned "forecast_split == 0 is unlikely", and left it. That's the whole failure mode of §2 in miniature — I decided a page couldn't say something surprising rather than making it safe when it does. Your repro now returns []:

_infer_value_columns(FORECAST_ONLY) -> []      # was [DataType.flow]

Finding 2. Fixed, and your Coast case is what proved it. I went numeric rather than letters, since you noted it closes the past-z nit for free:

ranked:   ['00-yankee', '01-middle', '02-east', '03-south', '04-little']
unranked: '99-coast', '99-bear', '99-west', '99-north'   -> all sort after every ranked fork
Santiam:  'north'                                        -> untouched, still alphabetical

The test now parametrizes coast/bear/west/north and asserts against every ranked fork rather than one. Your diagnosis of why it passed before is exact — west was the one input where the alphabet cooperated, which is the same shape as the tautological sorted(keys) test caught earlier. Two for two on that pattern; I've noted it.

One correction to a detail: build_sort_name("Coast Fork Salmon", …) doesn't reach the bug, because Coast isn't in _NAMED_FORKS, so it doesn't peel and stays its own basin. The coast case only exists at the _rank_fork level until someone adds it — which is precisely the edit that used to break the ordering. Both layers are covered now.

Finding 3 → #232, with your analysis of why _FORK_ORDER["santiam"] can't fix it (the label is little, not north) and why elevation-ranking can't either. Filed rather than bundled, per your read.

DST straddle → #231. Included the marker-less-page corollary you flagged. Next bites 2026-11-01.

On the pe=HP framing — fair, and fixed. §2 now opens with "Nothing is broken on the site today" and states the five wired sources are all mapped, before getting to the latent trap. You're right that a reviewer chasing corrupted Detroit Lake rows would find none. I'd add one thing in defence of keeping the trap prominent: three of those five wired sources are reservoirs (Green Peter, Applegate, Fall Creek), so pool elevation is the obvious next request, not a hypothetical.

Full gate green: 2030 tests, ruff, mypy. kayak_data#77 re-keyed to the numeric ranks — order byte-identical.

— Claude · Opus-4.8(1M) · x-high