Fix unit check#14
Merged
Merged
Conversation
The units check was a string comparison against the data request, so a file carrying `kg m^-2 s^-1` was reported as an error telling the modeler it "should be kg m-2 s-1" -- two spellings of the same unit. CF asks only that the attribute be "a string that can be recognized by the UDUNITS package", and UDUNITS recognises both, so the error was a false positive. It is not a hypothetical one: PISM writes `m^2` precisely because pint does not recognise `m2`, and the data request itself writes `m^2` for iareagr and iareafl while writing `m-2`/`s-1` everywhere else, so the checker was demanding the caret in one place and rejecting it in another. `_parse_units` reduces a units string to a canonical (scale, factors) form, where factors is a sorted tuple of (base unit, exponent). That makes the comparison insensitive to the spelling of exponents (`m2`, `m^2`, `m**2`), to the separator between factors (space, `.`, `*`, middle dot), to `/` for negative exponents, and to the order of the factors -- all of which UDUNITS treats as the same unit. Division follows UDUNITS' left-to-right associativity, so `kg/m2 s` is kg m-2 s rather than kg m-2 s-1. The parser is deliberately small, and says so: parentheses, timestamp offsets such as `days since 1850-01-01`, and decimal scale factors return None, and `_units_match` then falls back to the literal string comparison that was there before. So the change can only ever accept more units than before, never fewer, and nothing it does not understand is guessed at. Prefixes are not units it knows: `km` still fails against `m`, as it should. A file spelled exactly as the request still logs the unchanged "The unit is correct: <units>" line, which is what keeps the golden reference log untouched; an equivalent spelling logs "(equivalent to the requested ...)" so the difference is visible rather than silent. Verified: 23 new tests, 31 in total, green. Five of them run the whole checker over a file whose units attribute has been rewritten to an equivalent spelling (`m2`, `m**2`, `kg s^-1`, `kg/s`, `kg.s-1`) and require zero errors; one requires that `m^3` where `m^2` was asked for is still an error, with the message unchanged. The rest table-test `_units_match` directly, including the cases that must not match and the ones the parser declines to interpret. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Compare units by what they mean, not how they are spelled
Fixes #8
The problem
The units check compared the
unitsattribute with the data request as plain strings, so a file carryingkg m^-2 s^-1was flagged with an error telling the modeler it "should bekg m-2 s-1" — two spellings of the same unit. CF requires only that the attribute be "a string that can be recognized by the UDUNITS package", and UDUNITS recognises both, so this was a false positive. It is not a hypothetical one: PISM writesm^2precisely becausepintdoes not recognisem2. The data request is itself of two minds about this — it writesm^2foriareagrandiareaflandm-2/s-1everywhere else — so the checker was demanding the caret in one place and rejecting it in another.The fix
_parse_unitsreduces a units string to a canonical(scale, factors)form, wherefactorsis a sorted tuple of(base unit, exponent)pairs. Comparing those canonical forms makes the check insensitive to everything UDUNITS treats as the same unit: the spelling of exponents (m2,m^2,m**2), the separator between factors (space,.,*, middle dot),/for negative exponents, and the order of the factors. Division follows UDUNITS' left-to-right associativity, sokg/m2 sis kg m-2 s rather than kg m-2 s-1.The parser is deliberately small, and says so. Parentheses, timestamp offsets such as
days since 1850-01-01, and decimal scale factors all returnNone, and_units_matchthen falls back to the literal string comparison that was there before. The change can therefore only ever accept more units than it did, never fewer, and nothing the parser does not understand is guessed at. Unit prefixes are not something it knows about:kmstill fails againstm, as it should.A file spelled exactly as the request still logs the unchanged
The unit is correct: <units>line — which is what leaves the golden reference log untouched — while an equivalent spelling logsThe unit is correct: m2 (equivalent to the requested m^2), so the difference is recorded rather than passing silently.Testing
23 new tests, 31 in total, all green. Five run the whole checker over a file whose
unitsattribute has been rewritten to an equivalent spelling (m2,m**2,kg s^-1,kg/s,kg.s-1) and require zero errors; one requires thatm^3wherem^2was asked for is still an error, with its message unchanged. The rest table-test_units_matchdirectly, covering the equivalences above, the cases that must not match (m3vsm2,kmvsm,Mvsm,kg/m2 svskg m-2 s-1), and the strings the parser declines to interpret.The golden-log regression test passes unchanged, confirming that no existing, exactly-spelled file changes its log output.
Run locally against the latest environment (python 3.14, xarray 2026.7); CI covers the floor environment as well.
Also updated
The check summary at the top of
isschecker/checker.pyand the corresponding bullet inREADME.md, so the documented behavior matches what the checker now does.