Handle "completely contained within geographic area" time zone rules

[Manishearth]

https://tc39.es/ecma402/#sec-use-of-iana-time-zone-database lists three rules. The first has been handled since #463, the second was handled in #624.

The third is this:

Any Link name that is not listed in the “TZ” column of file zone.tab and that represents a geographical area entirely contained within the territory of a single ISO 3166-1 Alpha-2 country code must resolve to a primary identifier that also represents a geographical area entirely contained within the territory of the same country code. For example, "Atlantic/Jan_Mayen" must resolve to "Arctic/Longyearbyen".

This cannot be easily done in a data-driven way using tzdb, I think. The spec recommends using timezone.xml. Perhaps we can also manually identify these discrepancies and implement them, since IIRC timezone.xml is out of date in other ways.

[Manishearth]

We fail intl402/Temporal/ZonedDateTime/prototype/equals/canonicalize-timezone because of this.

[Manishearth]

Seems like the packrat stuff is intended to handle this. Unfortunately:

        let moredata = ZoneInfoData::from_filepath(tzdata_path.join("backzone")).unwrap();
        std::dbg!(moredata.pack_rat);

Produces

[provider/src/tzdb/datagen.rs:74:9] moredata.pack_rat = {
    "America/Coral_Harbour": "America/Atikokan",
    "Atlantic/Jan_Mayen": "Europe/Oslo",
    "Africa/Timbuktu": "Africa/Bamako",
}

I think these are the right "packed" timezones BUT they're packed to the wrong thing: Jan_Mayen should produce Longyearbyen, even though Longyearbyen shares data with Oslo.

[justingrant]

I think these are the right "packed" timezones BUT they're packed to the wrong thing: Jan_Mayen should produce Longyearbyen, even though Longyearbyen shares data with Oslo.

Yep, there are no build options of the IANA TZDB that will result in spec-compliant output. There's always some transformation needed to get to match the spec.

If it were me implementing the algorithm defined in https://tc39.es/ecma402/#sup-availablenamedtimezoneidentifiers, then I'd probably want to build a mapping of case-normalized time zone IDs to their country codes. Then you can use that mapping as part of your build process to create the ID => primary ID mapping you'll actually want to include in temporal_rs distributions.

To build the ID => country code mapping, probably the easiest way would be to build TZDB with PACKRATDATA=backzone PACKRATLIST=zone.tab and special-case Atlantic/Jan_Mayen, which last time I looked was the only case where a Link in backzone points to the wrong country code. It seems highly unlikely that we'll see another special case like this in the future.

[justingrant]

This cannot be easily done in a data-driven way using tzdb, I think. The spec recommends using timezone.xml. Perhaps we can also manually identify these discrepancies and implement them, since IIRC timezone.xml is out of date in other ways.

If you mean that if you find problems with timezone.xml you fix them in CLDR, and then temporal_rs uses timezones.xml as its source of time zone IDs, that sounds like a good solution because that will help all CLDR clients and make temporal_rs's job easier because you can just use timezone.xml instead of parsing IANA files directly.

[nekevss]

which last time I looked was the only case where a Link in backzone points to the wrong country code.

Why is there a Link in backzone that is pointing to the wrong country code? Is this a known issue?

[justingrant]

(deleted the older version of this comment because it was inaccurate)

special-case Atlantic/Jan_Mayen, which last time I looked was the only case where a Link in backzone points to the wrong country code.

FWIW, if this special case didn't exist in backzone then we might have written the spec more simply. But given that it does exist, we felt that it would be clearer to look at zone.tab and country codes first, which eliminates that special case.

algorithm defined in https://tc39.es/ecma402/#sup-availablenamedtimezoneidentifiers

In case it's helpful, here's a plain-english explanation of what that algorithm is doing:

1. Look for the ID in zone.tab. If it's there, then it's primary.
2. If the ID isn't in zone.tab, then it's a non-primary ID. But we need to figure out what ID is primary, which is hard because IANA has merged multiple countries' IDs together. What we want to know is where that Link used to point before the inter-country merges happened.
3. So first we figure out what country code this ID corresponds to. If there's only one ID in zone.tab for that country code, then that's the primary ID.
4. If there are multiple IDs in zone.tab for that country code, then it's not clear which of them to use. So look at backzone to see where that ID used to point before all merges happened, excluding cases where the backzone link crosses country boundaries, which AFAIK only Atlantic/Jan_Mayen does (in this line: #PACKRATLIST zone.tab Link Europe/Oslo Atlantic/Jan_Mayen) but because of (3) there's only one zone.tab row for Atlantic/Jan_Mayen's country code, so there's no ambiguity.

[Manishearth]

I think I figured out a good way to do this. We can hardcode the three packrat timezones and add canonical ids for them. We parse the backzone file's packrat info and ensure that the list matches. We can proactively include mappings for the two timezones that don't show up in links entries if we want.

With this fix, everything works, except we also get this failure:

test/test262/data/harness/assert.js:18: Test262Error: Time zone name Pacific/Truk should be canonicalized to Pacific/Chuuk

which I think means we should just be including backzone data?

I don't know if we should be including all the regular data plus backzone data, or just build using the backzone file.

[Manishearth]

@justingrant oh, that's useful.

I'm still unclear on whether we should be parsing all the regular files and then merging in backzone data, or just parsing backzone data. If we don't merge backzone data then the Truk/Chuuk issue doesn't work.

[nekevss]

I'm still not sure ... to be clear, I think the pack_rat portion of zoneinfo_rs is currently incorrect.

If I'm understanding correctly, the PACKRATLIST along with vanguard & rearguard are all preprocessing steps completed in tzdb that happens with awk scripts.

We can hard code it, but it would be nice if we were able to resolve this in a preprocessing stage.

[Manishearth]

We can hard code it, but it would be nice if we were able to resolve this in a preprocessing stage.

I think we do not have the information available to do this in a preprocessing stage: we don't have "what region a TZ is in" for tzs not in zone.tab.

[nekevss]

iso3166.tab lists the country code and regions. The line for SJ is: "SJ Svalbard & Jan Mayen". The question is whether we can assume that "&" is a delimiter between regions, so SJ would resolve to ["Svalbard", "Jan_Mayen"] or maybe just special case the values.

[Manishearth]

Ah. That seems like a potential improvement over hardcoding, but I already have hardcoding implemented and it matches CLDR, which seems good?