Use ERA5 ensemble bounds

[treigerm]

This PR adjusts the script to create the error bounds to use the ERA5 derived error bounds when possible. There's a couple of things we need to make decisions on before merging it, mostly about how to best integrate with the https://github.com/juntyr/era5-ensemble/ repo:

1. How do we access the CSV of error bounds? Right now I am using a raw.github link but that is suboptimal because the CSV is in a private repo and the access token copied in the URL expires after some time (don't know what the exact cut off is but it's not valid for more than a couple of hours).
2. I have copied the code to compute the error bounds from the notebooks at https://github.com/juntyr/era5-ensemble/. I guess this is the best option until the ERA5 ensemble paper and the code is published? Happy to have another solution! The code is slightly adjusted to not compute the quadratic error bounds because we don't have any compressors that support these error bounds.

[juntyr]

I think, for now, I could also host the error bounds csv file as a private gist that we can then access. It would probably contain fewer columns though.

The second feedback I have is to not depend on the pick column. The method for picking is ok but certainly not absolute so I think we should just always test both the absolute and relative error bound (especially since there are some cases where which one to pick is debatable and since some people may want to always use one or the other).

When you're mapping the variables to ERA5, could you also check that the units match?

[juntyr]

Here's the gist:
https://gist.github.com/juntyr/bbe2780256e5f91d8f2cb2f606b7935f

and here's the raw data link:
https://gist.githubusercontent.com/juntyr/bbe2780256e5f91d8f2cb2f606b7935f/raw/table-raw.csv

[juntyr]

The second feedback I have is to not depend on the pick column.

This will also simplify the code a lot for this PR since the ensemble error bounds method is really simple by itself

[treigerm]

Thanks @juntyr !

I've manually checked that the units matched and actually caught a bug: the precipation and OLR variables in ERA5 were using accumulated instead of instantaneous values. I fixed that now. I didn't add an automated check because it would add quite a bit of code with little added benefit for now. There's not that many variables and we don't plan to add many more.

I've also avoided using the pick column now. For now I manually added a lookup table for when to use an absolute and when to use a relative error bound. I would like to avoid running all compressors with both absolute and relative error bounds because it just adds further complexity in interpreting the results and one of the goals of this benchmark in the end is to have results that are interpretable. For now the OLR and the biomass variables use the relative error (this is also consistent with the "pick" column) so we ensure that we evaluate all compressors on both relative and absolute errors. In the evaluation we can then always check both the absolute and relative error bounds.

[juntyr]

I'll review this later, this is a reminder to myself to check the absolute vs relative selection (personally I'd probably pick relative for wind).

@treigerm Could you please plot the biomass value vs uncertainty scatter plot and attach the picture to the PR? With the error bounds project I think the visual intuition is usually the most important.

[treigerm]

@juntyr Here's a quick plot for the biomass mean-spread (with plotting code from https://github.com/juntyr/era5-ensemble/blob/main/mean-spread-plots.ipynb ):

So clearly point towards wanting a relative error bound!

[treigerm]

There were some additional niggles that popped up when I ran the downstream pipeline. So I had to make some minor adjustments.

[juntyr]

For rlut, I think the benefit of a relative error bound is a bit difficult to see

For tp, a relative error fits better visually, though an absolute error has a lower cost since so many points are near zero

Temperature has some different bounds at different pressure levels

Here's t2m for comparison

We should probably go with an absolute error bound (in degC a relative error bound also works, but ERA5 is in Kelvin)

sst is a bit weird since temperature around ~<0C has much lower uncertainty

For u10/v10 I'd recommend a relative error bound

I agree using an absolute error for the mean sea level pressure

[juntyr]

(these are plots with slightly outdated visuals but still give the same insights)

[treigerm]

For rlut:
Would you be able to regenerate the plot but let the y-axis go only from 0 to 5 so that we zoom into the bottom of the plot a bit more? It's hard to tell visually the slop of the lines.

For temperature:
Seems like we should do something a bit more involved than just picking the t2m temperature. Especially when comparing with the 50hPa plot, the t2m 95% error bound would probably be too lenient for that pressure level. For the benchmark, I think it should be enough to then take the absolute error bounds for each pressure level and then take the mean. Do you agree with this @juntyr ?

For wind:
Changing to relative makes sense to me!

[juntyr]

For temperature: Seems like we should do something a bit more involved than just picking the t2m temperature. Especially when comparing with the 50hPa plot, the t2m 95% error bound would probably be too lenient for that pressure level. For the benchmark, I think it should be enough to then take the absolute error bounds for each pressure level and then take the mean. Do you agree with this @juntyr ?

Either the mean or the min (no preference from my end), we can justify both

[treigerm]

Okay, now made the adjustments to the error bound calculation to take the mean over pressure levels (mainly I was going for the mean because I want to avoid having an ultra conservative error bound for the benchmark purpose). Also fixed some plotting bugs that cropped up with relative errors.

I think that's all I want to add in this PR, so would be ready to merge if there's no further objections!

[juntyr]

LGTM