Add pit functions

[TeemuSailynoja]

Summary

Add functions for computing probability integral transforms. This includes both the regular PIT of observations with respect to posterior predictive draws, as well as, a weighted version, where LOO weights can optionally be supplied.

Copyright and Licensing

By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the following licenses:

• Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
• Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

[TeemuSailynoja]

This would close #338.

[paul-buerkner]

Thank you! Are there tests in place already?

[TeemuSailynoja]

No, this is still a draft PR as I only have the default method and have not rested with rvars or draws objects.

[TeemuSailynoja]

A lot of failing checks. I'm still working on the PR, but should I be worried with those?

[TeemuSailynoja]

@paul-buerkner I think this should now be at a state ready for a review.

[paul-buerkner]

Thank you! Looks good overall. I have noted down a couple of questions and suggestions.

[paul-buerkner]

Are weights only valid if they come from loo? Or can they be, in theory, any kind of weights? If so, can we just call them weights?

[TeemuSailynoja]

I have changed them to weights. The difference to the weights already available to draws objects in .weights is, that this function requires a weight for each draw and each variable.
We were discussing with @n-kall that in the presence of the per-draw .weights they should also be taken into account.

[paul-buerkner]

Okay. Makes sense.

[paul-buerkner]

Is this incorporation of .weights something you want to do as part of this PR?

[TeemuSailynoja]

This could be a future extension for the functionality of pit().

[paul-buerkner]

highlight that by default these are expected to be log weights? What is more, do is log = TRUE a good default? Can we assume users to see this?

[TeemuSailynoja]

I changed the default to FALSE, and used the same doc-string as weight_draws to highlight the default value.

[paul-buerkner]

for extensibility, please add ... to all methods and the generic

[TeemuSailynoja]

added.

[paul-buerkner]

this signature is very restrictive. I would suggest the signature (x, y, ...) for better extensibility.

[TeemuSailynoja]

I changed the default signature to the suggested one.

pit <- function(x, y, ...) UseMethod("pit")

[paul-buerkner]

Could y be better validated? E.g., check this it is numeric or something along these lines?

[TeemuSailynoja]

I added a function for validate_y

validate_y <- function(y, x = NULL) {
  if (!is.numeric(y)) {
    stop_no_call("`y` must be numeric.")
  }
  if (anyNA(y)) {
    stop_no_call("NAs not allowed in `y`.")
  }
  if (is_rvar(x)) {
    if (length(x) != length(y) || any(dim(y) != dim(x))) {
      stop_no_call("`dim(y)` must match `dim(x)`.")
    }
  } else if (is_draws(x)) {
    if (!is.vector(y, mode = "numeric") || length(y) != nvariables(x)) {
      stop_no_call("`y` must be a vector of length `nvariables(x)`.")
    }
  }
  y
}

Currently, there is no other function in posterior that would use this validation, and the requirement of matching dimensions with x seems specific to this pit().
Just in case of future extensions, I left x as an optional argument.

[paul-buerkner]

Approved. Looks good now.