Skip to content

Weighted diagnostics#357

Merged
paul-buerkner merged 29 commits intostan-dev:rvar_weightsfrom
n-kall:weighted_diagnostics
Oct 3, 2024
Merged

Weighted diagnostics#357
paul-buerkner merged 29 commits intostan-dev:rvar_weightsfrom
n-kall:weighted_diagnostics

Conversation

@n-kall
Copy link
Copy Markdown
Collaborator

@n-kall n-kall commented Mar 21, 2024

Summary

Adds weighted ESS, MCSE and Pareto diagnostics.

Copyright and Licensing

By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the following licenses:

@github-actions
Copy link
Copy Markdown

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 77cdf1a is merged into rvar_weights:

  •   :rocket:as_draws_array: 141ms -> 139ms [-2.13%, -0.83%]
  •   :ballot_box_with_check:as_draws_df: 64.8ms -> 65.1ms [-1.67%, +2.43%]
  •   :ballot_box_with_check:as_draws_list: 158ms -> 158ms [-0.74%, +1.11%]
  •   :ballot_box_with_check:as_draws_matrix: 62.7ms -> 63.4ms [-0.79%, +2.97%]
  •   :ballot_box_with_check:as_draws_rvars: 80.5ms -> 81.6ms [-0.13%, +2.77%]
  •   :ballot_box_with_check:summarise_draws_100_variables: 709ms -> 710ms [-0.24%, +0.47%]
  •   :ballot_box_with_check:summarise_draws_10_variables: 111ms -> 111ms [-1.44%, +0.6%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

@paul-buerkner
Copy link
Copy Markdown
Collaborator

Thank you! @avehtari would you have time to review this PR?

@n-kall
Copy link
Copy Markdown
Collaborator Author

n-kall commented Mar 21, 2024

I should still add some documentation, but I think the core functionality is there (plus tests)

@n-kall n-kall marked this pull request as ready for review March 21, 2024 10:14
@github-actions
Copy link
Copy Markdown

This is how benchmark results would change (along with a 95% confidence interval in relative change) if d56a5c9 is merged into rvar_weights:

  •   :rocket:as_draws_array: 151ms -> 149ms [-1.92%, -0.49%]
  • ❗🐌as_draws_df: 72.8ms -> 73.7ms [+0.27%, +2.25%]
  •   :ballot_box_with_check:as_draws_list: 168ms -> 168ms [-0.58%, +0.7%]
  •   :ballot_box_with_check:as_draws_matrix: 67.8ms -> 67.3ms [-1.9%, +0.47%]
  •   :ballot_box_with_check:as_draws_rvars: 84.5ms -> 84.2ms [-1.43%, +0.69%]
  •   :ballot_box_with_check:summarise_draws_100_variables: 719ms -> 721ms [-0.14%, +0.53%]
  •   :ballot_box_with_check:summarise_draws_10_variables: 116ms -> 116ms [-0.98%, +0.35%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2024

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 1079cef is merged into rvar_weights:

  • ✔️as_draws_array: 137ms -> 137ms [-0.89%, +0.54%]
  • ✔️as_draws_df: 60.9ms -> 61.2ms [-1.14%, +2.31%]
  • ❗🐌as_draws_list: 156ms -> 157ms [+0.53%, +1.79%]
  • ✔️as_draws_matrix: 60ms -> 59.4ms [-2.16%, +0.01%]
  • ✔️as_draws_rvars: 82ms -> 82.5ms [-0.5%, +1.7%]
  • ❗🐌summarise_draws_100_variables: 707ms -> 741ms [+4.44%, +5.01%]
  • ✔️summarise_draws_10_variables: 110ms -> 111ms [-0.2%, +1.03%]
    Further explanation regarding interpretation and methodology can be found in the documentation.

@n-kall
Copy link
Copy Markdown
Collaborator Author

n-kall commented Oct 3, 2024

I think this can be merged into the weighted rvars branch, as @avehtari and I went through and reviewed this some time ago

@paul-buerkner
Copy link
Copy Markdown
Collaborator

Thanks. Merging now.

@paul-buerkner paul-buerkner merged commit ffc9c6a into stan-dev:rvar_weights Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants