Improve some thermo calculation bottlenecks by refactoring how units are handled#2064
Conversation
|
I added a comment to |
|
On an old laptop, I am going from 3:00 to 0:13 with this branch and my test file. Very nice! |
be2a1f2 to
52600c2
Compare
52600c2 to
e39fdbb
Compare
|
I wasn't all that happy with the private routine + public wrapper approach...it would get really messy if applied to the whole calculation suite. So, I went back and came up with a decorator-based alternative: https://gist.github.com/jthielen/93ec9fcd8f3892a83215309fe5cda2c6. Doing it this way moves all unit specification into the decorator (which does carry with it the slight issue of default arguments in the signature needing to be in base units and not Quanitities), and leaves the unitless version attached to the parent function on the The existing approach is passing all checks, but let me know if it'd be useful and go back and redo it in the decorator-based way. If going that route, I'd want to clean it up to allow the existing |
01592ce to
25b3567
Compare
|
This has now all been rewritten to use the decorator-based approach! Performance improvements from my tests are about the same as before. |
17d124c to
00745ba
Compare
|
This should be ready for final review once it is adapted after the merge of #2263 |
…r unit handling on decorator
00745ba to
3289182
Compare
|
We can ignore Code Climate. |
dopplershift
left a comment
There was a problem hiding this comment.
Wish we had some actual benchmarks in place, but alas.
dopplershift
left a comment
There was a problem hiding this comment.
Wish we had a benchmark, but alas.
Description Of Changes
I noticed that there have been two major performance bottlenecks caused by the existing approaches to unit handling: iteration in
moist_lapse/lcl(#1169) and looping ofthickness_hydrostaticinio.gempak's_interp_*functions (#2062). In an attempt to improve this situation, this PR takes the approach of refactoring the relevant thermo calculations to have private versions that only use base units (nopint.Quantityhandling), and wrapping those while handling units. No robust benchmarks were evaluated, but this did speed uppytest tests/calc/test_thermo.pyon my workstation from about 3.5 seconds to 0.69 seconds (similar to #1980), and @akrherz's test file in #2041 (comment) from 35.88 seconds to 1.78 seconds.I tried to be careful to not touch any tests in this refactor (other than the fix to the flake8 checker), but do let me know if any tests should be added.
Future work building on this PR could examine if all/most of the calculations should be refactored in this way (which may also reveal some cleaner implementation approaches) and if numba
jit/guvectorizecould meaningfully accelerate any of these private routines.@sgdecker, could you evaluate how much performance improvement this gives you relative to your earlier tests in #2062?
@nawendt, these changes caused some failing tests in the GEMPAK reader due to what looks to be marginally different output values. Would you be willing to take a look and see if these differences are significant or not?
Checklist