Improve memory consumption by only passing resolver args to resolver and canceller if callback requires them#113
Merged
WyriHaximus merged 1 commit intoreactphp:2.xfrom Apr 24, 2018
Conversation
jsor
approved these changes
Apr 24, 2018
WyriHaximus
approved these changes
Apr 24, 2018
kelunik
reviewed
Apr 25, 2018
| { | ||
| // Use reflection to inspect number of arguments expected by this callback. | ||
| // We did some careful benchmarking here: Using reflection to avoid unneeded | ||
| // function arguments is actually faster than blindly passing them. |
There was a problem hiding this comment.
The reason for it being faster isn't avoiding the arguments being passed, but rather the three closure creations that are saved.
Member
Author
There was a problem hiding this comment.
@kelunik Do you feel that this helps with understanding the motivation for this change? If so, I encourage you to file this as a new documentation PR and maybe back this with some numbers ![]()
There was a problem hiding this comment.
@clue I think the motivation is completely covered by the circular references, the rest is just additional background on the performance of the patch. It's not super important to change, just a minor note I had while reading.
This was referenced May 4, 2018
jsor
added a commit
to jsor-labs/pact
that referenced
this pull request
May 16, 2018
This incorporates the work done by @clue in the following PR's for reactphp/promise: * reactphp/promise#113 * reactphp/promise#115 * reactphp/promise#116 * reactphp/promise#117 * reactphp/promise#118 * reactphp/promise#119 Co-authored-by: Christian Lück <christian@lueck.tv>
This was referenced Jun 9, 2018
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While debugging some very odd memory issues in a live application, I noticed that this component shows some unexpected memory consumption and memory would not immediately be freed as expected when a pending promise is cancelled. Let's not call this a "memory leak", because memory was eventually freed, but this clearly caused some unexpected and significant memory growth.
I've used the following script to demonstrate unreasonable memory growth:
Initially this peaked somewhere around 15 MB on my system taking 3.4s. After applying this patch, this script reports a constant memory consumption of around 0.7 MB taking 1.8s
Empirical evidence suggests that many common use cases do not use the
$reject(etc.) arguments and instead simply throw anException. The Promise class now uses reflection to check whether the$resolverand$cancellerfunctions actually define any arguments and if they are not expected, doesn't actually pass them. This does involve some minor runtime overhead for using reflection, but this is actually compensated by not passing any closure arguments (1M invocations that do no use arguments show ~40% performance improvements, 1M invocations that do use arguments show a ~3% performance degradation).More importantly, not passing these arguments has the side effect of these arguments not showing up in the call stack anymore. This is particularly important when throwing an Exception, as it would otherwise keep a reference to the promise instance in its call stack (while the promise stores the reference to this exception) and as such would cause a cyclic garbage reference.
Previously, these cyclic garbage references were eventually freed by the cyclic garbage collector after accumulating around 10000 entries with somewhere between 8MB to 15 MB. These can now be freed immediately and thus no longer cause any unexpected memory growth. This implementation includes some of the ideas discussed in reactphp/promise-timer#32, #46 and reactphp/socket#113.
Note that this PR does not resolve all unexpected memory issues. However, it addresses a very common problem for many consumers of this library and makes many of the higher level work-arounds obsolete. My vote would to be get this in here now as it addresses a relevant memory issue and eventually address any additional issues on top of this.