RequestBody draining still allocates per-request buffers

[rynowak]

This change: 4bba074

Fixed a serious allocation issue, but also added a 4kb buffer to each frame, which is more per-request allocation. Simply just making this static as well may not do what we want, as this might lead to a lot of contention for this cache line.

I think we need to revisit the original proposal which was to implement CopyToStreamAsync in a smarter way.

Allocation data for 3000 requests:

In my sample app, this is responsible for 12/82mb allocated

[rynowak]

/cc @halter73 @benaadams

[benaadams]

This is only on fresh connections, not keep alive already open connections?

Might be worth pooling it; since its just a reusable sink.

[benaadams]

Raised PR #335 "Use pooled array"

[halter73]

I guess I didn't look at the PR that added _nullBuffer closely enough. I thought _nullBuffer was static. Is there any reason it can't be? I think that would even be more efficient.

[rynowak]

All of these solutions involve copy bytes from one location to another in memory. Wouldn't it be better to override CopyToAsync and special case Stream.Null?

[halter73]

@rynowak That's definitely an option. We were considering a change to make CopyToAsync bufferless in all cases, but that seemed to be a little too complex for a last minute rc1 fix.

It might make more sense just to add a new method to FrameRequestStream that efficiently consumes the rest of the request rather than overriding CopyToAsync and checking if the argument is Stream.Null only to do the same thing.

[rynowak]

⬆️ either of those would work.

[benaadams]

I thought _nullBuffer was static. Is there any reason it can't be? I think that would even be more efficient.

Because you'll have multiple threads writing to it on different cores and cause cpu cache thrashing.

[benaadams]

add a new method to FrameRequestStream that efficiently consumes the rest of the request

👍

Could even just call it directly in Frame; it constructs FrameRequestStream as a strong type in the same function

[benaadams]

Something that fasts forwards to the end rather than doing the copy to a buffer would be better; as should be less work? (Less work for CPU rather than; for dev)