Skip to content

Adjusting ssl_read_from_net to read limited amount of data in one go.#2249

Merged
shinrich merged 1 commit intoapache:masterfrom
shinrich:ssl_read_loop_rework
Jul 12, 2017
Merged

Adjusting ssl_read_from_net to read limited amount of data in one go.#2249
shinrich merged 1 commit intoapache:masterfrom
shinrich:ssl_read_loop_rework

Conversation

@shinrich
Copy link
Member

@shinrich shinrich commented Jul 7, 2017

Addressing problem seen by pdm. Large POSTS would cause the inbound inactivity timeout to go off killing the transaction.

Working with @oknet and @maskit, we reworked the ssl_read_from_net to not read as much as possible. Rather read enough to fill one buffer block then give control back to the event loop. The theory is that a fast client would keep ATS in the read loop and buffering up the post body before passing along the data. Setting the read trigger if there is still data on the wire.

@shinrich shinrich added the TLS label Jul 7, 2017
@shinrich shinrich added this to the 7.1.0 milestone Jul 7, 2017
@shinrich shinrich self-assigned this Jul 7, 2017
@shinrich shinrich requested review from maskit and oknet July 7, 2017 17:03
maskit
maskit previously approved these changes Jul 7, 2017
Copy link
Member

@maskit maskit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried 100MB POST request and it works fine for me.

@shinrich
Copy link
Member Author

shinrich commented Jul 7, 2017

pdm also verified that this fix works in his environment.

@zwoop
Copy link
Contributor

zwoop commented Jul 8, 2017

@oknet can you review this please as well, so we can get this shipped into 7.1.0 too?

@zwoop
Copy link
Contributor

zwoop commented Jul 9, 2017

Should we land this, or wait for @oknet ?

@shinrich
Copy link
Member Author

shinrich commented Jul 9, 2017

I think we can push on. This addresses the issues that @oknet raised in the previous PR's.

@zwoop
Copy link
Contributor

zwoop commented Jul 9, 2017

Ship it!

@zwoop
Copy link
Contributor

zwoop commented Jul 9, 2017

Naieve question though: Is this going to limit or affect throughput and performance on TLS sessions?

@oknet
Copy link
Member

oknet commented Jul 10, 2017

The ssl_read_from_net is a static function. It is designed to receiving data from net.
In my opinion, it is a wrap function for SSLReadBuffer to support watermark of IOBuffer.

The SSLNetVC::net_read_io is a method of SSLNetVC. It is designed to callback SM and manager the read side of SSLNetVC.

The implement of these functions in UnixNetVC are very vague.

event = (s->vio.ntodo() <= 0) ? SSL_READ_COMPLETE : SSL_READ_READY;
if (sslErr == SSL_ERROR_NONE && s->vio.ntodo() > 0) {
// We stopped with data on the wire (to avoid overbuffering). Make sure we are triggered
sslvc->read.triggered = 1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The triggered is only cleared in net_read_io and set in NetHandler::mainNetEvent.
Here, just return SSL_READ_READY to inform net_read_io and it will call readReschedule.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we should return SSL_READ_READY. Will fix that. However, we should also set read trigger. Otherwise, we will be waiting until the next read event. If there is read data left, we will not get another read event on the epoll.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The triggered is set in NetHandler::mainNetEvent().
There is no one to clear triggered in ssl_read_from_net().
It is only cleared in SSLNetVC::net_read_io() if SSL_READ_WOULD_BLOCK or SSL_READ_EOS or SSL_READ_ERROR is returned from ssl_read_from_net().

In another word, the read.triggered is always TRUE here, you can try ink_assert(sslvc->read.triggered) here.

Warning("Cannot add new block");
// If we filled up one block, give back to the event loop so we don't
// overbuffer.
if (bytes_read > 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this condition check breaks the watermark design in IOBuffer.

We can use readv() in UnixNetVC::read_from_net to read data into multi IOBufferBlocks if the rest space in the first IOBufferBlock is less than watermark size.

But there is only SSLReadBuffer, ATS should read() data for every IOBufferBlocks that is attached into MIOBuffer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I will replace my logic that is going per block and explicitly adding blocks and call buf.writer()->writer_avail() instead and rely on that to do the appropriate water mark calculations and add new blocks as necessary.

@shinrich
Copy link
Member Author

@zwoop By keeping the trigger bit set in the case where data is left on the wire we should not run into performance problems.

@shinrich shinrich force-pushed the ssl_read_loop_rework branch 2 times, most recently from 9f60842 to 42f598f Compare July 11, 2017 00:57
@zwoop
Copy link
Contributor

zwoop commented Jul 11, 2017

Do we really want the tsxs changes in this PR? That seems odd, ought to be a separate PR I think. Also, before we land this, make sure to squash into one commit.

This is running on Docs btw.

@zwoop
Copy link
Contributor

zwoop commented Jul 11, 2017

@oknet Any thoughts on this last revision? @shinrich I've tested this on docs a bunch, but before merging, I think you should remove the tsxs changes, as well as squashing the commits down to one.

oknet
oknet previously approved these changes Jul 12, 2017
@shinrich
Copy link
Member Author

Removed the tsxs changes, squashed, and pushed clearing previous approves.

@zwoop
Copy link
Contributor

zwoop commented Jul 12, 2017

What y'all say, should we land this? I've run it on Docs right problems.

@shinrich
Copy link
Member Author

I think we should land it. Someone will need to re-approve since my last push cleared @oknet 's approval.

Copy link
Contributor

@zwoop zwoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, lets land this and start testing in real production.

@shinrich shinrich merged commit f75dbf6 into apache:master Jul 12, 2017
@zwoop
Copy link
Contributor

zwoop commented Jul 12, 2017

Cherry-picked to 7.1.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants