Skip to content

Add a retry with wait interval for SSH operator #14489#19981

Merged
potiuk merged 6 commits into
apache:mainfrom
nshetty15:feature-14489-sshConnectionRetry
Feb 1, 2022
Merged

Add a retry with wait interval for SSH operator #14489#19981
potiuk merged 6 commits into
apache:mainfrom
nshetty15:feature-14489-sshConnectionRetry

Conversation

@nshetty15

Copy link
Copy Markdown
Contributor

closes: #14489

Retry attempts are added if there are any errors connecting or establishing SSH session.
This will NOT retry failures like wrong host, auth errors, socket timeouts.

@nshetty15

Copy link
Copy Markdown
Contributor Author

@eladkal Not sure why this build failed here.. how can this be fixed?

Comment thread airflow/providers/ssh/hooks/ssh.py Outdated
Comment thread airflow/providers/ssh/hooks/ssh.py Outdated
Comment thread airflow/providers/ssh/hooks/ssh.py Outdated
@potiuk

potiuk commented Dec 4, 2021

Copy link
Copy Markdown
Member

That was an intermittent problem - but there are some things to addresss, so let's see next time.

@nshetty15
nshetty15 requested a review from potiuk January 30, 2022 17:49
@nshetty15

Copy link
Copy Markdown
Contributor Author

@potiuk could you please help with the build failure here?

@potiuk potiuk closed this Jan 31, 2022
@potiuk potiuk reopened this Jan 31, 2022
@potiuk

potiuk commented Jan 31, 2022

Copy link
Copy Markdown
Member

I think it was an intermittent problem. Close/Reopen triggers rebuild in this case - you can always do it yourself or commit --amend and push when you suspect the problem is "flakiness"

@nshetty15 nshetty15 closed this Jan 31, 2022
@nshetty15 nshetty15 reopened this Jan 31, 2022
@nshetty15 nshetty15 closed this Jan 31, 2022
@nshetty15 nshetty15 reopened this Jan 31, 2022
@nshetty15 nshetty15 closed this Jan 31, 2022
@nshetty15 nshetty15 reopened this Jan 31, 2022
@nshetty15

Copy link
Copy Markdown
Contributor Author

@potiuk could you review this change?
I've noticed intermittent banner issues with ssh (postgres integration) when running tests on the cloud CI. Is this expected?

@potiuk
potiuk merged commit b6edc3b into apache:main Feb 1, 2022
@potiuk

potiuk commented Feb 1, 2022

Copy link
Copy Markdown
Member

Cool. I am just preparing to release next wave of provtders. Sometimes we have flakiness. we actively fight with it but it's an uphill (and constant) battle (however recently we have many more "green" builds than we used to have.

@nshetty15
nshetty15 deleted the feature-14489-sshConnectionRetry branch June 29, 2022 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a retry with wait interval for SSH operator

2 participants