Skip to content

aws-lib version upgrade + pagination support#296

Merged
oren-zohar merged 9 commits into
elastic:mainfrom
oren-zohar:aws-lib-upgrade
Jul 26, 2022
Merged

aws-lib version upgrade + pagination support#296
oren-zohar merged 9 commits into
elastic:mainfrom
oren-zohar:aws-lib-upgrade

Conversation

@oren-zohar

@oren-zohar oren-zohar commented Jul 24, 2022

Copy link
Copy Markdown
Collaborator

In this PR:

  • Updating aws-sdk-go-v2 version to 1.16.7
  • Pagination support for DescribeRepositories and DescribeLoadBalancer
  • General code refactoring

pagination support
@mergify

mergify Bot commented Jul 24, 2022

Copy link
Copy Markdown
Contributor

This pull request does not have a backport label. Could you fix it @oren-zohar? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit
    NOTE: backport-skip has been added to this pull request.

@oren-zohar
oren-zohar marked this pull request as ready for review July 25, 2022 08:51
@oren-zohar
oren-zohar requested a review from a team as a code owner July 25, 2022 08:51
@oren-zohar
oren-zohar requested a review from ofiriro3 July 25, 2022 08:51

@ofiriro3 ofiriro3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huge work + a lot of refactoring 💯

I found a small breaking change that is critical and we have to address it, the rest is some small comments.

But I'd like to go over it from the beginning just to make sure I hadn't miss anything

Comment thread go.mod
Comment thread resources/providers/awslib/eks_provider.go Outdated
/// If we will ever wish to change it, DescribeRepositories returns results in paginated manner
func (provider *ECRProvider) DescribeAllECRRepositories(ctx context.Context) ([]ecr.Repository, error) {
/// When repoNames is nil, it will describe all the existing repositories
// If we will ever wish to change it, DescribeRepositories returns results in paginated manner

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for this comment anymore

}

return response.Repositories, err
return repos, nil

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's good for a first iteration solution, but in the long term, I think that we might want to consider doing it on the infrastructure level.

Comment thread go.mod
github.com/Microsoft/go-winio => github.com/bi-zone/go-winio v0.4.15
github.com/Shopify/sarama => github.com/elastic/sarama v1.19.1-0.20210823122811-11c3ef800752
github.com/apoydence/eachers => github.com/poy/eachers v0.0.0-20181020210610-23942921fe77 //indirect, see https://github.com/elastic/beats/pull/29780 for details.
github.com/cucumber/godog => github.com/cucumber/godog v0.8.1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was it removed on purpose?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we don't use it

Comment thread resources/fetchers/ecr_fetcher.go Outdated
}

type EcrRepository ecr.Repository
type EcrRepository types.Repository

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To completely finish the amazing segregation that you made, can we replace the types. Repository with a type of our own?

Same as you did in the IAM and the EKS fetchers

That way, only the AWS provider will be aware of the AWS interface

Comment thread resources/fetchers/ecr_fetcher_test.go Outdated
Comment thread resources/fetchers/eks_fetcher.go
Comment thread resources/fetchers/elb_fetcher.go Outdated
}

type LoadBalancersDescription elasticloadbalancing.LoadBalancerDescription
type LoadBalancersDescription types.LoadBalancerDescription

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we replace types.LoadBalancerDescription with awslib.***?

Same way as you did for EKS fetcher and the IAM fetcher?

)

type EksClusterOutput eks.DescribeClusterOutput
type EksCluster types.Cluster

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we don't use the field anymore, are we?

@ofiriro3 ofiriro3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No further review is necessary! Amazing job

But please take a look at my last two comments 🥇

@@ -26,6 +26,7 @@ import (
)

type ElbLoadBalancerDescriptions []types.LoadBalancerDescription

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: isn't the ElbLoadBalancerDescriptions an array of ElbLoadBalancersDescription?

Do we need them both?

"github.com/aws/aws-sdk-go-v2/service/ecr/types"
)

type EcrRepository types.Repository

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: isn't the EcrRepositories an array of EcrRepository?

Do we need them both?

@oren-zohar
oren-zohar enabled auto-merge (squash) July 25, 2022 17:08
@mergify

mergify Bot commented Jul 25, 2022

Copy link
Copy Markdown
Contributor

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b aws-lib-upgrade upstream/aws-lib-upgrade
git merge upstream/main
git push upstream aws-lib-upgrade

@github-actions

Copy link
Copy Markdown

# Conflicts:
#	resources/fetchers/ecr_factory_test.go
#	resources/fetchers/ecr_fetcher.go
#	resources/fetchers/ecr_fetcher_test.go
#	resources/providers/awslib/ecr_provider.go
#	resources/providers/awslib/ecr_provider_mock.go
#	resources/providers/awslib/ecr_public_provider.go
@oren-zohar
oren-zohar merged commit 33c1989 into elastic:main Jul 26, 2022
orouz added a commit to orouz/cloudbeat that referenced this pull request Sep 6, 2023
orestisfl pushed a commit to orestisfl/cloudbeat that referenced this pull request Oct 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pagination support for EKS requests

2 participants