Skip to content
This repository was archived by the owner on Jul 10, 2021. It is now read-only.

docs(aws): AWS EC2 provider instructions, diagram and code for manual AWS IAM set up.#1996

Merged
dorbin merged 34 commits intospinnaker:masterfrom
aleon1220:patch-1
Sep 18, 2020
Merged

docs(aws): AWS EC2 provider instructions, diagram and code for manual AWS IAM set up.#1996
dorbin merged 34 commits intospinnaker:masterfrom
aleon1220:patch-1

Conversation

@aleon1220
Copy link
Contributor

@aleon1220 aleon1220 commented Aug 7, 2020 •
edited
Loading

Something happened in PR1964 i am re-doing the changes and re-committing them.
Please help @dorbin @brian-armory

@aleon1220
branch patch from PR1964
0ba8453 
Something happened in [PR1964](spinnaker#1964) i am readding the changes and re-committing them.
Please help @dorbin @brian-armory
@aleon1220 aleon1220 mentioned this pull request Aug 7, 2020
Closed
dorbin
dorbin reviewed Aug 7, 2020
View reviewed changes
Copy link
Contributor

@dorbin dorbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aleon1220 thanks for re-doing this!

I've left some suggestions. Sorry if some of them are repetitive.

dorbin
dorbin reviewed Aug 20, 2020
View reviewed changes
Copy link
Contributor

@dorbin dorbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aleon1220 Oops, looks like I responded some days ago but didn't submit my review. Looks like just the one comment about linking.

Andres Leon-Rangel added 3 commits August 21, 2020 16:49
Adds instructions with an example
ad5f7b1 
the instructions allow Spinnaker operators to
deploy to AWS Accounts develop and staging.
arranged the wording of the instructions.
Adds the AWS provider diagram with extension .drawio
c42d00c 
The diagram can be opened online with https://www.draw.io/
Adds the image in .svg format for the example
0317d63 
The example shows an architecture with 3 AWS accounts
1 Spinnaker Managing where Spinnaker Lives
2 develop
3 staging
@aleon1220 aleon1220 changed the title branch patch from PR1964. AWS EC2 provider branch patch from PR1964. AWS EC2 provider instructions, diagram and code for manual AWS IAM set up. Aug 21, 2020
@aleon1220
Copy link
Contributor Author

aleon1220 commented Aug 21, 2020

Hello, @dorbin
I have added enough detail to make the AWS EC2 provider work. I added an example all throughout the instructions to make it simple to understand. A diagram that i added in format .svg based on the .drawio diagram done by @anshrma back in 2018. These instructions work in AWS as this is what i have working. It was based on documentation and experience found throughout the community. Thanks to Armory for writing a wonderful guide on the aws provider. They have instructions for set up with halyard and operator.

Many of the new links are not working because they are not created but in general terms i checked this page after running it locally in my docker container dev set up.

image

@aleon1220
Copy link
Contributor Author

aleon1220 commented Aug 21, 2020

@dorbin Hi Dave, I have added details for the EC2 provider. By using it with these instructions one can successfully deploy Instances to EC2. It is a big change because i think his document was not updated much. In summary there are 3 options to set up AWs IAM structure for Spinnaker.
1-AWS CF templates (this doesnt work)
2-AWS CLI to deploy the CF templates (this doesnt work)
3-Set up AWS IAM manually. These are the instructions i am contributing. Is been tested 4 times in my environments. It works but it requires many steps. I used an example to make it simple to understand and relate.

@dorbin
Copy link
Contributor

dorbin commented Aug 22, 2020

@brian-armory can you take a look at this when you get a chance? I'm not on AWS, so I can't test these instrux. Thanks!

@dorbin dorbin requested a review from 317brian August 22, 2020 17:19
@317brian
Copy link
Contributor

317brian commented Sep 1, 2020

Hi @aleon1220, I've scheduled time for myself to work on this during this sprint (this week and next). Sorry for the delay.

@aleon1220
Copy link
Contributor Author

aleon1220 commented Sep 2, 2020

Hi @aleon1220, I've scheduled time for myself to work on this during this sprint (this week and next). Sorry for the delay.

Thanks Brian. I am still preparing my AWS personal account to test this and record a short video. Other than that. Thanks for your review.

@aleon1220 aleon1220 requested a review from dorbin September 3, 2020 01:58
dorbin
dorbin reviewed Sep 3, 2020
View reviewed changes
Copy link
Contributor

@dorbin dorbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @aleon1220 . A few more suggestions. Also, it looks like you've got a conflict in one of the files.

setup/install/providers/aws/aws-ec2.md Outdated
{% include toc %}

> :warning: These instructions are out-of-date and a new version is being
> :warning: These instructions were updated to manually set up the AWS provider with [Option-3](#option-3--configure-with-aws-iam-console) on **2020-08-22**. <br>
Copy link
Contributor

@dorbin dorbin Sep 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
> :warning: These instructions were updated to manually set up the AWS provider with [Option-3](#option-3--configure-with-aws-iam-console) on **2020-08-22**. <br>
> :warning: These instructions were updated, on **2020-08-22**, to manually set up the AWS provider with [Option-3](#option-3--configure-with-aws-iam-console). <br>

@aleon1220 @dorbin
Update setup/install/providers/aws/aws-ec2.md
5d90df4 
Great suggestion

Co-authored-by: Dave Dorbin <ddorbin@google.com>
@aleon1220
Copy link
Contributor Author

aleon1220 commented Sep 3, 2020

Hi, @aleon1220 . A few more suggestions. Also, it looks like you've got a conflict in one of the files.

sadly yes. This PR is been here for a while. Maybe someone updated the page. I dont understand. This AWS provider really needs some attention.

@aleon1220 @dorbin
Update setup/install/providers/aws/aws-ec2.md
0418aae 
simple and concise. Thanks

Co-authored-by: Dave Dorbin <ddorbin@google.com>
aleon1220 and others added 2 commits September 4, 2020 10:08
@aleon1220 @dorbin
Update setup/install/providers/aws/aws-ec2.md
30ea48e 
grammar fix.

Co-authored-by: Dave Dorbin <ddorbin@google.com>
@aleon1220 @dorbin
Update setup/install/providers/aws/aws-ec2.md
be3f5d2 
grammar fix

Co-authored-by: Dave Dorbin <ddorbin@google.com>
@aleon1220 aleon1220 mentioned this pull request Sep 6, 2020
Open
@317brian 317brian changed the title branch patch from PR1964. AWS EC2 provider instructions, diagram and code for manual AWS IAM set up. docs(aws): AWS EC2 provider instructions, diagram and code for manual AWS IAM set up. Sep 11, 2020
317brian
317brian suggested changes Sep 12, 2020
View reviewed changes
@aleon1220 aleon1220 requested a review from 317brian September 16, 2020 01:03
@aleon1220
Copy link
Contributor Author

aleon1220 commented Sep 17, 2020

@brian-armory Hi Brian, Thanks for all the help. I have improved the doc based on your suggestions. I have been keeping this branch updated. Would you like to do a review?

317brian
317brian approved these changes Sep 18, 2020
View reviewed changes
Copy link
Contributor

@317brian 317brian left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for all your hardwork on this @aleon1220. I pushed a commit to fix some indenting on lines 214-16 for the nested list.

aleon1220
aleon1220 commented Sep 18, 2020
View reviewed changes
Copy link
Contributor Author

@aleon1220 aleon1220 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much!

@dorbin dorbin merged commit fbfd466 into spinnaker:master Sep 18, 2020
@dorbin
Copy link
Contributor

dorbin commented Sep 18, 2020

Thanks for all the work, Andres! And all the re-doing of the work. 😃

avram
avram reviewed Oct 2, 2020
View reviewed changes
setup/install/providers/aws/aws-ec2.md
2. Switch to the AWS Account you want to create the roll for.
3. Go to **Roles > Create Role**.
4. Select EC2. You can change this later, because we want to specify an explicit consumer of this role in a later stage.
5. For permissions, search for "PowerUserAccess" and select this policy. This gives the role permission to access AWS services.
Copy link
Contributor

@avram avram Oct 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PowerUserAccess is broader access than Spinnaker really needs; there used to be a very specific policy document doc that had what Spinnaker needs; we should refer to that here. I think that Armory at least has an up-to-date, narrower policy to refer to.

Copy link
Contributor Author

@aleon1220 aleon1220 Oct 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what access do you recommend?
the policy is AWS managed it should be ok...

Copy link
Contributor

@avram avram Oct 5, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PowerUserAccess policy is extremely broad. Unfortunately it looks like the Armory docs also now recommend PowerUserAccess.

This is the policy I'm using now, which at least limits access to the core services Spinnaker needs, although it would need to be broadened to account for CloudFormation and Lambda and potentially other small revisions. This or a similar policy used to be present in the Spinnaker docs (or perhaps the Armory docs).

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1486065689000",
            "Effect": "Allow",
            "Action": [
                "autoscaling:CreateAutoScalingGroup",
                "autoscaling:CreateLaunchConfiguration",
                "autoscaling:CreateOrUpdateTags",
                "autoscaling:DeleteAutoScalingGroup",
                "autoscaling:DeleteLaunchConfiguration",
                "autoscaling:DeletePolicy",
                "autoscaling:DeleteScheduledAction",
                "autoscaling:DeleteTags",
                "autoscaling:DescribeAutoScalingGroups",
                "autoscaling:DescribeLaunchConfigurations",
                "autoscaling:DescribeLoadBalancers",
                "autoscaling:DescribePolicies",
                "autoscaling:DescribeScalingActivities",
                "autoscaling:DescribeScheduledActions",
                "autoscaling:DetachInstances",
                "autoscaling:DisableMetricsCollection",
                "autoscaling:EnableMetricsCollection",
                "autoscaling:PutLifecycleHook",
                "autoscaling:PutNotificationConfiguration",
                "autoscaling:PutScalingPolicy",
                "autoscaling:PutScheduledUpdateGroupAction",
                "autoscaling:ResumeProcesses",
                "autoscaling:SuspendProcesses",
                "autoscaling:TerminateInstanceInAutoScalingGroup",
                "autoscaling:UpdateAutoScalingGroup",
                "cloudwatch:DeleteAlarms",
                "cloudwatch:DescribeAlarms",
                "cloudwatch:GetMetricStatistics",
                "cloudwatch:ListMetrics",
                "cloudwatch:PutMetricAlarm",
                "ec2:AttachClassicLinkVpc",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CreateNetworkInterface",
                "ec2:CreateSecurityGroup",
                "ec2:CreateTags",
                "ec2:DeleteSecurityGroup",
                "ec2:DeleteTags",
                "ec2:DescribeAccountAttributes",
                "ec2:DescribeAddresses",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeClassicLinkInstances",
                "ec2:DescribeImages",
                "ec2:DescribeInstances",
                "ec2:DescribeKeyPairs",
                "ec2:DescribeRegions",
                "ec2:DescribeReservedInstances",
                "ec2:DescribeReservedInstancesOfferings",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSpotPriceHistory",
                "ec2:DescribeSubnets",
                "ec2:DescribeTags",
                "ec2:DescribeVpcClassicLink",
                "ec2:DescribeVpcs",
                "ec2:GetConsoleOutput",
                "ec2:ModifyImageAttribute",
                "ec2:ModifyInstanceAttribute",
                "ec2:RebootInstances",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:TerminateInstances",
                "elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
                "elasticloadbalancing:ConfigureHealthCheck",
                "elasticloadbalancing:CreateListener",
                "elasticloadbalancing:CreateLoadBalancer",
                "elasticloadbalancing:CreateLoadBalancerListeners",
                "elasticloadbalancing:CreateLoadBalancerPolicy",
                "elasticloadbalancing:CreateRule",
                "elasticloadbalancing:CreateTargetGroup",
                "elasticloadbalancing:DeleteListener",
                "elasticloadbalancing:DeleteLoadBalancer",
                "elasticloadbalancing:DeleteLoadBalancerListeners",
                "elasticloadbalancing:DeleteRule",
                "elasticloadbalancing:DeleteTargetGroup",
                "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
                "elasticloadbalancing:DeregisterTargets",
                "elasticloadbalancing:DescribeInstanceHealth",
                "elasticloadbalancing:DescribeListeners",
                "elasticloadbalancing:DescribeLoadBalancerAttributes",
                "elasticloadbalancing:DescribeLoadBalancerPolicies",
                "elasticloadbalancing:DescribeLoadBalancers",
                "elasticloadbalancing:DescribeRules",
                "elasticloadbalancing:DescribeTargetGroupAttributes",
                "elasticloadbalancing:DescribeTargetGroups",
                "elasticloadbalancing:DescribeTargetHealth",
                "elasticloadbalancing:ModifyListener",
                "elasticloadbalancing:ModifyLoadBalancerAttributes",
                "elasticloadbalancing:ModifyTargetGroup",
                "elasticloadbalancing:ModifyTargetGroupAttributes",
                "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
                "elasticloadbalancing:RegisterTargets",
                "elasticloadbalancing:SetLoadBalancerPoliciesOfListener",
                "elasticloadbalancing:SetSecurityGroups",
                "iam:ListServerCertificates",
                "iam:PassRole"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Copy link
Contributor Author

@aleon1220 aleon1220 Oct 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A customer managed policy is cumbersome for an organization. Is an interesting concept to try to reduce risk but is just too much to handle.

Copy link
Contributor

@avram avram Oct 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With cloudformation stacksets and AWS organizations it is quite feasible to have a customer managed policy.

The power user policy is an irresponsible one and it reflects poorly on the product to recommend it.

Copy link
Contributor Author

@aleon1220 aleon1220 Oct 6, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@avram I understand. If that's the case I highly encourage you to improve the Cloud formation templates that use the AWS IAM policies at https://github.com/spinnaker/spinnaker.github.io/tree/master/downloads/aws

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

3 more reviewers

@avram avram avram left review comments

@dorbin dorbin dorbin approved these changes

@317brian 317brian 317brian approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

target-release/1.23

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@aleon1220 @dorbin @317brian @avram @spinnakerbot