Skip to content

Commit 6f57ff6

Browse files
prometherionmjuraga
prometherion
authored and
mjuraga
committed
DOC/MINOR: discovery: Service Discovery on AWS
1 parent e2cce94 commit 6f57ff6

File tree

3 files changed

+222
-0
lines changed

3 files changed

+222
-0
lines changed

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,10 @@ For more docs how to use the Data Plane API check our [documentation](https://ww
134134

135135
Alternatively, dataplaneapi serves its own interactive documentation relevant for the current build on the `/v2/docs` uri. Just point your browser to the host/port dataplane was started with (i.e. `http://localhost:5555/v2/docs`)
136136

137+
## Service Discovery
138+
139+
Check the documentation in the [README](./discovery/README.md).
140+
137141
## Contributing
138142

139143
If you wish to contribute to this project please check [Contributing Guide](CONTRIBUTING.md)

‎discovery/AWS.md‎

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
# AWS Service Discovery
2+
3+
Data Plane API allows performing EC2 instances discovery, self-registering IP addresses as backend servers.
4+
5+
## Required tags
6+
7+
All instances must be tagged with the following tags:
8+
9+
- `HAProxy:Service:Name`: the service name will compose the HAProxy `backend` name.
10+
- `HAProxy:Service:Port`: the default service port is listening to.
11+
12+
> The said tags are mandatory, otherwise, the instance will be ignored.
13+
14+
An additional tag is provided, in case of override for the single instance
15+
16+
- `HAProxy:Instance:Port`: allows to override the default Service port.
17+
18+
## Filtering
19+
20+
By default, all instances in the selected AWS region will be considered.
21+
22+
Selection of specific instances can be achieved using the `allowlist` functionality, specifying the desired EC2 filter to consider according to the [AWS documentation](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options).
23+
24+
```hcl
25+
service_discovery {
26+
awsRegions = [
27+
{
28+
Description = "Allowlist example"
29+
Allowlist = [
30+
{
31+
Key = "tag-key"
32+
Value = "Must:Have:This:Tag:Key"
33+
},
34+
]
35+
Enabled = false
36+
ID = "96b14c57-b011-42e5-8d01-b58feba07319"
37+
Name = "john.doe"
38+
Region = "us-east-1"
39+
RetryTimeout = 10
40+
ServerSlotsBase = 10
41+
ServerSlotsGrowthIncrement = 10
42+
ServerSlotsGrowthType = "exponential"
43+
},
44+
]
45+
}
46+
```
47+
48+
As `allowlist`, the `denylist` option allows to filter out specific instances matching the desired filters.
49+
50+
```hcl
51+
service_discovery {
52+
awsRegions = [
53+
{
54+
Description = "Denylist example"
55+
Allowlist = [
56+
{
57+
Key = "tag-key"
58+
Value = "Must:Have:This:Tag:Key"
59+
},
60+
]
61+
Denylist = [
62+
{
63+
Key = "tag:Environment"
64+
Value = "Development"
65+
},
66+
]
67+
Enabled = false
68+
ID = "96b14c57-b011-42e5-8d01-b58feba07319"
69+
Name = "john.doe"
70+
Region = "us-east-1"
71+
RetryTimeout = 10
72+
ServerSlotsBase = 10
73+
ServerSlotsGrowthIncrement = 10
74+
ServerSlotsGrowthType = "exponential"
75+
},
76+
]
77+
}
78+
```
79+
80+
## Authorization
81+
82+
Data Plane API needs the plain AWS credentials to interact with it.
83+
84+
```hcl
85+
service_discovery {
86+
awsRegions = [
87+
{
88+
Description = "Credentials example"
89+
SecretAccessKey = "************************************soLl"
90+
AccessKeyID = "****************L7GT"
91+
Enabled = false
92+
ID = "96b14c57-b011-42e5-8d01-b58feba07319"
93+
Name = "john.doe"
94+
Region = "us-east-1"
95+
RetryTimeout = 10
96+
ServerSlotsBase = 10
97+
ServerSlotsGrowthIncrement = 10
98+
ServerSlotsGrowthType = "exponential"
99+
},
100+
]
101+
}
102+
```
103+
104+
> In case of Data Plane API running in an EC2 with a IAM Role attached (as [`AmazonEC2ReadOnlyAccess`](https://console.aws.amazon.com/iam/home#/policies/arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess$serviceLevelSummary)), there's no need for additional credentials.
105+
106+
## Server options
107+
108+
Upon a Service discovery, Data Plane API will create the corresponding `backend` section using the following options:
109+
110+
- `ServerSlotsBase`: the minumum amount of `server` entries per `backend`
111+
- `ServerSlotsGrowthIncrement`: the additional slots allocating for `server` in case of additional entries
112+
- `ServerSlotsGrowthType`: the function type to implement in case of `server` slots growth
113+
114+
## Instances IP address
115+
116+
Using the HCL `IPV4Address` option (or the JSON `ipv4_address` one) you can specify which kind of IP address Data Plane API has to consider for the backend `server`.
117+
118+
Available values can be `private` (for the private one, reachable inside the AWS VPC) or `public`.
119+
120+
> If the instances doesn't have a public IPv4 address, and the service discovery configuration claims the `public` type, In case of `public` type, the EC2 will be ignored.
121+
122+
## Retry timeout
123+
124+
With the HCL `RetryTimeout` option (`retry_timeout` in the JSON counterpart) you can specify the interval of time elapsing between the reconciliation and the following.
125+
126+
Unit is expressed in __seconds__.
127+
128+
# Examples
129+
130+
## Creating a discovery on a selected AWS region
131+
132+
```json
133+
// curl -XPOST "http://localhost:5555/v2/service_discovery/aws" -H 'content-type: application/json' -d @/path/to/payload.json
134+
{
135+
"access_key_id": "****************L7GT",
136+
"enabled": true,
137+
"name": "my-service-discovery",
138+
"region": "us-east-1",
139+
"secret_access_key": "****************soLl",
140+
"ipv4_address": "private",
141+
"retry_timeout": 60
142+
}
143+
```
144+
145+
```hcl
146+
service_discovery {
147+
awsRegions = [
148+
{
149+
AccessKeyID = "****************L7GT"
150+
Enabled = true
151+
Name = "my-service-discovery"
152+
Region = "us-east-1"
153+
SecretAccessKey = "****************soLl"
154+
IPV4Address = "private"
155+
RetryTimeout = 60
156+
},
157+
]
158+
}
159+
```
160+
161+
The resulting output will be the following, YMMV.
162+
163+
```
164+
backend aws-us-east-1-my-service-discovery-my-service-name-8080
165+
server SRV_L17LT 172.31.68.158:8080 check weight 128
166+
server SRV_lsVqM 127.0.0.1:80 disabled weight 128
167+
server SRV_NTZyL 127.0.0.1:80 disabled weight 128
168+
server SRV_KMIFS 127.0.0.1:80 disabled weight 128
169+
server SRV_D2x28 127.0.0.1:80 disabled weight 128
170+
server SRV_MlgPJ 127.0.0.1:80 disabled weight 128
171+
server SRV_0SDZV 127.0.0.1:80 disabled weight 128
172+
server SRV_HnHJP 127.0.0.1:80 disabled weight 128
173+
server SRV_xMKi0 127.0.0.1:80 disabled weight 128
174+
server SRV_tWxu3 127.0.0.1:80 disabled weight 128
175+
```
176+
177+
The `backend` name pattern is built with the following format:
178+
`aws-<REGION>-<SERVICE_DISCOVERY_CONFIGURATION_NAME>-<SERVICE_NAME>-<SERVICE_PORT>`
179+
180+
## Pausing the discovery on a selected AWS region
181+
182+
```json
183+
// curl -XPUT "http://localhost:5555/v2/service_discovery/aws/96b14c57-b011-42e5-8d01-b58feba07319" -H 'content-type: application/json' -d @/path/to/payload.json
184+
{
185+
"access_key_id": "****************L7GT",
186+
"enabled": false,
187+
"name": "my-service-discovery",
188+
"region": "us-east-1",
189+
"secret_access_key": "****************soLl",
190+
"ipv4_address": "private",
191+
"retry_timeout": 60
192+
}
193+
```
194+
195+
```hcl
196+
service_discovery {
197+
awsRegions = [
198+
{
199+
AccessKeyID = "****************L7GT"
200+
Enabled = false
201+
Name = "my-service-discovery"
202+
Region = "us-east-1"
203+
SecretAccessKey = "****************soLl"
204+
IPV4Address = "private"
205+
RetryTimeout = 60
206+
},
207+
]
208+
}
209+
```
210+
211+
As a result of this action, Data Plane API will not update the discovered `backend` sections and their `server` entries: no data will be lost.
212+
213+
> Potentially, due to the spawn of newer EC2 instances or reboots with a change of the IPv4 address, data could be outdated.

‎discovery/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Data Plane API Service Discovery
2+
3+
## AWS
4+
5+
Check the AWS Service Discovery [README](./AWS.md).

0 commit comments

Comments
 (0)