Couchbase on Amazon Marketplace showed how to setup a single Couchbase node using EC2 Console. But typically you provision these nodes en masse, and more commonly create a cluster of them. Couchbase clusters are homogenous, scales horizontally and thus ensure that database does not become a bottleneck for your high performing application.
This blog will show how to create, scale, and rebalance a Couchbase cluster using AWS Command Line Interface (CLI).
Install AWS CLI
Install the AWS CLI provide complete details, but here is what worked on my machine:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
|
tools > pip install awscli
Collecting awscli
Downloading awscli-1.10.27-py2.py3-none-any.whl (937kB)
100% |████████████████████████████████| 937kB 561kB/s
Collecting rsa<=3.3.0,>=3.1.2 (from awscli)
Downloading rsa-3.3-py2.py3-none-any.whl (44kB)
100% |████████████████████████████████| 45kB 7.7MB/s
Collecting s3transfer==0.0.1 (from awscli)
Downloading s3transfer-0.0.1-py2.py3-none-any.whl
Collecting colorama<=0.3.3,>=0.2.5 (from awscli)
Downloading colorama-0.3.3.tar.gz
Collecting botocore==1.4.18 (from awscli)
Downloading botocore-1.4.18-py2.py3-none-any.whl (2.3MB)
100% |████████████████████████████████| 2.3MB 221kB/s
Collecting docutils>=0.10 (from awscli)
Downloading docutils-0.12.tar.gz (1.6MB)
100% |████████████████████████████████| 1.6MB 328kB/s
Collecting pyasn1>=0.1.3 (from rsa<=3.3.0,>=3.1.2->awscli)
Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Collecting futures<4.0.0,>=2.2.0 (from s3transfer==0.0.1->awscli)
Using cached futures-3.0.5-py2-none-any.whl
Collecting jmespath<1.0.0,>=0.7.1 (from botocore==1.4.18->awscli)
Downloading jmespath-0.9.0-py2.py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.4.18->awscli)
Downloading python_dateutil-2.5.3-py2.py3-none-any.whl (201kB)
100% |████████████████████████████████| 204kB 2.4MB/s
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore==1.4.18->awscli)
Using cached six-1.10.0-py2.py3-none-any.whl
Building wheels for collected packages: colorama, docutils
Running setup.py bdist_wheel for colorama ... done
Stored in directory: /Users/arungupta/Library/Caches/pip/wheels/21/c5/cf/63fb92293f3ad402644ccaf882903cacdb8fe87c80b62c84df
Running setup.py bdist_wheel for docutils ... done
Stored in directory: /Users/arungupta/Library/Caches/pip/wheels/db/de/bd/b99b1e12d321fbc950766c58894c6576b1a73ae3131b29a151
Successfully built colorama docutils
Installing collected packages: pyasn1, rsa, futures, jmespath, six, python-dateutil, docutils, botocore, s3transfer, colorama, awscli
Successfully installed awscli-1.10.27 botocore-1.4.18 colorama-0.3.3 docutils-0.12 futures-3.0.5 jmespath-0.9.0 pyasn1-0.1.9 python-dateutil-2.5.3 rsa-3.3 s3transfer-0.0.1 six-1.10.0
|
Configure the CLI:
|
1
2
3
4
5
6
7
|
aws configure
AWS Access Key ID [****************Q5FQ]: YOUR ACCESS KEY ID
AWS Secret Access Key [****************thBR]: YOUR SECRET ACCESS KEY
Default region name [us-west-1]:
Default output format [None]:
|
Enter your access key id and secret access key. These can be obtained as explained in Getting Your Access Key Id and Secret Access Key.
Create AWS Security Group
If you provisioned a server earlier using Amazon 1-click then a security group by the name Couchbase Server Community Edition-4-0-0-AutogenByAWSMP- is created for you. This security group has all the ports authorized required for creating a Couchbase cluster and can be used for creating the instance.
Alternatively, you can create a new security group and explicitly authorize ports.
Create a security group:
|
1
2
3
4
5
6
|
aws ec2 create-security-group --group-name my-couchbase-sg --description "My Couchbase Security Group"
{
"GroupId": "sg-c9bb19ad"
}
|
Authorize ports in the security group:
|
1
2
3
4
5
6
|
aws ec2 authorize-security-group-ingress --group-name my-couchbase-sg --port 8091-8093 --protocol tcp --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-name my-couchbase-sg --port 11209-11211 --protocol tcp --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-name my-couchbase-sg --port 21100-21299 --protocol tcp --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-name my-couchbase-sg --port 4369 --protocol tcp --cidr 0.0.0.0/0
|
Create an AWS Key Pair
Read more about creating key pair. Create a key pair:
|
1
2
3
|
aws ec2 create-key-pair --key-name my-couchbase-key > my-couchbase-key.pem
|
Note the key name as that will be used later.
Create Couchbase Nodes on Amazon
Create two instances using the newly created security group as:
|
1
2
3
|
aws ec2 run-instances --image-id ami-db95ffbb --count 2 --instance-type m3.large --key-name my-couchbase-key --security-groups my-couchbase-sg
|
Note, the number of instances are specified using --count 2.
AMI ID can be obtained using EC2 console: https://us-west-1.console.aws.amazon.com/ec2/v2/home?region=us-west-1#Images:visibility=public-images;search=couchbase;sort=desc:name.
Or create two instances using the pre-created security group as:
|
1
2
3
|
aws ec2 run-instances --image-id ami-db95ffbb --count 2 --instance-type m3.large --key-name my-couchbase-key --security-groups "Couchbase Server Community Edition-4-0-0-AutogenByAWSMP-"
|
This will show the output as:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
|
{
"OwnerId": "598307997273",
"ReservationId": "r-cb952179",
"Groups": [],
"Instances": [
{
"Monitoring": {
"State": "disabled"
},
"PublicDnsName": "",
"RootDeviceType": "ebs",
"State": {
"Code": 0,
"Name": "pending"
},
"EbsOptimized": false,
"LaunchTime": "2016-05-16T16:11:59.000Z",
"PrivateIpAddress": "172.31.15.200",
"ProductCodes": [],
"VpcId": "vpc-c561f9a0",
"StateTransitionReason": "",
"InstanceId": "i-90199025",
"ImageId": "ami-db95ffbb",
"PrivateDnsName": "ip-172-31-15-200.us-west-1.compute.internal",
"KeyName": "my-couchbase-key",
"SecurityGroups": [
{
"GroupName": "my-couchbase-sg",
"GroupId": "sg-6172dd05"
}
],
"ClientToken": "",
"SubnetId": "subnet-625bd23b",
"InstanceType": "m3.large",
"NetworkInterfaces": [
{
"Status": "in-use",
"MacAddress": "06:b1:52:a6:87:f7",
"SourceDestCheck": true,
"VpcId": "vpc-c561f9a0",
"Description": "",
"NetworkInterfaceId": "eni-86ac6fdb",
"PrivateIpAddresses": [
{
"PrivateDnsName": "ip-172-31-15-200.us-west-1.compute.internal",
"Primary": true,
"PrivateIpAddress": "172.31.15.200"
}
],
"PrivateDnsName": "ip-172-31-15-200.us-west-1.compute.internal",
"Attachment": {
"Status": "attaching",
"DeviceIndex": 0,
"DeleteOnTermination": true,
"AttachmentId": "eni-attach-530b7293",
"AttachTime": "2016-05-16T16:11:59.000Z"
},
"Groups": [
{
"GroupName": "my-couchbase-sg",
"GroupId": "sg-6172dd05"
}
],
"SubnetId": "subnet-625bd23b",
"OwnerId": "598307997273",
"PrivateIpAddress": "172.31.15.200"
}
],
"SourceDestCheck": true,
"Placement": {
"Tenancy": "default",
"GroupName": "",
"AvailabilityZone": "us-west-1c"
},
"Hypervisor": "xen",
"BlockDeviceMappings": [],
"Architecture": "x86_64",
"StateReason": {
"Message": "pending",
"Code": "pending"
},
"RootDeviceName": "/dev/xvda",
"VirtualizationType": "hvm",
"AmiLaunchIndex": 0
},
{
"Monitoring": {
"State": "disabled"
},
"PublicDnsName": "",
"RootDeviceType": "ebs",
"State": {
"Code": 0,
"Name": "pending"
},
"EbsOptimized": false,
"LaunchTime": "2016-05-16T16:11:59.000Z",
"PrivateIpAddress": "172.31.15.201",
"ProductCodes": [],
"VpcId": "vpc-c561f9a0",
"StateTransitionReason": "",
"InstanceId": "i-93199026",
"ImageId": "ami-db95ffbb",
"PrivateDnsName": "ip-172-31-15-201.us-west-1.compute.internal",
"KeyName": "my-couchbase-key",
"SecurityGroups": [
{
"GroupName": "my-couchbase-sg",
"GroupId": "sg-6172dd05"
}
],
"ClientToken": "",
"SubnetId": "subnet-625bd23b",
"InstanceType": "m3.large",
"NetworkInterfaces": [
{
"Status": "in-use",
"MacAddress": "06:83:e8:06:2a:f1",
"SourceDestCheck": true,
"VpcId": "vpc-c561f9a0",
"Description": "",
"NetworkInterfaceId": "eni-81ac6fdc",
"PrivateIpAddresses": [
{
"PrivateDnsName": "ip-172-31-15-201.us-west-1.compute.internal",
"Primary": true,
"PrivateIpAddress": "172.31.15.201"
}
],
"PrivateDnsName": "ip-172-31-15-201.us-west-1.compute.internal",
"Attachment": {
"Status": "attaching",
"DeviceIndex": 0,
"DeleteOnTermination": true,
"AttachmentId": "eni-attach-680b72a8",
"AttachTime": "2016-05-16T16:11:59.000Z"
},
"Groups": [
{
"GroupName": "my-couchbase-sg",
"GroupId": "sg-6172dd05"
}
],
"SubnetId": "subnet-625bd23b",
"OwnerId": "598307997273",
"PrivateIpAddress": "172.31.15.201"
}
],
"SourceDestCheck": true,
"Placement": {
"Tenancy": "default",
"GroupName": "",
"AvailabilityZone": "us-west-1c"
},
"Hypervisor": "xen",
"BlockDeviceMappings": [],
"Architecture": "x86_64",
"StateReason": {
"Message": "pending",
"Code": "pending"
},
"RootDeviceName": "/dev/xvda",
"VirtualizationType": "hvm",
"AmiLaunchIndex": 1
}
]
}
|
Status of the instances can be checked as:
|
1
2
3
|
aws ec2 describe-instance-status --instance-ids `aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId]' --filters Name=instance-state-name,Values=running --output text`
|
And shows the output as:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
|
{
"InstanceStatuses": [
{
"InstanceId": "i-90199025",
"InstanceState": {
"Code": 16,
"Name": "running"
},
"AvailabilityZone": "us-west-1c",
"SystemStatus": {
"Status": "initializing",
"Details": [
{
"Status": "initializing",
"Name": "reachability"
}
]
},
"InstanceStatus": {
"Status": "initializing",
"Details": [
{
"Status": "initializing",
"Name": "reachability"
}
]
}
},
{
"InstanceId": "i-93199026",
"InstanceState": {
"Code": 16,
"Name": "running"
},
"AvailabilityZone": "us-west-1c",
"SystemStatus": {
"Status": "initializing",
"Details": [
{
"Status": "initializing",
"Name": "reachability"
}
]
},
"InstanceStatus": {
"Status": "initializing",
"Details": [
{
"Status": "initializing",
"Name": "reachability"
}
]
}
}
]
}
|
Here the status is shown as initializing. It takes a few minutes for the instances to be provisioned. Instances that have passed all the checks can be verified as:
|
1
2
3
|
aws ec2 describe-instance-status --filters Name=instance-status.reachability,Values=passed --instance-ids `aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId]' --filters Name=instance-state-name,Values=running --output text`
|
At first, it shows the result as:
|
1
2
3
4
5
|
{
"InstanceStatuses": []
}
|
But once all the instances pass the check, then the results look like:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
|
{
"InstanceStatuses": [
{
"InstanceId": "i-90199025",
"InstanceState": {
"Code": 16,
"Name": "running"
},
"AvailabilityZone": "us-west-1c",
"SystemStatus": {
"Status": "ok",
"Details": [
{
"Status": "passed",
"Name": "reachability"
}
]
},
"InstanceStatus": {
"Status": "ok",
"Details": [
{
"Status": "passed",
"Name": "reachability"
}
]
}
},
{
"InstanceId": "i-93199026",
"InstanceState": {
"Code": 16,
"Name": "running"
},
"AvailabilityZone": "us-west-1c",
"SystemStatus": {
"Status": "ok",
"Details": [
{
"Status": "passed",
"Name": "reachability"
}
]
},
"InstanceStatus": {
"Status": "ok",
"Details": [
{
"Status": "passed",
"Name": "reachability"
}
]
}
}
]
}
|
Here the status is shown as passed.
Configure Couchbase Nodes
Each Couchbase node needs to be provisioned with the following details:
- Memory
- Services (index, data and query)
- Auth credentials (username:
Administrator, password:password) - Loads
travel-samplebucket
This can be done using the script:
|
1
2
3
4
5
6
7
8
9
|
for ip in "$@"
do
curl -v -X POST http://$ip:8091/pools/default -d memoryQuota=300 -d indexMemoryQuota=300
curl -v http://$ip:8091/node/controller/setupServices -d services=kv%2Cn1ql%2Cindex
curl -v http://$ip:8091/settings/web -d port=8091 -d username=Administrator -d password=password
curl -v -u Administrator:password -X POST http://$ip:8091/sampleBuckets/install -d '["travel-sample"]'
done
|
This is available at: https://github.com/arun-gupta/couchbase-amazon/blob/master/configure-instance.sh.
It can be invoked as:
|
1
2
3
|
./configure-instance.sh `aws ec2 describe-instances --query 'Reservations[*].Instances[*].[PublicDnsName]' --filters Name=instance-state-name,Values=running --output text`
|
And shows the output as:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
|
* Trying 52.53.171.49...
* Connected to ec2-52-53-171-49.us-west-1.compute.amazonaws.com (52.53.171.49) port 8091 (#0)
> POST /pools/default HTTP/1.1
> Host: ec2-52-53-171-49.us-west-1.compute.amazonaws.com:8091
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 36
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 36 out of 36 bytes
< HTTP/1.1 200 OK
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 16 May 2016 16:18:47 GMT
< Content-Length: 0
< Cache-Control: no-cache
<
* Connection #0 to host ec2-52-53-171-49.us-west-1.compute.amazonaws.com left intact
* Trying 52.53.171.49...
* Connected to ec2-52-53-171-49.us-west-1.compute.amazonaws.com (52.53.171.49) port 8091 (#0)
> POST /node/controller/setupServices HTTP/1.1
> Host: ec2-52-53-171-49.us-west-1.compute.amazonaws.com:8091
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 26
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 26 out of 26 bytes
< HTTP/1.1 200 OK
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 16 May 2016 16:18:47 GMT
< Content-Length: 0
< Cache-Control: no-cache
<
* Connection #0 to host ec2-52-53-171-49.us-west-1.compute.amazonaws.com left intact
* Trying 52.53.171.49...
* Connected to ec2-52-53-171-49.us-west-1.compute.amazonaws.com (52.53.171.49) port 8091 (#0)
> POST /settings/web HTTP/1.1
> Host: ec2-52-53-171-49.us-west-1.compute.amazonaws.com:8091
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 50
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 50 out of 50 bytes
< HTTP/1.1 200 OK
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 16 May 2016 16:18:47 GMT
< Content-Type: application/json
< Content-Length: 78
< Cache-Control: no-cache
<
* Connection #0 to host ec2-52-53-171-49.us-west-1.compute.amazonaws.com left intact
{"newBaseUri":"http://ec2-52-53-171-49.us-west-1.compute.amazonaws.com:8091/"}* Trying 52.53.171.49...
* Connected to ec2-52-53-171-49.us-west-1.compute.amazonaws.com (52.53.171.49) port 8091 (#0)
* Server auth using Basic with user 'Administrator'
> POST /sampleBuckets/install HTTP/1.1
> Host: ec2-52-53-171-49.us-west-1.compute.amazonaws.com:8091
> Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 17
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 17 out of 17 bytes
< HTTP/1.1 202 Accepted
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 16 May 2016 16:18:47 GMT
< Content-Type: application/json
< Content-Length: 2
< Cache-Control: no-cache
<
* Connection #0 to host ec2-52-53-171-49.us-west-1.compute.amazonaws.com left intact
[]* Trying 52.53.239.100...
* Connected to ec2-52-53-239-100.us-west-1.compute.amazonaws.com (52.53.239.100) port 8091 (#0)
> POST /pools/default HTTP/1.1
> Host: ec2-52-53-239-100.us-west-1.compute.amazonaws.com:8091
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 36
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 36 out of 36 bytes
< HTTP/1.1 200 OK
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 16 May 2016 16:18:48 GMT
< Content-Length: 0
< Cache-Control: no-cache
<
* Connection #0 to host ec2-52-53-239-100.us-west-1.compute.amazonaws.com left intact
* Trying 52.53.239.100...
* Connected to ec2-52-53-239-100.us-west-1.compute.amazonaws.com (52.53.239.100) port 8091 (#0)
> POST /node/controller/setupServices HTTP/1.1
> Host: ec2-52-53-239-100.us-west-1.compute.amazonaws.com:8091
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 26
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 26 out of 26 bytes
< HTTP/1.1 200 OK
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 16 May 2016 16:18:48 GMT
< Content-Length: 0
< Cache-Control: no-cache
<
* Connection #0 to host ec2-52-53-239-100.us-west-1.compute.amazonaws.com left intact
* Trying 52.53.239.100...
* Connected to ec2-52-53-239-100.us-west-1.compute.amazonaws.com (52.53.239.100) port 8091 (#0)
> POST /settings/web HTTP/1.1
> Host: ec2-52-53-239-100.us-west-1.compute.amazonaws.com:8091
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 50
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 50 out of 50 bytes
< HTTP/1.1 200 OK
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 16 May 2016 16:18:48 GMT
< Content-Type: application/json
< Content-Length: 79
< Cache-Control: no-cache
<
* Connection #0 to host ec2-52-53-239-100.us-west-1.compute.amazonaws.com left intact
{"newBaseUri":"http://ec2-52-53-239-100.us-west-1.compute.amazonaws.com:8091/"}* Trying 52.53.239.100...
* Connected to ec2-52-53-239-100.us-west-1.compute.amazonaws.com (52.53.239.100) port 8091 (#0)
* Server auth using Basic with user 'Administrator'
> POST /sampleBuckets/install HTTP/1.1
> Host: ec2-52-53-239-100.us-west-1.compute.amazonaws.com:8091
> Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 17
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 17 out of 17 bytes
< HTTP/1.1 202 Accepted
< Server: Couchbase Server
< Pragma: no-cache
< Date: Mon, 16 May 2016 16:18:48 GMT
< Content-Type: application/json
< Content-Length: 2
< Cache-Control: no-cache
<
* Connection #0 to host ec2-52-53-239-100.us-west-1.compute.amazonaws.com left intact
[]
|
This is invoking Couchbase REST API to configure each Couchbase node.
Now that each Couchbase node is configured, lets access them. Find public IP address of the instances:
|
1
2
3
|
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[PublicDnsName]' --filters Name=instance-state-name,Values=running --output text
|
It shows the output as:
|
1
2
3
4
|
ec2-52-53-171-49.us-west-1.compute.amazonaws.com
ec2-52-53-239-100.us-west-1.compute.amazonaws.com
|
Pick one of the IP addresses and access it at <public-ip-1>:8091 to see the output:
Each Couchbase node is configured with username as Administrator and password as password. Entering the credentials shows Couchbase Web Console:
Click on Server Nodes to see that only a single node is in the cluster:
Create and Rebalance Couchbase Cluster
All Couchbase server nodes are created equal. This allows the Couchbase cluster to truly scale horizontally to meet your growing application demands. Independently running Couchbase nodes can be added to a cluster by invoking the server-add CLI command.
This is typically a two step process. The first step is to add one or more nodes. The second step is to rebalance the cluster where data on the existing nodes is rebalanced across the updated cluster.
In our case, a Couchbase node is running on each AMI. Lets pick IP address of any one Couchbase node and add IP address of the other node. This can be done using the script:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
export primary=$1
shift
for ip in "$@"
do
$COUCHBASE_CLI \
server-add \
--cluster=$primary:8091 \
--user Administrator \
--password password \
--server-add=$ip \
--server-add-username=Administrator \
--server-add-password=password
done
|
This script is available at https://github.com/arun-gupta/couchbase-amazon/blob/master/create-cluster.sh and can be invoked as:
|
1
2
3
|
./create-cluster.sh `aws ec2 describe-instances --query 'Reservations[*].Instances[*].[PublicDnsName]' --filters Name=instance-state-name,Values=running --output text`
|
And shows the output as:
|
1
2
3
|
SUCCESS: server-add ec2-52-53-239-100.us-west-1.compute.amazonaws.com:8091
|
Couchbase Web Console is updated to show:
Finally, rebalance the cluster using the script:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
export primary=$1
shift
for ip in "$@"
do
$COUCHBASE_CLI \
rebalance \
--cluster=$primary:8091 \
--user Administrator \
--password password \
--server-add-username=Administrator \
--server-add-password=password
done
|
This shows the output as:
|
1
2
3
4
|
INFO: rebalancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
SUCCESS: rebalanced cluster
|
Couchbase Web Console is now updated:
Once the cluster is up and running, try out Hello Couchbase Example.
Terminate Couchbase Nodes
Finally, killing the cluster is fairly straight forward:
|
1
2
3
|
aws ec2 terminate-instances --instance-ids `aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId]' --filters Name=instance-state-name,Values=running --output text`
|
This blog showed how to spin up, scale and rebalance a Couchbase cluster using AWS CLI. All scripts are available at https://github.com/arun-gupta/couchbase-amazon.
Further references …
- Couchbase Server Developer Portal
- Hello Couchbase Example
- Questions on StackOverflow, Forums or Slack Channel
- Follow us @couchbasedev
- Couchbase 4.5 Beta
Source: http://blog.couchbase.com/2016/may/couchbase-cluster-amazon-using-cli
