How to set up an OpenGeo Suite cluster on AWS¶
This page will show how to automatically set up and configure a cluster of servers on Amazon Web Services (AWS).
This script is available to Enterprise clients only. Please contact us to become an Enterprise client.
- Machine running the script must use Ubuntu, Red Hat-based Linux or Mac OS X
- Python 2.7 or higher (not guaranteed on Python 3)
- Ansible 1.6.2 or higher (installed via pip)
- Java JDK 7 or higher (not a JRE)
- AWS account with EC2 access
- Super user access
Mac OS X only: Install pip:
sudo easy_install pip
Ansible must be installed via pip. If you have Ansible installed through your standard package manager, run apt-get remove –purge ansible first before running pip install ansible.
Install the python-dev package through your package manager. For example, on Ubuntu:
apt-get install python-dev
Install boto with pip:
pip install boto
Set the $JAVA_HOME variable to the location of Java. For example:
The command which java may not give the true path. For more information on how to find the real location of Java, see the Amazon EC2 documentation.
To verify, type the following command:
Download and install the EC2 API tools.
Extract the EC2 API tools into /usr/local/ec2/ec2-api-tools-a.b.c.d, where a.b.c.d is the specific version of the EC2 API tools.
Add the path where you extracted the EC2 API tools to the $EC2_HOME environment variable. For example:
Add the $EC2_HOME/bin path to the $PATH:
Verify that the EC2 tools are on your path by entering the following command:
If you see a list of regions, your EC2 interface is correct:
REGION eu-west-1 ec2.eu-west-1.amazonaws.com REGION sa-east-1 ec2.sa-east-1.amazonaws.com REGION us-east-1 ec2.us-east-1.amazonaws.com REGION ap-northeast-1 ec2.ap-northeast-1.amazonaws.com REGION us-west-2 ec2.us-west-2.amazonaws.com REGION us-west-1 ec2.us-west-1.amazonaws.com REGION ap-southeast-1 ec2.ap-southeast-1.amazonaws.com REGION ap-southeast-2 ec2.ap-southeast-2.amazonaws.com
For more information about setting up the EC2 tools, please see the Amazon documentation.
EC2 remote access¶
In order to be able to use the EC2 tools, you will need to have AWS access keys. These consist of a pair of credentials, called an Access Key and a Secret Access Key.
Retrieve these keys (or make a new pair) by logging into your AWS account.
Set these keys as local environment variables:
export AWS_ACCESS_KEY=your_access_key export AWS_SECRET_KEY=your_secret_access_key
You will also need to have a key pair set up on your EC2 account.
- To create a key pair, navigate to the EC2 key pairs page.
- Click Create key pair.
- You will then be asked to download a file (with the extension .pem). Save this file in a safe place.
The following are other setup tasks that don’t fall under any specific category.
Add another environment variable:
Open $HOME/ssh_config in a text editor.
Add the following line to the bottom of the file:
Save and close the file.
Download and extract the clustering script archive to a directory.
If you are an Enterprise client and do not have the script, please let us know.
Open the file roles/aws/vars/main.yml in a text editor.
Change the line that starts with aws_keypair to contain the name of your key file (omitting the .pem extension):
Save and close the file.
Launching the cluster¶
With setup complete, you can now launch the cluster.
Run the following command from the root of the clustering directory:
ansible-playbook aws-launch.yml -i hosts.aws -e "use_aws=true" --private-key=key.pem
substituting the name and path of the key file as downloaded in a previous step for key.pem.
During the script, there will be a pause where you are asked to set up your SSH configuration. Add the following to the $HOME/.ssh/config
Host 10.1.2?.* IdentityFile key.pem User ubuntu Port 22 ProxyCommand ssh -o "ControlMaster no" -p 22 -i key.pem ec2-user@INSTANCE_IP -W %h:%p
subsituting the the name of the key file for key.pem, and the IP given by the script for INSTANCE_IP.
Details on the cluster created, including AWS-specific information, will be available in the log file /tmp/informationoutput.
- If you encounter errors while running the script, you can run the script in “debug mode” by appending -vvvv to the command. The individual commands run by the script will be displayed in the terminal.
- Try running the script again. Sometimes, due to issues with AWS connectivity, a script may fail the first time but succeed the second. (There is a fix checked into the latest development version of Ansible which should fix a lot of these problems)
Shutting down the cluster¶
For security reasons, there is no script to shut down a cluster.
To shut down the cluster:
- Log into your AWS EC2 console.
- Click Instances.
- Find the instances generated by the script. If you have many instances, look at the Launch Time or the Key Pair. You can also match up by IP address.
- Terminate these instances.
- Open the AWS VPC console.
- Find the VPC (or VPCs) created by the script and Delete them.
- Open the AWS RDS console.
- Find the RDS instances created by the script and Delete them.