Preface
This article exists because Docker for Mac is not yet mature and this provides a way to run docker containers in AWS, but be able to work with them like you would locally.
The advantages?
- Container pull speed
- Disk/Memory/CPU use moved to an AWS instance
- Overall build speed tends to be quicker
The disadvantages?
- Running an AWS instance just to run docker (cost)
- Copying files relies on upload speed of your internet link
- Potential to leave instance running when you're not using it (again, cost)
With that in mind, let's get started to see how it can be done!
Getting started
First, you'll need to have docker-machine installed on your laptop. I use homebrew for this:
$ brew install docker-machine
I'm testing this with version 0.4.1, so depending on what version you're using, YMMV.
Now, go into your AWS account and create an IAM user and then attach the AdministratorAccess IAM role to it. When you create the user, take note of the Access key and secret, then add it to your ~/.aws/credentials file, as the [default] entry, like so:
[default]
aws_access_key_id = **************************
aws_secret_access_key = ******************************
While still in the AWS console, find out the VPC ID and Subnet ID in that VPC you wish to place the instance into that you'll connect to with docker-machine. They will look something like:
vpc-XXXXXXXX
subnet-XXXXXXXX
Ideally, look for the subnet in Availability Zone A, as that's what docker-machine defaults to, but you can use a subnet in another AZ, but you need to tell docker-machine which subnet it's in.
Now that you have the credentials configured and the VPC/Subnet IDs, we can create the docker-machine instance so we can start using docker on it. We can do this like so:
$ docker-machine create --amazonec2-region ap-southeast-2 --amazonec2-vpc-id vpc-XXXXXXXX --amazonec2-subnet-id subnet-XXXXXXXX --amazonec2-instance-type t2.large --amazonec2-root-size 32 --amazonec2-volume-type gp2 --driver amazonec2 aws01
You can modify the above values to suit, if you want to call it something other than aws01, or you want a bigger instance or a bigger root volume for example.
If you specified a subnet in another availability zone, add this option after specifying the subnet ID:
--amazonec2-zone b
Where 'b' is the AZ of the subnet that you have used.
You can read up on all the options by visiting this article at the Docker Docs site.
Once you've kicked off the create, you should see something like the following:
$ docker-machine create --amazonec2-region ap-southeast-2 --amazonec2-vpc-id vpc-XXXXXXXX --amazonec2-subnet-id subnet-XXXXXXXX --amazonec2-instance-type t2.large --amazonec2-root-size 32 --amazonec2-volume-type gp2 --driver amazonec2 aws01
Running pre-create checks...
Creating machine...
(aws01) Launching instance...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with ubuntu(systemd)...
Installing Docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env aws01
As the last comment says, you need to configure your terminal to point docker-machine to the newly created instance. You can do this by:
$ docker-machine env aws01
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://XX.XXX.XX.XXX:2376"
export DOCKER_CERT_PATH="/Users/USER/.docker/machine/machines/aws01"
export DOCKER_MACHINE_NAME="aws01"
# Run this command to configure your shell:
# eval $(docker-machine env aws01)
Then by running the eval or exporting the environment variables. You can then check that you're using the remote docker instance by running docker info, which should show the 'Name' field as the remote instance name.
Note: If you open up a new terminal, it will need the ENV vars set to point to the remote docker instance, otherwise it will default to running locally.
Securing the instance
By default, docker-machine creates a new security group in your AWS account and opens up port 22 (SSH) and port 2376 (Docker) to the world, but I would recommend locking it down to only your IP address by editing the security group from allowing those two ports on 0.0.0.0/0 to selecting the 'My IP' option which will use your current IP address.
Logging into the instance
It's extremely easy to login to the instance if you need to check anything host-related. All you need to do is use the following command, substituting the 'aws01' for whatever name you have given your instance:
docker-machine ssh aws01
Stopping and starting the instance
To stop the instance, so that it turns it off and puts it in a 'Stopped' state in AWS to reduce costs and allow it to be used later, use the following command:
$ docker-machine stop aws01
Stopping "aws01"...
Machine "aws01" was stopped.
And to restart it again is simply:
$ docker-machine start aws01
Starting "aws01"...
Machine "aws01" was started.
Waiting for SSH to be available...
Detecting the provisioner...
Started machines may have new IP addresses. You may need to re-run the `docker-machine env` command.
Removing the instance
$ docker-machine rm aws01
About to remove aws01
Are you sure? (y/n): y
Successfully removed aws01
