This post is a simple guide on setting up an EC2 server for deep learning as quickly as possible.
Launching an Instance
In EC2, select ‘Launch Instance’. Select your AMI, choose Deep Learning AMI (Ubuntu). You could use the Amazon Linux version, I just personally prefer Ubuntu.
For actual deep learning, you will want to select a p* instance, ideally a p3 instance. If you are initially testing and writing prototype code, you will probably want to select a small t* instance. You can change this later on the EC2 Dashboard.
Move to Review and Launch. On the security groups you will need to open ports 22, 443 and 8888
Click ‘Launch’ and select your key pair (or create a new one).
On security, while we are opening 22, 443 and 8888 to the world it should not be a security risk. For SSH only the person with your key pair should be able to SSH in. For 443 and 8888, these are for Jupyter. As long as your Jupyter instance remains password or token protected (it is by default) it should not be a security risk.
Configuring HTTPS
SSH into the server with:
| ssh -i |
It is a good idea to update any packages using
Create a Jupyter config
| jupyter notebook –generate-config |
Create a certificate directory
Create a self-signed certificate valid for one year
| sudo openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem |
Edit your Jupyter config
| vim ~/.jupyter/jupyter_notebook_config.py |
Edit the file to include the following four lines
Note: I don’t set a password as I think it is safer to use the token (you will need to be SSH’d into the server to obtain the token)
Running Notebooks
Now run:
Copy and paste the URL into your browser. It will look like https://localhost:8888/?token=ef0f…
Change ‘localhost’ to be the IP or DNS of your server.
You will get a warning screen
As this is your own self-signed certificate, click Advanced and Proceed.
At this point you will have your notebook interface!
Installing additional packages
The AMI comes with most packages you need installed. If you need to install anything else, go to the terminal. Ensure you activate the conda environment you are using in your notebook e.g. if you are using Tensorflow Python 3, run
| source activate tensorflow_p36 |
Then install your package via conda