Here are some useful tips for DevOps teams using AWS CLI on a regular basis.
Amazon Linux - Frequently used commands (CLI)
# restart Apache $ sudo service httpd restart # edit vhosts file $ sudo nano /etc/httpd/conf.d/vhost.conf # edit httpd.conf file $ sudo nano /etc/httpd/conf/httpd.conf # edit PHP .ini file $ sudo nano /etc/php.ini # copy all from one dir to another $ sudo cp -R /var/www/html/* /var/www/vhosts/staging # to delete everything within a directory but not the directory itself (including inner sub-directories) $ sudo rm -rf /var/www/html/* # the command leaves a .htaccess file behind if there is one there $ sudo rm -rf /var/www/html/.htaccess # staging (if applicable) $ sudo rm -rf /var/www/vhosts/staging/* $ sudo rm -rf /var/www/vhosts/staging/.htaccess # zip a directory and everything in it $ sudo zip -r html.zip /var/www/html/* # aws configure IAM credentials $ aws configure AWS Access Key ID [None]: accesskey AWS Secret Access Key [None]: secretkey Default region name [None]: eu-west-1 Default output format [None]: # add/remove securiyt groups entries (example opens port 80 to all) $ aws ec2 authorize-security-group-ingress --group-id sg-******** --protocol tcp --port 80 --cidr 0.0.0.0/0 $ aws ec2 revoke-security-group-ingress --group-id sg-******** --protocol tcp --port 80 --cidr 0.0.0.0/0 # mysqldb export/dump $ mysqldump -hyour-host-name --port=3306 -uroot -pyour-password --databases example > /var/www/backup/daily/daily_`date +\%Y-\%m-\%d_\%H-\%M-\%S`.sql # mysqldb write export/dump $ mysql -hyour-host-name --port=3306 -uroot -pyour-password example < /var/www/backup/daily/daily_`date +\%Y-\%m-\%d_\%H-\%M-\%S`.sql # logs - global system messages, including the messages that are logged during system startup. Includes mail, cron, daemon, kern, auth, etc. (use your preferred text editor e.g. Vim, Gedit, Atom) $ sudo nano /var/log/message # logs - Authenication logs $ sudo nano /var/log/auth # logs - Kernel logs $ sudo nano /var/log/kern # logs - Cron logs $ sudo nano /var/log/cron # logs - Access Logs $ sudo nano /var/log/httpd/access_log
How to setup an SSL cert on an EC2 instance
$ sudo su root $ yum install python27-devel git $ git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt $ /opt/letsencrypt/letsencrypt-auto --debug ---- #if using an AMI where letsencrypt is already installed you may come across zope.interface issue in which case run these commands $ rm -rf ~/.local/share/letsencrypt $ rm -rf /opt/eff.org/certbot/ $ unset PYTHON_INSTALL_LAYOUT; $ rm -rf /root/.local/share/letsencrypt/; ----- $ echo "rsa-key-size = 4096" >> /etc/letsencrypt/config.ini $ echo "email = example@example.com" >> /etc/letsencrypt/config.ini # the following commands are assuming you have a UAT/Staging version of your site which you want to run over HTTPS also $ /opt/letsencrypt/letsencrypt-auto certonly --webroot -w /var/www/vhosts/staging -d uat.example.com --config /etc/letsencrypt/config.ini --agree-tos $ /opt/letsencrypt/letsencrypt-auto certonly --webroot -w /var/www/vhosts/staging -d www.uat.example.com --config /etc/letsencrypt/config.ini --agree-tos $ /opt/letsencrypt/letsencrypt-auto certonly --webroot -w /var/www/html -d example.com --config /etc/letsencrypt/config.ini --agree-tos $ /opt/letsencrypt/letsencrypt-auto certonly --webroot -w /var/www/html -d www.example.com --config /etc/letsencrypt/config.ini --agree-tos $ rmdir /var/www/vhosts/staging/.well-known $ rmdir /var/www/html/.well-known
# Your certs should now be validated, so here is the VirtualHost settings for your vhosts.conf file
# Listen 443 <VirtualHost *:443> ServerName uat.example.com DocumentRoot "/var/www/vhosts/staging" SSLEngine on SSLCertificateFile /etc/letsencrypt/live/uat.example.com/cert.pem SSLCertificateKeyFile /etc/letsencrypt/live/uat.example.com/privkey.pem SSLCertificateChainFile /etc/letsencrypt/live/uat.example.com/chain.pem SSLProtocol All -SSLv2 -SSLv3 SSLHonorCipherOrder on SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS" <Directory /var/www/vhosts/staging> AllowOverride All </Directory> </VirtualHost> <VirtualHost *:443> ServerName example.com DocumentRoot "/var/www/html" SSLEngine on SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem SSLProtocol All -SSLv2 -SSLv3 SSLHonorCipherOrder on SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS" <Directory /var/www/html> AllowOverride All </Directory> </VirtualHost> # www alternative on live <VirtualHost *:443> ServerName www.example.com DocumentRoot "/var/www/html" SSLEngine on SSLCertificateFile /etc/letsencrypt/live/www.example.com/cert.pem SSLCertificateKeyFile /etc/letsencrypt/live/www.example.com/privkey.pem SSLCertificateChainFile /etc/letsencrypt/live/www.example.com/chain.pem SSLProtocol All -SSLv2 -SSLv3 SSLHonorCipherOrder on SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS" <Directory /var/www/html> AllowOverride All </Directory> </VirtualHost> #restart apache and your site should be running over HTTPS. However, if your vhosts file throws up and error related to un-named VirtualHosts, go to your httpd.conf file and un-comment out: NameVirtualHost *:80 and NameVirtualHost *:443 (you may have to add 443). To find the line in nano use CTL + w # To renew, run this command $ /opt/letsencrypt/letsencrypt-auto renew --config /etc/letsencrypt/config.ini --agree-tos # if you are using an instance from an AMI you may have previous certs on this instance which need to be removed for renewals rm -rf /etc/letsencrypt/live/remove.com rm /etc/letsencrypt/renewal/remove.com.conf
Crontab settings for backups (pushed to S3)
$ crontab -e (not sudo) # press i to INSERT onto crontab 30 4 * * * aws s3 sync /var/www/backup/ s3://example-backup/ 0 2 * * * mysqldump -hyour-host-name --port=3306 -uroot -pyour-password --databases example > /var/www/backup/daily/daily_`date +\%Y-\%m-\%d_\%H-\%M-\%S`.sql 45 1 * * * zip -r /var/www/backup/daily/html.zip /var/www/html 10 3 * * * find /var/www/backup/daily* -mtime +2 -exec rm {} \; 1 2 * * 0 mysqldump -hyour-host-name --port=3306 -uroot -pyour-password --databases example > /var/www/backup/weekly/weekly_`date +\%Y-\%m-\%d_\%H-\%M-\%S`.sql 1 2 * * 0 zip -r /var/www/backup/weekly/html.zip /var/www/html 1 3 * * 0 find /var/www/backup/weekly* -mtime +8 -exec rm {} \; 0 1 1 * * mysqldump -hyour-host-name --port=3306 -uroot -pyour-password --databases example> /var/www/backup/monthly/monthly_`date +\%Y-\%m-\%d_\%H-\%M-\%S`.sql 0 1 1 * * zip -r /var/www/backup/monthly/html.zip /var/www/html 0 2 1 * * find /var/www/backup/montly* -mtime +32 -exec rm {} \; 30 5 1 * * /opt/letsencrypt/letsencrypt-auto renew --config /etc/letsencrypt/config.ini --agree-tos #run a few days before your SSL registration date each month to be sure its covered ESC - :wq to save
Steps in launching EC2 from AMI
- Select AMI and Launch - Select IAM Role (or create new) #!/bin/bash service httpd start chkconfig httpd on - Select Security Group (or create new) - Select Key Pair (or create new) # If new Convert key to PPK - Connect to Workbench # distinct user for each DB - never use root - 3rd party App to schedule backups
Upgrade to PHP7 - EC2 instance (from 5.6, 7.0, 7.1)
# Remove current php & apache - removed guide $ so you can copy and paste commands directly into CLI sudo service httpd stop sudo yum remove httpd* php* # Remove any third party repos that aren't relevant sudo yum repolist sudo yum remove remi-safe # Install Apache24 for Amazon AMI sudo yum install httpd24 # Download webtatic mkdir -p /tmp/php7 cd /tmp/php7 wget https://mirror.webtatic.com/yum/el6/latest.rpm # Install webtatic repo sudo yum install latest.rpm sudo vi /etc/yum.repos.d/webtatic.repo - set repo to enabled sudo yum clean all # Install php7 sudo yum install --enablerepo=webtatic php72 php -v # outputs build data sudo yum install php72-opcache php72-xml php72-pdo php72-mysqlnd php72-gd php72-pecl-apcu php72-mbstring php72-imap php72-mcrypt sudo php72-intl sudo yum install mod24_ssl # these edits are needed on your php.ini file, values are for demo purposes post_max_size 25M upload_max_filesize 10M max_execution_time 180 max_input_time 180 memory_limit = 25M short_open_tag = On
Connect to an RDS in one AWS account from an EC2 in another: The following example allows an EC2 instance in your 'Source' account to connect to an RDS instance in your 'Target' account (destination). This can be done via CLI but I find it easier through the console.
1. Create VPC peering: [1]
You will need to create a new VPC to avoid a conflict between the default CIDR block IP's. This will require the creation of new subnets for the new VPC (created automatically with default VPC)
2. Route Tables [2] [3]
a. Source: Update the route table(to which the subnet is associated with your EC2 instance) with the Destination IP address of target and Target as VPC Peering connection(target).
b. Target: Update the route table(to which the subnet is associated with your RDS instance) with the Destination IP address of source and Target as VPC Peering connection(source).
3. Security Groups [4]
a. Source instance's security group outbound rule is allowing ALL to Anywhere.
b. Destination RDS security group's inbound rule must allow Port (1024) from source IP. Add a security rule in the security groups in master account.
CodeDeploy: How to install the codedeploy agent on an EC2 instance
A. On an existing instance:
$ sudo yum install ruby
$ sudo yum install wget
$ cd /home/ec2-user
$ wget https://aws-codedeploy-eu-west-1.s3.amazonaws.com/latest/install
$ chmod +x ./install
$ sudo ./install auto
B. booting new instance (bootstrap user data):
#!/bin/bash
yum -y update
yum install -y ruby
yum install -y aws-cli
cd /home/ec2-user
aws s3 cp s3://aws-codedeploy-us-east-2/latest/install . --region us-east-2
chmod +x ./install
./install auto
Check Status:
$ sudo service codedeploy-agent status
Logs:
/var/log/aws/codedeploy-agent/codedeploy-agent.log
/opt/codedeploy-agent/deployment-root
WordPress HTTPS Redirect:
After adding an SSL (above), add the following to your WP .htaccess to redirect all to HTTPS
RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301,NE]
Here is a complete example including the default WordPress code:
RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301,NE]
# BEGIN WordPress
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
# END WordPress
Enabling ENA support on your AWS EC2
Enhanced networking with the Elastic Network Adapter (ENA) is required for the 't3.****' instance type. Ensure that your instance 'i-XXXXXXXXXXXXXXXXX' is enabled for ENA - You may experience this issue if trying to switch to the t3 range of EC2 instances. To remedy:
$ aws ec2 stop-instances --instance-ids i-XXXXXXXXXXXXXXXXX --region eu-west-1
$ aws ec2 modify-instance-attribute --instance-id i-XXXXXXXXXXXXXXXXX --region eu-west-1 --ena-support
$ aws ec2 modify-instance-attribute --instance-id i-XXXXXXXXXXXXXXXXX --instance-type "{\"Value\": \"t3.*****\"}"
$ aws ec2 start-instances --instance-ids i-XXXXXXXXXXXXXXXXX --region eu-west-1
Cross-site cookie error messages when running WordPress on EC2
How to add cookie HTTP header flag with HTTPOnly, Secure & SameSite to remove the browser warnings in WordPress and protect your instance from XSS attacks
# using your preferred CLI editor
$ sudo nano /etc/httpd/conf/httpd.conf
# Add this to the top of the conf file
$ Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure;SameSite=strict
Issues with email throttling when using EC2
SMTP Email failing intermittantly and nothing showing on your application/WordPress logs. Here is how to easily troubleshoot the problem:
# install telnet, ncat and openssl (if using SMTP over starttls)
$ sudo yum install telnet
$ sudo yum install nmap-ncat -y
$ sudo yum install openssl
# checking SMTP connection using telnet (this example uses AWS hostnames but use whatever is applicable in your case)
$ telnet email-smtp.us-east-1.amazonaws.com 587
$ telnet email-smtp.us-east-1.amazonaws.com 25
$ telnet email-smtp.us-east-1.amazonaws.com 465
# checking SMTP connection using openssl
$ openssl s_client -connect email-smtp.us-east-1.amazonaws.com:587 -starttls smtp