Earlier this week I looked at my web hosting’s usage stats and decided that I needed to move a bunch of static assets (mostly PDFs) somewhere else because they were eating my available bandwidth. A bunch of hot links around the web caused some usage spikes.
I’m a firm believer in learning by doing, so I decided to use this opportunity to learn a bit about Amazon Web Services and set up an S3 bucket from which to serve my static assets. To my surprise it took less than an hour to get everything up and running.
This guide is for people who want to use S3 to host some static files to lower their hosting costs. If you want to host your whole static site (Jekyll, Hugo, etc) on S3, follow this guide. If you are using WordPress and want to host your images on S3, follow this guide instead.
Setting up S3
Set up an account with the Free Tier. If you are getting a decent amount of traffic, you’ll quickly run over the free limit of 5 GB of Standard Storage,20000 Get Requests, and 2000 Put Requests. It took me about a week to pass it up. Thankfully, S3 is much cheaper than paying for additional bandwidth on A Small Orange, so I’m still ahead.
One note that I’ll expand upon later: By checking the logs, I discovered that the majority of the hits are coming from 5 IP addresses that all resolve to Cloudflare. I have Cloudflare running on my site for caching and SSL, which means they regularly crawl my site and cache the assets. This ate through my Free Tier limits in just one week. If you use Cloudflare for caching, I suggest turning it off unless you know you need it.
Create a new bucket. The region mostly doesn’t matter unless you know that the bulk of your traffic comes from one place. If it does, pick the closest region. Make sure you enable logs when it prompts you. This will be useful for tracking & analytics later.
Uploading Assets
Now that you have your first bucket, it is time to put stuff in there. If you
have a local backup of your website (and you should), identify the directories
where you store static content like images, PDFs, HTML, CSS, Javascript, etc. No
files ending in .php or .rb - those are server side files that can’t be
executed on S3.
You can upload straight from the browser or you can set up access keys to connect via a file transfer client. I use Transmit for my file transfer needs, but CyberDuck is a good free alternative. Both have options for connecting to S3 buckets.
When you upload files, try your best to keep the same folder structure from your website root. That will make writing redirects to S3 easier.
Once you’ve uploaded some files, make sure you can access them through the
browser by setting the permissions to Public. You can do this on a
file-by-file basis
or by selecting all of your files, clicking the “More ↓” dropdown, and
clicking “Make Public”. Now verify that you can reach your asset by going to
http://s3.amazonaws.com/[YourBucketName]/[filepath]. For example, my website’s
avatar is available at http://s3.amazonaws.com/cagrimmett/img/avatar.png
If that worked, your assets are now available on S3. Now it is time to redirect traffic to them.
Redirecting Requests
I’m assuming here that your server is using Apache and you have access to your
.htaccess file. If you are using IIS instead,
refer to this.
Remember, always make website backups before doing anything that some guy on the internet tells you to do.
Here are the rewrite settings I’m using in my main .htaccess file:
RewriteEngine On
RewriteRule ^justanswer/(.*) http://s3.amazonaws.com/cagrimmett/justanswer/$1 [QSA,NC,NE,R,L]
RewriteRule ^img/(.*) http://s3.amazonaws.com/cagrimmett/img/$1 [QSA,NC,NE,R,L]
RewriteRule ^static/(.*) http://s3.amazonaws.com/cagrimmett/static/$1 [QSA,NC,NE,R,L]
Here is what each line does:
RewriteEngine On- Turns on the ReWriteEngine so we can use it.RewriteRule ^justanswer/(.*) http://s3.amazonaws.com/cagrimmett/justanswer/$1 [QSA,NC,NE,R,L]- Takes any request to any file inhttp://cagrimmett.com/justanswer/and forwards that request tohttp://s3.amazonaws.com/cagrimmett/justanswer/. Since I kept my folder structure, this works perfectly. The other two entries are similar. - QSA = Query String Append - NC = No Case - NE = No Escape - R = Redirect - L = Last - More info here: https://wiki.apache.org/httpd/RewriteFlags
After I turned it on, I gave it a few minutes to update and then checked my site. When I right clicked to open images in a new tab or tried to download an existing PDF, I got forwarded to AWS! Success!
Tracking & Analytics
If you enabled logging back in the setup step, you can plug in some third party tools to parse the logs into useful analytics. I’m currently trying out two tools: S3Stat and Qloudstat.
Pros/cons:
- S3Stat is designed better and easier to use than Qloudstat.
- Qloudstat is cheaper than S3Stat for small-time use.
- I prefer using S3Stat, but right now I can’t justify spending $10/mo on it when there are other options.
- I might roll my own with one of the many log parsing scripts on Github and my knowledge of D3.js. We’ll see.
Questions? Issues?
First, check out the docs.
If you still have questions or run into issues, drop a comment below and I’ll get back within a few days to help.
