Playing with Ruby: Silly backup tool for Amazon S3


To celebrate the New Year, I played a little with Ruby. I made this small Ruby script to download all original image files of PicLyf stored in Amazon S3:

#!/usr/bin/env ruby

require 'rubygems'
require 'aws/s3'

bucket = 'bucket_name'

AWS::S3::Base.establish_connection!(
  :access_key_id     => 'our_access_key',
  :secret_access_key => 'our_secret_key'
)

last_key = nil
while true do
  puts "Getting objects " + (last_key == nil ? "" : "after #{last_key}")
  objects = AWS::S3::Bucket.objects bucket, :max_keys => 10, :marker => last_key

  objects.each do |object|
    # download file if it is the original and if it doesn't exist locally yet
    if !File.exist?(object.key) && object.key.match(/^\w+_o.\w+$/)
      puts "Downloading #{object.key} "
      open(object.key, 'w') do |file|
        AWS::S3::S3Object.stream(object.key, bucket) do |chunk|
          file.write chunk
        end
      end
    end
  end

  last_key = objects.last.key if !objects.empty?
end

The scripts loops through all objects from S3 and downloads only the original image files (ours have the “*_o.jpg” pattern). If it reaches the end of the loop, it will wait for new objects.

This is really just a simple silly script but it’s a good Ruby exercise :)