We’re evaluating using Amazon’s EC2 (Elastic Computing Cloud) to fire off full instances of our system for both dev and QA. That involves multiple servers and databases, it’s a non trivial task. I think RightScale is a great service which we’ll probably use if we move forward but at this point during testing and discovery I decided to muck around at a lower level. RightScale provides a very nice Ruby library to access AWS services.
So I wrote a simple ruby script that tracks changes in aws assets. Stuck it in a cron job and it notifies me by e-mail whenever there’s a change on our account.
To get started grab the ruby gem
gem install right_aws
I decided to use SimpleDB to store state and history. Turned out to work really well. I’m digging the schemaless DB approach for this type of thing. It’s something I’m going to explore for other uses.
The script is simple
- Iterate over the assets tracked
- Load the info from EC2
- Load the stored info in SDB and compare
- Report any changes
- Update SDB
- Rinse, repeat
Lots more to build but this was a nice start and showed the right_aws gem is a capable library for building a complete set of tools to manage your Amazon Web Services account. The best bet is to go with RightScale and then just add functionality you need that RightScale doesn’t provide (shouldn’t be much).
require 'rubygems' require 'right_aws' LOG_FILE = '/tmp/ec2.log' class AWS def initialize (argv, id, key) if argv.size > 1 print_help exit 1 end # List of resources to track # 0 - resource name # 1 - resources AWS id # 2 - [optional] name of the describe function in right_aws describ_[name] # Only needed if it's not describe_[resource name] # 3 - [optional] name of parameter to the describe function @items = [ ['images', 'aws_id', "images_by_owner", "self"], ['instances', 'aws_instance_id'], ['volumes', 'aws_id'], ['security_groups', 'aws_group_name'], ['snapshots', 'aws_id'], ['key_pairs', 'aws_key_name'] ] if argv.size == 1 if argv[0] == '-h' print_help exit 0 elsif argv[0] == '-d' create_sdb_domains id, key exit 0 else print "Unknown option: ", argv[0], "\n" print_help exit 0 end end @sdb = RightAws::SdbInterface.new(id, key, {:multi_thread => true, :logger => Logger.new(LOG_FILE)}) @ec2 = RightAws::Ec2.new id, key, {:logger => Logger.new(LOG_FILE)} check_for_updates end def print_help print "usage: #{$0} [options] \n" print "\n" print "\t -h this message\n" print "\t -d create sdb domains\n\n" end def check_for_updates @ec2_data = {} @sdb_data = {} @items.each do |i| key = i[0] meth_name = 'describe_' + i[2]||=key ec2m = @ec2.method(meth_name) @ec2_data[key] = convert_to_hash(ec2m.call(i[3]), i[1]) @sdb_data[key] = sdb_get(key.to_s.upcase) compare(@ec2_data[key], @sdb_data[key], key.to_s.upcase) end end def create_sdb_domains(id, key) @sdb = RightAws::SdbInterface.new(id, key, {:multi_thread => true, :logger => Logger.new(LOG_FILE)}) @items.each do |i| domain = i[0].to_s.upcase print 'Creating domain: ', domain, "\n" @sdb.create_domain domain print 'Creating domain: ', domain, "_ARCHIVE\n" @sdb.create_domain domain + '_ARCHIVE' end print 'Domains created.\n' end #--------------- SDB Routines ------------------# def sdb_save(domain, id, attributes) @sdb.put_attributes domain, id, attributes end def sdb_delete(domain, id, v) # Archive it first sdb_save(domain+'_ARCHIVE', id, v) @sdb.delete_attributes(domain, id) end def sdb_get(domain) v = {} results = @sdb.query(domain)[:items] results.each do |id| v[id] = @sdb.get_attributes(domain, id)[:attributes] end v end #--------------- EC2 Routines ------------------# # ec2 describe calls just returns an array of attributes for each entry # pull out the key and use that in a hash key, attributes def convert_to_hash(list, id_var) h = {} list.each do |v| if v[:aws_created_at] != nil v[:aws_created_at] = v[:aws_created_at].to_s end h[v[id_var.intern]] = v end h end def report_change (id, name, action, note) print "Asset #{id} : #{name} : #{action} : #{note}\n" end def compare(ec2_list, sdb_list, domain) # see if everything ec2 reports is in our db ec2_list.each do |k, v| if sdb_list[k] == nil v['added_date'] = Time.new.gmtime.to_s sdb_list[k] = v sdb_save(domain, k, sdb_list[k]) report_change domain.capitalize.chop, k, "added", v['added_date'] end end # see if everything the db reports is in ec2 sdb_list.each do |k, v| if ec2_list[k] == nil v['removed_date'] = Time.new.gmtime.to_s report_change domain.capitalize.chop, k, "removed", "Created: #{v['added_date']} Removed: #{v['removed_date']}" sdb_delete(domain, k, v) end end end end aws = AWS.new ARGV, 'your access key', 'your secret key' |



{ 1 comment… read it below or add one }
I will suggest using http://www.monitis.com for monitoring your EC2 instances both externally and internally.