AWS Instance Automated AMI Creation & Deletion System

UPDATE: 14 JUN 2017 – In step 5 below, please do select 5 minutes timeout for the script. We did have a few issues for users who had a lot of instances taking over the standard 2 minutes for the script to run.

Just a quick blog post to link our new Github script on automated AWS AMI management system.

Motivation

Creating an AMI automatically snapshots all the associated EBS volumes for that instance. This makes instance recovery much more reliable and faster. A daily or weekly backup schedule is recommended for instances and to make sure you have a backup if ever needed. The system also removes the automatically created AMIs and any associated snapshots as per the settings.

Setup / Installation of the Lambda script (one time setup)

1) Setting up the Lambda script by opening the Lambda console

2) Select “CloudWatch Events – Schedule” and click Next
3) Select a schedule you like to setup. Ideally the script has to run every day4) Paste the script for createAMI.js (See github link above)

5) Create a new role and paste the roles/roles.json (See github link above) also increase timeout.

6) Save this and follow step above again now using deleteAMI.js (See github link above).

Setting the tags for EC2 instances (repeat for each instance)

1) Login to your AWS console and open the EC2 section. Then select the instance

2) Select the add or edit tags as per settings

3) Set the three tags with the following Keys & Value a) Key: AutoDigiBackup with Value: yes. This marks the instances that need to be backed up. b) Key: AutoDigiBackupRetentionDays with Value: from 1 to as many days as you want the backup to be stored. c) Key: AutoDigiBackupSchedule with Value: * (for every day) or a mix of number from 0 (Saturday), 1 (Monday) and all the way to 6 (Sunday). You can set value to say 012 which means run on Saturday, Monday and Tuesday. You can set just 6 to run only on Sunday.

Example below is set to create the AMI every day

Save the settings that are approriate for this instance and then save the tags. Please repeat steps 1-3 for all instances that you like to be included.

Also read...

Comments

  2. Jim Boissonnault said on :

    Does the script include a stop-start feature?

    • php-manual said on :

      yes. the instances are automatically stopped and started after Ami is created. You can also easily change to avoid rebooting server if you want!

  3. Seth Fuller said on :

    Thanks very much for sharing this. Worked great!

  4. anto said on :

    didnot worked for me 🙁

    Test run of L-function completed successfully, but it didnt created any AMIs.. I’ve created all tags as said here.
    ===
    The area below shows the result returned by your function execution. Learn more about returning results from your function.

    null

    ===

    Also will it create “Name” Tag in AMI’s that will match up with instance name ?
    or do we need to identify the AMI based on instance id ?

    thanks
    -Anto

    • php-manual said on :

      Do you have the lambda in same region? did you add the role as mentioned for lambda to access?

      if both above were set then please open a github issue on the repo for this project link in the post.

  5. Jim Boissonnault said on :

    I am manually testing/running the scripts.

    AWS Lambda complains about putting a function within a loop:

    Create script line 78: ec2.createImage(imageparams, function(err, data)
    Delete script line 62: ec2.deregisterImage(derigisterparams,function(err, data01)

    I put /* jshint loopfunc:true */ just before the function to suppress the alert.

    Question 1: Is there any problem with the function where it is or should the scripts be updated?

    I ran the scripts successfully. I created two AMIs for the same EC2.
    When I run the delete script, I see 4 log messages “Not yet time to delete…” (2 per AMI); See below:

    START RequestId: 398ee4c9-19f4-11e7-8333-1fdbf4bd45b7 Version: $LATEST
    2017-04-05T11:37:20.415Z 398ee4c9-19f4-11e7-8333-1fdbf4bd45b7 Not yet time to delete Recovery Linux Box_DATETODEL-1492547526340
    2017-04-05T11:37:20.416Z 398ee4c9-19f4-11e7-8333-1fdbf4bd45b7 Not yet time to delete Recovery Linux Box_DATETODEL-1492549876699
    2017-04-05T11:37:30.421Z 398ee4c9-19f4-11e7-8333-1fdbf4bd45b7 Not yet time to delete Recovery Linux Box_DATETODEL-1492547526340
    2017-04-05T11:37:30.421Z 398ee4c9-19f4-11e7-8333-1fdbf4bd45b7 Not yet time to delete Recovery Linux Box_DATETODEL-1492549876699
    END RequestId: 398ee4c9-19f4-11e7-8333-1fdbf4bd45b7
    REPORT RequestId: 398ee4c9-19f4-11e7-8333-1fdbf4bd45b7 Duration: 11212.09 ms Billed Duration: 11300 ms Memory Size: 128 MB Max Memory Used: 42 MB

    Question 2: Is it normal to have 2 per AMI or is this because of the function nesting within the loop?

    • php-manual said on :

      Answer 1) What is the value of AutoDigiBackupRetentionDays set? The deletion happens based on the number of days set.

      Answer 2) You should only have one AMI per run. Is it possible that it was triggered twice?

      • Jim Boissonnault said on :

        1) Originally set to 14. Can this value be changed after creating an AMI? So if you set the tag to 14 and later to 1, will the script delete all AMIs older than 1 day?

        2) I don’t think it’s running twice. I’m selecting the Lambda Test button.

        Thanks for the quick response!

        • php-manual said on :

          1) Yes the value can be changed but it will only affect newer AMIs created.

          2) Please do let me know if this happens again as all our tests only creates this once.

          • Jim Boissonnault said on :

            To be clear, the script isn’t creating two AMIs per EC2 instance. I stated I created two on purpose.

            I’ve just created a new AMI with 1 day delete set and will try the delete script again after 1 day. If I wait, say, 3 days and run the delete script, will it delete the new AMI that was created with the 1 day deletion parameter?

            If I should email directly and/or not put all this on this page, let me know.

          • php-manual said on :

            Ok, thanks for that. It clears up that the AMI creation works as expected.

            Regarding deletion, the number of days set for deletion is used at the time of creation of the AMI. So if you have set a tag to store the AMI for 3 days then that is used when the AMI is created. So if you change it back to 1 day it will only affect AMIs that you create in the future.

            It is best to please leave a comment on the github page for any issues you find.

Comments are closed.