S3-Backed Images on Amazon Web Services
S3 (Simple Storage Service) images are stored on Amazon Web Service's Elastic Compute Cloud (EC2) as multiple 10MB files, along with a special XML file called a manifest. The manifest file is similar to the assembly instructions in an Ikea flat-pack piece of furniture — it gives AWS the information it needs to construct a running instance from the collection of 10MB S3 objects.
When you give AWS the command to launch an S3-backed image, the system reads the manifest file and uses it to construct and launch the instance by downloading it to the instance's local drive. It then becomes an operational instance and, depending on what was in the image when it was created, starts executing the software that was part of the image.
S3-backed images come with limitations, and you should fully understand what they are. The following list spells them out for you:
Root device limitations: It's crucial to understand precisely what's stored in the image proper. With S3-backed images, all you have is the root device — the part of the system containing system files (including the operating system) — plus, any other software that was installed when the image was created.
An S3-backed image is limited to 10GB in the root device. All other parts of the file system are constructed at the time of launch; for example, in a small instance with 170GB of disk space, 160GB of the instance storage is created at launch-time, and only 10GB is persistent. If you want to include a lot of software packages or data in the root device, you may exceed this 10GB limit.
Long launch times: Because the instance has to be created from the various 10MB files making up a collection, it takes a while to assemble them, which extends the launch times.
Removing an S3-backed instance from production requires terminating it — no ifs, ands, or buts: Doing so discards all data written to its file system since the launch.
The AWS management console doesn't support the creation of a new image from an S3-backed instance. If you want to create a new image from an S3-backed instance, first install AWS AMI tools on the instance, and then run scripts to create the image. Though this task is perfectly possible, it's not a trivial matter, so it's a definite limitation.
S3-backed images are widely used, but they carry operational implications that you should be aware of if you plan to use them.