Overview

"Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance". At a high level S3 has the following characteristics:

  • Object based storage for static files that do not change. EG: JPEG, MOV, etc..
  • Objects are stored in "buckets"
  • Files from 0 Bytes - 5TB in size
  • S3 has a universal namespace across all tenants and must be unique
  • S3 buckets and objects have a HTTPS endpoint associated to them
  • Successful uploads to an S3 buckets return a HTTP 200 Ok message

Note

S3 is not suitable for databases or operating systems.

Features

S3 has the following features.

  • Globally resiliant service
  • Tiered storage classes
  • Lifecycle management
  • Object versioning
  • Encryption
  • Security Policies

Objects

Objects are just files stored as key / value pairs

json
{
  "Key": "Name",
  "Value": "FileData",
  "VersionID": "VersionNumber",
  "Subresources": [
    "ACLs",
    "Torrent"
  ]
}

Data Consistency

There are two methods to maintain data consistency.

Read After Write Consistency

  • For PUTs of a new object
  • Can immediately read object after PUTing it to S3

Eventual Consistency

  • For overwrite PUTs and DELETEs
  • Can take some time to propagate changes
  • May be reading the old file on GETs or still see the old file on DELETEs

Storage Classes (Tiers)

There are 6 storage classes for S3 services. See link for further details on each tier.

S3 - Standard

  • The original S3 service
  • 99.99% uptime
  • 11 x 9s durability

S3 - IA (Infrequently Accessed)

  • Infrequent accessed data but requires rapid access when needed

S3 - One Zone - IA

  • Lower cost IA in a single availability zone

S3 - Intelligent Tiering

  • Uses AI to automatically move data to the most cost effective storage tier

S3 - Glacier

  • Low cost storage with configuratable retreival times from minutes to hours

S3 - Glacier Deep Archive

  • Lowest cost storage with a minimum of 12 hours retrieval time

Cost

Cost of the service is calculated on a number of factors.

  • Storage volume
  • Number of requests
  • Storage management
  • Data transfer

Transfer acceleration

  • Uses AWS cloudfront to upload files to an AWS pop close to the user
  • The file is then transferred to the S3 bucket across the AWS backbone

Cross region replication

  • Users upload a file to a "primary" bucket the file is then replicated to a "secondary" bucket in another AWS region

Transfer acceleration

  • Uses AWS cloudfront to upload files to an AWS pop close to the user
  • The file is then transferred to the S3 bucket across the AWS backbone

Cross region replication

  • Users upload a file to a "primary" bucket the file is then replicated to a "secondary" bucket in another AWS region

Security

S3 buckets can be secured in the following ways.

Bucket Policies

  • Applies to the whole bucket

Object Policies

  • Applies to individual files

IAM Policies for Users and Groups

  • Applies to users and groups

Tags