AWS S3 Cross Region Replication

What is AWS S3 Cross-Region Replication (CRR)?

Amazon S3 allows cross region replication of the newly uploaded objects to S3 destination buckets asynchronously and automatically across different AWS Regions.

Users can configure the replication rule so the rule can identify the objects to replicate using prefix, tag, or bucket through AWS CLI, Management Console, and AWS SDK.

When to use S3 Cross-Region Replication (SRR)

• When a user wants to store a replica of S3 objects in the different regions for data compliance requirements
• To minimize the latency in reading the objects from other AWS regions if the user's customer is present in the different geographical regions.
• If the user has clusters in two different AWS Regions that analyze the same objects, the user might choose to maintain object copies in those Regions.

What are the prerequisites for AWS to replicate your S3 objects across cross region?

• Amazon S3 source and destination buckets must have versioning enabled.
• AmazonS3 must have permission to replicate the objects on the source bucket's behalf to the destination bucket
• If the source bucket has object lock enabled, the destination bucket also must have enabled the same

Steps to create and configure the AWS S3 Cross Region Replication

a. Create a S3 Source and Destination Buckets

• Create a S3 source and destination bucket in your AWS Management Console in different regions.
• Make sure your S3 bucket's name is unique and DNS compatible; you must enable bucket versioning while creating buckets.

As you can see in the above screenshot, I have created my source S3 bucket in Mumbai Region and destination S3 bucket in Tokyo Region.

b. Set up AWS Identity and Access Management (IAM) permissions

• Go to IAM Console and select Roles from the left navigation under Access Management.
• Choose to create a role from Role Dashboard, select AWS Service as a trusted Entity, and choose S3 from the bottom dropdown for Use Case and press Next.

• Choose Create Policy, select the JSON tab, and copy-paste the below policy code.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetReplicationConfiguration",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::aws-crr-source-bucket"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObjectVersionForReplication",
                "s3:GetObjectVersionAcl",
                "s3:GetObjectVersionTagging"
            ],
            "Resource": [
                "arn:aws:s3:::aws-crr-source-bucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ReplicateObject",
                "s3:ReplicateDelete",
                "s3:ReplicateTags"
            ],
            "Resource": "arn:aws:s3:::aws-crr-destniation-bucket/*"
        }
    ]
}

• Click on Tags, add tags if you want to, and Press Next.
• Name the policy and click on Create policy.

• On Add Permission page, refresh at Permission policies. You can see your new policy created listed. Select your policy and press Next.
• Name your role and description, and press Create role.

c. Create Replication Configuration

• Open the S3 console, https://console.aws.amazon.com/s3/
• Select your source bucket, choose the Management tab, and scroll down to Replication Rules. Click on Create Replication Rule.
• Provide Replication Rule Name

• Under Source Bucket
  • You can provide the prefix of objects you want to replicate.
  • We will select "Apply to all objects in the bucket for this tutorial.

• Under Destination Bucket
  • Select "Choose a bucket in this account" and browse S3 to get the bucket list.
  • Select your destination bucket created in Tokyo Region and click on Choose Path.

• Under IAM Role, Select "Choose from existing IAM roles" and select role created earlier.
• Under Destination Storage Class, you have an option to change the storage class of replicating objects. For this tutorial, let's keep the storage class the same as the source bucket.
• Additional replication options
  • Check - Replication Time Control (RTC) - S3 Replication Time Control (S3 RTC) helps you meet compliance or business requirements for data replication and provides visibility into Amazon S3 replication times. S3 RTC replicates most objects you upload to Amazon S3 in seconds and 99.99 percent of those objects within 15 minutes.
  • Check - Delete marker replication - if you want to replicate the delete marker to your destination buckets. Amazon S3 behaves as if the object was deleted in both source and destination S3 buckets.
  • Check - Replica modification sync - Amazon S3 replica modification sync can help you keep object metadata such as tags, ACLs, and Object Lock settings replicated between replicas and source objects. When replica modification sync is enabled, Amazon S3 replicates metadata changes made to the replica copies back to the source object, making the replication bidirectional.
  
• Press Save to create your replication rule. AWS prompts if you want to replicate the existing objects in the bucket. For now, please select No and press to submit.
• Once you create the replication rule, your configured rule will be visible under the Replication Rule with the status Enabled.

d. Test the S3 bucket Cross-Region Replication

• Now, it's time to test the replication. Move to your source bucket and upload any file using the upload option.
• Wait for a couple of minutes and check your destination bucket in the Tokyo AWS region. The object from the S3 source bucket in the Mumbai Region is replicated and present in the destination bucket.
• You can also check the status of Object replication at the console.
  • Go S3 source bucket and select the object you uploaded.
  • Under the Object management overview, you can see the status of Replication as COMPLETED.
  

Amazon S3 Cross Region Pricing

• The S3 Storage charge at S3 Source and Destination bucket
• The PUT Request charges on the S3 destination bucket
• The inter-region Data Transfer OUT from S3 to each destination region
• If the user is using S3 Replication Time Control, the user also needs to pay a Replication Time Control - Data Transfer charge and S3 Replication Metrics charges

Conclusion

This article teaches you how to set up AWS S3 Cross-Region Replication easily and answers all your queries regarding it. Furthermore, it provides a brief introduction of various concepts related to it & helps the users understand them better and use them to perform data replication. Feel free to provide your feedback and subscribe to our newsletter for upcoming posts.

Don't forget to hit like on the post.. :)

The related documents and files are present on Github URL

To learn about AWS S3 Same Region Replication

Resources Used: AWS S3 Replication Documents AWS S3 Pricing