Mastering Data Redundancy: Amazon S3 Replication Demystified.
In today’s data-driven world, safeguarding your digital assets is paramount. As the preferred choice for scalable and secure data storage, Amazon S3 has revolutionized the way we manage and protect our valuable information. However, the true power of data resilience lies in Amazon S3 Replication, an often-underutilized feature that holds the key to an unshakeable data strategy.
INTRODUCTION
Replication is a process of automatically copying objects between buckets in the same or different AWS Regions.
This copying happens when you create new objects or update existing ones. Amazon Simple Storage Service (S3) replication allows you to have an exact copy of your objects stored in other buckets.
PURPOSE OF S3 REPLICATION
Reduce Latency.
Enhance availability.
Disaster Recovery.
Copy Objects to cost-effective storage class.
Data redundancy.
Meet compliance requirements.
REPLICATION OPTIONS:
AMAZON S3 SAME REGION REPLICATION
S3 Same Region Replication (SRR) will automatically replicate objects from a source bucket to a destination bucket within the same AZ or different AZ in the same region. It uses asynchronous replication, which means objects are not copied to the destination bucket as soon as it is created or modified.
AMAZON S3 CROSS-REGION REPLICATION
S3 Cross Region Replication (CRR) will automatically replicate objects or data from the source bucket to the destination bucket in different regions. It minimizes latency for data access in different geographic regions.
S3 BATCH REPLICATION:
With CRR and SRR, Amazon S3 protects your data by automatically replicating new objects that you upload. S3 Batch Replication, on the other hand, lets you replicate existing objects using S3 Batch Operations, which are managed jobs.
Let’s get Practical
We will show steps in setting up s3 CRR
Conditions for enabling Cross-region replication
- 1. Versioning must be enabled on both the source and destination buckets.
- 2. Regions must be unique.
- 3. Amazon S3 must have permission to replicate objects from the source bucket to the destination bucket on your behalf.
Setting up CRR:
Log in to the AWS s3 console then in the search box type s3, then select s3 under services.
In the s3 console, select buckets then click Create Bucket.
In the create bucket console we will name our bucket as sourcedemobucket11, for region, we will keep it in the Asia Pacific (Mumbai) ap-south 1 region. Also, note that the S3 bucket name needs to be globally unique. scroll down.
Under bucket versioning, it’s disabled by default. Enable it by checking the radio button. Then leave everything as default, scroll down, and click Create bucket.
Now following the same steps create a destination bucket: destinationdemobucket12 with versioning enabled, but this time let’s put it in US-east-1
After versioning is enabled leave everything as default and click Create bucket.
Next, click on your source bucket head over to the management tab then scroll down to the replication rule.
Now, click on “Create a replication rule”
Give your replication rule a name as “replicatedemo11’’
Choose the destination bucket as “destinationdemobucket12”.
Notice that you have an option to choose a destination bucket in another account.
In order to replicate objects from the source bucket to the destination bucket, you need to create an IAM role. So, select the drop-down button under the I AM role and click Create New Role.
If you want your S3 objects to be replicated within 15 minutes you need to check the “Replication Time Control (RTC) box. But you will be charged for this. So, we will move forward without enabling that for now and click on save.
As soon as you click on save, a screen will pop up asking if you want to replicate existing objects in the S3 bucket. But that will incur charges so we will proceed without replicating existing objects and click on submit.
After completing this setup, you can see a screen saying “Replication configuration successfully updated”.
head over to our destination bucket: destinationdemobucket12 to check if the uploaded file is replicated to our destination bucket. You can see that our uploaded file is successfully copied to the destination bucket. if you are not seeing it just click the refresh button.
Note: pull everything down
Facts about CRR
- 1. Files in an existing bucket are not replicated automatically but all the subsequent updated files will be automatically replicated.
- 2. You cannot replicate to multiple buckets or use daisy chaining (from bucket A to bucket B to bucket C).
- 3. Delete markers are replicated.
- 4. Deleting individual versions or delete markers will not be replicated.
Thanks for your attention and stay tuned for more.
If you have any questions concerning this article or have an AWS project that require our assistance, please reach out to us by leaving a comment below or email us at
[email protected].
Thank you!