Tag Archives: backup

AWS, Cloud

SIMPLE BACKUP SOLUTION WITH AWS S3

1 Comment

Data availability is one of the biggest concern in IT industry. After moving most of my services to the AWS cloud I was thinking how I can ensure data availability and accuracy in case of AWS data center failure or what if my EC2 EBS volume gets corrupted.

A case study

I have a Oracle RDS running on EC2 instance.

  • I need to ensure I can restore data from backup in case of user demand, in case of data center failure or in case of instance failure
  • On the other hand I need to ensure it will not increase my AWS monthly charges unexpectedly
  • I will only run that service during the business hours

Solution could be

  • Use AWS Oracle RDS. The service will take care of everything including backup and patch update. This is really a very reliable service AWS is providing. But to fulfil my last requirement it will be a lot of work for me, since RDS can’t be stopped, you can only terminate RDS (yes, you can take snapshot before terminating)
  • Use EC2 instance and take snapshot backup of your EC2 EBS volume. But my EBS volume is 120 GB, much bigger than the original SQL DB backup. Which means it will cost me more to store multiple snapshots in S3 (120 GB x 7days).

The solution I am using

  • Created a maintenance plan in SQL Server to take daily db backup
  • Created an AWS CLI script to sync data from SQL server backup location to a S3 bucket
  • aws s3 sync \\SERVER_NAME\backup$ s3://BUCKETNAME –exclude * –include *.bak
  • Created a batch job to move local SQL server backup data to another folder for old data clean-up
  • move \\SERVER_NAME\backup$\*.* \\SERVER_NAME\backup$\movedS3
  • Create a maintenance plan in SQL Server to delete older files from movedS3 folder. It will help me to control unwanted data growth
  • Created a lifecycle policy to delete older files from my S3 bucketS3-Lifecycle

 

What this solution will ensure

  • First of all I can sleep tight during night. I don’t need to worry about my backup data.😉
  • S3 provides me 99.999999999% data durability. It means I will be able to access my S3 data in case of AWS availability zone failure also. Because S3 data synchronizes between multiple availability zone.
  • S3 is the cheapest cloud data storage solution. That’s why drop box dare to give you such storage space as free😉
Aix, HP-UX, Linux, Solaris

Unix Backup Key Files !!

Leave a comment
  1. Kernel This should be the first file backed up. If it gets deleted you can restore it back. This file is often named /kernel, /unix, or /vmunix.

  2. Password File Having a backup of the password file is a good idea, in case the system password file gets compromised. This is the file referred to as /etc/passwd.
  3. Group File Back this up for similar reasons as you do a password file. This file is /etc/group.
  4. Host Table Make an extra copy of the host table, in case the file is corrupted or entries are removed and then needed in the future. This file is named /etc/hosts.
  5. Filesystem Table This is for recovery of the filesystem configurations. This file is often called /etc/fstab or /etc/vfstab.
  6. Sendmail Config Files If anyone works with sendmail, back these up for sure. If mail breaks, the quickest recovery is to restore from the backed up config files. These would be /usr/lib/sendmail.cf, /usr/lib/sendmail.fc, and /usr/lib/sendmail.mc.
  7. Inetd Configs This file has been known to get misconfigured and is a target for hackers. This is the file /etc/inetd.conf.
  8. TTY settings If you have any specialized devices attached, these should be backed up. These settings are typically found in /etc/inittab, /etc/ttytab, and /etc/ttys.
  9. Start Up Scripts Any special scripts that get started at boot time should be kept in another area. These files would usually reside in the /etc/init.d or /etc/rc#.d directories.