Contained in this wiki are instructions for backing your data on the Klauda Lab Backup Server. The name of this server is klauda-bkup1.umd.edu and can only be reached on computers on the UMD network. You'll need to logon to DT2 first if you need to access this server outside campus without VPN.

If you have simulation data, then it MUST be backed up in two places in the event of failure of hardware. Data is categorized as either being published or unpublished. As you start in the lab your work is in progress for publication but essential for a potential future publication. Published data is after you've had your work accepted by a scientific journal and no longer requires more simulation or analysis.

Unpublished Data

This will be the data you are currently running simulations and analysis on. This will be backed up on a weekly incremental basis and a full backup every month. If by chance you mistakenly delete data it may be recoverable from these sources but must be done before the full backup that occurs the beginning of the month. You will need to setup backup scripts (see below) on DT2, as appropriate for your simulations. If you run on MARCC or XSEDE resources, this data needs to be copied to DT2 for data analysis and backups.

Published Data

Once you have data that is published, you need to reduce the trajectory file sizes and make a single tarred and compressed file (.tar.gz). The name of this file should make it clear to me what publication this is from and be placed in your backup directory on the backup server. Then you will create another backup on an external hard drive (see Dr. Klauda) and remove data from DT2. So you still have two locations (external HD and backup server). The DCD/trajectory files should be reduced to save space. If you have 100-400ns of data, then reduce this to have only 10ps frames. If longer, consult with Dr. Klauda on the frame rate.

Download the following directory to reduce dcd files. It is adapted from recenter script from EDP analysis. Adjust crystal parameters in cryst.str and file range in recenter1.inp. Also change the “skip” option as needed.

truncate.tar.gz

The following instructions apply to DT2 (or any new resource) and you need to adjust files per each cluster.

1. Create your login and password on backup server

We will create a username as your UMD username and provide you with an initial password. Then you can ssh to klauda-bkup1.umd.edu with this password. You MUST change this to a stronger password once you login for the first time with

 passwd 

2. Setting up password-less login to backup server from DT2

This is key for doing automatic backups and allows you to login to the server without prompting the password.

• login to DT2
• From you home on DT open .ssh/known_hosts file and delete the line starting with klauda-bkup1.umd.edu, if it exists (likely not). Save and exit
• Type ssh-keygen
• Press y (when asked y/n?) and 'return' key otherwise
• ssh-copy-id -i ~/.ssh/id_rsa.pub username@klauda-bkup1.umd.edu (Your username on backup machine)
• Try ssh username@klauda-bkup1.umd.edu and you should not be asked for a password

3. Creating backup directories

• Once you are logged in on the backup server, go to /local0/backup and create a directory with your username
• Inside that create two directories named full and weekly
• Inside weekly create 12 directories corresponding to 12 months 01, 02, 03 … 12

4. Creating backup scripts on DT2

• Create a directory named backup in your home directory on DT2
• Copy backup-full.csh and backup-weekly.csh from /homes/jbklauda/backup to this backup directory
• There will be two paths:
  • Path of your directory on deepthought that will be backed up on the server. It should generally be “/lustre/username”
    
  • Path of directory on server where it will be backed up. This will be “/local0/backup/username/full” or “weekly/xx”
• Double check whether you have made these changes. Otherwise when you run your backup, data will not be backed up.

5. Setting up crontab

Crontab will schedule your weekly and monthly backups automatically on DT2

• When you are on DT, type crontab -e, a crontab window will open up
• Use the following as a template to create your crontab

 SHELL=/bin/csh
# Weekly Incremental Backups
# Sat at 0X:0X am
0X 0X * * 6 /homes/usrname/backup/backup-weekly.csh
# Monthly Full Backups
# 1st day of month at 11:01 am
0X 0X 1 * * /homes/usrname/backup/backup-full.csh 

• 0X:0X am is the time at which your data will be automatically backed up every Saturday and first day of the month. We keep our backup times few hours apart from each other. Pick random numbers for X (single digit) to prevent everyone starting this at the same time)
• When you save and exit, it will install crontab
• Check whether crontab is installed or not using crontab -l

6. Run backup-full to backup your data:

./backup-full.csh & 

and check when it is completed correctly on the backup machine several hours later.

Note: You must do this for DT2 or any new resource at UMD if you have data on these machines. Make sure you check on Monday that you've backup up the data on the weekly backup.