Manually Migrating Files from InfiniDisc to Vault

In some cases, you may wish to move a lot of files stored on an InfiniDisc, via an external hard drive, rather than over a network connection to an OSVault server.   There are several reasons for doing this, including initial setup of an InfiniDisc/OSVault configuration or trying to reduce a large backlog on the InfiniDisc, or just to keep from swamping an Internet connection.

To manually move(migrate) files, you follow a four step process.  First you have to select and copy files to the external hard drive attached to the InfiniDisc system.  This is probably best done in  Command Line Interface (CLI) rather than via a GUI.  Then the second step is to ship the drives to the OSVault.  The third step is to copy off the files from the external drive to the OSVault cache; and the fourth step is to confirm that the files are properly copied and to mark them migrated on the InfiniDisc.

Step One
This assumes that you have hooked up an external drive and mounted it as /tmp/mnt on the InfiniDisc system.  Once this is done, you can run the "find" command on the InfiniDisc to move files to the external drive.  The "find" command used is something like this:
          find /cache/ -type f +mtime +30  -exec /root/test_migrated {} \; -exec cp {} /tmp/mnt/ \; -exec attr -s nomigrate -V 1 {} \;
This single command will search the /cache directory on your InfiniDisc for regular files ( -type f ) older than 30 days (+mtime +30), will see if they have been migrated (-exec /root/test_migrated {} \;) and if they are NOT migrated, will copy them to the external hard drive (-exec cp {} /tmp/mnt \;) and then mark the file as non-migratable (-exec attr -s nomigrate -V 1 {} \;).  Setting the "nomigrate" extended attribute on the file will insure that the InfiniDisc does not try to migrate the file while the external hard drive is in-transit.

Step Two
Unmount the external hard drive, disconnect it, and move it to the OSVault server and plug it in there.  It then gets mounted a /tmp/mnt on the OSVault system.

Step Three
You now need to copy the files from the external hard drive to the OSVault /cache for management.  There are many ways to do this, but a favorite for us is to use RSYNC, so that if anything happens to the copy operation, it can be restarted where it left off, rather than starting from scratch again.  The RSYNC command would be:
           rsync -avz --ignore-existing /tmp/mnt /cache
The "--ignore-existing" flag keeps the system from copying any files that may have already migrated to the OSVault system, or allows you to kill this command and restart it anytime.

Step Four
You now need to tell the InfiniDisc system to check that all the files marked "nomigrate" in step one are successfully copied to the OSVault and to mark them "copied".  You do this with the "net_verify" CLI command.  Net_+verify will search a directory and sub-directories, will find any files marked "nomigrate" and will run a SSH check to the OSVault to make sure that the file is truly copied to the OSVault.  The syntax of the net_verify command is:

                    net_verify -n -s /cache  -f

The "-n" flag tells net_verify to look for "nomigrate" files, and the "-f" flag tells net_verify to mark the files as "copied" if they are confirmed to be on the remote Vault.

The /root/test_migrated script may not be on your InfiniDisc box.  If you, you can create it with a text editor in the /root directory and mark it executable (chmod 0755 /root/test_migrated).  The contents of test_migrated are:

attr -q -g migrated "$1">/dev/null

if [ $? -eq 0 ]

  then

exit 1

  else

exit 0

fi