Windows Azure Cloud Drive–Dealing with large VHD’s and Blob Snapshot Restores

I ran into a scenario where I needed to transfer data from an Azure Cloud Drive VHD that was 250GB in Size from Production to Preproduction. So this means that I want to transfer the VHD from one azure account to another.

The thing with VHD’s and Cloud Drive, is that it is a Page Blob, and the VHD has to be a fixed size, so even though I might have 200MB of data in a 250GB VHD, you would need to download 250GB worth of data.

The Solution?

Remove desktop into the Azure Virtual Machine, Mount the VHD Manually, copy the data, zip it up and send it through other means, so in essence, I only send or download the data that is USED in the VHD i.e. 200MB and not 250GB.

This utility can use the concept of a blobsnapshot VHD backup to restore, and what it will do it mount it. This is ideal when you using blob snapshots as a backup mechanism and you need a way to restore the blob snapshot VHD data, as fast as possible to another Azure Account.

Below is the code for the helper and you can download the source code for the project here:

NOTE: This application must be run in a Windows Azure Virtual Machine, it will NOT WORK on a development machine/emulator.

hg clone ssh://hg@bitbucket.org/romiko/mountclouddrive
hg clone https://romiko@bitbucket.org/romiko/mountclouddrive

using System;
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.StorageClient;
using MountCloudDrive.Properties;

namespace MountCloudDrive
{
    public class CloudDriveHotAssistant
    {
        private string restoreVHDName;
        private readonly StorageCredentialsAccountAndKey storageCredentials;
        private readonly CloudStorageAccount cloudStorageAccount;
        private CloudBlobClient blobClient;

        public CloudDriveHotAssistant(string accountName, string accountKey)
        {
            restoreVHDName = Settings.Default.DefaultRestoreDestination;
            storageCredentials = new StorageCredentialsAccountAndKey(accountName, accountKey);
            cloudStorageAccount = new CloudStorageAccount(storageCredentials, false);
            blobClient = new CloudBlobClient(cloudStorageAccount.BlobEndpoint, storageCredentials);
        }

        public CloudPageBlob GetBlobToRestore(string blobContainer, string blobFileName)
        {
            DateTime snapShotTime;
            var converted = DateTime.TryParse(Settings.Default.BlobSnapShotTime, out snapShotTime);

            CloudPageBlob pageBlob;

            if (converted)
                pageBlob = blobClient.GetPageBlobReference(string.Format(@"{0}\{1}", blobContainer, blobFileName), snapShotTime);
            else
                pageBlob = blobClient.GetPageBlobReference(string.Format(@"{0}\{1}", blobContainer, blobFileName));

            try
            {
                pageBlob.FetchAttributes();
            }
            catch (Exception ex)
            {
                Console.WriteLine("\r\nBlob Does Not Exist!");
                Console.WriteLine(ex);
                return null;
            }
            return pageBlob;
        }

        public void UnMountCurrentDrives()
        {
            foreach (var driveName in CloudDrive.GetMountedDrives())
            {
                var drive = new CloudDrive(driveName.Value, storageCredentials);
                Console.WriteLine(string.Format("\r\nUnmounting {0}", driveName.Key));
                Console.WriteLine("\r\nAre you sure Y/N");
                var key = Console.ReadKey();
                var decision = key.Key == ConsoleKey.Y;

                if (!decision)
                    continue;

                try
                {
                    drive.Unmount();
                }
                catch (Exception ex)
                {
                    Console.WriteLine(string.Format("\r\nUnmounting {0} FAILED.\r\n {1}", driveName.Key, ex));
                }
            }
        }

        public CloudPageBlob GetBlobReferenceToRestoreTo(string blobContainer)
        {
            return blobClient.GetPageBlobReference(string.Format(@"{0}\{1}", blobContainer, restoreVHDName));
        }

        public void MountCloudDrive(CloudPageBlob pageBlobSource, CloudPageBlob pageBlobDestination, string blobContainer)
        {
            pageBlobDestination.CopyFromBlob(pageBlobSource);
            Console.WriteLine(string.Format("\r\nAttempting to mount {0}", pageBlobDestination.Uri.AbsoluteUri));
            var myDrive = cloudStorageAccount.CreateCloudDrive(pageBlobDestination.Uri.AbsoluteUri);
            var drivePath = myDrive.Mount(0, DriveMountOptions.None);
            Console.WriteLine(string.Format("\r\nVHD mounted at {0}", drivePath));
        }
    }
}
About these ads

2 thoughts on “Windows Azure Cloud Drive–Dealing with large VHD’s and Blob Snapshot Restores

  1. Hi Romiko,
    I’ve found your blog while searching for online backup software using Azure or Amazon S3 storage.
    BTW, your Powershell scripts are really cool!
    Do you have any ideas on how to do block level backup to Azure or S3? For example of my 500GB of data there is only 1 or 2 GB of changes a day. I would only want to upload those changes instead of the whole thing. (Some of the files are really large, and only have small changes to them.)
    Making a local disk based backup and syncing it up to the cloud would be even better, as long as only the changed blocs would sync.

    • Hi George,

      Perhaps, you could put a boot strapper on the virtual machine that has mounted the VHD that loads backup software/agent that can then do the differential backups or you could could manually write a script that enumerates all files and folders and creates a list of files that changed. Easiest is of course a backup solution from a 3rd party, currently I am not aware of differential backup in Azure blob storage for page blobs, would be cool though :)

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s