Active Directory disaster recovery with DPM

Question

I have a sort of catch-22 question here.

Suppose I'm using Microsoft System Center Data Protection Manager (2010 or 2012, it works the same way) to backup, amongst various other things, my Active Directory environment (as in "the System State of my domain controllers").

Then, a complete data center lost occurs. I have to start fresh on new hardware, I only have my tape backups available because they were stored off-site. So I buy some new servers, a new tape library, new storage, and so on.

Now, everyone knows (or should know) that in order to perform an Active Directory disaster recovery I need to at least restore the system state of a domain controller; of course, this can get... tricky if I need to restore it on different hardware from the original server, but let's also assume this point is covered.

Howewer, and here's the catch, DPM needs Active Directory in order work; it won't even install on a standalone server. But, of course, a working DPM server is needed in order to get back those backups from the tapes.

How can I restore my Active Directory environment starting with only new servers and DPM tape backups?

N.B. Using virtual domain controllers and backing up the full VMs could make the restore easier, but actually doesn't change the question at all: a working AD environment is still needed in order to even install DPM.

Solution 1

So far, I've been able to come up with the following procedure, but I really hope there is some simpler way:

• Install the operating system on a new server
• Create a new "dummy" domain and make the server its domain controller
• Install the operating system on a second server
• Join the server to the "dummy" domain
• Install DPM on the second server and connect it to the tape library
• Restore the DPM database (*)
• Find the tape with a system state backup of a domain controller
• Restore the system stabe backup to a network location
• Throw away everything except the restored backup
• Install the operating system on the new domain controller
• Restore the system state backup on the new domain controller
• Verify that the restored AD is working properly
• Install the operating system on the new DPM server
• Join the new DPM server to the restored domain
• Install DPM on the new DPM server and connect it to the tape library
• Restore the DPM database
• Start restoring everything else according to your DR plan

This solution is clumsy, long and somewhat awkward, but it should work; my only concern is about restoring the DPM database for the first time (the step marked with (*) in the list), because I don't know if this could work when running on a different AD domain. If this doesn't work, then the only solution would be manually importing the tape containing the system state backup of a DC... and good luck finding it if you have decent-sized backups.
But of course, this also applies to finding the backup of the DPM database in the first place...

Solution 2

We backup the DPM server separately (via command line schedeuled task) weeklyt, and DPM database daily.

That way we can bootstrap the DPM server from non-DPM managed backups, and logon works with cached domain credentials. Then I can start restoring "real" backups from our virtual tape library.

This works because the DPM server uses a local database with local logon, because we wanted the unit to be as standalone as possible. If your server uses a remote database, this may not work for you.

Solution 3

Backup your DCs to Azure. It's extremely cheap (100GB costs $10/month) and super easy to use. Then the recovery of AD only requires the following:

• access to your Azure subscription - shouldn't be a problem
• the passphrase used for encrypting the Azure backups - save it offsite, to your pendrive where you store SSH/BitLocker/etc keys, or something

Then you can recover on a completely new, temporary Windows Server without any domains (new or existing) involved. That's right, you don't need to join it to any domain. The procedure looks like this:

1. Go to Azure / Recovery Services

2. Open the appropriate Backup Vault

   • Download the Azure Backup Agent for Windows Server
   • Download Vault credentials

3. Install agent on the temp server

4. Register Server Wizard

   • specify downloaded credentials
   • Generate Passphrase <- save it, though shouldn't be too important as this server is only for temporary use

5. Start / Microsoft Azure Backup / Recover Data

6. Recover Data Wizard

   • Another server / specify the downloaded credentials again
   • Select Backup Server / (your old DPM server)
   • Browse for files
   • VM storage will be specified as full paths instead of friendly names but it will work nevertheless
   • Once you select the data to recover, it will ask for the passphrase you used on the OLD DPM server to encrypt your stuff in the cloud, so that's why you absolutely need offsite backup of this passphrase. If you don't have it, you're scr*wed.

And that's it. I've tested it, it works :)

Comments

[pauska]

Honestly - Doing system state of a domain controller is to protect yourself against accidental deleting of objects, or corruption of the database. You really should have a second (geographically different) site with a domain controller in it for disaster recovery scenarios.

[Massimo]

And I usually have. But this is a complete disaster recovery scenario, and needs to go in a disaster recovery plan even if it's very unlikely. Also, not every business spans multiple sites or can afford a spare "hot" datacenter.

[alx9r]

"We backup the DPM server separately..." -- For this to work in the case where the whole site is wiped out, some of these separate backups must be offsite. I'm curious how you achieved that.

[alx9r]

It seems like the first 5-1/2 bullets could be implemented in advance as VMs on a netbook that is kept in the same vault as the tapes -- a sort of bootstrap recovery workstation if you will. You could also have a prepared VM for the DC on the netbook ready to accept the system state and another ready to join the domain and become the new DPM server. You might as well have a fileshare on that netbook with all the install media that you'll need as well.

[Juan Penaloza]

Hi, with a good old scheduled task of wbadmin start backup -quiet -allCritical -systemState -vssFull . We also back up the local MSDPM2010 instance with BACKUP DATABASE [DPMDB] TO DISK ... That way, the DPM server can be standalone bootstrapped and reinventoried for recovery.