Error adding child Active Directory domain to existing forest

Question

I'm building a test environment containing multiple Active Directory domains in the same forest, but I'm having strange issues while trying to add a child domain to the forest root domain.

All servers are Windows Server 2012 R2 VMs running on the Azure cloud platform, connected to the same virtual network; they have statically reserved IP addresses and they can talk to each other without any networking issue.

My domain structure is (or at least should be) as follows:

    A0.lab (forest root)            B0.lab
   /  \                            /  \
  A1  A2                          B1  B2
  |                               |
  A3                              B3

Thus:

• A0.lab (forest root)
• A1.A0.lab
• A2.A0.lab
• A3.A1.A0.lab
• B0.lab
• B1.B0.lab
• B2.B0.lab
• B3.B1.B0.lab

I've created the forest root domain (A0.lab) successfully and I've defined an AD site and its subnet; the domain is operating correctly.

Next, I've configured the server which should become the domain controller for the first child domain (A1.A0.lab) to use the root DC as its DNS server, and I've started the promotion wizard; I've filled in all the parameters, including the user account of the domain admin for the root domain and the option to create a DNS delegation; all the prerequisite checks are successful.

When I start the actual promotion process, it stalls at the "Replicating the schema directory partition" stage. The "Directory Service" event log is repeatedly filled with several errors:

Event ID 1963, source ActiveDirectory_DomainService, task category DS RPC Client:

Internal event: The following local directory service received an exception from a
remote procedure call (RPC) connection. Extensive RPC information was requested. This
is intermediate information and might not contain a possible cause. 

Process ID:  
540  

Reported error information:  
Error value:  
Could not find the domain controller for this domain. (1908)  
directory service:  
DCA0.a0.lab  

Extensive error information:  
Error value:  
A security package specific error occurred. 1825  
directory service:  
DCA1  

Additional Data  
Internal ID:  
5000e02

Event ID 1961, source ActiveDirectory_DomainService, task category DS RPC Client:

Internal event: This log entry is a continuation from the preceding extended error
information entry on the following error and directory service. 

Extended information:  
Error value:  
A security package specific error occurred. (1825)  
directory service:  
DCA1  

Supplemental information:  
Detection location:  
1461  
Generating component:  
RPC Runtime  
Time at directory service:  
2015-03-19 21:44:04  

Additional Data  
Error value:  
A security package specific error occurred. (1825)

Event ID 2839, source ActiveDirectory_DomainService, task category DS RPC Client:

Internal event: This log entry is a continuation from the preceding extended error
information entry. 

Extended information:  
Extended Error Parameters:  
0  
Parameter 1:  
(NULL)  
Parameter 2:  
(NULL)  
Parameter 3:  
(NULL)  
Parameter 4:  
(NULL)  
Parameter 5:  
%6  
Parameter 6:  
%7  
Parameter 7:  
%8

Event ID 1962, source ActiveDirectory_DomainService, task category DS RPC Client:

Internal event: The local directory service received an exception from a remote
procedure call (RPC) connection. Extended error information is not available. 

directory service:  
DCA0.a0.lab  

Additional Data  
Error value:  
Could not find the domain controller for this domain. (1908)

Event ID 1125, source ActiveDirectory_DomainService, task category Setup:

The Active Directory Domain Services Installation Wizard (Dcpromo) was unable to
establish connection with the following domain controller. 

Domain controller:
DCA0.a0.lab 

Additional Data  
Error value:  
1908 Could not find the domain controller for this domain.

Those errors are repeated again and again, but there is no progress or failure, the promotion process just remains stalled.

Here are the contents of the dcpromo.log file:

03/19/2015 22:43:35 [INFO] Promotion request for domain controller of new domain
03/19/2015 22:43:35 [INFO] DnsDomainName  a1.a0.lab
03/19/2015 22:43:35 [INFO]  FlatDomainName  A1
03/19/2015 22:43:35 [INFO]  SiteName  Lab
03/19/2015 22:43:35 [INFO]  SystemVolumeRootPath  C:\Windows\SYSVOL
03/19/2015 22:43:35 [INFO]  DsDatabasePath  C:\Windows\NTDS, DsLogPath  C:\Windows\NTDS
03/19/2015 22:43:35 [INFO]  ParentDnsDomainName  a0.lab
03/19/2015 22:43:35 [INFO]  ParentServer  DCA0.a0.lab
03/19/2015 22:43:35 [INFO]  Account A0\AdmA0
03/19/2015 22:43:35 [INFO]  Options  5243072
03/19/2015 22:43:35 [INFO] Validate supplied paths
03/19/2015 22:43:35 [INFO] Validating path C:\Windows\NTDS.
03/19/2015 22:43:35 [INFO]  Path is a directory
03/19/2015 22:43:35 [INFO]  Path is on a fixed disk drive.
03/19/2015 22:43:35 [INFO] Validating path C:\Windows\NTDS.
03/19/2015 22:43:35 [INFO]  Path is a directory
03/19/2015 22:43:35 [INFO]  Path is on a fixed disk drive.
03/19/2015 22:43:35 [INFO] Validating path C:\Windows\SYSVOL.
03/19/2015 22:43:35 [INFO]  Path is on a fixed disk drive.
03/19/2015 22:43:35 [INFO]  Path is on an NTFS volume
03/19/2015 22:43:35 [INFO] Child domain creation -- check the new domain name is child of parent domain name.
03/19/2015 22:43:35 [INFO] Domain Creation -- check that the flat name is unique.
03/19/2015 22:43:40 [INFO] Start the worker task
03/19/2015 22:43:40 [INFO] Request for promotion returning 0
03/19/2015 22:43:42 [INFO] Using supplied domain controller: DCA0.a0.lab
03/19/2015 22:43:42 [INFO] Using supplied site: Lab
03/19/2015 22:43:42 [INFO] Forcing time sync
03/19/2015 22:43:42 [INFO] Forcing a time sync with DCA0.a0.lab
03/19/2015 22:43:42 [INFO] Reading domain policy from the domain controller DCA0.a0.lab
03/19/2015 22:43:42 [INFO] Stopping service NETLOGON
03/19/2015 22:43:42 [INFO] Stopping service NETLOGON
03/19/2015 22:43:42 [INFO] ControlService(STOP) on NETLOGON returned 0(gle=1062)
03/19/2015 22:43:42 [INFO] Exiting service-stop loop after service NETLOGON entered STOPPED state
03/19/2015 22:43:42 [INFO] StopService on NETLOGON returned 0
03/19/2015 22:43:42 [INFO] Configuring service NETLOGON to 1 returned 0
03/19/2015 22:43:42 [INFO] Stopped NETLOGON
03/19/2015 22:43:42 [INFO] Creating the System Volume C:\Windows\SYSVOL
03/19/2015 22:43:42 [INFO] Deleting current sysvol path C:\Windows\SYSVOL 
03/19/2015 22:43:44 [INFO] Preparing for system volume replication using root C:\Windows\SYSVOL
03/19/2015 22:43:44 [INFO] Created the system volume
03/19/2015 22:43:44 [INFO] Copying initial Directory Service database file C:\Windows\system32\ntds.dit to C:\Windows\NTDS\ntds.dit
03/19/2015 22:43:44 [INFO] Installing the Directory Service
03/19/2015 22:43:44 [INFO] Calling NtdsInstall for a1.a0.lab
03/19/2015 22:43:44 [INFO] Starting Active Directory Domain Services installation
03/19/2015 22:43:44 [INFO] Validating user supplied options
03/19/2015 22:43:44 [INFO] Determining a site in which to install
03/19/2015 22:43:44 [INFO] Examining an existing forest...
03/19/2015 22:43:44 [INFO] Configuring the local computer to host Active Directory Domain Services
03/19/2015 22:43:48 [INFO] EVENTLOG (Informational): NTDS General / Service Control : 1094  
Software write caching for the following disk drive has been disabled to prevent possible data loss during system failures such as power outages or hardware component failures that can cause a sudden shutdown of the system. The disk drive that stores Active Directory Domain Services log files is the only drive affected by this change.

Disk drive:  
c:

03/19/2015 22:43:59 [INFO] EVENTLOG (Informational): NTDS Database / Internal Processing : 2013  
Active Directory Domain Services is rebuilding the following number of indices as part of the initialization process.

Number of indices:  
1

Indices:  
LCL_ABVIEW_index00000410 +ATTb590468 

03/19/2015 22:43:59 [INFO] EVENTLOG (Informational): NTDS Database / Internal Processing : 2014  
Active Directory Domain Services successfully completed rebuilding the following number of indices.

Indices:  
1

03/19/2015 22:44:00 [INFO] EVENTLOG (Informational): NTDS General / Internal Configuration : 2120  
This Active Directory Domain Services server does not support the Recycle Bin. Deleted objects may be undeleted, however, when an object is undeleted, some attributes of that object may be lost.  Additionally, attributes of other objects that refer to the object being undeleted may also be lost.

03/19/2015 22:44:00 [INFO] EVENTLOG (Informational): NTDS General / Internal Configuration : 2405  
This Active Directory Domain Services server does not support the "Recycle Bin Feature" optional feature.

03/19/2015 22:44:00 [INFO] Replicating the schema directory partition

After this, the same errors reported in the event log are logged.

I've found this article which states this error can happen if the Administrator account has the same password on the new DC and on the domain you're logging on to; I'm not using at all the built-in Administrator account, since these are Azure VMs, but I was actually using the same username and password on all servers during my first test, thus I guessed this could indeed have been the reason for the error; however, I've since rebuilt all servers, and created a distinct local admin account on each one (AdmA0, AdmA1, AdmA2...), with a distinct password; I've also made sure to specify the credentials for the parent domain in the form A0\AdmA0; but the error happened again.

What's happening, and how can I fix it?

Solution

Looks like I'm running into (a variant of?) this issue: the promotion completes successfully if I use "long" logon credentials, i.e. A0.lab\AdmA0 instead of A0\AdmA0.

However, based on the article, this issue should only happen if NetBIOS over TCP/IP is disabled, but it's actually enabled, and this can be verified in the ipconfig output. I also tried configuring the VMs with static network settings instead of using DHCP (which is required by Azure), and forcing NetBIOS over TCP/IP to "Enabled", but the error always happens; the only way for the promotion process to work is by using "long" credentials.

However, this definitely seems to be an Azure-specific quirk: I have created an identical test environment on a local Hyper-V server, and everything works as it should.

Looks like either Azure is doing something strange at the network level which block NetBIOS, or the Azure Windows Server 2012 R2 VM templates have some strange NetBIOS-related behavior which makes DC promotion fail in this peculiar way.

---

Update:

Culprit found: https://msdn.microsoft.com/en-us/library/azure/dn133803.aspx.

Does Virtual Network support multicast or broadcast?

No. We do not support multicast or broadcast.

Azure virtual networks don't support broadcast; thus, even if NetBIOS is enabled, it just doesn't work. And it looks like Windows Server 2012 R2 really needs it for a DC promotion to work.

Workaround: use "long" logon credentials during DC promotion (full.domain.fqdn\username instead of NetBIOSDomain\username).

---

As for why Azure virtual networks don't support broadcast, and how can they manage to do that while still relying so heavily on DHCP... that's beyond my ability to understand. And I'm not quite sure I really want to understand; Azure networking is well known to be rather peculiar.

Comments

[Mathias R. Jessen]

Is the new child Domain Controller in the same network segment as the A0.lab DC's? You can broadcast NetBIOS requests for A0 all you want, but if nobody is answering, you won't get far :-)

[Massimo]

Yes, all servers are in the same subnet.

[Massimo]

(Also, domain authentication in 2015 shouldn't really be relying on NetBIOS at all...)

[Ben]

Thank you very much for sharing this. Encountered the same issue but with an on premises AD physical server. Spent a long time looking at DNS but everything appeared to look fine. Using the UPN suffix instead of domain\username appears to be key in getting this work. Added info. To stop the dcpromo progress, I logged off the server. I then uninstalled ADDS roles, restarted. I then had to disjoin the domain in didn't finish creating and switched to a workgroup. Restarted. Installed ADDS roles, promoted as child DC in forest using UPN suffix instead of domain\username.