Installing Windows NT4 PDC or BDC in a VMware vSphere 4 VM

Problem: When attempting to install Windows NT4 as a PDC or BDC, you receive the error "The domain controller for this Domain can not be located." and you cannot proceed with installation.

Reason: The AMD PCnet driver that ships with NT4 does not work properly with the new version of the virtual NIC in vSphere and the device fails to initialize.

Solution: Power off the VM (do not shut down). In the VM "Edit Settings" dialog, add a virtual USB device. Power on the VM and the configuration phase should re-start. Now the driver should function properly, the device (virtual NIC) should initialize and TCP/IP stack should be working which then allows you to authenticate against the Domain.

So I recently came across a situation where I needed to upgrade a customer from a Windows NT Domain to a Windows 2003 Domain. The PDC and BDC were on ancient hardware that was flaky and it was decided that the best way to proceed was to create a new NT4 VM, make it a BDC, promote it to a PDC, then do an in-place upgrade to Server 2003.

There were several gotchas with this approach, not the least of which was that NT4 will only allow you to install to a 4GB partition (unless you want to jump through a bunch of hoops). With NT4 you also have to select to make the server a PDC, BDC or Stand Alone server at install. You cannot promote the server via something like DCPROMO. There is a 3rd party utility that will do this, but it costs money and I just don't trust it. I won't mention what that is here on purpose because of this.

In vSphere (ESX/i 4.x branch) there seems to be some kind of bug in which NT4 can 'see' the AMD PCnet virtual network card, but the driver that NT4 uses will not work with it. Because of this, the virtual NIC device does not initialize properly and there is no TCP/IP traffic. So, when you get to the point where you enter credentials to join the NT4 server to the Domain as a PDC or BDC, you get the error "The domain controller for this domain can not be located." and you cannot proceed.

I do know that this was not the case with the VI3 (ESX 3.x branch) versions of the virtual NIC. So somewhere between VI3 and vSphere, something was changed that hoses the networking configuration during installation. I remembered running into something like this a while back on the free VMware Server product and the solution was very odd. After searching around, I found that the 'tweak' was to add a virtual USB device to the VM via the "Edit Settings" dialog. I did this to the VM and wouldn't you know the stock NT4 AMD PCnet driver worked! I'm not sure why this is and I don't really have time to trace the root cause right now, but if you run into this problem, that is a solution.

I hope this helps some other poor soul out there, but I would imagine not too many people are messing with NT4 now a days anyway. haha