Virtualization help needed (You too, lurkers)

Freebase Dali · 10-11-2012, 05:34 PM

First, before you read all of this, if you don't know anything about server virtualization, networking, and in particular: Server 2008 R2 and the Hyper-V role, I will save you the anger of having read through all of it for nothing.

TL;DR
011101010010101011101010001010somethingaboutafluxc apacitor10101001010101...

Ok.
Some of you may be able to give me some fresh ideas about how to continue troubleshooting a problem I've been having for the last few days at work when trying to get network connectivity from some VM guests in Hyper-V.
I've literally spent 8 hours straight getting absolutely nowhere. Here is a breakdown:

I have an HP Proliant server (exact model may be important if it's driver issues, but I'll leave that out for now) with 16 gigs of RAM and 2 1TB data drives in a RAID 1.
I have Server 2008 R2, Enterprise installed on it, with the Hyper-V role installed. Integration services are active. There's 1 NIC that connects at 10gig, which only works at auto-negotiation speed and duplex for some reason. Not a problem, though.
The Hyper-V network manager is all set up, and the VMs are using the virtual network that's connected to the physical NIC.
The physical NIC has a static private address, and the router/firewall outside it has 1-to-1 NAT so that a public, routable address is mapped to it. All necessary ports are open to any. From the host server, connectivity is established to the outside world no problem.

All updates are current.

Now the VMs.
I have 3 VMs, all with 2008R2 installed, fulfilling various roles. One VM is the DC with AD installed and DNS installed. That server uses a trusted external DNS server for queries it has no records for. The other is the Hub Transport Exchange server with the typical roles installed for that, is joined to the AD domain, and uses it as its primary DNS server, and the other is the Edge Transport Server with that role installed, not joined to the domain, with the trusted external DNS server for name resolution. I've already attained good sync between the ETS and Hub Transport.
All updates current. They too have public IPs mapped to their static addresses. All necessary ports are open for those mappings as well.
Upon initial installation and TCP/IP configuring of all three VMs, connectivity is good both inside the network, and to the outside world. I'm able to remote into all three VMs, and ping back and forth. At that point, it's basically all good.

Here is where it gets weird.
For whatever reason, suddenly the mail server (hub transport) loses internet connectivity. No changes are made to prompt this. The connection just drops out. Connection inside the network is still ok, and each VM sees one another, and the host server, as they are all inside the same network.
At the same time, the DC will still have connectivity. Then, the DC will lose connectivity and the Edge Transport server will suddenly have it. Then eventually, just the DC will retain connectivity, and the remaining two will not.

I'm convinced there's no DNS issue here, as the Edge Transport uses an external DNS, and I've tried different servers for that. Besides, that's just for name resolution. I should still be able to ping the server, provided there's connectivity through and through.

I've trawled the web and tried every viable solution I've found regarding the issue, but the majority of the solutions involve installing a particular update that doesn't apply to my situation (I have no network load, as the server is not in production right now.) and upon trying to install it, it says it does not apply to my server.

I've tried various disabling of offloading features, to no effect. Restarting the VMs produces no result. Restarting the host server does, SOMETIMES. This leads me to believe it's an issue between the host server and Hyper-V, possibly a driver conflict for the particular NIC and Hyper-V's integration services. The problem is, I'd literally have to install a different NIC into the server to really test that theory, and that involves both getting the company to buy a new server NIC, and driving all the way to the datacenter to install it.

Since I want to exhaust all possible avenues prior to going that route, I'm simply wondering if any of our resident tech-nerds and possibly some lurkers that don't post may have any ideas, because I need something fresh after spending 8 hours exhausting my own ideas and those I've found on the web.

Thanks in advance.

10-11-2012, 05:34 PM	#1 (permalink)
Freebase Dali Partying on the inside Join Date: Mar 2009 Posts: 5,584	Virtualization help needed (You too, lurkers) First, before you read all of this, if you don't know anything about server virtualization, networking, and in particular: Server 2008 R2 and the Hyper-V role, I will save you the anger of having read through all of it for nothing. TL;DR 011101010010101011101010001010somethingaboutafluxc apacitor10101001010101... Ok. Some of you may be able to give me some fresh ideas about how to continue troubleshooting a problem I've been having for the last few days at work when trying to get network connectivity from some VM guests in Hyper-V. I've literally spent 8 hours straight getting absolutely nowhere. Here is a breakdown: I have an HP Proliant server (exact model may be important if it's driver issues, but I'll leave that out for now) with 16 gigs of RAM and 2 1TB data drives in a RAID 1. I have Server 2008 R2, Enterprise installed on it, with the Hyper-V role installed. Integration services are active. There's 1 NIC that connects at 10gig, which only works at auto-negotiation speed and duplex for some reason. Not a problem, though. The Hyper-V network manager is all set up, and the VMs are using the virtual network that's connected to the physical NIC. The physical NIC has a static private address, and the router/firewall outside it has 1-to-1 NAT so that a public, routable address is mapped to it. All necessary ports are open to any. From the host server, connectivity is established to the outside world no problem. All updates are current. Now the VMs. I have 3 VMs, all with 2008R2 installed, fulfilling various roles. One VM is the DC with AD installed and DNS installed. That server uses a trusted external DNS server for queries it has no records for. The other is the Hub Transport Exchange server with the typical roles installed for that, is joined to the AD domain, and uses it as its primary DNS server, and the other is the Edge Transport Server with that role installed, not joined to the domain, with the trusted external DNS server for name resolution. I've already attained good sync between the ETS and Hub Transport. All updates current. They too have public IPs mapped to their static addresses. All necessary ports are open for those mappings as well. Upon initial installation and TCP/IP configuring of all three VMs, connectivity is good both inside the network, and to the outside world. I'm able to remote into all three VMs, and ping back and forth. At that point, it's basically all good. Here is where it gets weird. For whatever reason, suddenly the mail server (hub transport) loses internet connectivity. No changes are made to prompt this. The connection just drops out. Connection inside the network is still ok, and each VM sees one another, and the host server, as they are all inside the same network. At the same time, the DC will still have connectivity. Then, the DC will lose connectivity and the Edge Transport server will suddenly have it. Then eventually, just the DC will retain connectivity, and the remaining two will not. I'm convinced there's no DNS issue here, as the Edge Transport uses an external DNS, and I've tried different servers for that. Besides, that's just for name resolution. I should still be able to ping the server, provided there's connectivity through and through. I've trawled the web and tried every viable solution I've found regarding the issue, but the majority of the solutions involve installing a particular update that doesn't apply to my situation (I have no network load, as the server is not in production right now.) and upon trying to install it, it says it does not apply to my server. I've tried various disabling of offloading features, to no effect. Restarting the VMs produces no result. Restarting the host server does, SOMETIMES. This leads me to believe it's an issue between the host server and Hyper-V, possibly a driver conflict for the particular NIC and Hyper-V's integration services. The problem is, I'd literally have to install a different NIC into the server to really test that theory, and that involves both getting the company to buy a new server NIC, and driving all the way to the datacenter to install it. Since I want to exhaust all possible avenues prior to going that route, I'm simply wondering if any of our resident tech-nerds and possibly some lurkers that don't post may have any ideas, because I need something fresh after spending 8 hours exhausting my own ideas and those I've found on the web. Thanks in advance.